ORC Source

The ORC source is a specific subset of the of the File Source. As such, File Source options may also be configured on an ORC source. As always, the path option of the File Source must be configured. Please review the File Source if you are not already familiar with it.

Options

spark.hadoop.orc.*

Allows specifying internal settings for the underlying ORC file readers. Typically, users will only modify these settings for use cases requiring fine grained tuning. If you are unfamiliar with these settings, you can use the OrcConf class as a reference.

-- spark.properties: spark.hadoop.orc.skip.corrupt.data=true
CREATE STREAM foo
FROM ORC
OPTIONS();

spark.sql.orc.filterPushdown

When true, enables predicate push-down optimizations when reading ORC files.

Defaults to false.

-- spark.properties: spark.sql.orc.filterPushdown=true
CREATE STREAM foo
FROM ORC
OPTIONS();