spark-compaction-plugin

Make Spark to compact small files like Hive, support Parquet/ORC/RC file, set spark.sql.sources.commitProtocolClass=com.github.yantzu.compaction.CompactFilesCommitProtocol to enable.