Preserve original field order during schema evolution

This overcomes a limitation with how Hudi syncs schemas to the Glue
catalog. Previously, if version `1-0-0` of a schema had fields `a` and
`b`, and then vesion `1-0-1` adds a field `c`, then the new field might
be added _before_ the original fields in the Hudi schema.  The new field
would get synced to Glue, but only for new partitions; it is not
back-filled to existing partitions.

After this change, the new field `c` is added _after_ the original
fields `a` and `b` in the Hudi schema.  Then there is no need to sync
the new field to existing partitions in Glue.

The problem manifested in AWS Athena with a message like:

> HIVE_PARTITION_SCHEMA_MISMATCH: There is a mismatch between the table and partition schemas.

This fix was implemented in snowplow/schema-ddl#213 and
snowplow-incubator/common-streams#98 and imported via a new version of
common-streams.

This change does not impact Delta or Iceberg, where nothing was broken.

modules/core/src/main/resources/reference.conf

@@ -57,9 +57,8 @@
         # -- Record key and partition settings. Chosen to be consistent with `hudiTableOptions`.
         "hoodie.keygen.timebased.timestamp.type": "SCALAR"
         "hoodie.keygen.timebased.output.dateformat": "yyyy-MM-dd"
-        "hoodie.datasource.write.reconcile.schema": "true"
         "hoodie.datasource.write.partitionpath.field": "load_tstamp"
-        "hoodie.schema.on.read.enable": "true"
+        "hoodie.write.set.null.for.missing.columns": "true"
         "hoodie.metadata.index.column.stats.column.list": "load_tstamp,collector_tstamp,derived_tstamp,dvce_created_tstamp,true_tstamp"
         "hoodie.metadata.index.column.stats.enable": "true"
         "hoodie.datasource.write.keygenerator.consistent.logical.timestamp.enabled": "true"

project/Dependencies.scala

@@ -49,7 +49,7 @@ object Dependencies {
     val awsRegistry = "1.1.20"
 
     // Snowplow
-    val streams    = "0.8.0"
+    val streams    = "0.8.2-M1"
     val igluClient = "4.0.0"
 
     // Transitive overrides