Maxwell changelog

Maxwell gets the ability to talk to bigtable! I have no idea how well it'll work. I hope it works for you!
upgrade protobuf to fix a rabbitmq issue with booleans, I think.
rabbitMQ timeouts on connection
other fixes.
I can't imagine the security department cares about my naming what with what's going on inside 1019. I guess we'll see.

Released 2022-07-29

v1.37.7

Bump viafoura/metrics-datadog 2.0.0-RC3

Released 2022-06-21

v1.37.6

In non-GTID mode, Verify that the master's server hasn't changed out from underneath us. thanks Tamin Khan

Released 2022-05-12

v1.37.5

Upgrade binlog-replicator. pulls in some minor fixes.

Released 2022-04-16

v1.37.4

configure custom producer via environment
sns and sqs producers take output config properly

Released 2022-04-08

v1.37.2

configurable binlog event queue size

Released 2022-03-14

v1.37.1

upgrade mysql-connector-j

Released 2022-03-07

v1.37.0

Change max size of RowMap buffer to unblock high-efficiency producers

Released 2022-01-26

v1.36.0

fix bug where the millionth binlog would kinda sort "overflow" and the binlog positions would stop moving.
My benefactor here asked that I stopped creating cute release names. The security department, mysteriously.

Released 2022-01-23

v1.35.5

log4j, again and agian.

Released 2021-12-29

v1.35.4

log4j turns 2.17.0, happy birthday

Released 2021-12-18

v1.35.3

log4j vulnerability #2

Released 2021-12-15

v1.35.2

better logging when we can't connect on startup

Released 2021-12-12

v1.35.1

log4j upgrade to upgrade past the giant security hole

Released 2021-12-10

v1.35.0

couple of parser fixes
docker builds are now multi-platform
replication_reconnection_retries configuration option
quote table names in bootstrapper properly

Released 2021-11-30

v1.34.1

support for mysql 8's visible/invisible columns
support mariadb's if-exists/if-not-exists for partition management
add an index for the http endpoint

Released 2021-09-21

v1.34.0

intern a bunch of objects in our in-memory representation of schema. Saves gobs of memory in cases where one has N copies of the same database. Note that this changes the API of Columns, should any embedded Maxwell application be using that.
go up to BIGINT for maxwell's auto-increment ids

Released 2021-07-29

v1.33.1

properties may now be fetched from a javascript blob in the env
RowMap provides access to primary keys
fix an odd NPE in mariaDB init

Released 2021-06-02

v1.33.0

Add HTTP endpoint for runtime reconfiguration

Released 2021-03-29

v1.32.0

Amazon SNS producer added, thanks Rober Wittman
kafka 2.7.0 supported
stackdriver metrics logging available

Released 2021-03-17

v1.31.0

Add producer for NATS streaming server

Released 2021-02-11

v1.30.0

support server-sent heartbeating on the binlog connection via --binlog-heartbeat
can connect to rabbitmq by URL, supports SSL connections
fix parser bug with multiline SQL
target JDK 11 -- we have dropped support for JDK 8
ability to send a microsecond timestamp via --output_push_timestamp
fixes for odd azure mysql connection failures

Released 2021-02-05

v1.29.2

fix for terrible performance regression in bootstrapping

Released 2021-01-27

v1.29.1

small bugfix release, fixes binlog event type processing in mysql 8

Released 2020-12-23

v1.29.0

High Availability support via jgroups-raft
rework --help text

Released 2020-12-15

v1.28.2

fix for encryption parsing error on table creation
some logging around memory usage in RowMapBuffer

Released 2020-12-02

v1.28.1

fix http server issue in 1.28.0

Released 2020-11-25

v1.28.0

schema compaction! with the new --max_schemas option, maxwell will periodically roll up the maxwell.schemas table, preventing it from growing infinitely long.
fix metricsAgeSloMS calculation
support SRID columns
fix parsing of complex INDEX(CAST()) statements
various dependency bumps

Released 2020-11-19

v1.27.1

redis producer gets sentinal support
fix a double-reconnect race condition
file producer honors javascript row-suppression
better error messaging when we lack REPLICATION SLAVE privs
miscellaneous dependency bumps

Released 2020-08-07

v1.27.0

better support for empty/null passwords
allow bootstrap utility to query replication_host
a few library upgrades, notably pubsub and kinesis library
bootstrap connection uses jdbc_options properly
add logging for when we hit out of sync schema exceptions
allow for partitioning by thread_id, thx @gogov
fresh and clean documentation

Released 2020-06-30

v1.26.4

support now() function with precision

Released 2020-06-08

v1.26.3

use pooled redis connections, fixes corruption when redis was accessed from multiple threads (bootstrap/producer), thanks @lucastex
fix date handling of '0000-01-01'
fix race condition in binlog reconnect logic

Released 2020-05-26

v1.26.2

bootstraps can be scheduled in the future by setting the started_at column, thanks @lucastex
two mysql 8 fixes; one for a DEFAULT(function()) parse error, one for supporting DEFAULT ENCRYPTION

Released 2020-05-18

v1.26.1

fixes for redis re-connection login, thanks much @lucastex

Released 2020-05-07

v1.26.0

We now support mysql 8's caching_sha2_password authentication scheme
support for converting JSON field names to camelCase

Released 2020-05-06

v1.25.3

fixes memory leak in mysql-binlog-connector
fixes exceptions that occur when a connection passes wait_timeout

Released 2020-05-02

v1.25.2

Fixes for a long standing JSON bug in 8.0.19+

Released 2020-05-01

v1.25.1

issue #1457, ALTER DATABASE with implicit database name
maxwell now runs on JDK 11 in docker
exit with status 2 when we can't find binlog files

Released 2020-04-22

v1.25.0

swap un-maintained snaq.db with C3P0.
support eu datadog metrics
protect against lost connections during key queries (bootstrapping, heartbeats, postition setting)

Released 2020-03-29

v1.24.2

bugfix parsing errors: compressed columns, exchange partitions, parenthesis-enclosed default values, drop column foo.t.
add partition-by-random feature.
update jackson-databind to get security patch
fix redis channel interpolation on RPUSH

Released 2020-03-25

v1.24.1

allow jdbc_options on secondary connections
fix a crash in bootstrapping / javascript filters
fix a regression in message.publish.age metric

Released 2020-01-21

v1.24.0

add comments field to bootstrapping, thanks Tom Collins
fix sql bug with #comments style comments

Released 2019-12-14

v1.23.5

Update bootstrap documentation
Bump drop wizard metrics to support Java versions 10+

Released 2019-12-12

v1.23.4

Bump and override dependencies to fix security vulnerabilities.
Update redis-key config options
list changes

Released 2019-12-03

v1.23.3

pubsubDelayMultiplier may now be 1.0
allow %{database} and %{topic} interpolation into redis producer
docs updates
setup default client_id in maxwell-bootstrap util

Released 2019-11-21

v1.23.2

upgrade jackson
stop passing maxwell rows through the JS filter. too dangerous.

Released 2019-10-18

v1.23.1

Add option for XADD (redis streams) operation
Add configuration flag for tuning transaction buffer memory
sectionalize help text

Released 2019-10-12

v1.23.0

Added AWS FIFO support
Add retry and batch settings to pubs producer
Add support for age SLO metrics

Released 2019-10-08

v1.22.6

upgrade mysql-connector-java to 8.0.17
use a newer docker image as base
list changes

Released 2019-09-20

v1.22.5

bugfix for bootstrapping off a split replica that doesn't contain a "maxwell" database
Fix a parser issue with db.table.column style column names

Released 2019-09-06

v1.22.4

Add row type to fallback message
Upgrade jackson-databind

Released 2019-08-23

v1.22.3

fix issue with google pubsub in 1.22.2

Released 2019-06-20

v1.22.2

fix an issue with bootstrapping-on-replicas
add --output_primary_keys and --output_primary_key_columns
fix a very minor memory leak with blacklists

Released 2019-06-18

v1.22.1

fix crash in rabbit-mq producer
better support for maxwell + azure-mysql
remove bogus different-host bootstrap check
some security upgrades

Released 2019-05-28

v1.22.0

Bootstrapping has been reworked and is now available in all setups, including those in which the maxwell store is split from the replicator.
cleanup and fix a deadlock in the kafka fallback queue logic
add .partition_string = to javascript filters

Released 2019-04-16

v1.21.1

Upgrade binlog connector. Should fix issues around deserialization errors.

Released 2019-03-29

v1.21.0

Bootstrapping output no longer contain binlog positions. Please update any code that relies on this.
Fix 3 parser issues.

Released 2019-03-23

v1.20.0

add support for partitioning by transaction ID thx @hexene
add support for a kafka "fallback" topic to write to when a message fails to write
add UJIS charset support
parser bug: multiple strings concatenate to make one default string
parser bug: deal with bizarre column renames which are then referenced in AFTER column statements

Released 2019-02-28

v1.19.7

fix a parser error with empty sql comments
interpret latin-1 as windows-1252, not iso-whatever, thx @borleaandrei

Released 2019-01-25

v1.19.6

Further fixes for GTID-reconnection issues.
Crash sanely when GTID-enabled maxwell is connected to clearly the wrong master, thanks @acampoh

Released 2019-01-20

v1.19.5

Fixes for unreliable connections wrt to GTID events; previously we restart in any old position, now we throw away the current transaction and restart the replicator again at the head of the GTID event.

Released 2019-01-15

v1.19.4

Fixes for a maxwell database not making it through the blacklist
Add output_null_zerodates parameter to control how we treat '0000-00-00'

Released 2019-01-12

v1.19.3

Add a universal backpressure mechanism. This should help people who were running into out-of-memory situations while bootstrapping.

Released 2018-12-19

v1.19.2

Include schema_id in bootstrap events
add more logging around binlog connector losing connection
add retry logic to redis
some aws fixes
allow pushing JS hashes/arrays into data from js filters
list changes

Released 2018-12-02

v1.19.1

Handle mysql bit literals in DEFAULT statements
blacklist out CREATE ROLE etc
upgrade dependencies to pick up security issues

Released 2018-11-12

v1.19.0

mysql 8 support!
utf8 enum values are supported now
fix #1125, bootstrapping issue for TINYINT(1)
fix #1145, nasty bug around SQL blacklist and columns starting with "begin"
only resume bootstraps that are targeted at this client_id
fixes for blacklists and heartbeats. Did I ever mention blacklists are a terrible idea?

Released 2018-10-27

v1.18.0

memory optimizations for large schemas (especially shareded schemas with lots of duplicates)
add support for an http endpoint to support Prometheus metrics
allow javascript filters to access the row query object
javascript filters now run in the bootstrap process
support for non-latin1 column names
add --output_schema_id option
better handling of packet-too-big errors from Kinesis
add message.publish.age metric

Released 2018-09-15

v1.17.1

fix a regression around filters + bootstrapping
fix a regression around filters + database-only-ddl

Released 2018-07-03

v1.17.0

v1.17.0 brings a new level of configurability by allowing you to inject a bit of javascript into maxwell's processing. Should be useful! Also:

fix regression for Alibaba RDS tables

Released 2018-06-28

v1.16.1

Fix Bootstrapping for JSON columns
add --recapture_schema flag for when ya wanna start over
add kafka 1.0 libraries, make them default

Released 2018-06-21

v1.16.0 brings a rewrite of Maxwell's filtering system, giving it a concise list of rules that are executed in sequence. It's now possible to exclude tables from a particular database, exclude columns matching a value, and probably some other use cases. See http://maxwells-daemon.io/config/#filtering for details.

Released 2018-06-15

v1.15.0

This is a bug-fix release, but it's big enough I'm giving it a minor version.

Fix a very old bug in which DDL rows were writing the start of the row into maxwell.positions, leading to chaos in some scenarios where maxwell managed to stop on the row and double-process it, as well as to a few well-meaning patches.
Fix the fact that maxwell was outputting "next-position" instead of "position" of a row into JSON.
Fix the master-recovery code to store schema that corresponds to the start of a row, and points the replicator at the next-position.

Much thanks to Tim, Likun and others in sorting this mess out.

Released 2018-06-13

v1.14.7

add RowMap#getRowQuery, thx @saimon7
revert alpine-linux docker image fiasco
fix RawJSONString not serializable, thx @niuhaifeng

Released 2018-06-03

v1.14.6

Fix docker image

Released 2018-05-15

v1.14.5

reduce docker image footprint
add benchmarking framework
performance improvements for date/datetime columns
fix parser error on UPGRADE PARTITIONING

Released 2018-05-15

v1.14.4

Fix race condition in SchemaCapturer

Released 2018-05-07

v1.14.3

Enable jvm metrics

Released 2018-05-04

v1.14.2

fix regression in 1.14.1 around bootstrapping host detection
fix heartbeating code around table includes

Released 2018-05-02

v1.14.1

bootstraps can now take a client_id
improved config validation for embedded mode

Released 2018-05-01

v1.14.0

new feature --output_xoffset to uniquely identify rows within transactions, thx Jens Gyti
Bug fixes around "0000-00-00" times.
Bug fixes around dates pre 1000 AD

Released 2018-04-24

v1.13.5

Support environment variable based configuration

Released 2018-04-11

v1.13.4

Added possibility to do not declare the rabbitmq exchange.

Released 2018-04-03

v1.13.3

Add logging for binlog errors
Maven warning fix
Do not include current position DDL schema to avoid processing DDL twice
Always write null fields in primary key fields
Bugfix: fix http_path_prefix command line option issue

Released 2018-04-03

v1.13.2

fix a bug with CHARACTER SET = DEFAULT
maxwell now eclipse-friendly.
configurable bind-address for maxwell's http server

Released 2018-03-06

v1.13.1

redis producer now supports LPUSH, thx @m-denton
RowMap can now contain artbitrary attributes for embedded maxwell, thx @jkgeyti
bugfix: fix jdbc option parsing when value contains =
bugfix: apparently the SQS producer was disabled
bugfix: fix a situation where adding a second client could cause schemas to become out of sync
support for --daemon

Released 2018-02-20

v1.13.0

proper SSL connection support, thanks @cadams5
support for including original SQL in insert/update/deletes, thanks @saimon7
fixes for float4, float8 and other non-mysql datatypes
bump kinesis lib to 0.12.8
fix for bug when two databases share a single table

Released 2018-02-01

v1.12.0

Support for injecting a custom producer, thanks @tomcollinsproject
New producer for Amazon SQS, thanks @vikrant2mahajan
Maxwell can now filter rows based on column values, thanks @finnplay
Fixes for the Google Pubsub producer (it was really broken), thanks @finnplay
DDL output can now optionally include the source SQL, thanks @sungjuly
Support for double-quoted table/database/etc names
rabbitmq option for persistent messages, thanks @d-babiak
SQL parser bugfix for values like +1.234, thanks @hexene

Released 2018-01-09

v1.11.0

 - default kafka client upgrades to 0.11.0.1
 - fix the encryption issue (https://github.com/zendesk/maxwell/issues/803)

Released 2017-11-22

v1.10.9

We recommend all v1.10.7 and v1.10.8 users upgrade to v1.10.9.

Add missing Kafka clients
Listen and report on binlog connector lifecycle events for better visibility
Reduce docker image size

Released 2017-10-30

v1.10.8

Fix docker builds
Add Google Cloud Pub/Sub producer
RabbitMQ producer enhancements

Released 2017-10-12

v1.10.7

Java 8 upgrade
Diagnostic health check endpoint
Encryption
Documentation update: encryption, kinesis producer, schema storage fundamentals, etc.

Released 2017-10-11

v1.10.6

Binlog-connector upgrade
Bug-fix: when using literal string for an option that accepts Regex, Regex characters are no longer special
If master recovery is enabled, Maxwell cleans up old positions for the same server and client id

Released 2017-08-14

v1.10.5

Shyko's binlog-connector is now the default and only replication backend available for maxwell.

Released 2017-07-25

v1.10.4

Notable changes:

Shutdown hardening. If maxwell can't shut down (because the kafka producer is in a bad state and close() never terminates, for example), it would previously stall and process no messages. Now, shutdown is run in a separate thread and there is an additional watchdog thread which forcibly kills the maxwell process if it can't shut down within 10 seconds.
Initial support for running maxwell from java, rather then as its own process. This mode of operation is still experimental, but we'll accept PRs to improve it (thanks Geoff Lywood).
Fix incorrect handling of negative (pre-epoch dates) when using binlog_connector mode (thanks Geoff Lywood).

Released 2017-07-10

v1.10.3

tiny release to fix a units error in the replication.lag metric (subtracting seconds from milliseconds)

Released 2017-06-06

v1.10.2

added metrics: "replication.queue.time" and "inflightmessages.count"
renamed "time.overall" metric to "message.publish.time"
documentation updates (thanks Chintan Tank)

Released 2017-06-04

v1.10.1

The observable changes in this minor release are a new configuration for Kafka/Kinesis producer to abort processing on publish errors, and support of Kafka 0.10.2. Also a bunch of good refactoring has been done for heartbeat processing. List of changes:

Support Kafka 0.10.2
Stop procesing RDS hearbeats
Keep maxwell heartbeat going every 10 seconds when database is quiet
Allow for empty double-quoted string literals for database schema changes
Ignore Kafka/Kinesis producer errors based on new configuration ignore_producer_error

Released 2017-05-26

v1.10.0

This is a small release, primarily around a change to how schemas are stored. Maxwell now stores the last_heartbeat_read with each entry in the schemas table, making schema management more resilient to cases where binlog numbers are reused, but means that you must take care if you need to roll back to an earlier version. If you deploy v1.10.0, then roll back to an earlier version, you should manually update all schemas.last_heartbeat_read values to 0 before redeploying v1.10.0 or higher.

Other minor changes:

allow negative default numbers in columns
only store final binlog position if it has changed
blacklist internal aurora table `rds_heartbeat*'
log4j version bump (allows for one entry per line JSON logging)

Released 2017-05-09

v1.9.0

Maxwell 1.9 adds one main feature: monitoring support, contributed by Scott Ferguson. Multiple backends can be configured, read the updated docs for full details.

There's also some bugfixes:

filter DDL messages based on config
determine newest schema from binlog order, not creation order
add task manager to shutdown cleanly on error
minor logging improvements

Released 2017-04-26

v1.8.2

Bugfix release.

maxwell would crash on a quoted partition name
fixes for alters on non-string tables containing VARCHAR
use seconds instead of milliseconds for DDL messages

Released 2017-04-11

v1.8.1

performance improves in capturing and restoring schema, thx Joren Minnaert
Allow for capturing from a separate mysql host (adds support for using Maxscale as a replication proxy), thx Adam Szkoda

Released 2017-02-20

v1.8.0

In version 1.8.0 Maxwell gains alpha support for GTID-based positions! All praise due to Henry Cai.

Released 2017-02-14

v1.7.2

Fix a bug found where maxwell could cache the wrong TABLE_MAP_ID for a binlog event, leading to crashes or in some cases data mismatches.

Released 2017-01-30

v1.7.1

bootstrapping now can take a --where clause
performance improvements in the kafka producer

Released 2017-01-24

v1.7.0

Maxwell 1.7 brings 2 major new, alpha features. The first is Mysql 5.7 support, including JSON column type support and handling of 5.7 SQL, but not including GTID support yet. This is based on porting Maxwell to Stanley Shyko's binlog-connector library. Thanks to Stanley for his amazing support doing this port.

The second major new feature is a producer for Amazon's Kinesis streams, This was contributed in full by the dogged and persistent Thomas Dziedzic. Check it out with --producer=kinesis.

There's also some bugfixes:

Amazon RDS heartbeat events now tick maxwell's position, thx Scott Ferguson
allow CHECK() statements inside column definitions

Released 2017-01-07

v1.6.0

This is mostly a bugfix release, but it gets a minor version bump due to a single change of behavior: dates and timestamps which mysql may accept, but are considered invalid (0000-00-00 is a notable example) previously had inconsistent behavior. Now we convert these to NULL. Other bugfixes:

heartbeats have moved into their own table
more fixes around alibaba rds
ignore DELETE statements that are output for MEMORY tables upon server restart
allow pointing maxwell to a pre-existing database

Released 2016-12-29

v1.5.2

add support for kafka 0.10.1 @ smferguson
master recovery: cleanup positions from previous master; prevent errors on flip-back.
fix a bug that would trigger in certain cases when dropping a column that was part of the primary-key

Released 2016-12-07

v1.5.1

This is a bugfix release.

fixes for bootstrapping with an alternative maxwell-schema name and an include_database filter, thanks Lucian Jones
fixes for kafka 0.10 with lz4 compression, thanks Scott Ferguson
ignore the RDS table mysql.ha_health_check table
Get the bootstrapping process to output NULL values.
fix a quoting issue in the bootstrap code, thanks @mylesjao.

Released 2016-11-24

v1.5.0

CHANGE: Kafka producer no longer ships with hard-coded defaults. Please ensure you have "compression.type", "metadata.fetch.timeout.ms", and "retries" configured to your liking.
bugfix: fix a regression in handling ALTER TABLE change c int after b statements
warn on servers with missing server_id

Released 2016-11-07

v1.4.2

kafka 0.10.0 support, as well as a re-working of the --kafka_version command line option.

Released 2016-11-01

v1.4.1

support per-table topics, Thanks @smferguson and @sschatts.
fix a parser issue with DROP COLUMN CASCADE, thanks @smferguson

Released 2016-10-27

v1.4.0

1.4.0 brings us two nice new features:

partition-by-column: see --kafka_partition_columns. Thanks @smferguson
output schema changes as JSON: see --output_ddl. Thanks @xmlking
As well as a fix around race conditions on shutdown.

Released 2016-10-21

v1.3.0

support for fractional DATETIME, TIME, TIMESTAMP columns, thanks @Dagnan
support for outputting server_id & thread_id, thanks @sagiba
fix a race condition in bootstrap support

Released 2016-10-03

v1.2.2

Maxwell will now include by default fields with NULL values (as null fields). To disable this and restore the old functionality where fields were omitted, pass --output_nulls=false
Fix an issue with multi-client support where two replicators would ping-pong heartbeats at each other
Fix an issue where a client would attempt to recover a position from a mismatched client_id
Fix a bug when using CHANGE COLUMN on a primary key

Released 2016-09-23

v1.2.1

This is a bugfix release.

fix a parser bug around ALTER TABLE CHARACTER SET
fix bin/maxwell to pull in the proper version of the kafka-clients library

Released 2016-09-15

v1.2.0

1.2.0 is a major release of Maxwell that introduces master recovery features; when a slave is promoted to master, Maxwell is now capable of recovering the position. See the --master_recovery flag for more details.

It also upgrades the kafka producer library to 0.9. If you're using maxwell with a kafka 0.8 server, you must now pass the --kafka0.8 flag to maxwell.

Released 2016-09-12

v1.1.6

minor bugfix in which maxwell with --replay mode was trying to write heartbeats

Released 2016-09-07

v1.1.5

@dadah89 adds --output_binlog_position to optionally output the position with the row
@dadah89 adds --output_commit_info to turn off xid/commit fields
maxwell now supports tables with partitions
maxwell now supports N maxwells per-server. see the client_id / replica_server_id options.
two parser fixes, for engine=innodb and CHARSET ASCII
lay the ground work for doing master recovery; we add a heartbeat into the positions table that we can co-ordinate around.

Released 2016-09-04

v1.1.4

add support for a bunch more charsets (gbk, big5, notably)
fix Maxwell's handling of kafka errors - previously we were trying to crash Maxwell by throwing a RuntimeException out of the Kafka Producer, but this was a failure. Now we log and skip all errors.

Released 2016-08-05

v1.1.3

This is a bugfix release, which fixes:

#376, a problem parsing RENAME INDEX
#371, a problem with the SERIAL datatype
#362, we now preserve the original casing of columns
#373, we were incorrectly expecting heartbeats to work under 5.1

Released 2016-07-14

v1.1.2

pick up latest mysql-connector-j, fixes #369
fix an issue where maxwell could skip ahead positions if a leader failed.
rework buffering code to be much kinder to the GC and JVM heap in case of very large transactions / rows inside transactions
kinder, gentler help text when you specify an option incorrectly

Released 2016-06-27

v1.1.1

fixes a race condition setting the binlog position that would get maxwell stuck

Released 2016-05-23

v1.1.0

much more efficient processing of schema updates storage, especially when dealing with large schemas.
@lileeyao added --exclude-columns and the --jdbc_options features
@lileeyao added --jdbc_options
can now blacklist entire databases
new kafka key format available, using a JSON array instead of an object
bugfix: unsigned integer columns were captured incorrectly. 1.1 will recapture the schema and attempt to correct the error.

Released 2016-05-20

v1.1.0-pre4

Eddie McLean gives some helpful patches around bootstrapping
Bugfixes for the patch-up-the-schema code around unsigned ints

Released 2016-05-06

v1.1.0-pre3

forgot to include some updates that back-patch unsigned column problems

Released 2016-05-05

v1.1.0-pre2

fix performance issues when capturing schema in AWS Aurora
fix a bug in capturing unsigned integer columns

Released 2016-05-04

v1.0.1

fixes a parsing bug with CURRENT_TIMESTAMP()

Released 2016-04-12

v1.0.0

Since v0.17.0, Maxwell has gotten:

bootstrapping support
blacklisting for tables
flexible kafka partitioning
replication heartbeats
GEOMETRY columns
a whole lotta lotta bugfixes

and I, Osheroff, think the damn thing is stable enough for a 1.0. So there.

Released 2016-03-11

v1.0.0-RC3

pull in support for replication heartbeats. helps in the flakier network environs.

Released 2016-03-08

v1.0.0-RC2

fixes the way ALTER DATABASE charset= was handled
adds proper handling of ALTER TABLE CONVERT TO CHARSET

Released 2016-02-20

v1.0.0-RC1

modifications to the way the bootstrap utility works
fix a race condition crash bug in bootstrapping
fix a parser bug

Released 2016-02-11

v1.0.0-PRE2

1.0.0-PRE2 brings in a lot of changes that got merged while we were testing out PRE1. so, hey.

Configurable names for the maxwell schema database (Kristian Kaufman)
Configurable key (primary key, id, database) into the kafka partition hash function (Kristian Kaufman)
Configurable Kafka partition hash function (java hashCode, murmur3) (Kristian Kaufman)
support GEOMETRY columns, output as well-known-text
add --blacklist_tables option to fully ignore excessive schema changes (Nicolas Maquet)
bootstrap rows now have 'bootstrap-insert' type

Released 2016-01-30

v1.0.0-PRE1

Here we have the preview release of @nmaquet's excellent work around bootstrapping initial versions of mysql tables.

Released 2016-01-09

v0.17.0

v0.17 is a large bugfix release with one new feature.

FEATURE: allow specifying an alternative mysql schema-storage server and replication server
BUGFIX: properly handle case-sensitivity by aping the behavior of the master server. Fixes #230.
BUGFIX: parse some forms of CHECK( ... ) statements. Fixes #203.
BUGFIX: many more SQL-parser fixes. We are mostly through some thousands of lines of SQL produced by mysql-test.

Released 2016-01-07

v0.16.2

This is a large-ish bugfix release.

Support, with reservations, binlog_row_image=MINIMAL
parser bug: handle table names that look like floating points
parser bug: fix for entity names that have '.', '', etc in them
handle UPPERCASE encoding names
support UCS2 (start trying to operate ok on the mysql-test suite)
use ObjectOutputStream.reset to fix memory leaks when buffering to disk

Released 2015-12-16

v0.16.1

This is a bug-fix-roundup release:

support ALTER DATABASE
fix a bunch of parse errors: we've started running mysql-test at maxwell and are fixing up failures.
some modifications to the overflow-to-disk logic; we buffer the input and output, and we fix a memory leak

Released 2015-12-11

v0.16.0

Version 0.16.0 introduces a feature where UPDATE statements will now show both the new row image and the old values of the fields that changed. Thanks @kristiankaufmann

Released 2015-12-10

v0.15.0

fix a parse problem with indices ordered by ASC/DESC

Released 2015-12-07

v0.15.0-RC1

large transactions now buffer to disk instead of crushing maxwell.
support ALGORITHM=[algo], LOCK=[lock] for 5.6 alters

Released 2015-12-04

v0.14.6

fix TIME column support
fix parsing on millisecond precision column defintions
fix CREATE SCHEMA parsing

Released 2015-11-27

v0.14.5

handle BOOLEAN columns with true/false defaults

Released 2015-11-25

v0.14.4

fixes parsing of "mysql comments" (/*! .. */)
More performance improvements, another 10% in a tight loop.

Released 2015-11-24

v0.14.3

fixes a regression in 0.14.2 that creates duplicate copies of the "mysql" database in the schema.

Released 2015-11-23

v0.14.2

capture the mysql database along with the rest of the schema. Eliding it was a bad premature optimization that led to crashes when tables in the mysql database changed.

Released 2015-11-20

v0.14.1

fixes a parser bug around named PRIMARY KEYs.

Released 2015-11-17

v0.14.0

This release introduces row filters, allowing you to include or exclude tables from maxwell's output based on names or regular expressions.

Released 2015-11-03

v0.13.1

v0.13.1 is a bug fix of v0.13.0 -- fixes a bug where long rows were truncated.

v0.13.0 contains:

Big performance boost for maxwell: 75% faster in some benchmarks
@davidsheldon contributed some nice bug fixes around CREATE TABLE ... IF NOT EXISTS, which were previously generating new, bogus copies of the schema.
we now include a "scavenger thread" that will lazily clean out old, deleted schemas.

Released 2015-10-29

v0.13.0

Lucky release number 13 brings some reasonably big changes:

Big performance boost for maxwell: 75% faster in some benchmarks
@davidsheldon contributed some nice bug fixes around CREATE TABLE ... IF NOT EXISTS, which were previously generating new, bogus copies of the schema.
we now include a "scavenger thread" that will lazily clean out old, deleted schemas.

This release has a pretty bad bug. do not use.

Released 2015-10-29

v0.12.0

add support for BIT columns.

Released 2015-10-16

v0.11.4

this is another bugfix release that fixes a problem where the replication thread can die in the middle of processing a transaction event. I really need to fix this at a lower level, ie the open-replicator level.

Released 2015-09-30

v0.11.3

this is a bugfix release:

fix problems with table creation options inside alter statements ( ALTER TABLE foo auto_increment=10 )
fix a host of shutdown-procedure bugs

the test suite should also be way more reliable, not like you care.

Released 2015-09-29

v0.11.2

This is a bugfix release. It includes:

soft deletions of maxwell.schemas to fix A->B->A master swapping without creating intense replication delay
detect and fail early if we see binlog_row_image=minimal
kill off maxwell if the position thread dies
fix a bug where maxwell could pick up a copy of schema from a different server_id (curse you operator precedence!)

Released 2015-09-18

v0.11.1

maxwell gets a very minimal pass at detecting when a master has changed, in which it will kill off schemas and positions from a server_id that no longer is valid. this should prevent the worst of cases.

Released 2015-09-16

v0.11.0

This release of Maxwell preserves transaction information in the kafka stream by adding a xid key in the JSON object, as well as a commit key for the final row inside the transaction.

It also contains a bugfix around server_id handling.

Released 2015-09-15

v0.10.1

proper support for BLOB, BINARY, VARBINARY columns (base 64 encoded)
fix a problem with the SQL parser where specifying encoding or collation in a string column in the wrong order would crash
make table option parsing more lenient

Released 2015-09-11

v0.11.0-RC1

merge master fixes

Released 2015-09-09

v0.11.0-PRE4

bugfix on v0.11.0-PRE3

Released 2015-09-09

v0.10.0

Mysql 5.6 checksum support!
some more bugfixes with the SQL parser

Released 2015-09-09

v0.11.0-PRE3

handle SAVEPOINT within transactions
downgrade unhandled SQL to a warning

Released 2015-09-08

v0.11.0-PRE2

fixes for myISAM "transactions"

Released 2015-09-03

v0.11.0-PRE1

fix a server_id bug (was always 1 in maxwell.schemas)
JSON output now includes transaction IDs

Released 2015-09-02

v0.10.0-RC4

deal with BINARY flag in string column creation.

Released 2015-08-31

v0.9.5

handle the BINARY flag in column creation

Released 2015-08-31

v0.10.0-RC3

handle "TRUNCATE [TABLE_NAME]" statements

Released 2015-08-27

v0.10.0-RC2

fixes a bug with checksum processing.

Released 2015-08-26

v0.10.0-RC1

upgrade to open-replicator 1.3.0-RC1, which brings binlog checksum (and thus easy 5.6.1) support to maxwell.

Released 2015-08-04

v0.9.4

allow a configurable number (including unlimited) of schemas to be stored

Released 2015-07-27

v0.9.3

bump open-replicator to 1.2.3, which allows processing of single rows greater than 2^24 bytes

Released 2015-07-14

v0.9.2

bump open-replicator buffer to 50mb by default
log to STDERR, not STDOUT
--output_file option for file producer

Released 2015-07-10

v0.9.1

Maxwell is now aware that column names are case-insenstive
fix a nasty bug in which maxwell would store the wrong position after it lost its connection to the master.

Released 2015-06-22

v0.9.0

Also, vanchi is so paranoid he's worried immediately about this.

mysql 5.6 support (without checksum support, yet)
fix a bunch of miscellaneous bugs @akshayi1 found (REAL, BOOL, BOOLEAN types, TRUNCATE TABLE)

Released 2015-06-18

v0.8.1

minor bugfix release around mysql connections going away.

Released 2015-06-16

v0.8.0

add "ts" field to row output
add --config option for passing a different config file
support int1, int2, int4, int8 columns

Released 2015-06-09

v0.7.2

handle inline sql comments
ignore more user management SQL

Released 2015-05-29

v0.7.1

only keep 5 most recent schemas

Released 2015-05-15

v0.7.0

handle CURRENT_TIMESTAMP parsing properly
better binlog position sync behavior

Released 2015-04-28

v0.6.3

better blacklist for CREATE TRIGGER

Released 2015-04-13

v0.6.2

maxwell now ignores SAVEPOINT statements.

Released 2015-04-13

v0.6.1

fixes a bug with parsing length-limited indexes.

Released 2015-04-13

v0.6.0

Version 0.6.0 has Maxwell outputting a JSON kafka key, so that one can use Kafka's neat "store the last copy of a key" retention policy. It also fixes a couple of bugs in the query parsing path.

Released 2015-04-09

v0.5.0

maxwell now captures primary keys on tables. We'll use this to form kafka key names later.
maxwell now outputs to a single topic, hashing the data by database name to keep a database's updates in order.

Released 2015-04-06

v0.4.0

v0.4.0 fixes some bugs with long-lived mysql connections by adding connection pooling support.

Released 2015-03-25

v0.3.0

This version fixes a fairly nasty bug in which the binlog-position flush thread was sharing a connection with the rest of the system, leading to crashes.

It also enables kafka gzip compression by default.

Released 2015-03-24

Files

CHANGELOG.md

Latest commit

History

CHANGELOG.md

File metadata and controls

Maxwell changelog