What is the main challenges to support Partitioned tables?

[Enach]

Hello,

We are considering using shadowing to migrate to PG. Most of our tables are partitioned and receive millions of transactions daily.
The readme mentions that this tool doesn't yet support partitioned tables. I assume you tried it and encountered issues.
Could you explain a bit about what are those issues, please?
It could help propose a proper PR to support.

Thanks for the tool & cheers!

[shayonj]

Hello, and thank you for taking the time to write this post. Regarding partitioned tables, support for maintaining the parent-child relationship is generally lacking. Based on the current mechanics of pg-osc, there is simply no support for it. This is less about particular challenges and more about implementation I would say.

That said, I am working on a new tool called pg_flo (open source) which has a concept of source and destination database (both PostgreSQL) and uses logical replication under the hood to listen for changes on a table.

You could set up the source and destination to be the same database in your case and, using the routing feature, to copy and stream all changes from the primary table to a new table (shadow table) using pg_flo in the same DB. It does require NATS currently; however, I am open to all feedback.

Thank you!