Fix PostgresToGCSOperator does not allow nested JSON

[pierrejeambrun]

fixes: #23040

This is due to a double json.dumps when exporting to json format:

• https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/postgres_to_gcs.py#L153
• https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/sql_to_gcs.py#L221

I added a parameter to the convert_type that allow us to chose what to do with the dict type objects. For parquet and csv we want to stringify them. But we want to keep them as dict when exporting to the json format.

csv and parquet export are hence not modified.

I added data to the tests so we assert json column export.

Regards,

[pierrejeambrun]

Hello @eladkal,

Here if a first draft of the PR just in case you want to take a look :)

Best,

[github-actions]

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.

[eladkal]

Thanks @pierrejeambrun hopeful someday we will have #21599 resolved so we won't need dedicated operators per db

[pierrejeambrun]

@eladkal, my pleasure. Matter of fact I was wondering about that as there are strong similarities on some part of the code.

Good to know there already is an issue tracking this, it's not going to be an easy one though 😄