Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not possible to create syncing DB connection #145

Open
gelin opened this issue Aug 26, 2019 · 1 comment
Open

Not possible to create syncing DB connection #145

gelin opened this issue Aug 26, 2019 · 1 comment

Comments

@gelin
Copy link

gelin commented Aug 26, 2019

The syncing DB connections, as described in Import API are POSTs with JSON content to "/api/v1/synchronizations" where both connector and interval are defined. Also, actually there are null data import urls (archive) there.

The code in datasets.py does not allow to do this:

    def is_sync_table(self, archive, interval, **import_args):
        return (hasattr(archive, "startswith") and archive.startswith("http")
                or "connection" in import_args)    # this is true for DB connections
            and interval is not None               # this also is true for syncing connections

    def create(self, archive, interval=None, **import_args):
        # ...
        if self.is_sync_table(archive, interval, **import_args):  # true for syncing DB connection
            manager = SyncTableJobManager(self.client)
        else:
            manager = FileImportJobManager(self.client)

        if interval is None:
            import_job = manager.create(archive)  # different signature for different managers (out of this issue)
            import_job.run(**import_args)
        else:
            import_job = manager.create(archive, interval)  # this is called for SyncTableJobManager
            # !!! import_args (including "connection") are just ignored and not POSTed later
        # ...
@gelin
Copy link
Author

gelin commented Aug 27, 2019

Also, it's even not possible to send necessary JSON to Carto API for this case.

It's necessary to POST JSON like:

{
  "connector": {
    "provider": "postgres",
    "connection": {
      "server": "postgres.server",
      "database": "db",
      "username": "dbuser",
      "password": "dbpass"
    },
    "schema": "dbschema",
    "table": "dbtable"
  },
  "interval": 86400
}

However, SyncTableJobManager does this:

resource = self.resource_class(url, interval, self.client)
resource.update_from_dict(kwargs)
resource.save(force_create=True)

It tries to create a resource of type SyncTableJob which has no "connector" property. So connector parameters are never sent to the server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant