Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GeoPackage as a Provider #161

Closed
fawick opened this issue Oct 26, 2017 · 6 comments
Closed

Add GeoPackage as a Provider #161

fawick opened this issue Oct 26, 2017 · 6 comments
Assignees
Milestone

Comments

@fawick
Copy link

fawick commented Oct 26, 2017

(Somewhat related to #115 and #160)

I wonder whether it would be of interest for tegola to have a mvt.Provider for the GeoPackage format. To my knowledge, there is no GeoPackage library for Go, yet. But since it is an open standard maybe the time is right to kick off one under the tegola roof.

@jj0hns0n
Copy link
Contributor

This one is in progress. @JivanAmara can you give us an update?

@JivanAmara
Copy link
Contributor

Using go-sqlite, I'm writing a basic library to read gpkg files (and the wkbgeometry format used inside them) as a tegola provider. Have a rough version working on a particular gpkg that loads layers and doesn't throw any errors, but something is preventing tiles from showing up in the map. Once the branch is working at a basic level and I've cleaned up a bit I'll share.

@ARolek
Copy link
Member

ARolek commented Oct 27, 2017

@fawick a bit more about this. Since Geopackage uses SQLite, we're waiting for an SQLite port to Go. In the mean time a bridge via cgo is necessary. There are two projects working on SQLite in pure go though, and we're watching those.

@fawick
Copy link
Author

fawick commented Oct 28, 2017

Although I am well aware of the call overhead that CGO induces I would like to make a case for using the official SQLite library via CGO. The main advantage I see is that the official SQLite sources are rigorously tested.

In fact, their testing suite suffices the very strict requirements of the DO-178 aviation standard for safety-critical software. As every aviation software creator is required by DO-178 to present and re-do these tests the folks behind SQLite sell this testing suite via a proprietary license. Nevertheless, any SQLite version they make is checked against it behind the curtains before it is released. It is mingboggingly to see such a thing in open-source (and even Public Domain) software and IMHO makes SQLite one of the (if not THE) most mature FOSS projects there are.

I reckon they put tens of man years of engineering into coming to that point. Any attempt to come up with a SQLite package in pure Go will be having a hard time ensuring that same level of robustness.

@ARolek
Copy link
Member

ARolek commented Oct 28, 2017

@fawick using CGO is the current strategy that @JivanAmara is working on for several of the reasons you have mentioned and the fact that it's the only way to leverage SQLite in Go. The pure Go projects we're watching:

  • c2go - If you take a look at their roadmap, the ultimate goal is to transpile SQLite3.
  • sqlite - this is another transpiler project with a different approach.

Ultimately this is a data provider and it needs to be stable. I would prefer tegola be pure Go, but we're going to implement some cgo while other packages mature.

@JivanAmara
Copy link
Contributor

JivanAmara commented Dec 11, 2017

  • Add support for "fields" config param
  • Add support for "sql" config param
  • Rewrite tests without testify
  • Replace logging calls with internal/log
  • Replace bounds literals with vars from projections.go - won't fix: the bounds in projections.go have normal bounding box arrangement (minx, miny, maxx, maxy) while there's no indication that tiles passed to WKTLayer() have switched from their arrangement (minx, maxy, maxx, miny)
  • Add support for zoom token to custom sql.
  • Add context checks.
  • Make requested changes & checks on PR Add testing helpers in 'internal/assert'. #220 .

@ARolek ARolek added this to the v0.6.0 milestone Dec 13, 2017
JivanAmara added a commit that referenced this issue Dec 14, 2017
JivanAmara added a commit that referenced this issue Dec 14, 2017
JivanAmara added a commit that referenced this issue Dec 18, 2017
JivanAmara added a commit that referenced this issue Jan 12, 2018
ARolek added a commit that referenced this issue Feb 9, 2018
…l logger format and updated logs in various server packages. fixed breaking tests. #161
ARolek added a commit that referenced this issue Feb 10, 2018
ARolek added a commit that referenced this issue Feb 10, 2018
ARolek added a commit that referenced this issue Feb 10, 2018
ARolek added a commit that referenced this issue Feb 12, 2018
#161 rewrite of the binary header code.
ARolek added a commit that referenced this issue Feb 12, 2018
…Close / Cleanup #161

- gpkg connection pool concept was converted into a Close() method which will be called by a higer level connection manager.
- travis had overlapping go version envrionments
- gpkg Provider now has a Close method for closing database connections
- gpkg Provider now has a Cleanup method for cleaning up database connection before tegola shuts down
gdey added a commit that referenced this issue Feb 12, 2018
There was a need for providers to have the ability to clean up when
a program is ending, or is reloading configs.
gdey added a commit that referenced this issue Feb 12, 2018
gdey added a commit that referenced this issue Feb 13, 2018
gdey added a commit that referenced this issue Feb 13, 2018
The build matrix did not seprate out each entry. It made six entries
instead of three.
gdey added a commit that referenced this issue Feb 13, 2018
The build matrix did not seprate out each entry. It made six entries
instead of three.
gdey added a commit that referenced this issue Feb 13, 2018
ARolek added a commit that referenced this issue Feb 14, 2018
gdey added a commit that referenced this issue Feb 14, 2018
The providers now have a clean up that needs to happen before the
binary exits. This is handle by trapping terminal signals and running
the clean up routines. The initial way we were doing this was not
correct.
gdey added a commit that referenced this issue Feb 15, 2018
On go 1.7 we will use Close() which will just kill the connection. It's
highly recommened that you use 1.8 or later, so that there is a graceful
shutdown.
gdey added a commit that referenced this issue Feb 15, 2018
gdey added a commit that referenced this issue Feb 15, 2018
The test directory was misnamed to test_data. Fixed.
ARolek added a commit that referenced this issue Feb 15, 2018
@ARolek ARolek closed this as completed Feb 15, 2018
gdey added a commit that referenced this issue Feb 16, 2018
Additional changes and fixes requested by Jivan
JivanAmara added a commit that referenced this issue Feb 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants