Supported Apache Spark version: 2.0+(Master branch), 1.0+(1.X branch)
GeoSpark is listed as Infrastructure Project on Apache Spark Official Third Party Project Page
GeoSpark is a cluster computing system for processing large-scale spatial data. GeoSpark extends Apache Spark with a set of out-of-the-box Spatial Resilient Distributed Datasets (SRDDs) that efficiently load, process, and analyze large-scale spatial data across machines. GeoSpark provides APIs for Apache Spark programmer to easily develop their spatial analysis programs with Spatial Resilient Distributed Datasets (SRDDs) which have in house support for geometrical and Spatial Queries (Range, K Nearest Neighbors, Join).
GeoSpark artifacts are hosted in Maven Central: Maven Central Coordinates
Version information (more)
| Version | Summary |
|---|---|
| 0.5.2 | Bug fix: Fix Issue #58 and Issue #60; Performance enhancement:: (1) Deprecate all old Spatial RDD constructors. See the JavaDoc here. (2) Recommend the new SRDD constructors which take an additional RDD storage level and automatically cache rawSpatialRDD to accelerate internal SRDD analyze step |
| 0.5.1 | Bug fix: (1) GeoSpark: Fix inaccurate KNN result when K is large (2) GeoSpark: Replace incompatible Spark API call Issue #55; (3) Babylon: Remove JPG output format temporarily due to the lack of OpenJDK support |
| 0.5.0 | Major updates: We are pleased to announce the initial version of Babylon a large-scale in-memory geospatial visualization system extending GeoSpark. Babylon and GeoSpark are integrated together. You can just import GeoSpark and enjoy! More details are available here: Babylon GeoSpatial Visualization |
Important features (more)
Supported Spatial RDDs: PointRDD, RectangleRDD, PolygonRDD, LineStringRDD
Native input format support: CSV, TSV, WKT, GeoJSON
User-supplied input format mapper: Any input formats
Supported Spatial Partitioning techniques: R-Tree, Voronoi diagram
Supported Spatial Indexes: Quad-Tree and R-Tree. Quad-Tree doesn't support Spatial K Nearest Neighbors query.
Inside, Overlap, DatasetBoundary, Minimum Bounding Rectangl, Polygon Union
Spatial Range Query, Spatial Join Query, and Spatial K Nearest Neighbors Query.
GeoSpark Tutorial (more)
GeoSpark full tutorial is available at GeoSpark GitHub Wiki: https://github.com/DataSystemsLab/GeoSpark/wiki
Babylon is a large-scale in-memory geospatial visualization system.
Babylon provides native support for general cartographic design by extending GeoSpark to process large-scale spatial data. It can visulize Spatial RDD and Spatial Queries and render super high resolution image in parallel.
Babylon and GeoSpark are integrated together. You just need to import GeoSpark and enjoy them! More details are available here: Babylon GeoSpatial Visualization
Jia Yu, Jinxuan Wu, Mohamed Sarwat. "A Demonstration of GeoSpark: A Cluster Computing Framework for Processing Big Spatial Data". (demo paper) In Proceeding of IEEE International Conference on Data Engineering ICDE 2016, Helsinki, FI, May 2016
Jia Yu, Jinxuan Wu, Mohamed Sarwat. "GeoSpark: A Cluster Computing Framework for Processing Large-Scale Spatial Data". (short paper) In Proceeding of the ACM International Conference on Advances in Geographic Information Systems ACM SIGSPATIAL GIS 2015, Seattle, WA, USA November 2015
GeoSpark makes use of JTS Plus (An extended JTS Topology Suite Version 1.14) for some geometrical computations.
Please refer to JTS Topology Suite website and JTS Plus for more details.
-
Please join
-
Email us!
-
Jia Yu (Email: [email protected])
-
Mohamed Sarwat (Email: [email protected])
Please visit GeoSpark project wesbite for latest news and releases.
GeoSpark is one of the projects under Data Systems Lab at Arizona State University. The mission of Data Systems Lab is designing and developing experimental data management systems (e.g., database systems).
We appreciate the help and suggestions from GeoSpark users: Thanks List