Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare the plugin to an elasticsearch 7.17.24 upgrade #16

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This plugin extends Elasticsearch with a `geo_point_clustering` aggregation, all
It is very similar to what is done with the official [geohash_grid aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/7.10/search-aggregations-bucket-geohashgrid-aggregation.html) except that final clusters are not bound to the geohash grid.

For example, at zoom level 1 with points across France, `geohash_grid` agg will output 3 clusters stuck to geohash cells u, e, s, while `geo_point_clustering` will merge these clusters into one.
This is done during the reduce phase.
This is done during the reduce phase.

Contrary to `geohash_grid` aggregation, buckets keys are a tuple(centroid, geohash cells) instead of geohash cells only, because one cluster can be linked to several geohash cells, due to the cluster merge process during the reduce phase.
Centroids are built during the shard collect phase.
Expand All @@ -16,7 +16,7 @@ Please note that [geo_shape data type](https://www.elastic.co/guide/en/elasticse
### Install

Install plugin with:
`./bin/elasticsearch-plugin install https://github.com/opendatasoft/elasticsearch-aggregation-geoclustering/releases/download/v7.17.6.0/geopoint-clustering-aggregation-7.17.6.0.zip`
`./bin/elasticsearch-plugin install https://github.com/opendatasoft/elasticsearch-aggregation-geoclustering/releases/download/v7.17.24.0/geopoint-clustering-aggregation-7.17.24.0.zip`


### Quickstart
Expand Down Expand Up @@ -146,9 +146,10 @@ Result:


## Development environment setup

### Build

Built with Java 17 and Gradle 7.5.1 (use the packaged gradlew included in this repo if you want to build yourself).
Built with Java 17 and Gradle 8.10.2 (use the packaged gradlew included in this repo if you want to build yourself).

### Development Environment Setup

Expand All @@ -162,12 +163,16 @@ or
./gradlew assemble # (to avoid the test suite)
```

In case you have to upgrade Gradle, you can do it with `./gradlew wrapper --gradler-version x.y.z`.

Then the following command will start a dockerized ES and will install the previously built plugin:
``` shell
docker-compose up
docker compose up
```

Please be careful during development: you'll need to manually rebuild the .zip using `./gradlew build` on each code
change before running `docker-compose` up again.
You can now check the Elasticsearch instance on `localhost:9200` and the plugin version at `localhost:9200/_cat/plugins`.

Please be careful during development: you'll need to manually rebuild the .zip using `./gradlew build` on each code
change before running `docker compose up` up again.

> NOTE: In `docker-compose.yml` you can uncomment the debug env and attach a REMOTE JVM on `*:5005` to debug the plugin.
4 changes: 4 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ dependencies {
yamlRestTestImplementation "org.apache.logging.log4j:log4j-core:2.17.1"
}

tasks.named("yamlRestTest").configure {
systemProperty 'tests.security.manager', 'false'
}

// Make sure the ES distribution used for rest tests is the "complete" variant
// Otherwise weirds errors occur (like the geo_shape type is not handled)
testClusters.configureEach {
Expand Down
2 changes: 0 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
version: "3.7"

services:
elasticsearch-plugin-debug:
build:
Expand Down
6 changes: 3 additions & 3 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM docker.elastic.co/elasticsearch/elasticsearch:7.17.6 AS elasticsearch-plugin-debug
FROM docker.elastic.co/elasticsearch/elasticsearch:7.17.24 AS elasticsearch-plugin-debug

COPY /build/distributions/geopoint-clustering-aggregation-7.17.6.0.zip /tmp/geopoint-clustering-aggregation-7.17.6.0.zip
RUN ./bin/elasticsearch-plugin install file:/tmp/geopoint-clustering-aggregation-7.17.6.0.zip
COPY /build/distributions/geopoint-clustering-aggregation-7.17.24.0.zip /tmp/geopoint-clustering-aggregation-7.17.24.0.zip
RUN ./bin/elasticsearch-plugin install file:/tmp/geopoint-clustering-aggregation-7.17.24.0.zip
4 changes: 2 additions & 2 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
es_version = 7.17.6
plugin_version = 7.17.6.0
es_version = 7.17.24
plugin_version = 7.17.24.0
Binary file modified gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
2 changes: 1 addition & 1 deletion gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.10.2-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
6 changes: 6 additions & 0 deletions gradlew
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,12 @@ set -- \
org.gradle.wrapper.GradleWrapperMain \
"$@"

# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi

# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
Expand Down
14 changes: 8 additions & 6 deletions gradlew.bat
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
@rem limitations under the License.
@rem

@if "%DEBUG%" == "" @echo off
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
Expand All @@ -25,7 +25,7 @@
if "%OS%"=="Windows_NT" setlocal

set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=.
if "%DIRNAME%"=="" set DIRNAME=.
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%

Expand All @@ -40,7 +40,7 @@ if defined JAVA_HOME goto findJavaFromJavaHome

set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if "%ERRORLEVEL%" == "0" goto execute
if %ERRORLEVEL% equ 0 goto execute

echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Expand Down Expand Up @@ -75,13 +75,15 @@ set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar

:end
@rem End local scope for the variables with windows NT shell
if "%ERRORLEVEL%"=="0" goto mainEnd
if %ERRORLEVEL% equ 0 goto mainEnd

:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
exit /b 1
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%

:mainEnd
if "%OS%"=="Windows_NT" endlocal
Expand Down