Skip to content

Commit

Permalink
Update es-restlog for ES 7.x and migrate to bazel build (#4)
Browse files Browse the repository at this point in the history
* Migrate to bazel build
* Add build and test actions
* Add manual release workflow
  • Loading branch information
cristi- authored Mar 10, 2021
1 parent 01ed4b9 commit c63300d
Show file tree
Hide file tree
Showing 19 changed files with 1,839 additions and 169 deletions.
6 changes: 6 additions & 0 deletions .bazelproject
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
directories:
.

targets:
//...

1 change: 1 addition & 0 deletions .bazelversion
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
4.0.0
40 changes: 40 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Release plugin
on:
workflow_dispatch:
inputs:
version:
description: 'Version'
required: true

jobs:
bazel:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: mishas/setup-bazelisk-action@v1

- name: Version
run: bazel version

- name: Bazel build
run: bazel build ...
- name: List plugin contents
run: |
FILE=$(bazel aquery 'outputs(".*zip", //src/main/java/com/etsy/elasticsearch/restlog:restlog_plugin_versioned)' 2>/dev/null|grep -oE 'Outputs: .+' | sed 's/Outputs: \[//' | sed 's/\]//' | tr "," "\n")
unzip -l $FILE
CHECKSUM=$(sha256sum $FILE | cut -d " " -f 1 )
echo "FILE=$FILE" >> $GITHUB_ENV
echo "CHECKSUM=$CHECKSUM" >> $GITHUB_ENV
- name: Bazel test
run: bazel test ... --test_output=streamed
- name: Create Release
id: create_release
uses: softprops/action-gh-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.event.inputs.version }}
name: Release for ${{ github.event.inputs.version }}
body: "sha256sum: ${{ env.CHECKSUM }}"
files: "${{ env.FILE }}"
# fail_on_unmatched_files: true
25 changes: 25 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Bazel build and test
on:
pull_request:
branches:
- master
jobs:
bazel:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: mishas/setup-bazelisk-action@v1

- name: Version
run: bazel version

- name: Bazel build
run: bazel build ...

- name: List plugin contents
run: |
FILE=$(bazel aquery 'outputs(".*zip", //src/main/java/com/etsy/elasticsearch/restlog:restlog_plugin_versioned)' 2>/dev/null|grep -oE 'Outputs: .+' | sed 's/Outputs: \[//' | sed 's/\]//' | tr "," "\n")
unzip -l $FILE
- name: Bazel test
run: bazel test ... --test_output=streamed
32 changes: 29 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,34 @@
target/

# gah macs
.DS_Store

# intellij
.idea/
# intellij user-specific stuff
.idea/workspace.xml
.idea/tasks.xml
.idea/dictionaries
.idea/saveactions_settings.xml
.idea/compiler.xml
.idea/sshConfigs.xml

# IntellJ IDEA Bazel plugin settings
.ijwb

# managed by gradle's intellij (or is it intellij's gradle) plugin
.idea/libraries

# intellij high-churn
.idea/deployment.xml
.idea/uiDesigner.xml
.idea/inspectionProfiles
.idea/webServers.xml
.idea/modules.xml
.idea/misc.xml
.idea/codeStyles/

*.iml

# See: https://docs.bazel.build/versions/master/best-practices.html#bazelrc
user.bazelrc
bazel-
bazel-*
build/
File renamed without changes.
59 changes: 29 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,28 +20,48 @@ Note that Elasticsearch provides the 'slowlog' mechanism for capturing index or

## Installation

The plugin is available for Elastisearch 2.x
The plugin is available for Elastisearch 7.x

### pre-packaged

The release naming scheme is `es-restlog-${plugin.version}-es_maj_min_patch_etc.zip`.

Head over to `Releases` on Github to find the latest plugin package.

### packaging

Use the [sbt](http://www.scala-sbt.org/#install) target `pack`, which will generate a plugin zip under `target/`.
We use [bazel](https://bazel.build/) to build and package the es-restlog elasticsearch plugin.

The ES version is overridable with the `esVersion` setting, so you can do:
1. Install [bazelisk](https://docs.bazel.build/versions/master/install-bazelisk.html), a simple bazel wrapper and add it in your `$PATH`.
Bazelisk will automatically download the required bazel version defined in `.bazelversion`.

```
$ sbt 'set esVersion := "2.0.1"' clean pack
```
2. To build the plugin locally:
```
bazelisk build //src/main/java/com/etsy/elasticsearch/restlog:restlog_plugin_versioned
```

The plugin `zip` file will be available in the following path:
```
bazel-bin/src/main/java/com/etsy/elasticsearch/restlog/restlog-plugin-X.Y.Z.zip
```

3. To update the version of `elasticsearch` before building the plugin:
- update the ES version in `version.bzl`
- update the jvm dependencies for the new version by running:
```bash
REPIN=1 bazelisk run @unpinned_jvm_deps//:pin
```
- build the plugin: `bazelisk build //src/main/java/com/etsy/elasticsearch/restlog:restlog_plugin_versioned`

## Configuration

### x-pack security module
`es-restlog` and X-Pack security module can't run together, so the latter needs to be disabled.
Add `xpack.security.enabled: false` to `elasticsearch.yml`
### plugin
The following plugin configuration can be added in `elasticsearch.yml`:
`restlog.category` the logger category to be used, defaults to "restlog".
`restlog.path_regex` allows for filtering what gets logged at the level of the HTTP request path, defaults to blank which implies matching everything. If you only want to include search requests for example, you could set this to `\/_search\/?\??.*`.
Expand All @@ -50,25 +70,4 @@ $ sbt 'set esVersion := "2.0.1"' clean pack
`restlog.null_value` how any value that is not available (e.g. if there was no request body) get encoded in the log line, defaults to "-".
### logging

Note that the plugin uses `INFO` level for logging at the configured category.

You will probably want to direct the restlog output to a dedicated logfile. The configuration can be based off how the slowlogs get configured in `logging.yml`, for example:

```yaml
logger:
restlog: INFO, restlog

additivity:
restlog: false

appender:
restlog:
type: dailyRollingFile
file: ${path.logs}/${cluster.name}_rest.log
datePattern: "'.'yyyy-MM-dd"
layout:
type: pattern
conversionPattern: "%m%n"
```
`restlog.uuid_header` log this request header value if present in the request (e.g. X-Request-ID), defaults to "" (disabled)
53 changes: 53 additions & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
workspace(name = "restlog")

load("//:version.bzl", "elasticsearch_version")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "6a3bd6a2cf02e96fd5041500bafa3a35731d4981" # master as of 9 Mar 2021
RULES_JVM_EXTERNAL_SHA = "42712b494220629e3dbd3d0edee1fb675fb41f507695ca80284b53c291a76299"

http_archive(
name = "rules_jvm_external",
strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
sha256 = RULES_JVM_EXTERNAL_SHA,
url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)

load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps")

rules_jvm_external_deps()

load("@rules_jvm_external//:setup.bzl", "rules_jvm_external_setup")

rules_jvm_external_setup()

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
name = "jvm_deps",
artifacts = [
"org.elasticsearch:elasticsearch:%s" % elasticsearch_version,
"org.elasticsearch.test:framework:%s" % elasticsearch_version,
"org.elasticsearch.client:elasticsearch-rest-high-level-client:%s" % elasticsearch_version,
'org.apache.logging.log4j:log4j-api:jar:2.11.1',
'org.apache.logging.log4j:log4j-slf4j-impl:jar:2.11.1',
'org.apache.logging.log4j:log4j-core:2.11.1',
'com.google.guava:guava:29.0-jre',
'org.mockito:mockito-core:3.5.13',
'org.mockito:mockito-inline:3.5.13',
'junit:junit:4.12',
],
repositories = [
"https://repo1.maven.org/maven2",
],
maven_install_json = "@//:jvm_deps_install.json",
# fails if maven_install_json was not regenerated after updating artifacts
fail_if_repin_required = True,
)


load("@jvm_deps//:defs.bzl", pinned_jvm_deps_install = "pinned_maven_install")

pinned_jvm_deps_install()

Loading

0 comments on commit c63300d

Please sign in to comment.