Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Latest commit

 

History

History
402 lines (321 loc) · 36.2 KB

opendistro-for-elasticsearch-release-notes-1.7.0.md

File metadata and controls

402 lines (321 loc) · 36.2 KB

Open Distro for Elasticsearch 1.7.0 Release Notes

Open Distro for Elasticsearch 1.7.0 is now available for download.

The release consists of Apache 2 licensed Elasticsearch version 7.6.1, Kibana version 7.6.1, new plugins for anomaly detection and SQL Kibana, a SQL ODBC driver and SQL CLI client. Other plugins in the distribution include alerting, index management, performance analyzer, security, SQL and machine learning with k-NN. A SQL JDBC driver and PerfTop, a client for Performance Analyzer are also available for download.

Release Highlights

  • The anomaly detection plugin has moved out of the preview phase and is now generally available. It comes with a easy-to-use Kibana user interface.
  • The SQL plugin has greatly expanded its supported operations, added a new Kibana user interface, and now has an interactive CLIwith autocomplete.
  • A new SQL ODBC driver is also now available.
  • A new package, the Open Distro for Elasticsearch 1.7.0 AMI, is now available for use with Amazon EC2. Search for “open distro” when you start a new instance.

Release Details

The release includes the following features, enhancements, infrastructure and documentation updates, and bug fixes.

Breaking Changes

SQL

  • Change #414: Invalidate HTTP GET method.

Features

New!🔥 ANOMALY DETECTION

  • Add state and error to profile API (#84)
  • Preview detector on the fly (#72)
  • Cancel query if given detector already have one (#54)
  • Support return AD job when get detector (#50)
  • Add AD job on top of Job Scheduler (#44)
  • Adding negative cache to throttle extra request (#40)
  • Add window delay support (#24)
  • Circuit breaker (#10 , #7)
  • Stats collection (#8)

New!🔥 ANOMALY DETECTION KIBANA UI

  • Add functionality to start & stop detector PR #12
  • Add side navigation bar PR #19
  • Add detector detail page PR #20
  • Add functionality to get detector state PR #16
  • Add dashboard page PR #17
  • Add edit feature page PR #52
  • Add detector configuration page PR #22
  • Add anomaly results page PR #62
  • Add detector state page PR #65
  • Add anomaly charts PR #50

SQL

  • Simple Query Cursor support. (#390)
  • New SQL cluster settings endpoint (#400)
  • Security Update: Escape comma for CSV header and all queries. (#456)
  • Security Update: Fix CSV injection issue. (#447)
  • Security Update: Anonymize sensitive data in queries exposed to RestSqlAction logs. (#419)

New!🔥 SQL CLI

  • Feature #12: Initial development for SQL CLI
    • prototype launch: app -> check connection -> take input -> query ES -> serve raw results(format=jdbc)
    • enrich auto-completion corpus
    • Convert to vertical output format if fields length is larger than terminal window
    • Add style to output fields name. Add logic to confirm choice from user for vertical output
    • Add single query without getting into console. Integrate "explain" api
    • Add config base logic. Add pagination for long output
    • Add nice little welcome banner.
    • Add params -f for format_output (jdbc/raw/csv), -v for vertical display
    • Initial implementation of connection to OD cluster and AES with auth
    • Create test module and write first test
    • Add fake test data. Add test utils to set up connection
    • [Test] Add pagination test and query test
    • Add Test plan and dependency list
    • [Test] Add test case for ConnectionFailExeption
    • [Feature] initial implementation of index suggestion during auto-completion
    • [Feature] display (data retrieved / total hits), and tell user to use "limit" to get more than 200 lines of data
    • Added legal and copyright files,
    • Added THIRD PARTY file
    • Added setup.py for packaging and releasing
  • Feature #24: Provide user option to toggle to use AWS sigV4 authentication (issue: #23)

SQL JDBC Driver

  • Cursor integration. (#76)

New!🔥 SQL KIBANA UI

  • Initialize SQL Kibana plugin (#1)
  • SQL Kibana plugin functionality implementation (#2)
  • Update plugin name and adjust dependency version (#3)
  • Added kibana plugin helper configuration (#4)
  • Adjust the service routes to call SQL plugin (#5)
  • Bumped up the version to support v7.3 compatibility (#6)
  • Updated configureation for v7.3 compatibility (#7)
  • Opendistro-1.3 compatible with ES and Kibana 7.3 (#8)
  • Changed kibana version to v7.3.2 (#9)
  • Support v7.1.1 Compatibility (#13)
  • Bump pr-branch to v1.3 (#21)
  • Support v7.4 compatibility for kibana and sql-plugin (#23)
  • Improve the performance by ridding of sending redundant requests (#24)
  • Added raw format in download options (#25)
  • Update the release notes for the first official release (#27)
  • Updated test cases and snapshots (#28)
  • Initial merge from development branches to master (#31)
  • Support the result table to display results of DESCRIBE/SHOW/DELETE queries (#37)
  • Adjust the appearance of SQL Workbench UI (#38)
  • Migrated the default request format from Json to JDBC (#41)
  • Support v7.6.1 compatibility (#42)
  • Removed the overriding settings in css (#44)
  • Updated outdated dependencies (#47)
  • Updated open source code compliance (#48)
  • Opendistro Release v1.7.0 (#56)
  • Set up workflows for testing, building artifacts and releasing (#58)
  • Update copyright from year 2019 to 2020 (#59)
  • Moved release notes to dedicated folder (#60)
  • Added contributors file (#61)
  • Updated lastest yarn.lock for accurate dependency alert (#62)
  • Updated snapshot (#63)

New!🔥 SQL ODBC Driver

  • Add support for connection string abbreviations (#7)
  • Connection string refactoring and registry updates (#2)
  • Simple Windows Installer (#27)
  • Add fetch_size for pagination support (#78)

INDEX STATE MANAGEMENT

  • Adds rollover conditions into info object (#208)

K-NN

  • Support cosine similarity (#90). Note: This is an experimental feature.

Enhancements

ALERTING KIBANA UI

  • Creates monitor with anomaly detector. - (#123)

New!🔥 ANOMALY DETECTION

  • Stats API: moved detector count call outside transport layer and make asynchronous PR #108
  • Change AD result index rollover setting PR #100
  • Add async maintenance PR #94
  • Add async stopModel PR #93
  • Add timestamp to async putModelCheckpoint PR #92
  • Add async clear PR #91
  • Use callbacks and bug fix PR #83
  • Add async trainModel PR #81
  • Add async getColdStartData PR #80
  • Change the default value of lastUpdateTime PR #77
  • Add async getThresholdingResult PR #70
  • Add async getRcfResult PR #69
  • Fix rcf random seed in preview PR #68
  • Fix empty preview result due to insufficient sample PR #65
  • Add async CheckpointDao methods. PR #62
  • Record error and execution start/end time in AD result; handle except… PR #59
  • Improve log message when we cannot get anomaly result PR #58
  • Write detection code path in callbacks PR #48
  • Send back error response when failing to stop detector PR #45
  • Adjust preview configuration for more data PR #39
  • Refactor using ClientUtil PR #32
  • Return empty preview results on failure PR #31
  • Allow non-negative window delay PR #30
  • Return no data error message to preview PR #29
  • Change AD setting name PR #26
  • Add async CheckpointDao methods. PR #17
  • Add async implementation of getFeaturesForSampledPeriods. PR #16
  • Add async implementation of getFeaturesForPeriod. PR #15
  • Add test evaluating anomaly results PR #13

New!🔥 ANOMALY DETECTION KIBANA UI

  • Add window delay PR #4
  • Add empty dashboard page PR #9
  • Update create/edit detector page PR #13
  • Add search monitor API PR #18
  • Add detector state support on dashboard PR #28
  • Fix dark mode readability on detector list page PR #39
  • Fix live anomaly chart time range PR #45
  • Make breadcrumbs consistent on home pages PR #41
  • Modify detector list tooltips PR #47
  • Change chart style PR #45
  • Add feature required detector state PR #48
  • Remove old preview detector code PR #51
  • Change live anomaly chart height PR #45
  • Add live anomaly reducer PR #55
  • Modify logic to delete detector PR #54
  • Add chart and ad result .css file PR #64
  • Make titles with counts consistent PR #74
  • Avoid label cutoff on sunburst chart PR #83
  • Remove tooltip icon on detector list page PR #93
  • Modify some wording PR #95
  • Change create detector link on dashboard PR #100
  • Tune AD result charts PR #102
  • Use annotation for live chart PR #119
  • Set fixed height for anomalies live chart PR #123
  • Use scientific notation when number less than 0.01 on live chart PR #124
  • Use bucket aggregation for anomaly distribution PR #126

INDEX STATE MANAGEMENT

  • Adds isIdempotent method to each step and updates ManagedIndexRunner to use it (#165)
  • Adds logs, fix for index creation date -1L, nullable checks (#170)
  • Update schema version for IndexManagementConfig mapping if available (#198)
  • Switches UpdateManagedIndexMetaData to batch tasks using custom executor (#209)

K-NN

  • Add stats to track the number of requests and errors for KNN query and index operations. (#89)
  • Switched the default value of the circuit breaker from 60% to 50%. (#92)

New!🔥SQL CLI

  • Added github action workflow for CI/CD (#31)
  • Update github action test and build workflow to spin up ES instance (#35)

Bug fixes

New!🔥ANOMALY DETECTION

  • Change setting name so that rpm/deb has the same name as zip PR #109
  • Can't start AD job if detector has no feature PR #76
  • Fix null pointer exception during preview PR #74
  • Add threadpool prefix and change threadpool name PR #56
  • Change setting name and fix stop AD request PR #41
  • Revert "merge changes from alpha branch: change setting name and fix … PR #38
  • Fix stop detector api to use correct request PR #25

New!🔥ANOMALY DETECTION KIBANA UI

  • Fix dashboard width PR #29
  • Fix dashboard bugs PR #35
  • Fix detector list bugs PR #43
  • Fix more dashboard bugs PR #45
  • Minor fix PR #45
  • Return correct AD result PR #57
  • Set max monitor size PR #59
  • Fix more dashboard bugs PR #61
  • Fix bugs on detector configuration page PR #66
  • Fix bugs on create/edit detector page PR #67
  • Fix blank anomaly results bug PR #69
  • Fix link to detector configuration page PR #71
  • Fix thin bar on anomaly results live chart PR #70
  • Fix sunburst chart undefined issue PR #73
  • Fix chart colors PR #76
  • Don't display legend value on chartt PR #79
  • Fix legend value bug on dashboard live chart PR #80
  • Fix typo and change save feature button title PR #81
  • Fix feature breakdown tabs PR #84
  • Fix stats on dashboard live chart to not be wrapped PR #82
  • Fix column truncation on detector list PR #86
  • Fix issue that 0 cannot be set in detector filter PR #68
  • Add -kibana suffix in links to prevent broken links PR #92
  • Fix bug where latest anomalous detector can get lost PR #98
  • Fix detector initializing message PR #106
  • Fix preview detector error message PR #108
  • Cover more detector state edge cases PR #110
  • Fix 2 issues related to detector state PR #111
  • Fix blank detector detail page PR #112
  • Fix issue of not resetting to first page after applying filters PR #115
  • Fix issue when live chart pulls anomaly results PR #113
  • Fix live chart bar width problem PR #116
  • Fix unnecessary filter when getting single anomaly result PR #118
  • Fix live chart bar height PR #121
  • Fix live chart time range PR #122
  • Fix more live chart bugs PR #125
  • Fix loading bug on live chart PR #129

INDEX STATE MANAGEMENT

  • Delete and close failing during snapshot in progress (#172)

K-NN

  • Added validation in VectorFieldMapper to check for vector values of NaN and throwing an Exception if so. (#100)
  • Fix debugging integration tests (#78)

SECURITY

  • Implemented APIs and datamodel to configure nodes_dn dynamically #445 (backported from master #362)
  • Performance improvement by memorizing results of resolveIndexPatterns for Bulk requests (backported from master #309)
  • Performance improvement by implementing faster version of implies type perm (#302)
  • Enabled limited OpenSSL support
  • Changed file permissions for securityconfig and tools (#387)
  • Fixed bug which caused user to lose roles on account password update (#333)
  • Refactored to use Greenrobot EventBus (#445) (backported from master #370)
  • Refactored Resolved class, dropped unused fields and simplified logic (backported from master #310)
  • Refactored audit logging related classes (#445) (backported from master #303, #445, #306, #373, #368)

SQL

  • Support using aggregation function in order by clause. (#452)
  • Count(distinct field) should translate to cardinality aggregation. (#442)
  • Enforce AVG return double data type. (#437)
  • Ignore the term query rewrite if there is no index found. (#425)
  • Support subquery in from doesn't have alias. (#418)
  • Add support for strict_date_optional_time. (#412)
  • Field function name letter case preserved in select with group by. (#381)

New!🔥SQL CLI

  • Initial development for SQL CLI (#12)
    • Fix the logic of passing self-constructed settings
    • [Fix] get rid of unicode warning. Fix meta info display
    • [fix] Refactor executor code
    • [Fix] Fix test cases corresponding to fraction display.
    • [Fix] fix code style using Black, update documentation and comments
  • Fix typos, remove unused dependencies, add .gitignore and legal file (#18)
  • Fix test failures (#19)
  • Update usage gif, fix http/https issue when connect to AWS Elasticsearch (#26)

New!🔥SQL KIBANA UI

  • Fixed the issue that response failed to be delivered in the console (#10)
  • Fixed the issue that aggregation result not delivered in the table (#26)
  • Fixed the issue that the table remains delivering the cached results table from the last query (#30)
  • Fixed some dependency issues (#36)
  • Fixed: fatal errors occur when import @elastic/eui modules in docker images (#43)
  • BugFix: Corrected the downloaded .csv/.txt files to proper format (#50)

New!🔥SQL ODBC Driver

Infra Updates

New!🔥ANOMALY DETECTION

  • Add release notes for ODFE 1.7.0 (#120 , #119)
  • Open Distro Release 1.7.0 (#106)
  • Create opendistro-elasticsearch-anomaly-detection.release-notes.md (#103)
  • Update test branch (#101)
  • Bump opendistroVersion to 1.6.1 (#99)
  • Change to mention we support only JDK 13 (#98)
  • AD opendistro 1.6 support (#87)
  • Added URL for jb_scheduler-plugin_zip instead of local file path (#82)
  • Change build instruction for JDK (#61)
  • ODFE 1.4.0 (#43)
  • Add spotless for code format (#22)
  • Update third-party (#14)
  • Build artifacts for rpm, deb, zip (#5)
  • Update test-workflow.yml (#2)

New!🔥ANOMALY DETECTION KIBANA UI

  • Initial commit here
  • Fix unit test issue PR #14
  • Update test snapshots PR #44
  • Add unit test workflow PR #42
  • Change workflow to run on pushes to master PR #72
  • Change default build artifact name PR #89
  • Fix test workflow PR #104

K-NN

  • Create Github action that automatically runs integration tests against docker image whenever code is checked into master or opendistro branch. (#73)

New!🔥SQL CLI

  • Feature #28 :Added tox scripts for testing automation

Documentation Updates

New!🔥ANOMALY DETECTION KIBANA UI

  • Create CONTRIBUTORS here
  • Update README here
  • Update README here
  • Update README here
  • Update CONTRIBUTORS here
  • Update README PR #88
  • Add release notes for ODFE 1.7.0 PR #109
  • Modify ODFE 1.7.0 release notes PR #132

SQL

  • More docs in reference manual and add architecture doc. (#417)

New!🔥SQL CLI

  • Update documentation and CLI naming (#22)
  • Update copyright to 2020 (#32)
  • Updated package naming and created folder for release notes (#33)
  • Added CONTRIBUTORS.md (#34)
  • Polish README.md and test_plan.md (#36)

New!🔥SQL ODBC Driver

In development

  1. Performance Analyzer RCA Engine

You can also track upcoming features in Open Distro for Elasticsearch by watching the code repositories or checking the [project website](https://opendistro.github.io/for-elasticsearch/features/comingsoon.html)