Added following string functions: regex, substr, substring, ltrim, rtrim, trim, upper, lower, concat, concat_ws, length, strcmp #750

lyndonbauto · 2020-09-19T23:13:17Z

Issue #, if available:
#710

Description of changes:
Added implementation unit testing, manual IT, and doc updates for the following string functions:

regex
substr
substring
ltrim
rtrim
trim
upper
lower
concat
concat_ws
length
strcmp

The following functions have limitations from the specification:

trim Does not support LEADING/TRAILING/BOTH/FROM commands, only supports string space character trimming
substr/substring Does not support FROM/TO commands, only comma separated values
concat/concat_ws Does not support variable arguments being specified, only two strings to concatenate

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…rch#522) * Bug fix, support long type for aggregation * change to datetime to JDBC format

* prepare odfe 1.9 * Fix all ES 7.8 compile and build errors * Revert changes as Lombok is working now * Update CustomExternalTestCluster.java * Fix license headers check * Use splitFieldsByMetadata to separate fields when calling SearchHit constructor * More fixes for ODFE 1.9 * Remove todo statement * Add ODFE 1.9.0 release notes

…earch#547)

…ticsearch#551) * Revert "Rename release notes to use 4 digit versions (opendistro-for-elasticsearch#547)" This reverts commit 33c6d3e. * Revert "Opendistro Release 1.9.0 (opendistro-for-elasticsearch#532)" This reverts commit 254f2e0. * Revert "Bug fix, support long type for aggregation (opendistro-for-elasticsearch#522)" This reverts commit fb2ed91.

Merge develop to master

…h#549) (opendistro-for-elasticsearch#554) * merge all sql repos * fix test and build workflows * fix workbench and odbc path * fix workbench and odbc path * restructure workbench dir and fix workflows * fix workbench workflow * fix workbench workflow * fix workbench workflow * fix workbench workflow * fix workbench workflow * revert workbench directory structure * fix workbench workflow * fix workbench workflow * fix workbench workflow * fix workbench workflow * update workbench workflow for release * Delete .github/ in sql-workbench directory * Add cypress to sql-workbench * Sync latest ODBC commits * Sync latest workbench commits (will add cypress in separate PR) * Add ignored ODBC libs

* add date and time support * update doc * update doc

…opendistro-for-elasticsearch#567) This reverts commit 4b33a2f.

…ticsearch#633) Merge develop to master for ODFE 1.9.0.1 release

…stro-for-elasticsearch#638) Merge fixes for github release actions from develop to master

…rch#642) Fix odbc win32 release workflow for master

…-elasticsearch#645) - add null check to avoid crashing if details not initialized

…stro-for-elasticsearch#645)" (opendistro-for-elasticsearch#653) This reverts commit c11125d.

…or-elasticsearch/develop Merge develop branch into master to cut odfe1.10 release

…lasticsearch#701) Merge develop branch into master for od1.10 release

…icsearch#704） Merge workflow fix to master for od1.10 release

…h#729)

…sticsearch#733) Merge develop to master for ODFE 1.10.1.0 release

[2] Added EMPTY_STRING [3] Removed unused function resolver

This reverts commit 3fa954c.

penghuo · 2020-09-23T15:49:41Z

core/src/main/java/com/amazon/opendistroforelasticsearch/sql/expression/text/TextFunction.java

+      return new ExprStringValue(EMPTY_STRING);
+    }
+    String str = exprValue.stringValue();
+    return exprSubStr(str, startIdx, 0);


the length=0 is misunderstanding, in Java SDK, the definition like below

public String substring(int beginIndex) { return substring(beginIndex, length()); }

penghuo

Thanks for the change.

penghuo and others added 30 commits June 17, 2020 09:17

Bug fix, support long type for aggregation (opendistro-for-elasticsea…

fb2ed91

…rch#522) * Bug fix, support long type for aggregation * change to datetime to JDBC format

Rename release notes to use 4 digit versions (opendistro-for-elastics…

33c6d3e

…earch#547)

Merge develop branch to master (opendistro-for-elasticsearch#553)

893cd18

Merge develop to master

add date and time support (opendistro-for-elasticsearch#560)

4b33a2f

* add date and time support * update doc * update doc

Revert "add date and time support (opendistro-for-elasticsearch#560)" (…

af74293

…opendistro-for-elasticsearch#567) This reverts commit 4b33a2f.

resolve conflict

baac103

Merge develop to master for ODFE 1.9.0.1 release (opendistro-for-elas…

35e37a3

…ticsearch#633) Merge develop to master for ODFE 1.9.0.1 release

Merge fixes for github release actions from develop to master (opendi…

0ed6594

…stro-for-elasticsearch#638) Merge fixes for github release actions from develop to master

Fix odbc win32 release workflow for master (opendistro-for-elasticsea…

e4981e3

…rch#642) Fix odbc win32 release workflow for master

add error details for all server communication errors (opendistro-for…

c11125d

…-elasticsearch#645) - add null check to avoid crashing if details not initialized

Revert "add error details for all server communication errors (opendi…

34b979e

…stro-for-elasticsearch#645)" (opendistro-for-elasticsearch#653) This reverts commit c11125d.

Merge pull request opendistro-for-elasticsearch#698 from opendistro-f…

5ab1bc8

…or-elasticsearch/develop Merge develop branch into master to cut odfe1.10 release

Merge develop branch into master for od1.10 release (opendistro-for-e…

8735723

…lasticsearch#701) Merge develop branch into master for od1.10 release

Merge branch 'develop' of github.com:opendistro-for-elasticsearch/sql

332ee9c

Merge workflow fix to master for od1.10 release （opendistro-for-elast…

2939081

…icsearch#704） Merge workflow fix to master for od1.10 release

Fix download link in package description (opendistro-for-elasticsearc…

9e82a91

…h#729)

Merge develop to master for ODFE 1.10.1.0 release (opendistro-for-ela…

89c6e05

…sticsearch#733) Merge develop to master for ODFE 1.10.1.0 release

[1] Initial commit, checking if server build passes

625a178

[1] Commiting expression documentation with REGEXP

715e0c1

[1] Failure with REGEXP doc

30e6768

[1] Moved testing for regexp to binary predicates.

6b52b2b

[1] Updating parser for REGEX

baae0fc

[1] Parser update

38a2bb9

[1] Making REGEXP like LIKE

da44ff9

[1] Reverting change to legacy

04dcacb

[1] Checking if same without NOT

0291223

[1] testing adding to Ast Expr

bd93eaa

lyndonbauto added 22 commits September 22, 2020 11:22

[1] Condensed logic

fdcd554

[2] Added EMPTY_STRING [3] Removed unused function resolver

[1] Removed SUBSTRING FunctionName.

99532e6

[1] Reverted failure issues

334509b

[1] Combined substring and substr test.

d3cef5b

[1] Added ppl test and edited caps in textfunctiontest

539fa2a

[1] Testing without source

fc24427

[1] Correcting format of string

e5e4d15

[1] Testing new queries

7fd259d

[1] Adding resource and fixed tests

6708826

[1] Added maapping and adjusted tests

e51aa81

[1] minor corrections

cba5aae

[1] Additional debug info

85a24e3

[1] Removing unsuspported ppl functions.

018da57

[1] Added back unsupported

24066a9

[1] Checking regex

10f143d

[1] Removing printout

b918bad

[1] Trying to fix commit

3fa954c

Revert "[1] Trying to fix commit"

0ceb735

This reverts commit 3fa954c.

[1] Pulling develop to fix conflicts

b03c6d6

[1] Adding rest of commit

3e87927

[1] Adding workflow files

8c67ee8

[1] fixed docs

7342fb9

rupal-bq approved these changes Sep 23, 2020

View reviewed changes

penghuo reviewed Sep 23, 2020

View reviewed changes

lyndonbauto added 2 commits September 23, 2020 09:07

[1] Updated based on PR comments.

d85c258

[1] Fixed code so tests pass

8dfd7a6

penghuo approved these changes Sep 24, 2020

View reviewed changes

dai-chen approved these changes Sep 24, 2020

View reviewed changes

chloe-zh merged commit 15bc237 into opendistro-for-elasticsearch:develop Sep 24, 2020

LantaoJin mentioned this pull request Jul 11, 2024

Fix regexp filter query opensearch-project/sql#2797

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added following string functions: regex, substr, substring, ltrim, rtrim, trim, upper, lower, concat, concat_ws, length, strcmp #750

Added following string functions: regex, substr, substring, ltrim, rtrim, trim, upper, lower, concat, concat_ws, length, strcmp #750

lyndonbauto commented Sep 19, 2020

penghuo Sep 23, 2020

penghuo left a comment

Added following string functions: regex, substr, substring, ltrim, rtrim, trim, upper, lower, concat, concat_ws, length, strcmp #750

Added following string functions: regex, substr, substring, ltrim, rtrim, trim, upper, lower, concat, concat_ws, length, strcmp #750

Conversation

lyndonbauto commented Sep 19, 2020

penghuo Sep 23, 2020

Choose a reason for hiding this comment

penghuo left a comment

Choose a reason for hiding this comment