Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-14673: Add bin/solr stream CLI #2479

Merged
merged 49 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
8708bfb
First pass of bringing in patch file for adding streaming expressions…
epugh May 25, 2024
6175602
tidy and precommit fixes
epugh May 25, 2024
14d6c5b
Actually run a simple expression!
epugh May 25, 2024
67dffff
Precommit burps, and honestly we don't need this messaging...
epugh May 27, 2024
87320b9
Wire new command into SolrCLI.
epugh May 27, 2024
f52773c
Now have headers and comments working, introducing some docs
epugh May 28, 2024
5ee82f6
Respond to warnings
epugh May 28, 2024
e916dbe
No longer require a "let" to be used since we have command line param…
epugh May 28, 2024
b5242eb
Add an integration test
epugh May 28, 2024
bcd9a90
tests back to working after attempting to support basic auth..
epugh May 28, 2024
0b84a01
Moving test into it's own class as it requires more setup etc..
epugh May 28, 2024
ed13d5c
working on precommit
epugh May 28, 2024
6160841
lets test variable interpolation
epugh May 28, 2024
0baafc8
Introduce Basic Auth support to Streaming expressions.
epugh May 28, 2024
5599d60
debugging snuck in
epugh May 28, 2024
f7216ec
wip
epugh May 29, 2024
83b7cd1
Introduce "worker" concept that lets you run it locally or on solr.
epugh Jun 20, 2024
c36317d
Remove unused method
epugh Jun 20, 2024
f9c03eb
This test has been merged into StreamToolTest
epugh Jun 20, 2024
b4023cf
precommit
epugh Jun 20, 2024
50d0897
need the zk!
epugh Jun 20, 2024
550c078
support the stdin() streaming source
epugh Jun 24, 2024
e144792
Logging of standard messages should go to system.err, not system.out
epugh Jun 24, 2024
fc60f6b
Mention the CLI tool for streaming
epugh Jun 24, 2024
ea7c0c9
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Jul 22, 2024
74a6c6d
update for how tools are now invoked
epugh Jul 22, 2024
28d004e
with out the " " wrapping the $@ you split args on white space!
epugh Jul 23, 2024
7c2a013
use the new and deprecated options
epugh Jul 23, 2024
b567d00
Change from "workers" to "execution" for the local / solr environment.
epugh Jul 23, 2024
de55505
Introduce ability to cat local data into a streaming expression, and …
epugh Jul 23, 2024
ec8e4e5
Missing annotation!
epugh Jul 23, 2024
b2a6c4d
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Aug 5, 2024
a0bed95
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Aug 10, 2024
665a799
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Sep 17, 2024
3cf55e3
Improve the text, and introduce a note about this being "Experimental"
epugh Sep 18, 2024
668e5a5
move up the note about experimental status
epugh Sep 24, 2024
407aa70
stress more what local is
epugh Sep 25, 2024
f7aca93
formatting
epugh Sep 25, 2024
11c71a7
Merge branch 'main' into SOLR-14673
epugh Nov 8, 2024
1835f8b
Update CHANGES.txt
epugh Nov 8, 2024
0ef3b18
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Nov 8, 2024
3c3b561
Updates
epugh Nov 8, 2024
0cfd69d
Lint.
epugh Nov 8, 2024
20b5a00
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Nov 13, 2024
a889f0b
Oops!
epugh Nov 13, 2024
c52b087
Update tests to match modern CLI params
epugh Nov 13, 2024
4e018a0
Respond to feedback, and polish
epugh Nov 13, 2024
f2693a9
Lint
epugh Nov 13, 2024
86722d1
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Nov 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions solr/CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ New Features

* SOLR-17467: Solr CLI bin/solr start defaults to starting Solr in Cloud mode, use --user-managed switch for User Managed (aka Standalone) mode. (Eric Pugh)

* SOLR-14673: Solr CLI now has bin/solr stream tool that executates streaming expressions via command line, either locally or on solr cluster. (Eric Pugh)

Improvements
---------------------

Expand Down
6 changes: 6 additions & 0 deletions solr/benchmark/.factorypath
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<factorypath>
<factorypathentry kind="EXTJAR" id="/Users/epugh/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-math3/3.6.1/e4ba98f1d4b3c80ec46392f25e094a6a2e58fcbf/commons-math3-3.6.1.jar" enabled="true" runInBatchMode="false"/>
<factorypathentry kind="EXTJAR" id="/Users/epugh/.gradle/caches/modules-2/files-2.1/org.openjdk.jmh/jmh-generator-annprocess/1.37/da93888682df163144edf9b13d2b78e54166063a/jmh-generator-annprocess-1.37.jar" enabled="true" runInBatchMode="false"/>
<factorypathentry kind="EXTJAR" id="/Users/epugh/.gradle/caches/modules-2/files-2.1/net.sf.jopt-simple/jopt-simple/5.0.4/4fdac2fbe92dfad86aa6e9301736f6b4342a3f5c/jopt-simple-5.0.4.jar" enabled="true" runInBatchMode="false"/>
<factorypathentry kind="EXTJAR" id="/Users/epugh/.gradle/caches/modules-2/files-2.1/org.openjdk.jmh/jmh-core/1.37/896f27e49105b35ea1964319c83d12082e7a79ef/jmh-core-1.37.jar" enabled="true" runInBatchMode="false"/>
</factorypath>
epugh marked this conversation as resolved.
Show resolved Hide resolved
1 change: 1 addition & 0 deletions solr/bin/solr.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,7 @@ IF "%1"=="restart" goto set_script_cmd
IF "%1"=="healthcheck" goto run_solrcli
IF "%1"=="create" goto run_solrcli
IF "%1"=="delete" goto run_solrcli
IF "%1"=="stream" goto run_solrcli
IF "%1"=="postlogs" goto run_solrcli

IF "%1"=="auth" (
Expand Down
4 changes: 2 additions & 2 deletions solr/core/src/java/org/apache/solr/cli/SolrCLI.java
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,7 @@ private static Tool newTool(String toolType) throws Exception {
else if ("post".equals(toolType)) return new PostTool();
else if ("postlogs".equals(toolType)) return new PostLogsTool();
else if ("version".equals(toolType)) return new VersionTool();
else if ("stream".equals(toolType)) return new StreamTool();
else if ("snapshot-create".equals(toolType)) return new SnapshotCreateTool();
else if ("snapshot-delete".equals(toolType)) return new SnapshotDeleteTool();
else if ("snapshot-list".equals(toolType)) return new SnapshotListTool();
Expand Down Expand Up @@ -511,8 +512,7 @@ private static void printHelp() {
print("Usage: solr COMMAND OPTIONS");
print(" where COMMAND is one of: start, stop, restart, status, ");
print(
" healthcheck, create, delete, auth, assert, config, export, api, package, post, ");

" healthcheck, create, delete, auth, assert, config, export, api, package, post, stream,");
print(
" zk ls, zk cp, zk rm , zk mv, zk mkroot, zk upconfig, zk downconfig,");
print(
Expand Down
Loading
Loading