Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-14673: Add bin/solr stream CLI #2479

Merged
merged 49 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
8708bfb
First pass of bringing in patch file for adding streaming expressions…
epugh May 25, 2024
6175602
tidy and precommit fixes
epugh May 25, 2024
14d6c5b
Actually run a simple expression!
epugh May 25, 2024
67dffff
Precommit burps, and honestly we don't need this messaging...
epugh May 27, 2024
87320b9
Wire new command into SolrCLI.
epugh May 27, 2024
f52773c
Now have headers and comments working, introducing some docs
epugh May 28, 2024
5ee82f6
Respond to warnings
epugh May 28, 2024
e916dbe
No longer require a "let" to be used since we have command line param…
epugh May 28, 2024
b5242eb
Add an integration test
epugh May 28, 2024
bcd9a90
tests back to working after attempting to support basic auth..
epugh May 28, 2024
0b84a01
Moving test into it's own class as it requires more setup etc..
epugh May 28, 2024
ed13d5c
working on precommit
epugh May 28, 2024
6160841
lets test variable interpolation
epugh May 28, 2024
0baafc8
Introduce Basic Auth support to Streaming expressions.
epugh May 28, 2024
5599d60
debugging snuck in
epugh May 28, 2024
f7216ec
wip
epugh May 29, 2024
83b7cd1
Introduce "worker" concept that lets you run it locally or on solr.
epugh Jun 20, 2024
c36317d
Remove unused method
epugh Jun 20, 2024
f9c03eb
This test has been merged into StreamToolTest
epugh Jun 20, 2024
b4023cf
precommit
epugh Jun 20, 2024
50d0897
need the zk!
epugh Jun 20, 2024
550c078
support the stdin() streaming source
epugh Jun 24, 2024
e144792
Logging of standard messages should go to system.err, not system.out
epugh Jun 24, 2024
fc60f6b
Mention the CLI tool for streaming
epugh Jun 24, 2024
ea7c0c9
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Jul 22, 2024
74a6c6d
update for how tools are now invoked
epugh Jul 22, 2024
28d004e
with out the " " wrapping the $@ you split args on white space!
epugh Jul 23, 2024
7c2a013
use the new and deprecated options
epugh Jul 23, 2024
b567d00
Change from "workers" to "execution" for the local / solr environment.
epugh Jul 23, 2024
de55505
Introduce ability to cat local data into a streaming expression, and …
epugh Jul 23, 2024
ec8e4e5
Missing annotation!
epugh Jul 23, 2024
b2a6c4d
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Aug 5, 2024
a0bed95
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Aug 10, 2024
665a799
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Sep 17, 2024
3cf55e3
Improve the text, and introduce a note about this being "Experimental"
epugh Sep 18, 2024
668e5a5
move up the note about experimental status
epugh Sep 24, 2024
407aa70
stress more what local is
epugh Sep 25, 2024
f7aca93
formatting
epugh Sep 25, 2024
11c71a7
Merge branch 'main' into SOLR-14673
epugh Nov 8, 2024
1835f8b
Update CHANGES.txt
epugh Nov 8, 2024
0ef3b18
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Nov 8, 2024
3c3b561
Updates
epugh Nov 8, 2024
0cfd69d
Lint.
epugh Nov 8, 2024
20b5a00
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Nov 13, 2024
a889f0b
Oops!
epugh Nov 13, 2024
c52b087
Update tests to match modern CLI params
epugh Nov 13, 2024
4e018a0
Respond to feedback, and polish
epugh Nov 13, 2024
f2693a9
Lint
epugh Nov 13, 2024
86722d1
Merge remote-tracking branch 'upstream/main' into SOLR-14673
epugh Nov 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions solr/bin/solr
Original file line number Diff line number Diff line change
Expand Up @@ -777,6 +777,12 @@ if [[ "$SCRIPT_CMD" == "delete" ]]; then
exit $?
fi

# run a streaming expression
if [[ "$SCRIPT_CMD" == "stream" ]]; then
run_tool $SCRIPT_CMD "$@"
exit $?
fi

# manage packages
if [[ "$SCRIPT_CMD" == "package" ]]; then
run_tool $SCRIPT_CMD $@
Expand All @@ -790,7 +796,7 @@ if [[ "$SCRIPT_CMD" == "auth" ]]; then
echo -e "\nSolr server directory $SOLR_SERVER_DIR not found!\n"
exit 1
fi

if [ -z "${SOLR_HOME:-}" ]; then
SOLR_HOME="$SOLR_SERVER_DIR/solr"
elif [[ $SOLR_HOME != /* ]]; then
Expand All @@ -801,7 +807,7 @@ if [[ "$SCRIPT_CMD" == "auth" ]]; then
SOLR_PID_DIR="$SOLR_HOME"
fi
fi

if [ -z "${AUTH_PORT:-}" ]; then
for ID in $(ps auxww | grep java | grep start\.jar | awk '{print $2}' | sort -r)
do
Expand All @@ -811,8 +817,8 @@ if [[ "$SCRIPT_CMD" == "auth" ]]; then
break
fi
done
fi
fi

run_tool auth $@ --solr-url "$SOLR_URL_SCHEME://$SOLR_TOOL_HOST:${AUTH_PORT:-8983}" --auth-conf-dir "$SOLR_HOME" "--solr-include-file" "$SOLR_INCLUDE"
exit $?
fi
Expand Down
1 change: 1 addition & 0 deletions solr/bin/solr.cmd
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,7 @@ IF "%1"=="restart" goto set_script_cmd
IF "%1"=="healthcheck" goto run_solrcli
IF "%1"=="create" goto run_solrcli
IF "%1"=="delete" goto run_solrcli
IF "%1"=="stream" goto run_solrcli
IF "%1"=="postlogs" goto run_solrcli
IF "%1"=="zk" (
set SCRIPT_CMD=zk
Expand Down
5 changes: 3 additions & 2 deletions solr/core/src/java/org/apache/solr/cli/SolrCLI.java
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,7 @@ private static Tool newTool(String toolType) throws Exception {
else if ("post".equals(toolType)) return new PostTool();
else if ("postlogs".equals(toolType)) return new PostLogsTool();
else if ("version".equals(toolType)) return new VersionTool();
else if ("stream".equals(toolType)) return new StreamTool();

// If you add a built-in tool to this class, add it here to avoid
// classpath scanning
Expand Down Expand Up @@ -549,7 +550,7 @@ private static void printHelp() {

print("Usage: solr COMMAND OPTIONS");
print(
" where COMMAND is one of: start, stop, restart, status, healthcheck, create, delete, version, zk, auth, assert, config, export, api, package, post");
" where COMMAND is one of: start, stop, restart, status, healthcheck, create, delete, version, zk, auth, assert, config, export, api, package, post, stream");
print("");
print(" Standalone server example (start Solr running in the background on port 8984):");
print("");
Expand Down Expand Up @@ -620,7 +621,7 @@ public static String normalizeSolrUrl(CommandLine cli) throws Exception {
cli.hasOption("zk-host") ? cli.getOptionValue("zk-host") : cli.getOptionValue("zkHost");
if (zkHost == null) {
solrUrl = SolrCLI.getDefaultSolrUrl();
CLIO.getOutStream()
CLIO.getErrStream()
.println(
"Neither --zk-host or --solr-url parameters provided so assuming solr url is "
+ solrUrl
Expand Down
Loading
Loading