Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added dynamic macros into Jenkins config #20

Merged
merged 6 commits into from
Mar 1, 2024
Merged

Added dynamic macros into Jenkins config #20

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f728d72
added a check whether the user is trying to escape directory when loo…
patrikcerbak Feb 21, 2024
caafcbb
fixed a bug, where the values in the dropdown menu were not saved whe…
patrikcerbak Feb 21, 2024
904f7a2
added more macros into Jenkins comparator link settings - for values …
patrikcerbak Feb 21, 2024
ed66398
reworked the argument parsing, so the dynamic argument can be used be…
patrikcerbak Feb 21, 2024
54e807e
updated the help in Jenkins reflecting the macros changes
patrikcerbak Feb 21, 2024
9fc9016
updated the comparator help to reflect changes to the argument parsing
patrikcerbak Feb 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
254 changes: 135 additions & 119 deletions ...main/java/io/jenkins/plugins/report/jtreg/main/comparator/arguments/ArgumentsParsing.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package io.jenkins.plugins.report.jtreg.main.comparator.arguments;

import io.jenkins.plugins.report.jtreg.Constants;
import io.jenkins.plugins.report.jtreg.main.comparator.HelpMessage;
import io.jenkins.plugins.report.jtreg.main.comparator.Options;
import io.jenkins.plugins.report.jtreg.main.comparator.jobs.DefaultProvider;
Expand All @@ -8,152 +9,162 @@
import io.jenkins.plugins.report.jtreg.formatters.ColorFormatter;
import io.jenkins.plugins.report.jtreg.formatters.HtmlFormatter;

import java.util.ArrayList;
import java.util.List;
import java.util.HashMap;
import java.util.Map;

public class ArgumentsParsing {
// parses the given arguments and returns instance of Options
public static Options parse(String[] arguments) {
Options options = new Options();

if (arguments.length >= 1) {
List<String> dynamicArgs = new ArrayList<>(); // a list for dynamic arguments
Map<String, String> otherArgs = new HashMap<>(); // a list for unmatched (for now) arguments

for (int i = 0; i < arguments.length; i++) {
// delete all leading - characters
String currentArg = arguments[i].replaceAll("^-+", "--");
if (arguments.length == 0) {
throw new RuntimeException("Expected some arguments.");
}

if (!currentArg.matches("^--.*")) {
throw new RuntimeException("Unknown argument " + arguments[i] + ", did you forget the leading hyphens (--)?");
}
for (int i = 0; i < arguments.length; i++) {
// delete all leading - characters
String currentArg = arguments[i].replaceAll("^-+", "--");

// parsing the arguments:
if (currentArg.equals(ArgumentsDeclaration.helpArg.getName()) || currentArg.equals("--h")) {
// --help
System.out.print(HelpMessage.HELP_MESSAGE);
Options tmpO = new Options();
tmpO.setDie(true);
return tmpO;
} else if (currentArg.equals(ArgumentsDeclaration.listArg.getName())) {
// --list
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --list with other operations.");
}
options.setOperation(Options.Operations.List);
if (!currentArg.matches("^--.*")) {
throw new RuntimeException("Unknown argument " + arguments[i] + ", did you forget the leading hyphens (--)?");
}

} else if (currentArg.equals(ArgumentsDeclaration.enumerateArg.getName())) {
// --enumerate
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --enumerate with other operations.");
}
options.setOperation(Options.Operations.Enumerate);
// parsing the arguments:
if (currentArg.equals(ArgumentsDeclaration.helpArg.getName()) || currentArg.equals("--h")) {
// --help
System.out.print(HelpMessage.HELP_MESSAGE);
Options tmpO = new Options();
tmpO.setDie(true);
return tmpO;
} else if (currentArg.equals(ArgumentsDeclaration.listArg.getName())) {
// --list
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --list with other operations.");
}
options.setOperation(Options.Operations.List);

} else if (currentArg.equals(ArgumentsDeclaration.compareArg.getName())) {
// --compare
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --compare with other operations.");
}
options.setOperation(Options.Operations.Compare);
} else if (currentArg.equals(ArgumentsDeclaration.enumerateArg.getName())) {
// --enumerate
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --enumerate with other operations.");
}
options.setOperation(Options.Operations.Enumerate);

} else if (currentArg.equals(ArgumentsDeclaration.printArg.getName())) {
// --print
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --print with other operations.");
}
options.setOperation(Options.Operations.Print);
} else if (currentArg.equals(ArgumentsDeclaration.compareArg.getName())) {
// --compare
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --compare with other operations.");
}
options.setOperation(Options.Operations.Compare);

} else if (currentArg.equals(ArgumentsDeclaration.virtualArg.getName())) {
// --virtual
options.setPrintVirtual(true);
} else if (currentArg.equals(ArgumentsDeclaration.printArg.getName())) {
// --print
if (options.getOperation() != null) {
throw new RuntimeException("Cannot combine --print with other operations.");
}
options.setOperation(Options.Operations.Print);

} else if (currentArg.equals(ArgumentsDeclaration.pathArg.getName())) {
// --path
options.setJobsPath(getArgumentValue(arguments, i++));
} else if (currentArg.equals(ArgumentsDeclaration.virtualArg.getName())) {
// --virtual
options.setPrintVirtual(true);

} else if (currentArg.equals(ArgumentsDeclaration.historyArg.getName())) {
// --history
options.setNumberOfBuilds(Integer.parseInt(getArgumentValue(arguments, i++)));
} else if (currentArg.equals(ArgumentsDeclaration.pathArg.getName())) {
// --path
options.setJobsPath(getArgumentValue(arguments, i++));

} else if (currentArg.equals(ArgumentsDeclaration.skipFailedArg.getName())) {
// --skip-failed
if (!Boolean.parseBoolean(getArgumentValue(arguments, i++))) {
// if skip failed = false, any result value is taken, so .*
options.getConfiguration("result").setValue(".*");
}
} else if (currentArg.equals(ArgumentsDeclaration.historyArg.getName())) {
// --history
options.setNumberOfBuilds(Integer.parseInt(getArgumentValue(arguments, i++)));

} else if (currentArg.equals(ArgumentsDeclaration.forceArg.getName())) {
// --force
options.setForceVague(true);

} else if (currentArg.equals(ArgumentsDeclaration.onlyVolatileArg.getName())) {
// --only-volatile
options.setOnlyVolatile(Boolean.parseBoolean(getArgumentValue(arguments, i++)));

} else if (currentArg.equals(ArgumentsDeclaration.exactTestsArg.getName())) {
// --exact-tests
options.setExactTestsRegex(getArgumentValue(arguments, i++));

} else if (currentArg.equals(ArgumentsDeclaration.formattingArg.getName())) {
// --formatting
String formatting = getArgumentValue(arguments, i++);
if (formatting.equals("color") || formatting.equals("colour")) {
options.setFormatter(new ColorFormatter(System.out));
} else if (formatting.equals("html")) {
options.setFormatter(new HtmlFormatter(System.out));
} else if (!formatting.equals("plain")) {
throw new RuntimeException("Unexpected formatting specified.");
}
} else if (currentArg.equals(ArgumentsDeclaration.skipFailedArg.getName())) {
// --skip-failed
if (!Boolean.parseBoolean(getArgumentValue(arguments, i++))) {
// if skip failed = false, any result value is taken, so .*
options.getConfiguration("result").setValue(".*");
}

} else if (currentArg.equals(ArgumentsDeclaration.useDefaultBuildArg.getName())) {
// --use-default-build
options.setUseDefaultBuild(Boolean.parseBoolean(getArgumentValue(arguments, i++)));
} else if (currentArg.equals(ArgumentsDeclaration.forceArg.getName())) {
// --force
options.setForceVague(true);

} else if (currentArg.equals(ArgumentsDeclaration.onlyVolatileArg.getName())) {
// --only-volatile
options.setOnlyVolatile(Boolean.parseBoolean(getArgumentValue(arguments, i++)));

} else if (currentArg.equals(ArgumentsDeclaration.exactTestsArg.getName())) {
// --exact-tests
options.setExactTestsRegex(getArgumentValue(arguments, i++));

} else if (currentArg.equals(ArgumentsDeclaration.formattingArg.getName())) {
// --formatting
String formatting = getArgumentValue(arguments, i++);
if (formatting.equals("color") || formatting.equals("colour")) {
options.setFormatter(new ColorFormatter(System.out));
} else if (formatting.equals("html")) {
options.setFormatter(new HtmlFormatter(System.out));
} else if (!formatting.equals("plain")) {
throw new RuntimeException("Unexpected formatting specified.");
}

} else if (currentArg.equals(ArgumentsDeclaration.buildConfigFindArg.getName()) ||
currentArg.equals(ArgumentsDeclaration.jobConfigFindArg.getName())) {
// --X-config-find
String[] values = getArgumentValue(arguments, i++).split(":");
Options.Configuration configuration = getConfiguration(values, currentArg);
// (whatToFind, configuration)
} else if (currentArg.equals(ArgumentsDeclaration.useDefaultBuildArg.getName())) {
// --use-default-build
options.setUseDefaultBuild(Boolean.parseBoolean(getArgumentValue(arguments, i++)));

} else if (currentArg.equals(ArgumentsDeclaration.buildConfigFindArg.getName()) ||
currentArg.equals(ArgumentsDeclaration.jobConfigFindArg.getName())) {
// --X-config-find
String[] values = getArgumentValue(arguments, i++).split(":");
Options.Configuration configuration = createConfiguration(values, currentArg);
// (whatToFind, configuration)
if (options.getConfiguration(values[1]) == null) {
options.addConfiguration(values[1], configuration);
dynamicArgs.add(values[1]);
} else {
throw new RuntimeException("The configuration for " + values[1] + " already exists.");
}

// parsing arguments of the jobs providers
} else if (JobsByQuery.getSupportedArgsStatic().contains(currentArg) || JobsByRegex.getSupportedArgsStatic().contains(currentArg)) {
// add a jobs provider to options, if there is none
if (options.getJobsProvider() == null) {
if (JobsByQuery.getSupportedArgsStatic().contains(currentArg)) {
options.setJobsProvider(new JobsByQuery());
} else if (JobsByRegex.getSupportedArgsStatic().contains(currentArg)){
options.setJobsProvider(new JobsByRegex());
}
} else if (JobsByQuery.getSupportedArgsStatic().contains(currentArg) || JobsByRegex.getSupportedArgsStatic().contains(currentArg)) {
// add a jobs provider to options, if there is none
if (options.getJobsProvider() == null) {
if (JobsByQuery.getSupportedArgsStatic().contains(currentArg)) {
options.setJobsProvider(new JobsByQuery());
} else if (JobsByRegex.getSupportedArgsStatic().contains(currentArg)){
options.setJobsProvider(new JobsByRegex());
}
// check if the argument is compatible with current jobs provider
if (options.getJobsProvider().getSupportedArgs().contains(currentArg)) {
options.getJobsProvider().parseArguments(currentArg, getArgumentValue(arguments, i++));
}
// check if the argument is compatible with current jobs provider
if (options.getJobsProvider().getSupportedArgs().contains(currentArg)) {
options.getJobsProvider().parseArguments(currentArg, getArgumentValue(arguments, i++));

} else {
throw new RuntimeException("Cannot combine arguments from multiple job providers.");
}
} else {
throw new RuntimeException("Cannot combine arguments from multiple job providers.");
}

} else {
if (!arguments[i + 1].matches("^--.*")) {
// these arguments can be dynamic arguments, that were not defined yet
// putting them into the map and checking later
otherArgs.put(currentArg, getArgumentValue(arguments, i++));
} else {
// check if the argument is one of the dynamic arguments
String strippedArg = currentArg.replaceAll("-", "");
if (dynamicArgs.contains(strippedArg)) {
Options.Configuration configuration = options.getConfiguration(strippedArg);
if (configuration != null) {
configuration.setValue(getArgumentValue(arguments, i++));
} else {
throw new RuntimeException("Cannot find configuration for argument " + currentArg + ". Please set it first with --build-config-find or --job-config-find.");
}
} else {
// unknown argument
throw new RuntimeException("Unknown argument " + currentArg + ", run with --help for info.");
}
// only arguments with some value can now be legal, so if no value was present, throw
throw new RuntimeException("Unknown argument " + currentArg + ", run with --help for info.");
}
}
}

for (Map.Entry<String, String> entry : otherArgs.entrySet()) {
// check if the argument is one of the dynamic arguments
String strippedArg = entry.getKey().replaceAll("-", "");

Options.Configuration configuration = options.getConfiguration(strippedArg);
if (configuration != null) {
configuration.setValue(entry.getValue());
} else {
// unknown argument
throw new RuntimeException("Unknown argument " + entry.getKey() + ", run with --help for info.");
}
} else {
throw new RuntimeException("Expected some arguments.");
}

// check for basic errors
Expand All @@ -177,10 +188,15 @@ public static Options parse(String[] arguments) {
return options;
}

private static Options.Configuration getConfiguration(String[] values, String currentArg) {
private static Options.Configuration createConfiguration(String[] values, String currentArg) {
Options.Configuration configuration;
// checks the number of items it got from the splitting of the value
if (values.length == 3) { // filename:whatToFind:findQuery
// check if the user is trying to escape the current directory with ../
if (values[0].matches(Constants.ESCAPE_DIRECTORY_REGEX)) {
throw new RuntimeException("Cannot escape from the directory with config file to its parent directory.");
}

if (currentArg.equals(ArgumentsDeclaration.buildConfigFindArg.getName())) {
configuration = new Options.Configuration(values[0], values[2], Options.Locations.Build);
} else {
Expand All @@ -193,7 +209,7 @@ private static Options.Configuration getConfiguration(String[] values, String cu
}

private static String getArgumentValue(String[] arguments, int i) {
if (i + 1 < arguments.length) {
if (i + 1 < arguments.length && !arguments[i + 1].matches("^--.*")) {
return arguments[i + 1];
} else {
throw new RuntimeException("Expected some value after " + arguments[i] + " argument.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trust in you, I have, young padawan. However this is becaoming regex hell :) I once implemented mathematic formula parser based on split and repalce... not any more!

Expand Down
2 changes: 1 addition & 1 deletion report-jtreg-lib/src/main/java/io/jenkins/plugins/report/jtreg/Constants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ final public class Constants {
public static final String REPORT_TESTS_LIST_JSON = "tests-list.json";
public static final String IRRELEVANT_GLOB_STRING = "report-{runtime,devtools,compiler}.xml.gz";
public static final double VAGUE_QUERY_THRESHOLD = 0.5;

public static final int VAGUE_QUERY_LENGTH_THRESHOLD = 4;
public static final String ESCAPE_DIRECTORY_REGEX = "((\\.\\./)+.*(\\.\\./)*.*)|((\\.\\./)*.*(\\.\\./)+.*)";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isnt better to simply check, that the fully resolved filename, the final abs path (java have methods for that), is simply still in desired workspace/cwd/app dir?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wil merge this as it is. But the rework of shoudl be needed.
Again,. from my own experiments, checking input like this may be missleading.

eg correctdir/correctdir/../fileInFirstCorrectDir is pretty correct
Eg I'm checking the bash injection in the CGI wrapper by checking for "|" and thus literally killing this tool. Btrw, any ideas how to fix that "|" (https://github.com/jenkinsci/report-jtreg-plugin/blob/master/report-jtreg-service/src/main/java/io/jenkins/plugins/report/jtreg/main/web/ContextExecutingHandler.java#L114)

}
Loading