[test] port linux package packaging tests #31943

andyb-elastic · 2018-07-10T22:36:31Z

Add packaging tests for the linux package distributions to the java test
project and remove them from bats. Most of the tests that lived in
30_deb_package.bats and 40_rpm_package.bats are applicable to both
package types and are combined into a single type of test case. Others
are separated out into separate cases to make their intent more clear

For #26741

Add packaging tests for the linux package distributions to the java test project and remove them from bats. Most of the tests that lived in 30_deb_package.bats and 40_rpm_package.bats are applicable to both package types and are combined into a single type of test case. Others are separated out into separate cases to make their intent more clear For elastic#26741

elasticmachine · 2018-07-10T22:36:32Z

Pinging @elastic/es-core-infra

andyb-elastic · 2018-07-10T22:40:53Z

qa/vagrant/src/main/java/org/elasticsearch/packaging/util/Cleanup.java

            sh.runIgnoreExitCode("rpm --quiet -e elasticsearch elasticsearch-oss");
        }

-        if (isYUM()) {
-            sh.runIgnoreExitCode("yum remove -y elasticsearch elasticsearch-oss");
-        }


On oel 6, yum downloads a lot of stuff when this is invoked. We don't call out to yum or apt get anywhere else in the tests, and we always interact with rpm or dpkg directly, so it seemed fine to take this out

andyb-elastic · 2018-07-10T22:45:50Z

qa/vagrant/src/main/java/org/elasticsearch/packaging/test/PackageTestCase.java

+            // The exception is Centos 7, where it returns exit code 4 for non-existent services from a systemd reporting a version
+            // earlier than 231. Centos 6 does not have an /etc/os-release, but that's fine because it also doesn't use systemd.
+            //
+            // [1] https://github.com/systemd/systemd/pull/3385


I did not dig too deeply into why centos behaves differently here. I figured it was worth still doing this (vs just accepting either exit code) because distinguishing between exists-but-stopped and nonexistent seems valuable if we're able to

centos-7:~$ sudo bash -c 'systemctl --version; systemctl status foooooo.service; echo $?' systemd 219 +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN Unit foooooo.service could not be found. 4

andyb-elastic · 2018-07-11T00:59:28Z

Looks like I made a mistake that causes it to fail when the package is still installed from the bats tests. I'll fix tomorrow

alpar-t

LGTM.
There are a lot of checks for file exists, not exists. permissions, may be more readable with some specialized methods like assertAllExists(Path... paths)

I still get that feeling we might want to have chainable conditionals like Platform.isDpkg(() -> {}).isRpm(() -> {}) etc. If/when something goes wrong with these checks and they won't work properly it will make it hard to debug those failures, but I don't feel strongly about it, we could wait for it to happen :)

alpar-t · 2018-07-11T08:50:53Z

qa/vagrant/src/main/java/org/elasticsearch/packaging/util/Packages.java

+            result = sh.runIgnoreExitCode("dpkg -s " + distribution.flavor.name);
+        }
+
+        if (result.exitCode > 1) {


Why is it ok to ignore exit code 1 but not the others ?

This ended up like this after some refactoring and doesn't really make sense now, so I'll remove it. Originally it was that I didn't want to conflate exit code 1 (package doesn't exist) with some other error from the package manager

andyb-elastic · 2018-07-11T18:19:34Z

may be more readable with some specialized methods like assertAllExists(Path... paths)

Good point I'll do that

Could have sworn I tested this

andyb-elastic · 2018-07-19T00:17:42Z

Ryan gave the okay to merge this with just Alpar's review

Add packaging tests for the linux package distributions to the java test project and remove them from bats. Most of the tests that lived in 30_deb_package.bats and 40_rpm_package.bats are applicable to both package types and are combined into a single type of test case. Others are separated out into separate cases to make their intent more clear For elastic#26741

@return

* master: Painless: Simplify Naming in Lookup Package (#32177) Handle missing values in painless (#32207) add support for write index resolution when creating/updating documents (#31520) ECS Task IAM profile credentials ignored in repository-s3 plugin (#31864) Remove indication of future multi-homing support (#32187) Rest test - allow for snapshots to take 0 milliseconds Make x-pack-core generate a pom file Rest HL client: Add put watch action (#32026) Build: Remove pom generation for plugin zip files (#32180) Fix comments causing errors with Java 11 Fix rollup on date fields that don't support epoch_millis (#31890) Detect and prevent configuration that triggers a Gradle bug (#31912) [test] port linux package packaging tests (#31943) Revert "Introduce a Hashing Processor (#31087)" (#32178) Remove empty @return from JavaDoc Adjust SSLDriver behavior for JDK11 changes (#32145) [test] use randomized runner in packaging tests (#32109) Add support for field aliases. (#32172) Painless: Fix caching bug and clean up addPainlessClass. (#32142) Call setReferences() on custom referring tokenfilters in _analyze (#32157) Fix BwC Tests looking for UUID Pre 6.4 (#32158) Improve docs for search preferences (#32159) use before instead of onOrBefore Add more contexts to painless execute api (#30511) Add EC2 credential test for repository-s3 (#31918) A replica can be promoted and started in one cluster state update (#32042) Fix Java 11 javadoc compile problem Fix CP for namingConventions when gradle home has spaces (#31914) Fix `range` queries on `_type` field for singe type indices (#31756) [DOCS] Update TLS on Docker for 6.3 (#32114) ESIndexLevelReplicationTestCase doesn't support replicated failures but it's good to know what they are Remove versionType from translog (#31945) Switch distribution to new style Requests (#30595) Build: Skip jar tests if jar disabled Painless: Add PainlessClassBuilder (#32141) Build: Make additional test deps of check (#32015) Disable C2 from using AVX-512 on JDK 10 (#32138) Build: Move shadow customizations into common code (#32014) Painless: Fix Bug with Duplicate PainlessClasses (#32110) Remove empty @param from Javadoc Re-disable packaging tests on suse boxes Docs: Fix missing example script quote (#32010) [ML] Wait for aliases in multi-node tests (#32086) [ML] Move analyzer dependencies out of categorization config (#32123) Ensure to release translog snapshot in primary-replica resync (#32045) Handle TokenizerFactory TODOs (#32063) Relax TermVectors API to work with textual fields other than TextFieldType (#31915) Updates the build to gradle 4.9 (#32087) Mute :qa:mixed-cluster indices.stats/10_index/Index - all’ Check that client methods match API defined in the REST spec (#31825) Enable testing in FIPS140 JVM (#31666) Fix put mappings java API documentation (#31955) Add exclusion option to `keep_types` token filter (#32012) [Test] Modify assert statement for ssl handshake (#32072)

This recreates a test that was added to the bats packaging tests in elastic#31343 but didn't make it over to the java project during when the linux package tests were ported in elastic#31943 When packages are installed but can not locate the java executable, they should fail with a descriptive message

This recreates a test that was added to the bats packaging tests in #31343 but didn't make it over to the java project during when the linux package tests were ported in #31943 When packages are installed but can not locate the java executable, they should fail with a descriptive message

Add packaging tests for the linux package distributions to the java test project and remove them from bats. Most of the tests that lived in 30_deb_package.bats and 40_rpm_package.bats are applicable to both package types and are combined into a single type of test case. Others are separated out into separate cases to make their intent more clear For #26741

This recreates a test that was added to the bats packaging tests in elastic#31343 but didn't make it over to the java project during when the linux package tests were ported in elastic#31943 When packages are installed but can not locate the java executable, they should fail with a descriptive message

* 6.x: Security: revert to old way of merging automata (#32254) Fix a test bug in RangeQueryBuilderTests introduced in the field aliases backport. Introduce Application Privileges with support for Kibana RBAC (#32309) Undo a debugging change that snuck in during the field aliases merge. [test] port linux package packaging tests (#31943) Painless: Update More Methods to New Naming Scheme (#32305) Tribe: Add error with secure settings copied to tribe (#32298) Add V_6_3_3 version constant Add ERR to ranking evaluation documentation (#32314) [DOCS] Added link to 6.3.2 RNs [DOCS] Updates 6.3.2 release notes with PRs from ml-cpp repo (#32334) [Kerberos] Add Kerberos authentication support (#32263) [ML] Extract persistent task methods from MlMetadata (#32319) Backport - Add Snapshots Status API to High Level Rest Client (#32295) Make release notes ignore the `>test-failure` label. (#31309) [DOCS] Adds release highlights for search for 6.4 (#32095) Allow Integ Tests to run in a FIPS-140 JVM (#32316) Add support for field aliases to 6.x. (#32184) Register ERR metric with NamedXContentRegistry (#32320) fixes broken build for third-party-tests (#32315) Relates #31918 / Closes infra/issues/6085 [DOCS] Rollup Caps API incorrectly mentions GET Jobs API (#32280) Rest HL client: Add put watch action (#32026) (#32191) Add WeightedAvg metric aggregation (#31037) Consistent encoder names (#29492) Switch monitoring to new style Requests (#32255) specify subdirs of lib, bin, modules in package (#32253) Rename ranking evaluation `quality_level` to `metric_score` (#32168) Add new permission for JDK11 to load JAAS libraries (#32132) Switch x-pack:core to new style Requests (#32252) Watcher: Store username on watch execution (#31873) Silence SSL reload test that fails on JDK 11 Painless: Clean up add methods in PainlessLookup (#32258) CCE when re-throwing "shard not available" exception in TransportShardMultiGetAction (#32185) Fail shard if IndexShard#storeStats runs into an IOException (#32241) Fix `range` queries on `_type` field for singe type indices (#31756) (#32161) AwaitsFix RecoveryIT#testHistoryUUIDIsGenerated Add new fields to monitoring template for Beats state (#32085) (#32273) [TEST] improve REST high-level client naming conventions check (#32244) Check that client methods match API defined in the REST spec (#31825)

This recreates a test that was added to the bats packaging tests in #31343 but didn't make it over to the java project during when the linux package tests were ported in #31943 When packages are installed but can not locate the java executable, they should fail with a descriptive message

andyb-elastic added >test Issues or PRs that are addressing/adding tests :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts v7.0.0 v6.4.0 labels Jul 10, 2018

andyb-elastic requested review from rjernst and alpar-t July 10, 2018 22:36

andyb-elastic commented Jul 10, 2018

View reviewed changes

alpar-t reviewed Jul 11, 2018

View reviewed changes

andyb-elastic added 5 commits July 11, 2018 11:51

fix issue with rpm package removal

083ba13

code review fixes

ae78673

checkstyle fixes

2e7b6d4

turn log level down here - will need a way to configure it later

7132987

oel 7 also has weird systemctl behavior

f22b17d

Could have sworn I tested this

alpar-t approved these changes Jul 12, 2018

View reviewed changes

This was referenced Jul 16, 2018

[test] use randomized runner in packaging tests #32109

Merged

java packaging tests #32143

Closed

andyb-elastic added 2 commits July 18, 2018 10:33

Merge branch 'master' into feature-java-package-1

dac3d0a

use packagingtestcase and randomizedrunner

463b517

andyb-elastic merged commit aae0133 into elastic:master Jul 19, 2018

andyb-elastic mentioned this pull request Jul 19, 2018

backport to 6.x - [test] port linux package packaging tests #32183

Merged

andyb-elastic mentioned this pull request Jul 20, 2018

[test] package pre-install java check #32259

Merged

jimczi added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[test] port linux package packaging tests #31943

[test] port linux package packaging tests #31943

andyb-elastic commented Jul 10, 2018

elasticmachine commented Jul 10, 2018

andyb-elastic Jul 10, 2018

andyb-elastic Jul 10, 2018 •

edited

Loading

andyb-elastic commented Jul 11, 2018

alpar-t left a comment

alpar-t Jul 11, 2018

andyb-elastic Jul 11, 2018

andyb-elastic commented Jul 11, 2018

andyb-elastic commented Jul 19, 2018

[test] port linux package packaging tests #31943

[test] port linux package packaging tests #31943

Conversation

andyb-elastic commented Jul 10, 2018

elasticmachine commented Jul 10, 2018

andyb-elastic Jul 10, 2018

Choose a reason for hiding this comment

andyb-elastic Jul 10, 2018 • edited Loading

Choose a reason for hiding this comment

andyb-elastic commented Jul 11, 2018

alpar-t left a comment

Choose a reason for hiding this comment

alpar-t Jul 11, 2018

Choose a reason for hiding this comment

andyb-elastic Jul 11, 2018

Choose a reason for hiding this comment

andyb-elastic commented Jul 11, 2018

andyb-elastic commented Jul 19, 2018

andyb-elastic Jul 10, 2018 •

edited

Loading