Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jepsen: ignore some install errors #56815

Merged
merged 1 commit into from
Dec 1, 2020

Conversation

andreimatei
Copy link
Contributor

@andreimatei andreimatei commented Nov 17, 2020

Two failures. Seem to have been transient.

| Could not transfer artifact org.clojure:clojure:jar:1.9.0 from/to central (https://repo1.maven.org/maven2/): GET request of: org/clojure/clojure/1.9.0/clojure-1.9.0.jar from central failed
| Could not find artifact org.clojure:clojure:jar:1.9.0 in clojars (https://repo.clojars.org/)
| This could be due to a typo in :dependencies, file system permissions, or network issues.
| If you are behind a proxy, try setting the 'http_proxy' environment variable.
| Failed to read artifact descriptor for commons-codec:commons-codec:jar:1.6
| This could be due to a typo in :dependencies, file system permissions, or network issues.

Fixes #56695
Fixes #56645

Release note: None

@andreimatei andreimatei requested review from knz and tbg November 17, 2020 19:02
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@andreimatei andreimatei force-pushed the jepsen-ignore-install-err branch 2 times, most recently from c44776d to 3ac636a Compare November 17, 2020 19:16
@andreimatei andreimatei changed the title jepsen: ignore install error jepsen: ignore some install errors Nov 17, 2020
@andreimatei andreimatei force-pushed the jepsen-ignore-install-err branch 4 times, most recently from 070c415 to 99b0bf9 Compare November 17, 2020 20:31
@knz
Copy link
Contributor

knz commented Nov 18, 2020

No objection for me - but I would like to highlight this is one more test that is too happy to go and fetch dependencies externally. Why don't we have our own local copies of these files?

cc @jlinder - what would be the better fix here?

@tbg
Copy link
Member

tbg commented Nov 18, 2020

LGTM

Andrei has long been beating the drum about making these self-contained, but we haven't had the resources to do it. Once we build abstractions here, they need to be maintained, and so unless dev-inf can take this on I think we'll just have to live with the status quo.

@jlinder
Copy link
Collaborator

jlinder commented Nov 18, 2020

Dev-inf hasn't had time to focus on roachtest since before July and won't until next year sometime.

cc @jlinder - what would be the better fix here?

The most obvious fix that doesn't download the file from the source location on every run is:

  1. create a google storage bucket
  2. place the files we need there
  3. download the files from the bucket
  4. add some tracking mechanism to check for new versions of the files periodically (assuming they get updated) and automatically download the new file and place it in the bucket

If there are files loaded by the runs in AWS, add this to the above:

  1. create an S3 bucket
  2. place the files in it that are used by the tests run in S3
  3. add a mechanism to roachtest that downloads the target files from the S3 bucket / gs bucket according to whether it's running in GCP or AWS
  4. check for new versions of the files with the above defined tracking mechanism + placing the new versions in the S3 bucket

@jlinder
Copy link
Collaborator

jlinder commented Nov 18, 2020

Why don't we have our own local copies of these files?

Added an issue to track this feature request. #56856

@nvanbenschoten
Copy link
Member

We'll want to backport this as well, as we've also seen this failure on other release branches: https://github.com/cockroachdb/cockroach/issues?q=is%3Aissue+is%3Aopen+%22If+you+are+behind+a+proxy%2C+try+setting+the+%27http_proxy%27+environment+variable.%22.

@andreimatei andreimatei force-pushed the jepsen-ignore-install-err branch 2 times, most recently from 6e54539 to 06e75f5 Compare December 1, 2020 18:23
Two failures. Seem to have been transient.

```
| Could not transfer artifact org.clojure:clojure:jar:1.9.0 from/to central (https://repo1.maven.org/maven2/): GET request of: org/clojure/clojure/1.9.0/clojure-1.9.0.jar from central failed
| Could not find artifact org.clojure:clojure:jar:1.9.0 in clojars (https://repo.clojars.org/)
| This could be due to a typo in :dependencies, file system permissions, or network issues.
| If you are behind a proxy, try setting the 'http_proxy' environment variable.
```

```
| Failed to read artifact descriptor for commons-codec:commons-codec:jar:1.6
| This could be due to a typo in :dependencies, file system permissions, or network issues.
```

Fixes cockroachdb#56695
Fixes cockroachdb#56645
Fixes cockroachdb#56644
Fixes cockroachdb#56565
Fixes cockroachdb#56519

Release note: None
@andreimatei andreimatei force-pushed the jepsen-ignore-install-err branch from 06e75f5 to efd1cbb Compare December 1, 2020 19:05
@andreimatei
Copy link
Contributor Author

bors r+

@craig craig bot merged commit 4e28ca3 into cockroachdb:master Dec 1, 2020
@craig
Copy link
Contributor

craig bot commented Dec 1, 2020

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roachtest: jepsen/bank/majority-ring failed roachtest: jepsen/bank-multitable/split failed
6 participants