Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Manager] Download snapshot artifacts from snapshots repo #18685

Merged
merged 7 commits into from
Jun 10, 2020

Conversation

michalpristas
Copy link
Contributor

@michalpristas michalpristas commented May 21, 2020

What does this PR do?

This PR introduces new downloader which tries to detect snapshot artifact repository in case agent is built as a snapshot.
If detection fails it just does not include snapshot downloader and proceed as usual (with disk backed up by configured downloader)

Why is it important?

More intuitive scenario when you build a snapshot without anything else and it downloads dependencies. Without this it would fail trying to download SNAPSHOT of 8.0.0 from official elastic.co

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

cc @111andre111

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels May 21, 2020
@michalpristas michalpristas requested review from ph, ruflin and blakerouse May 21, 2020 09:57
@michalpristas michalpristas changed the title download from snapshot repo [Ingest Manager] Download snapshot artifacts from snapshots repo May 21, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 21, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18685 updated]

  • Start Time: 2020-05-21T14:07:32.901+0000

  • Duration: 31 min 12 sec

Test stats 🧪

Test Results
Failed 0
Passed 497
Skipped 127
Total 624

@blakerouse
Copy link
Contributor

Overall code looks good. I would like to run it locally to test it out, how would I go about making agent think its a snapshot build?

@michalpristas
Copy link
Contributor Author

@blakerouse agent uses same machinery as beats so SNAPSHOT=true mage build works

@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 2, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18685 updated]

  • Start Time: 2020-06-10T08:17:38.865+0000

  • Duration: 38 min 54 sec

Test stats 🧪

Test Results
Failed 0
Passed 509
Skipped 127
Total 636

Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay on the review.

This worked perfectly!

@michalpristas michalpristas merged commit 3c0eeab into elastic:master Jun 10, 2020
@michalpristas michalpristas added the needs_backport PR is waiting to be backported to other branches. label Jun 12, 2020
michalpristas added a commit to michalpristas/beats that referenced this pull request Jun 12, 2020
…stic#18685)

[Ingest Manager] Download snapshot artifacts from snapshots repo (elastic#18685)
michalpristas added a commit that referenced this pull request Jun 12, 2020
) (#19155)

[Ingest Manager] Download snapshot artifacts from snapshots repo (#18685)
v1v added a commit to v1v/beats that referenced this pull request Jun 12, 2020
…ngs-archive

* upstream/master: (119 commits)
  Update filebeat input docs (elastic#19110)
  Add ECS fields from log pipeline of PostgreSQL (elastic#19127)
  Init package libbeat/statestore (elastic#19117)
  [Ingest Manager] Retryable downloads of beats (elastic#19102)
  [DOCS] Add output.console to Functionbeat doc and Functionbeat reference file (elastic#18965)
  Add compatibility info (elastic#18929)
  Set ecszap version to v0.2.0 (elastic#19106)
  [filebeat][httpjson] Fix unit test function call (elastic#19124)
  [Filebeat][httpjson] Adds oauth2 support for httpjson input (elastic#18892)
  Allow host.* fields to be disabled in Suricata module (elastic#19107)
  Make selector string casing configurable (elastic#18854)
  Switch the GRPC communication where Agent is running the server and the beats are connecting back to Agent (elastic#18973)
  Disable host.* fields by default for netflow module (elastic#19087)
  Automatically fill zube teams on backports if available (elastic#18924)
  Fix crash on vsphere module (elastic#19078)
  [Ingest Manager] Download snapshot artifacts from snapshots repo (elastic#18685)
  [Ingest Manager] Basic Elastic Agent documentation (elastic#19030)
  Make user.id a string in system/users, in line with ECS (elastic#19019)
  [docs] Add 7.8 release highlights placeholder file (elastic#18493)
  Fix translate_sid's empty target field handling (elastic#18991)
  ...
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
…stic#18685)

[Ingest Manager] Download snapshot artifacts from snapshots repo (elastic#18685)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Ingest Management:beta1 Group issues for ingest management beta1 needs_backport PR is waiting to be backported to other branches. review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants