Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented Docker Image caching for integration-unreleased workflow #498

Merged
merged 1 commit into from
Apr 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 61 additions & 17 deletions .github/workflows/integration-unreleased.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,41 +19,85 @@ jobs:
fail-fast: false
matrix:
entry:
- { opensearch_ref: '1.x' }
- { opensearch_ref: '2.0' }
- { opensearch_ref: '2.x' }
- { opensearch_ref: 'main' }
- { node_version: '16.x', opensearch_ref: '1.x' }
- { node_version: '16.x', opensearch_ref: '2.0' }
- { node_version: '16.x', opensearch_ref: '2.x' }
- { node_version: '16.x', opensearch_ref: 'main' }
steps:
- name: Checkout OpenSearch
uses: actions/checkout@v2
uses: actions/checkout@v3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess do we never want to just download the snapshots directly instead of building?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate? If we don't even have to build the newest OS ourselves that would be great!

with:
repository: opensearch-project/OpenSearch
ref: ${{ matrix.entry.opensearch_ref }}
path: opensearch

- name: Get OpenSearch branch top
id: get-key
working-directory: opensearch
run: echo key=`git log -1 --format='%H'` >> $GITHUB_OUTPUT

- name: Restore cached build
id: cache-restore
uses: actions/cache/restore@v3
with:
path: opensearch/distribution/archives/linux-tar/build/distributions
key: ${{ steps.get-key.outputs.key }}

- name: Assemble OpenSearch
if: steps.cache-restore.outputs.cache-hit != 'true'
working-directory: opensearch
run: ./gradlew :distribution:archives:linux-tar:assemble

- name: Save cached build
if: steps.cache-restore.outputs.cache-hit != 'true'
uses: actions/cache/save@v3
with:
path: opensearch/distribution/archives/linux-tar/build/distributions
key: ${{ steps.get-key.outputs.key }}

- name: Run OpenSearch
working-directory: opensearch/distribution/archives/linux-tar/build/distributions
run: |
cd opensearch
./gradlew assemble
# This step runs the docker image generated during gradle assemble in OpenSearch. It is tagged as opensearch:test.
# Reference: https://github.com/opensearch-project/OpenSearch/blob/2.0/distribution/docker/build.gradle#L190
- name: Run Docker Image
run: |
docker run -p 9200:9200 -p 9600:9600 -d -e "discovery.type=single-node" -e "bootstrap.memory_lock=true" opensearch:test
sleep 90
tar xf opensearch-min-*
./opensearch-*/bin/opensearch \
-Enode.attr.testattr=test \
-Epath.repo=/tmp \
-Erepositories.url.allowed_urls='http://snapshot.*' \
-d
for attempt in {1..20}; do
sleep 5
if curl -s localhost:9200; then
echo '=====> ready'
exit 0
fi
echo '=====> waiting...'
done
exit 1

- name: Checkout Javascript Client
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
path: js-client

- name: Use Node.js 16.x
uses: actions/setup-node@v1
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: 16.x
node-version: ${{ matrix.entry.node_version }}

- name: Install
working-directory: js-client
run: |
npm install

- name: Integration test
working-directory: js-client
run: |
npm run test:integration:helpers

- name: Save server logs
if: failure()
uses: actions/upload-artifact@v3
with:
name: opensearch-logs-${{ matrix.entry.opensearch_ref }}-js-${{ matrix.entry.node_version }}
path: |
opensearch/distribution/archives/linux-tar/build/distributions/**/logs/*
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Added
### Dependencies
### Changed
- Implemented Docker Image caching for `integration-unreleased` workflow ([#387](https://github.com/opensearch-project/opensearch-js/issues/387))
### Deprecated
### Removed
### Fixed
Expand Down