Introduce Conan, JFrog and GitHub Actions to build and manage Cura (and her dependencies #12708
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| name: Create and Upload Conan package | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| recipe_id_full: | ||
| required: true | ||
| type: string | ||
|
|
||
| runs_on: | ||
| required: true | ||
| type: string | ||
|
|
||
| python_version: | ||
| required: true | ||
| type: string | ||
|
|
||
| conan_config_branch: | ||
| required: false | ||
| type: string | ||
|
|
||
| conan_logging_level: | ||
| required: false | ||
| type: string | ||
|
|
||
| conan_clean_local_cache: | ||
| required: false | ||
| type: boolean | ||
| default: false | ||
|
|
||
| env: | ||
| CONAN_LOGIN_USERNAME_CURA: ${{ secrets.CONAN_USER }} | ||
| CONAN_PASSWORD_CURA: ${{ secrets.CONAN_PASS }} | ||
| CONAN_LOGIN_USERNAME_CURA_CE: ${{ secrets.CONAN_USER }} | ||
| CONAN_PASSWORD_CURA_CE: ${{ secrets.CONAN_PASS }} | ||
| CONAN_LOG_RUN_TO_OUTPUT: 1 | ||
| CONAN_LOGGING_LEVEL: ${{ inputs.conan_logging_level }} | ||
| CONAN_NON_INTERACTIVE: 1 | ||
|
|
||
| jobs: | ||
| conan-package-create: | ||
| runs-on: ${{ inputs.runs_on }} | ||
|
|
||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v3 | ||
|
|
||
| - name: Setup Python and pip | ||
| uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: ${{ inputs.python_version }} | ||
| cache: 'pip' | ||
| cache-dependency-path: .github/workflows/requirements-conan-package.txt | ||
|
|
||
| - name: Install Python requirements and Create default Conan profile | ||
| run: | | ||
| pip install -r .github/workflows/requirements-conan-package.txt | ||
| conan profile new default --detect | ||
|
|
||
| - name: Use Conan download cache (Bash) | ||
| if: ${{ runner.os != 'Windows' }} | ||
| run: conan config set storage.download_cache="$HOME/.conan/conan_download_cache" | ||
|
|
||
| - name: Use Conan download cache (Powershell) | ||
| if: ${{ runner.os == 'Windows' }} | ||
| run: conan config set storage.download_cache="C:\Users\runneradmin\.conan\conan_download_cache" | ||
casperlamboo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| - name: Cache Conan local repository packages (Bash) | ||
| uses: actions/cache@v3 | ||
| if: ${{ runner.os != 'Windows' }} | ||
| with: | ||
| path: | | ||
| $HOME/.conan/data | ||
| $HOME/.conan/conan_download_cache | ||
| key: conan-${{ runner.os }}-${{ runner.arch }} | ||
|
|
||
| - name: Cache Conan local repository packages (Powershell) | ||
| uses: actions/cache@v3 | ||
| if: ${{ runner.os == 'Windows' }} | ||
| with: | ||
| path: | | ||
| C:\Users\runneradmin\.conan\data | ||
casperlamboo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| C:\.conan | ||
| C:\Users\runneradmin\.conan\conan_download_cache | ||
| key: conan-${{ runner.os }}-${{ runner.arch }} | ||
|
|
||
| - name: Install MacOS system requirements | ||
| if: ${{ runner.os == 'Macos' }} | ||
| run: brew install autoconf automake ninja | ||
|
|
||
| - name: Install Linux system requirements | ||
| if: ${{ runner.os == 'Linux' }} | ||
| run: sudo apt install build-essential checkinstall libegl-dev zlib1g-dev libssl-dev ninja-build autoconf libx11-dev libx11-xcb-dev libfontenc-dev libice-dev libsm-dev libxau-dev libxaw7-dev libxcomposite-dev libxcursor-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxi-dev libxinerama-dev libxkbfile-dev libxmu-dev libxmuu-dev libxpm-dev libxrandr-dev libxrender-dev libxres-dev libxss-dev libxt-dev libxtst-dev libxv-dev libxvmc-dev libxxf86vm-dev xtrans-dev libxcb-render0-dev libxcb-render-util0-dev libxcb-xkb-dev libxcb-icccm4-dev libxcb-image0-dev libxcb-keysyms1-dev libxcb-randr0-dev libxcb-shape0-dev libxcb-sync-dev libxcb-xfixes0-dev libxcb-xinerama0-dev xkb-data libxcb-dri3-dev uuid-dev libxcb-util-dev libxkbcommon-x11-dev -y | ||
|
There was a problem hiding this comment. Maybe these dependencies can be stored similar to the There was a problem hiding this comment. Not an Ubuntu-man myself if we can store these in a system deps file, then I'm all for it. But I myself don't have a ready made solution for this in mind. Maybe you guys can look into that |
||
|
|
||
| - name: Clean Conan local cache | ||
| if: ${{ inputs.conan_clean_local_cache }} | ||
| run: conan remove "*" -f | ||
|
|
||
| - name: Get Conan configuration from branch | ||
| if: ${{ inputs.conan_config_branch != '' }} | ||
| run: conan config install https://github.com/Ultimaker/conan-config.git -a "-b ${{ inputs.conan_config_branch }}" | ||
|
|
||
| - name: Get Conan configuration | ||
| if: ${{ inputs.conan_config_branch == '' }} | ||
| run: conan config install https://github.com/Ultimaker/conan-config.git | ||
|
|
||
| - name: Create the Packages | ||
| run: conan install ${{ inputs.recipe_id_full }} --build=missing --update | ||
|
|
||
| - name: Upload the Package(s) | ||
| if: always() | ||
|
There was a problem hiding this comment.
There was a problem hiding this comment. If creating a package from the current project fails, then we still could have build binaries from depending Conan packages. By ensuring that these are uploaded you will have a quicker subsequent run, were it doesn't need to rebuild those but they can be downloaded. There was a problem hiding this comment. The |
||
| run: | | ||
| conan upload "*" -r cura --all -c | ||
| conan upload "*" -r cura-ce -c | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| --- | ||
| name: conan-package | ||
|
|
||
| # Exports the recipe, sources and binaries for Mac, Windows and Linux and upload these to the server such that these can | ||
| # be used downstream. | ||
| # | ||
| # It should run on pushes against main or CURA-* branches, but it will only create the binaries for main and release branches | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| create_binaries_windows: | ||
| required: true | ||
| default: false | ||
| description: 'create binaries Windows' | ||
| create_binaries_linux: | ||
| required: true | ||
| default: false | ||
| description: 'create binaries Linux' | ||
| create_binaries_macos: | ||
| required: true | ||
| default: false | ||
| description: 'create binaries Macos' | ||
|
|
||
| push: | ||
| paths: | ||
| - 'plugins/**' | ||
| - 'resources/**' | ||
| - 'cura/**' | ||
| - 'icons/**' | ||
| - 'tests/**' | ||
| - 'packaging/**' | ||
| - '.github/workflows/conan-*.yml' | ||
| - '.github/workflows/notify.yml' | ||
| - '.github/workflows/requirements-conan-package.txt' | ||
| - 'requirements*.txt' | ||
| - 'conanfile.py' | ||
| - 'conandata.yml' | ||
| - 'GitVersion.yml' | ||
| - '*.jinja' | ||
| branches: | ||
| - main | ||
| - 'CURA-*' | ||
| - '[1-9]+.[0-9]+' | ||
casperlamboo marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| tags: | ||
| - '[0-9]+.[0-9]+.[0-9]+' | ||
| - '[0-9]+.[0-9]+-beta' | ||
|
|
||
| jobs: | ||
| conan-recipe-version: | ||
| uses: ultimaker/cura/.github/workflows/conan-recipe-version.yml@CURA-9365 | ||
| with: | ||
| project_name: cura | ||
|
|
||
| conan-package-export-macos: | ||
| needs: [ conan-recipe-version ] | ||
| uses: ultimaker/cura/.github/workflows/[email protected] | ||
| with: | ||
| recipe_id_full: ${{ needs.conan-recipe-version.outputs.recipe_id_full }} | ||
| recipe_id_latest: ${{ needs.conan-recipe-version.outputs.recipe_id_latest }} | ||
| recipe_id_pr: ${{ needs.conan-recipe-version.outputs.recipe_id_pr }} | ||
| runs_on: 'macos-10.15' | ||
| python_version: '3.10.4' | ||
| conan_config_branch: 'master' | ||
| conan_logging_level: 'info' | ||
| conan_export_binaries: true | ||
| secrets: inherit | ||
|
|
||
| conan-package-export-linux: | ||
| needs: [ conan-recipe-version ] | ||
| uses: ultimaker/cura/.github/workflows/[email protected] | ||
| with: | ||
| recipe_id_full: ${{ needs.conan-recipe-version.outputs.recipe_id_full }} | ||
| recipe_id_latest: ${{ needs.conan-recipe-version.outputs.recipe_id_latest }} | ||
| recipe_id_pr: ${{ needs.conan-recipe-version.outputs.recipe_id_pr }} | ||
| runs_on: 'ubuntu-20.04' | ||
| python_version: '3.10.4' | ||
| conan_config_branch: 'master' | ||
| conan_logging_level: 'info' | ||
| conan_export_binaries: true | ||
| secrets: inherit | ||
|
|
||
| conan-package-export-windows: | ||
| needs: [ conan-recipe-version ] | ||
| uses: ultimaker/cura/.github/workflows/[email protected] | ||
| with: | ||
| recipe_id_full: ${{ needs.conan-recipe-version.outputs.recipe_id_full }} | ||
| recipe_id_latest: ${{ needs.conan-recipe-version.outputs.recipe_id_latest }} | ||
| recipe_id_pr: ${{ needs.conan-recipe-version.outputs.recipe_id_pr }} | ||
| runs_on: 'windows-2022' | ||
| python_version: '3.10.4' | ||
| conan_config_branch: 'master' | ||
| conan_logging_level: 'info' | ||
| conan_export_binaries: true | ||
| secrets: inherit | ||
Joeydelarago marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| notify-export: | ||
| if: ${{ always() }} | ||
jellespijker marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| needs: [ conan-package-export-linux, conan-package-export-macos, conan-package-export-windows ] | ||
|
|
||
| uses: ultimaker/cura/.github/workflows/[email protected] | ||
| with: | ||
| success: ${{ contains(join(needs.*.result, ','), 'success') }} | ||
| success_title: "New Conan recipe exported in ${{ github.repository }}" | ||
| success_body: "Exported ${{ needs.conan-recipe-version.outputs.recipe_id_full }}" | ||
| failure_title: "Failed to export Conan Export in ${{ github.repository }}" | ||
| failure_body: "Failed to exported ${{ needs.conan-recipe-version.outputs.recipe_id_full }}" | ||
| secrets: inherit | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| name: Export Conan Recipe to server | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| recipe_id_full: | ||
| required: true | ||
| type: string | ||
|
|
||
| recipe_id_latest: | ||
| required: false | ||
| type: string | ||
|
|
||
| recipe_id_pr: | ||
| required: false | ||
| type: string | ||
|
|
||
| runs_on: | ||
| required: true | ||
| type: string | ||
|
|
||
| python_version: | ||
| required: true | ||
| type: string | ||
|
|
||
| conan_config_branch: | ||
| required: false | ||
| type: string | ||
|
|
||
| conan_logging_level: | ||
| required: false | ||
| type: string | ||
|
|
||
| conan_export_binaries: | ||
| required: false | ||
| type: boolean | ||
|
|
||
| env: | ||
| CONAN_LOGIN_USERNAME_CURA: ${{ secrets.CONAN_USER }} | ||
| CONAN_PASSWORD_CURA: ${{ secrets.CONAN_PASS }} | ||
| CONAN_LOGIN_USERNAME_CURA_CE: ${{ secrets.CONAN_USER }} | ||
| CONAN_PASSWORD_CURA_CE: ${{ secrets.CONAN_PASS }} | ||
| CONAN_LOG_RUN_TO_OUTPUT: 1 | ||
| CONAN_LOGGING_LEVEL: ${{ inputs.conan_logging_level }} | ||
| CONAN_NON_INTERACTIVE: 1 | ||
|
|
||
| jobs: | ||
| package-export: | ||
| runs-on: ${{ inputs.runs_on }} | ||
|
|
||
| steps: | ||
| - name: Checkout project | ||
| uses: actions/checkout@v3 | ||
|
|
||
| - name: Setup Python and pip | ||
| uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: ${{ inputs.python_version }} | ||
| cache: 'pip' | ||
| cache-dependency-path: .github/workflows/requirements-conan-package.txt | ||
|
|
||
| - name: Install Python requirements and Create default Conan profile | ||
| run: | | ||
| pip install -r .github/workflows/requirements-conan-package.txt | ||
| conan profile new default --detect | ||
|
|
||
| - name: Cache Conan local repository packages | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: $HOME/.conan/data | ||
| key: ${{ runner.os }}-conan | ||
|
|
||
| - name: Get Conan configuration from branch | ||
| if: ${{ inputs.conan_config_branch != '' }} | ||
| run: conan config install https://github.com/Ultimaker/conan-config.git -a "-b ${{ inputs.conan_config_branch }}" | ||
|
|
||
| - name: Get Conan configuration | ||
| if: ${{ inputs.conan_config_branch == '' }} | ||
| run: conan config install https://github.com/Ultimaker/conan-config.git | ||
|
|
||
| - name: Export the Package (binaries) | ||
| if: ${{ inputs.conan_export_binaries == 'true' }} | ||
| run: conan export-pkg . ${{ inputs.recipe_id_full }} | ||
|
|
||
| - name: Export the Package | ||
| if: ${{ inputs.conan_export_binaries != 'true' && github.event_name != 'pull_request' }} | ||
| run: conan export . ${{ inputs.recipe_id_full }} | ||
|
|
||
| - name: Create the latest alias | ||
| if: ${{ inputs.recipe_id_latest != '' && github.event_name != 'pull_request' }} | ||
| run: conan alias ${{ inputs.recipe_id_latest }} ${{ inputs.recipe_id_full }} | ||
|
|
||
| - name: Create the pull request alias | ||
| if: ${{ inputs.recipe_id_pr != '' && github.event_name == 'pull_request' }} | ||
| run: conan alias ${{ inputs.recipe_id_latest }} ${{ inputs.recipe_id_full }} | ||
|
|
||
| - name: Upload the Package(s) | ||
| run: | | ||
| # Only use --all (upload binaries) for the cura repository | ||
| conan upload "*" -r cura --all -c | ||
| conan upload "*" -r cura-ce -c |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not use
%HOMEPATH%here?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because I ran to issues with it early on.
Try changing it, but
If anything we should use the PowerShell env naming scheme:
$Env:<name>, since Windows uses the PowerShell by default