Introduce Conan, JFrog and GitHub Actions to build and manage Cura (and her dependencies #12708
Conversation
Still WIP, but needs to be committed due to not copying sources (1gb saving) Contributes to CURA-9365
Something similar should be done for Uranium Contributes to CURA-9365
Contributes to CURA-9365
The following options shouldn't be used to determine the hash, since these are only used to set the CuraVersion.py which will als be generated by the deploy method during the `conan install cura/5.1.0@_/_` Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
[CURA-8849] more intent fixes
Intents in packages
It's the same hack that we already use for the other instances where we use the tableModel. For some reason it wasn't done here, so we also had the not updating bug CURA-9270
PVA is stringing a lot due to the too small retract distance. This oozed material is causing under extrusion elsewhere, reducing the strength of the support. Too long retractions could cause reliability issues as well. However, since we reduced the number of retractions by ~40% for the smaller models (for bigger models it is ~0%, but then it is also not relevant since a lot of material is printed in between) with the new settings of PP-108 increasing the retract distance to 6.5mm is safe. PP-185
A bigger horizontal expansion is improving the stability of the support structure. Setting the Support Horizontal Expansion > Support X/Y distance causes the support folding around the object, making it way harder to remove and introducing a lot of additional stringing. PP-185
Setting this value smaller could cause scarring when the nozzle offset calibration is not performed. A higher value causes a too big gap between the model and support, resulting in support not being attached to the model on sloped surfaces. PP-185
Contributes to CURA-9365
Some paths in dependencies were moved around due to the new deploy functionality this should fix it. Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
Also added share/<dep>/resources releative to sys.executable Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
We use word-capitalisation in these setting labels. And we should put a period after sentences in the description. Contributes to issue CURA-9432.
We already had to customize to much, it isn't worth the effort to continue with GitVersion The script still has some issues when a tag is pushed Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
This can result in a strange versioning compared to the previous 5.1.0 beta, but it will only be cosmetic Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
Contributes to CURA-9365
|
I had to rewrite the reusable workflow for the semver. We needed to customize the GitVersion Action to much, so I decided to write a simple Python script. This means however that the build metadata is slightly different then we currently use. Since this doesn't affect semver, it will purely be an aesthetic thing. The next Cura-5.1.0-beta+1234 will have a lower version for the build metadata then we had yesterday. |
|
|
||
| jobs: | ||
| conan-recipe-version: | ||
| uses: ultimaker/cura/.github/workflows/conan-recipe-version.yml@CURA-9365 |
There was a problem hiding this comment.
Should we change this to reference latest after/just before merge?
There was a problem hiding this comment.
Also merge cura first,
Then you can change it before in all the other repo's
If you don't immediately delete this branch then it will still be used after the merge.
As a clean up action we should still fix afterwards.
There was a problem hiding this comment.
Changes look good from what I can see, definitely an improvement compared to the old build system and we can already benefit from the github runners and it works great!
Posted some remarks through out the code. We did have a somewhat bigger remark regarding the regex of the versioning numbering. The pattern we want to match on is usually something like x.x (where x is any digit). The regex for doing so is commonly expressed in this PR as [0-9]+.[0-9]+. The dot in regex does not mean the character dot but is a special token that matches on anything. If we want to use the character . the dot needs to be escaped.
|
|
||
| - name: Use Conan download cache (Powershell) | ||
| if: ${{ runner.os == 'Windows' }} | ||
| run: conan config set storage.download_cache="C:\Users\runneradmin\.conan\conan_download_cache" |
There was a problem hiding this comment.
| run: conan config set storage.download_cache="C:\Users\runneradmin\.conan\conan_download_cache" | |
| run: conan config set storage.download_cache="%HOMEPATH%\.conan\conan_download_cache" |
why not use %HOMEPATH% here?
There was a problem hiding this comment.
Because I ran to issues with it early on.
Try changing it, but
If anything we should use the PowerShell env naming scheme: $Env:<name>, since Windows uses the PowerShell by default
|
|
||
| - name: Install Linux system requirements | ||
| if: ${{ runner.os == 'Linux' }} | ||
| run: sudo apt install build-essential checkinstall libegl-dev zlib1g-dev libssl-dev ninja-build autoconf libx11-dev libx11-xcb-dev libfontenc-dev libice-dev libsm-dev libxau-dev libxaw7-dev libxcomposite-dev libxcursor-dev libxdamage-dev libxdmcp-dev libxext-dev libxfixes-dev libxi-dev libxinerama-dev libxkbfile-dev libxmu-dev libxmuu-dev libxpm-dev libxrandr-dev libxrender-dev libxres-dev libxss-dev libxt-dev libxtst-dev libxv-dev libxvmc-dev libxxf86vm-dev xtrans-dev libxcb-render0-dev libxcb-render-util0-dev libxcb-xkb-dev libxcb-icccm4-dev libxcb-image0-dev libxcb-keysyms1-dev libxcb-randr0-dev libxcb-shape0-dev libxcb-sync-dev libxcb-xfixes0-dev libxcb-xinerama0-dev xkb-data libxcb-dri3-dev uuid-dev libxcb-util-dev libxkbcommon-x11-dev -y |
There was a problem hiding this comment.
Maybe these dependencies can be stored similar to the requirements.txt we have for our python dependencies. In my opinion such a file is much more maintainable.
There was a problem hiding this comment.
Not an Ubuntu-man myself if we can store these in a system deps file, then I'm all for it. But I myself don't have a ready made solution for this in mind.
Maybe you guys can look into that
| 'CFBundleShortVersionString': {{ short_version }}, | ||
| 'CFBundleDocumentTypes': [{ | ||
| 'CFBundleTypeRole': 'Viewer', | ||
| 'CFBundleTypeExtensions': ['*'], |
There was a problem hiding this comment.
This is the list of extensions supported with cura. I think we should be a little more specific with the extensions. From MacOS right click menu I'm able to open for instance scad files with cura, only to be prompted with an error message saying the file format is not supported.
There was a problem hiding this comment.
This is the same list as we previously used Cura 4.13.1 and lower. It just got left behind when we moved away from the cura-build to cura-build-environment I reintroduced it since there were also some GH issues with the Version set to 0.0.0
I agree with you that it is a very blunt approach, but I thought it best to use the proven settings for now.
I'm open to suggestions btw
| for i, v in enumerate(sys.argv): | ||
| print(f"{i} = {v}") |
There was a problem hiding this comment.
are these left-over debug statements? or are these used to repeat/duplicate the path variables?
There was a problem hiding this comment.
Imo that whole script needs a rehaul, since it takes way to many arguments and it should be setup in a similar fashion as the AppImage and DMG scripts.
This was currently the least amount of effort copy-paste from cura-build-environment
| nsis_content = template.render( | ||
| app_name = sys.argv[3], | ||
| main_app = sys.argv[4], | ||
| version_major = sys.argv[5], | ||
| version_minor = sys.argv[6], | ||
| version_patch = sys.argv[7], | ||
| version_build = sys.argv[8], | ||
| company = sys.argv[9], | ||
| web_site = sys.argv[10], | ||
| year = datetime.now().year, | ||
| cura_license_file = Path(sys.argv[11]), | ||
| compression_method = sys.argv[12], # ZLIB, BZIP2 or LZMA | ||
| cura_banner_img = Path(sys.argv[13]), | ||
| cura_icon = Path(sys.argv[14]), | ||
| mapped_out_paths = mapped_out_paths, | ||
| rmdir_paths = rmdir_paths, | ||
| destination = Path(sys.argv[15]) | ||
| ) |
There was a problem hiding this comment.
can't we use python deconstructor statements here?
| nsis_content = template.render( | |
| app_name = sys.argv[3], | |
| main_app = sys.argv[4], | |
| version_major = sys.argv[5], | |
| version_minor = sys.argv[6], | |
| version_patch = sys.argv[7], | |
| version_build = sys.argv[8], | |
| company = sys.argv[9], | |
| web_site = sys.argv[10], | |
| year = datetime.now().year, | |
| cura_license_file = Path(sys.argv[11]), | |
| compression_method = sys.argv[12], # ZLIB, BZIP2 or LZMA | |
| cura_banner_img = Path(sys.argv[13]), | |
| cura_icon = Path(sys.argv[14]), | |
| mapped_out_paths = mapped_out_paths, | |
| rmdir_paths = rmdir_paths, | |
| destination = Path(sys.argv[15]) | |
| ) | |
| _program, dist_loc, jinja_template_path, app_name, main_app, version_major, version_minor, version_patch, version_build, company, company, web_site, cura_license_file, compression_method, cura_banner_img, cura_icon, destination, icon_path, *args = sys.argv | |
| nsis_content = template.render( | |
| app_name = app_name, | |
| main_app = main_app, | |
| version_major = version_major, | |
| version_minor = version_minor, | |
| version_patch = version_patch, | |
| version_build = version_build, | |
| company = company, | |
| web_site = web_site, | |
| year = datetime.now().year, | |
| cura_license_file = Path(cura_license_file), | |
| compression_method = compression_method, # ZLIB, BZIP2 or LZMA | |
| cura_banner_img = Path(cura_banner_img), | |
| cura_icon = Path(cura_icon), | |
| mapped_out_paths = mapped_out_paths, | |
| rmdir_paths = rmdir_paths, | |
| cura_icon = Path(cura_icon) | |
| ) |
There was a problem hiding this comment.
See previous comment
Co-authored-by: Casper Lamboo <[email protected]>
Co-authored-by: Casper Lamboo <[email protected]>
Co-authored-by: Casper Lamboo <[email protected]>
Co-authored-by: Casper Lamboo <[email protected]>
Co-authored-by: Casper Lamboo <[email protected]>
Overview
pushevent occurs a Conan package is either created or exported for the following repositories:conan_install_info.jsonis added to the installer such that we can trace exactly which dependencies are usedalphaversion.curaonly accessible with Ultimaker account. Hosts Conan recipes and binariescura-cecan be used without an account. Hosts only Conan recipes (to limit bandwidth usage) Dependencies will be built once on the local community dev his/her computer and stored in the local Conan cacheImportant
GitVersion.ymlfiles to bump up the version to 5.2conandata.ymlfiles to add the correct blocksAnd then we see what breaks. Don't hesitate to contact me.
Closes issues
@nallath I think it would be a good idea to create a label specific to build/development.
x86_64instead ofarmv8forarchandarch_hostin yourdefaultprofileRelated issues and pull requests