-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: Add rust compiler to charms #1178
Comments
Thank you for reporting us your feedback! The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6645.
|
KFPKFP tests fail due to space in the runner as seen here https://github.com/canonical/kfp-operators/actions/runs/12162787081/job/33958507596?pr=631#step:5:468 (same as canonical/kfp-operators#602)
This means that adding rust compiler to the charms needs first this feature to land canonical/kfp-operators#627. |
* build: Enable building rust dependencies Ref canonical/bundle-kubeflow#1178 * ci: remove destructive-mode
Note that as done in mysql-k8s (see canonical/charmcraft#1722 (comment)), the build packages ...
:: Running external command ['python3', '-m', 'venv', '/root/parts/charm/build/staging-venv']
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--force-reinstall', 'pip@https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz']
:: :: Collecting pip@ https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz
:: :: Downloading pip-24.1.1.tar.gz (1.9 MB)
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing wheel metadata: started
:: :: Preparing wheel metadata: finished with status 'done'
:: :: Building wheels for collected packages: pip
:: :: Building wheel for pip (PEP 517): started
:: :: Building wheel for pip (PEP 517): finished with status 'done'
:: :: Created wheel for pip: filename=pip-24.1.1-py3-none-any.whl size=1824282 sha256=e8294edf16c52b89a0eccd34036d30218ae3bdac7a2acf323b6dfe8d3d8e88e1
:: :: Stored in directory: /root/.cache/pip/wheels/a2/93/80/a8fa207c102102e4c7f0ea7965db2509466086aa06a927a5ce
:: :: Successfully built pip
:: :: Installing collected packages: pip
:: :: Attempting uninstall: pip
:: :: Found existing installation: pip 20.0.2
:: :: Uninstalling pip-20.0.2:
:: :: Successfully uninstalled pip-20.0.2
:: :: Successfully installed pip-24.1.1
:: Installing Python pre-dependencies from source.
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', 'wheel', 'setuptools', 'pip']
:: :: Collecting wheel
:: :: Downloading wheel-0.45.1.tar.gz (107 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 107.5/107.5 kB 19.5 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Requirement already satisfied: setuptools in ./staging-venv/lib/python3.8/site-packages (44.0.0)
:: :: Requirement already satisfied: pip in ./staging-venv/lib/python3.8/site-packages (24.1.1)
:: :: Building wheels for collected packages: wheel
:: :: Building wheel for wheel (pyproject.toml): started
:: :: Building wheel for wheel (pyproject.toml): finished with status 'done'
:: :: Created wheel for wheel: filename=wheel-0.45.1-py3-none-any.whl size=72494 sha256=708e7481cc80179af0e556bbf0cc00b8444c7321e2700b8d8580231d13017248
:: :: Stored in directory: /root/.cache/pip/wheels/1b/00/e6/216671cc7f33efcb5fc4e2b16b86f829cc57d2e0e28d9b90e8
:: :: Successfully built wheel
:: :: Installing collected packages: wheel
:: :: Successfully installed wheel-0.45.1
:: ::
Notice: :: :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1
Notice: :: :: [notice] To update, run: python3 -m pip install --upgrade pip
:: Installing packages from requirements files and charm lib dependencies.
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt']
:: :: Collecting annotated-types==0.7.0 (from -r requirements.txt (line 7))
:: :: Downloading annotated_types-0.7.0.tar.gz (16 kB)
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting anyio==4.5.2 (from -r requirements.txt (line 9))
:: :: Downloading anyio-4.5.2.tar.gz (171 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 171.3/171.3 kB 20.4 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting attrs==24.2.0 (from -r requirements.txt (line 11))
:: :: Downloading attrs-24.2.0.tar.gz (792 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 792.7/792.7 kB 96.0 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting backports-strenum==1.3.1 (from -r requirements.txt (line 13))
:: :: Downloading backports_strenum-1.3.1.tar.gz (7.3 kB)
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting bcrypt==4.2.1 (from -r requirements.txt (line 15))
:: :: Downloading bcrypt-4.2.1.tar.gz (24 kB)
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting cachetools==5.5.0 (from -r requirements.txt (line 17))
:: :: Downloading cachetools-5.5.0.tar.gz (27 kB)
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting certifi==2024.8.30 (from -r requirements.txt (line 19))
:: :: Downloading certifi-2024.8.30.tar.gz (168 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.5/168.5 kB 40.9 MB/s eta 0:00:00
:: :: Preparing metadata (setup.py): started
:: :: Preparing metadata (setup.py): finished with status 'done'
:: :: Collecting cffi==1.17.1 (from -r requirements.txt (line 25))
:: :: Downloading cffi-1.17.1.tar.gz (516 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 516.6/516.6 kB 53.3 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting charmed-kubeflow-chisme==0.4.3 (from -r requirements.txt (line 29))
:: :: Downloading charmed_kubeflow_chisme-0.4.3.tar.gz (43 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.3/43.3 kB 13.4 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting charset-normalizer==3.4.0 (from -r requirements.txt (line 31))
:: :: Downloading charset_normalizer-3.4.0.tar.gz (106 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 29.3 MB/s eta 0:00:00
:: :: Preparing metadata (setup.py): started
:: :: Preparing metadata (setup.py): finished with status 'done'
:: :: Collecting cosl==0.0.45 (from -r requirements.txt (line 33))
:: :: Downloading cosl-0.0.45.tar.gz (67 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.5/67.5 kB 16.9 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: still running...
:: :: Installing build dependencies: still running...
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting cryptography==44.0.0 (from -r requirements.txt (line 35))
:: :: Downloading cryptography-44.0.0.tar.gz (710 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 710.7/710.7 kB 91.0 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: still running...
:: :: Installing build dependencies: still running...
:: :: Installing build dependencies: finished with status 'error'
:: :: error: subprocess-exited-with-error
:: ::
:: :: × pip subprocess to install build dependencies did not run successfully.
:: :: │ exit code: 1
:: :: ╰─> [89 lines of output]
:: :: Collecting maturin<2,>=1
:: :: Downloading maturin-1.7.8.tar.gz (195 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 195.7/195.7 kB 2.5 MB/s eta 0:00:00
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting cffi>=1.12
:: :: Using cached cffi-1.17.1.tar.gz (516 kB)
:: :: Installing build dependencies: started
:: :: Installing build dependencies: finished with status 'done'
:: :: Getting requirements to build wheel: started
:: :: Getting requirements to build wheel: finished with status 'done'
:: :: Preparing metadata (pyproject.toml): started
:: :: Preparing metadata (pyproject.toml): finished with status 'done'
:: :: Collecting setuptools!=74.0.0,!=74.1.0,!=74.1.1,!=74.1.2
:: :: Using cached setuptools-75.3.0-py3-none-any.whl
:: :: Collecting tomli>=1.1.0 (from maturin<2,>=1)
:: :: Using cached tomli-2.2.1-py3-none-any.whl
:: :: Collecting pycparser (from cffi>=1.12)
:: :: Downloading pycparser-2.22.tar.gz (172 kB)
:: :: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 172.7/172.7 kB 9.1 MB/s eta 0:00:00
:: :: Preparing metadata (setup.py): started
:: :: Preparing metadata (setup.py): finished with status 'done'
:: :: Building wheels for collected packages: maturin, cffi, pycparser
:: :: Building wheel for maturin (pyproject.toml): started
:: :: Building wheel for maturin (pyproject.toml): still running...
:: :: Building wheel for maturin (pyproject.toml): still running...
:: :: Building wheel for maturin (pyproject.toml): finished with status 'done'
:: :: Created wheel for maturin: filename=maturin-1.7.8-cp38-cp38-linux_x86_64.whl size=5995658 sha256=e9cbdec65db8641ca6bfcfc3c1f7c89ee91357d80a058e10f54fdbf57c44c[549](https://github.com/canonical/argo-operators/actions/runs/12199502801/job/34033534048#step:4:550)
:: :: Stored in directory: /root/.cache/pip/wheels/f1/2a/cc/94408ea6aec8dad6addae6c4bb4fb04597715b4315e43aca2d
:: :: Building wheel for cffi (pyproject.toml): started
:: :: Building wheel for cffi (pyproject.toml): finished with status 'error'
:: :: error: subprocess-exited-with-error
:: ::
:: :: × Building wheel for cffi (pyproject.toml) did not run successfully.
:: :: │ exit code: 1
:: :: ╰─> [35 lines of output]
:: :: running bdist_wheel
:: :: running build
:: :: running build_py
:: :: creating build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/error.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/vengine_gen.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/recompiler.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/model.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/vengine_cpy.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/verifier.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/setuptools_ext.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/_shimmed_dist_utils.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/__init__.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/cffi_opcode.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/ffiplatform.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/cparser.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/backend_ctypes.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/lock.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/pkgconfig.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/api.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/_imp_emulation.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/commontypes.py -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/_cffi_include.h -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/parse_c_type.h -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/_embedding.h -> build/lib.linux-x86_64-cpython-38/cffi
:: :: copying src/cffi/_cffi_errors.h -> build/lib.linux-x86_64-cpython-38/cffi
:: :: running build_ext
:: :: building '_cffi_backend' extension
:: :: creating build/temp.linux-x86_64-cpython-38/src/c
:: :: x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/parts/charm/build/staging-venv/include -I/usr/include/python3.8 -c src/c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-38/src/c/_cffi_backend.o
:: :: src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
:: :: 15 | #include <ffi.h>
:: :: | ^~~~~~~
:: :: compilation terminated.
:: :: error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
:: :: [end of output]
:: ::
:: :: note: This error originates from a subprocess, and is likely not a problem with pip.
:: :: ERROR: Failed building wheel for cffi
:: :: Building wheel for pycparser (setup.py): started
:: :: Building wheel for pycparser (setup.py): finished with status 'done'
:: :: Created wheel for pycparser: filename=pycparser-2.22-py3-none-any.whl size=117532 sha256=0c29a4cd2b8e5899651b460a8dfa20524f1a3ead0f26e7b24cbfee87b84bad9d
:: :: Stored in directory: /root/.cache/pip/wheels/9d/ac/d9/e5fa4cb20ee9c45d0664af14d6a9da2664[558](https://github.com/canonical/argo-operators/actions/runs/12199502801/job/34033534048#step:4:559)3dc723e3ab868
:: :: Successfully built maturin pycparser
:: :: Failed to build cffi
:: :: ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (cffi)
:: ::
Notice: :: :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1
Notice: :: :: [notice] To update, run: python3 -m pip install --upgrade pip
:: :: [end of output]
:: ::
:: :: note: This error originates from a subprocess, and is likely not a problem with pip.
:: :: error: subprocess-exited-with-error
:: ::
:: :: × pip subprocess to install build dependencies did not run successfully.
:: :: │ exit code: 1
:: :: ╰─> See above for output.
:: ::
:: :: note: This error originates from a subprocess, and is likely not a problem with pip.
:: ::
Notice: :: :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1
Notice: :: :: [notice] To update, run: python3 -m pip install --upgrade pip
:: Traceback (most recent call last):
:: File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 483, in <module>
:: main()
:: File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 478, in main
:: builder.build_charm()
:: File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 93, in build_charm
:: self.handle_dependencies()
:: File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 354, in handle_dependencies
:: self._install_dependencies(staging_venv_dir)
:: File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/instrum.py", line 152, in _f
:: return func(*args, **kwargs)
:: File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 284, in _install_dependencies
:: _process_run(
:: File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 414, in _process_run
:: raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}")
:: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt'] execution failed with retcode 1
:: error:
:: Traceback (most recent call last):
:: File "/snap/charmcraft/current/lib/python3.10/site-packages/craft_parts/ctl.py", line 110, in main
:: ret = CraftCtl.run(cmd, args)
:: File "/snap/charmcraft/current/lib/python3.10/site-packages/craft_parts/ctl.py", line 44, in run
:: _client(cmd, args)
:: File "/snap/charmcraft/current/lib/python3.10/site-packages/craft_parts/ctl.py", line 97, in _client
:: raise RuntimeError(message)
:: RuntimeError: Failed to run the build script for part 'charm'.
'override-build' in part 'charm' failed with code 1.
Recommended resolution: Review the scriptlet and make sure it's correct.
Failed to execute charmcraft in instance.
Full execution log: '/home/runner/.local/state/charmcraft/log/charmcraft-20241206-132340.499833.log' |
Context
Hit during #1156 and #1177, we hit the issue where charms fail to be built due to the absence of a rust compiler during build time.
What needs to get done
Add appropriate dependencies as we previously did canonical/kubeflow-dashboard-operator@c31941b or follow what mysql-k8s team is doing seen here canonical/charmcraft#1722 (comment). The latter proved to be building faster the charm ( 12 minutes compared to 18 minutes for argo-controller)
Definition of Done
Charms can be built with dependencies requiring a rust compiler.
The text was updated successfully, but these errors were encountered: