[BUG] Can't package map because of non-exising docker build cache

[jordimassaguerpla]

Describe the bug

I am trying to package an example map inside a github action. However, I am getting this error:

#31 exporting cache to client directory
#31 preparing build cache for export
#31 writing layer sha256:01749e67f491daf18c325986a55fd8c93925995e90a214bd6631f11424ee0a64
#31 writing layer sha256:01749e67f491daf18c325986a55fd8c93925995e90a214bd6631f11424ee0a64 0.2s done
#31 writing layer sha256:0709800848b4584780b40e7e81200689870e890c38b54e96b65cd0a3b1942f2d
#31 writing layer sha256:0709800848b4584780b40e7e81200689870e890c38b54e96b65cd0a3b1942f2d 0.5s done
#31 writing layer sha256:0ce020987cfa5cd1654085af3bb40779634eb3d792c4a4d6059036463ae0040d 0.0s done
#31 writing layer sha256:0f65089b284381bf795d15b1a186e2a8739ea957106fa526edef0d738e7cda70
#31 writing layer sha256:0f65089b284381bf795d15b1a186e2a8739ea957106fa526edef0d738e7cda70 0.0s done
#31 writing layer sha256:12a47450a9f9cc5d4edab65d0f600dbbe8b23a1663b0b3bb2c481d40e074b580 0.0s done
#31 writing layer sha256:1a1e81012eba314b7a7a4e831cd5118e372d4b968ecad66cc1b63c4db6e3202d 0.0s done
#31 writing layer sha256:25603df9ade58979074493d0b28da44262748ef9cea551b4acd0e1c98b6210fa 0.0s done
#31 writing layer sha256:289ecdd915ad254bab577a7aafe095de2f2e7398eead330cde01c051d3c8b36a
#31 writing layer sha256:289ecdd915ad254bab577a7aafe095de2f2e7398eead330cde01c051d3c8b36a 0.0s done
#31 writing layer sha256:2d42104dbf0a7cc962b791f6ab4f45a803f8a36d296f996aca180cfb2f3e30d0
#31 writing layer sha256:2d42104dbf0a7cc962b791f6ab4f45a803f8a36d296f996aca180cfb2f3e30d0 0.4s done
#31 writing layer sha256:2fa1ce4fa3fec6f9723380dc0536b7c361d874add0baaddc4bbf2accac82d2ff
#31 writing layer sha256:2fa1ce4fa3fec6f9723380dc0536b7c361d874add0baaddc4bbf2accac82d2ff 21.5s done
#31 writing layer sha256:369aa34c082692623cf8ae6a08899bdcde789074709fcaee69bd3e6620172d1e 0.0s done
#31 writing layer sha256:38794be1b5dc99645feabf89b22cd34fb5bdffb5164ad920e7df94f353efe9c0 0.0s done
#31 writing layer sha256:38f963dc57c1e7b68a738fe39ed9f9345df7188111a047e2163a46648d7f1d88
#31 writing layer sha256:38f963dc57c1e7b68a738fe39ed9f9345df7188111a047e2163a46648d7f1d88 0.3s done
#31 writing layer sha256:3e7e4c9bc2b136814c20c04feb4eea2b2ecf972e20182d88759931130cfb4181
#31 writing layer sha256:3e7e4c9bc2b136814c20c04feb4eea2b2ecf972e20182d88759931130cfb4181 0.0s done
#31 writing layer sha256:3fd77037ad585442cd82d64e337f49a38ddba50432b2a1e563a48401d25c79e6 0.0s done
#31 writing layer sha256:45893188359aca643d5918c9932da995364dc62013dfa40c075298b1baabece3
#31 writing layer sha256:45893188359aca643d5918c9932da995364dc62013dfa40c075298b1baabece3 2.4s done
#31 writing layer sha256:49bc651b19d9e46715c15c41b7c0daa007e8e25f7d9518f04f0f06592799875a 0.0s done
#31 writing layer sha256:4c12db5118d8a7d909e4926d69a2192d2b3cd8b110d49c7504a4f701258c1ccc 0.0s done
#31 writing layer sha256:4cc43a803109d6e9d1fd35495cef9b1257035f5341a2db54f7a1940815b6cc65
#31 writing layer sha256:4cc43a803109d6e9d1fd35495cef9b1257035f5341a2db54f7a1940815b6cc65 51.3s done
#31 writing layer sha256:4d32b49e2995210e8937f0898327f196d3fcc52486f0be920e8b2d65f150a7ab
#31 writing layer sha256:4d32b49e2995210e8937f0898327f196d3fcc52486f0be920e8b2d65f150a7ab 0.9s done
#31 writing layer sha256:4d6fe980bad9cd7b2c85a478c8033cae3d098a81f7934322fb64658b0c8f9854 0.0s done
#31 writing layer sha256:4e59aa461d4cc09006c603c01b4a0bacbd3cb04cbf17205f67ea9bed133d4ce7
#31 writing layer sha256:4e59aa461d4cc09006c603c01b4a0bacbd3cb04cbf17205f67ea9bed133d4ce7 2.0s done
#31 writing layer sha256:4ea6adb5442daa819b67da3532b61fe5bb255652a0fe57829e7ce4eab09a2626
#31 writing layer sha256:4ea6adb5442daa819b67da3532b61fe5bb255652a0fe57829e7ce4eab09a2626 0.1s done
#31 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 done
#31 writing layer sha256:51dea13ec469daef160238c70f28e30c87512b6a365a3b803cb9755ac7692e0e 0.0s done
#31 writing layer sha256:586405150a1f976400275459675d0011c365cddaef463cbba5640d459b4396ed
#31 writing layer sha256:586405150a1f976400275459675d0011c365cddaef463cbba5640d459b4396ed 0.1s done
#31 writing layer sha256:595c38fa102c61c3dda19bdab70dcd26a0e50465b986d022a84fa69023a05d0f 0.1s done
#31 writing layer sha256:59d451175f6950740e26d38c322da0ef67cb59da63181eb32996f752ba8a2f17
#31 writing layer sha256:59d451175f6950740e26d38c322da0ef67cb59da63181eb32996f752ba8a2f17 1.4s done
#31 writing layer sha256:5a69f2143021a189f74efa7159fba34604cb2b1c773cb60d2b894f3cdfbf81e1 0.0s done
#31 writing layer sha256:5ad1f2004580e415b998124ea394e9d4072a35d70968118c779f307204d6bd17
#31 writing layer sha256:5ad1f2004580e415b998124ea394e9d4072a35d70968118c779f307204d6bd17 3.4s done
#31 writing layer sha256:62598eafddf023e7f22643485f4321cbd51ff7eee743b970db12454fd3c8c675
#31 writing layer sha256:62598eafddf023e7f22643485f4321cbd51ff7eee743b970db12454fd3c8c675 2.2s done
#31 writing layer sha256:63d7e616a46987136f4cc9eba95db6f6327b4854cfe3c7e20fed6db0c966e380 0.0s done
#31 writing layer sha256:6939d591a6b09b14a437e5cd2d6082a52b6d76bec4f72d960440f097721da34f
#31 writing layer sha256:6939d591a6b09b14a437e5cd2d6082a52b6d76bec4f72d960440f097721da34f 0.1s done
#31 writing layer sha256:698318e5a60e5e0d48c45bf992f205a9532da567fdfe94bd59be2e192975dd6f 0.0s done
#31 writing layer sha256:6ddc1d0f91833b36aac1c6f0c8cea005c87d94bab132d46cc06d9b060a81cca3
#31 writing layer sha256:6ddc1d0f91833b36aac1c6f0c8cea005c87d94bab132d46cc06d9b060a81cca3 0.0s done
#31 writing layer sha256:74ac1f5a47c0926bff1e997bb99985a09926f43bd0895cb27ceb5fa9e95f8720 0.0s done
#31 writing layer sha256:7577973918dd30e764733a352a93f418000bc3181163ca451b2307492c1a6ba9
#31 writing layer sha256:7577973918dd30e764733a352a93f418000bc3181163ca451b2307492c1a6ba9 2.7s done
#31 writing layer sha256:8a7451db9b4b817b3b33904abddb7041810a4ffe8ed4a034307d45d9ae9b3f2a
#31 writing layer sha256:8a7451db9b4b817b3b33904abddb7041810a4ffe8ed4a034307d45d9ae9b3f2a 0.2s done
#31 writing layer sha256:8dde51aa1b226eda81a5290c94d85d1e7d4e3ca3dbb45b9a72a557bffb738541
#31 writing layer sha256:8dde51aa1b226eda81a5290c94d85d1e7d4e3ca3dbb45b9a72a557bffb738541 0.0s done
#31 writing layer sha256:916f4054c6e7f10de4fd7c08ffc75fa23ebecca4eceb8183cb1023b33b1696c9 0.0s done
#31 writing layer sha256:9463aa3f56275af97693df69478a2dc1d171f4e763ca6f7b6f370a35e605c154 0.0s done
#31 writing layer sha256:955fd173ed884230c2eded4542d10a97384b408537be6bbb7c4ae09ccd6fb2d0
#31 writing layer sha256:955fd173ed884230c2eded4542d10a97384b408537be6bbb7c4ae09ccd6fb2d0 0.3s done
#31 writing layer sha256:9c42a4ee99755f441251e6043b2cbba16e49818a88775e7501ec17e379ce3cfd 0.0s done
#31 writing layer sha256:9c63be0a86e3dc4168db3814bf464e40996afda0031649d9faa8ff7568c3154f 0.0s done
#31 writing layer sha256:9e04bda98b05554953459b5edef7b2b14d32f1a00b979a23d04b6eb5c191e66b
#31 writing layer sha256:9e04bda98b05554953459b5edef7b2b14d32f1a00b979a23d04b6eb5c191e66b 8.4s done
#31 writing layer sha256:a4a0c690bc7da07e592514dccaa26098a387e8457f69095e922b6d73f7852502 0.0s done
#31 writing layer sha256:a4aafbc094d78a85bef41036173eb816a53bcd3e2564594a32f542facdf2aba6
#31 writing layer sha256:a4aafbc094d78a85bef41036173eb816a53bcd3e2564594a32f542facdf2aba6 0.1s done
#31 writing layer sha256:ae36a4d38b76948e39a5957025c984a674d2de18ce162a8caaa536e6f06fccea 0.0s done
#31 writing layer sha256:b1296608d03bf15aad726d41dfdc73215f8bab4332e5da25f92a3034f9db10f6 0.0s done
#31 writing layer sha256:b2fa40114a4a0725c81b327df89c0c3ed5c05ca9aa7f1157394d5096cf5460ce 0.0s done
#31 writing layer sha256:b48a5fafcaba74eb5d7e7665601509e2889285b50a04b5b639a23f8adc818157 0.0s done
#31 writing layer sha256:b7c37675757fa3db4b7d54e03b3dbd74a435405ca7d7c4ee4584d79abecbd7ae
#31 writing layer sha256:b7c37675757fa3db4b7d54e03b3dbd74a435405ca7d7c4ee4584d79abecbd7ae 1.5s done
#31 writing layer sha256:c55b79b2f79815a627a54dc28b93e302e63728bc338477c5341cb93055a51064
#31 writing layer sha256:c55b79b2f79815a627a54dc28b93e302e63728bc338477c5341cb93055a51064 0.3s done
#31 writing layer sha256:c86976a083599e36a6441f36f553627194d05ea82bb82a78682e718fe62fccf6
#31 writing layer sha256:c86976a083599e36a6441f36f553627194d05ea82bb82a78682e718fe62fccf6 1.4s done
#31 writing layer sha256:cb506fbdedc817e3d074f609e2edbf9655aacd7784610a1bbac52f2d7be25438
#31 writing layer sha256:cb506fbdedc817e3d074f609e2edbf9655aacd7784610a1bbac52f2d7be25438 0.3s done
#31 writing layer sha256:d8d16d6af76dc7c6b539422a25fdad5efb8ada5a8188069fcd9d113e3b783304 0.0s done
#31 writing layer sha256:dc50837d89c783c6152c0b456e55de4efc5197ab8962b5bc7001b06334741532 0.0s done
#31 writing layer sha256:ddc2ade4f6fe866696cb638c8a102cb644fa842c2ca578392802b3e0e5e3bcb7
#31 writing layer sha256:ddc2ade4f6fe866696cb638c8a102cb644fa842c2ca578392802b3e0e5e3bcb7 5.2s done
#31 writing layer sha256:e2cfd7f6244d6f35befa6bda1caa65f1786cecf3f00ef99d7c9a90715ce6a03c
#31 writing layer sha256:e2cfd7f6244d6f35befa6bda1caa65f1786cecf3f00ef99d7c9a90715ce6a03c 14.0s done
#31 writing layer sha256:e75fc65e79abafc5219c10f5d47949f2a57d2673757c184a3f3450d0f63df228
#31 writing layer sha256:e75fc65e79abafc5219c10f5d47949f2a57d2673757c184a3f3450d0f63df228 2.6s done
#31 writing layer sha256:e94a4481e9334ff402bf90628594f64a426672debbdfb55f1290802e52013907 0.0s done
#31 writing layer sha256:eaf45e9f32d1f5a9983945a1a9f8dedbb475bc0f578337610e00b4dedec87c20
#31 writing layer sha256:eaf45e9f32d1f5a9983945a1a9f8dedbb475bc0f578337610e00b4dedec87c20 3.3s done
#31 writing layer sha256:eb411bef39c013c9853651e68f00965dbd826d829c4e478884a2886976e9c989 0.0s done
#31 writing layer sha256:edfe4a95eb6bd3142aeda941ab871ffcc8c19cf50c33561c210ba8ead2424759
#31 writing layer sha256:edfe4a95eb6bd3142aeda941ab871ffcc8c19cf50c33561c210ba8ead2424759 1.5s done
#31 writing layer sha256:ef4466d6f927d29d404df9c5af3ef5733c86fa14e008762c90110b963978b1e7 0.0s done
#31 writing layer sha256:f346e3ecdf0bee048fa1e3baf1d3128ff0283b903f03e97524944949bd8882e5
#31 writing layer sha256:f346e3ecdf0bee048fa1e3baf1d3128ff0283b903f03e97524944949bd8882e5 0.2s done
#31 writing layer sha256:f3f9a00a1ce9aadda250aacb3e66a932676badc5d8519c41517fdf7ea14c13ed
#31 preparing build cache for export 141.9s done
#31 writing layer sha256:f3f9a00a1ce9aadda250aacb3e66a932676badc5d8519c41517fdf7ea14c13ed 11.6s done
#31 writing layer sha256:ff1f8b9a0c990345bbe729337990068f4adb7afc45ee2988b5633b4ad05249ac 0.0s done
#31 writing layer sha256:ff9b9c0b1efe5f0b8a3e2110d3215ccf1d77ea119c8c2477afb17b71cacdad6b 0.0s done
#31 writing config sha256:041e0789f6b9aefe9606d6ba014955a2e08333eb81b7beb24613e7947a3f03c9 0.0s done
#31 writing cache manifest sha256:5f1942909871242855c0f73d8efc9f76589b521747525d9d514799d3e48ed40a 0.0s done
#31 DONE 141.9s
WARNING: local cache import at /home/adminuser/.holoscan_build_cache not found due to err: could not read /home/adminuser/.holoscan_build_cache/index.json: open /home/adminuser/.holoscan_build_cache/index.json: no such file or directory
[2023-11-13 11:27:08,371] [INFO] (packager) - Build Summary:

Platform: x64-workstation/dgpu
    Status: Failure
    Error:  Error building image: see Docker output for additional details.

See workflow file and terraform file

Looks very similar at this bug docker/buildx#206 . However, according to that bug, this is supposed to be fixed in buildx already.

Could you help me?

Steps/Code to reproduce bug

I am having this issue when running the package subcommand inside a vm in Azure, which is deployed using terraform inside of a github action.

The vm is Ubuntu-22_04-lts, thus in principle you should be able to reproduce it on a fresh install of Ubuntu-22_04-lts. The issue seems to be that, when you run the package subcommand, it uses a --from-cache and --to-cache options when there is not yet a cache. If you run it twice, the second time it will find a cache, so it is super important to run it on a fresh install.

You can see the exact commands in the github action jordimassaguerpla@279dcbd

I have tried using the "--no-cache" flag of the package subcommand, but I get the same behaviour.

Expected behavior
The package subcommand should not try to use cache if you specify the "--no-cache" flag.

Environment details (please complete the following information)

• OS/Platform: Ubuntu-22_04_lts
• Python Version: 2.10
• Method of MONAI Deploy App SDK install: pip
• SDK Version: 0.6.0

[MMelQin]

@jordimassaguerpla Thanks for reporting this issue as well as the investigation. The PR will be handled in the Holoscan SDK, though unfortunately nvidia-holoscan is read only, pushed from Nvidia internal gitlab repo. The fix will then have to wait till the next public release of Holoscan SDK, ETA Jan, 2024.

[mocsharp]

@jordimassaguerpla, another option to try is to specify the cache location, setting it to a writable directory using --build-cache option.

[nitin271080]

when using monai-depoy package command
| # Copy user-specified MONAI Deploy SDK file
78 | >>> COPY ./None /tmp/None
79 | RUN pip install /tmp/None
this error generate

use | # Copy user-specified MONAI Deploy SDK file
78 | >>> COPY ./None /tmp/None
79 | RUN pip install /tmp/None

use sub option ----monai-deploy-sdk-file monai_deploy_app_sdk-0.6.0-py3-none-any.whl with monai-deploy package command

like

monai-deploy package simple_imaging_app -c simple_imaging_app/app.yaml -t {tag_prefix}:1.0 --platform x64-workstation --monai-deploy-sdk-file monai_deploy_app_sdk-0.6.0-py3-none-any.whl -l DEBUG

i will work