Skip to content

Commit

Permalink
Run up-to 10 nb in parallel
Browse files Browse the repository at this point in the history
  • Loading branch information
volkerstampa committed Oct 30, 2023
1 parent 5b65bb1 commit 6507a16
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion scripts/notebook_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
[ -f .env ] && source .env
# Find all .ipynb files in the directory and pass them to xargs for parallel execution
rm --force --recursive src/examples/*.nbconvert.ipynb src/examples/.ipynb_checkpoints
find src/examples/ -name "*.ipynb" | xargs -I {} --max-procs 1 bash -c "AA_TOKEN=\"$AA_TOKEN\" poetry run jupyter nbconvert --to notebook --execute {}"
find src/examples/ -name "*.ipynb" | xargs -I {} --max-procs 10 bash -c "AA_TOKEN=\"$AA_TOKEN\" poetry run jupyter nbconvert --to notebook --execute {}"
rm --force src/examples/*.nbconvert.ipynb

0 comments on commit 6507a16

Please sign in to comment.