Adding Progress Bars #1703

atharva-2001 · 2021-07-09T06:56:24Z

This PR adds progress bars to TARDIS.

Description

Motivation and context

How has this been tested?

Other.
Testing pipeline.

Examples

Please visit the documentation here:
https://atharva-2001.github.io/tardis/branch/progress_bar/io/output/index.html
https://atharva-2001.github.io/tardis/branch/progress_bar/quickstart/quickstart.html

Type of change

Bug fix.
New feature.
Breaking change.
None of the above.

Checklist

My change requires a change to the documentation.
- I have updated the documentation accordingly.
- (optional) I have built the documentation on my fork following the instructions.
I have assigned and requested two reviewers for this pull request.

tardis-bot · 2021-07-09T07:00:22Z

Before a pull request is accepted, it must meet the following criteria:

Is the necessary information provided?
Is this a duplicate PR?
- If a new PR is clearly a duplicate, ask how this PR is different from the original PR?
- If this PR is about to be merged, close the original PR with a link to this new PR that solved the issue.
Does it pass existing tests and are new tests provided if required?
- The test coverage should not decrease, and for new features should be close to 100%.
Is the code tidy?
- No unnecessary print lines or code comments.

codecov · 2021-07-09T07:11:14Z

Codecov Report

Merging #1703 (54d82ee) into master (e21ecf7) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1703      +/-   ##
==========================================
- Coverage   62.65%   62.63%   -0.02%     
==========================================
  Files          65       65              
  Lines        6036     6089      +53     
==========================================
+ Hits         3782     3814      +32     
- Misses       2254     2275      +21

Impacted Files	Coverage Δ
tardis/tardis/util/base.py	`72.19% <0.00%> (-2.16%)`	⬇️
tardis/tardis/base.py	`57.14% <0.00%> (ø)`
tardis/tardis/montecarlo/base.py	`88.77% <0.00%> (ø)`
tardis/tardis/simulation/base.py	`83.25% <0.00%> (+0.07%)`	⬆️
tardis/tardis/montecarlo/montecarlo_numba/base.py	`26.21% <0.00%> (+0.44%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e21ecf7...54d82ee. Read the comment docs.

MarkMageeAstro

The progress bars look great, but I personally think this should not be a separate PR. To me, this is still part of the convergence plots because it's showing the progress of the model. I would like to see the layout of the progress bars and convergence plots together.

What is the impact on performance using these bars, if any?

Minor changes:

The packets bar should read "XX% of packets propagated". Perhaps even "XX% of real packets propagated".
There are hardcoded values for iterations and packet numbers, but these should come from the input yaml file. You can get the total packet number from (iterations - 1) * no_of_packets + last_no_of_packets

atharva-2001 · 2021-07-09T12:20:43Z

Thank you @MarkMageeAstro!
I tried 5 iterations in each case and the simulation took me around 30 seconds using progress bars and around 27 seconds without them, using tardis_example.yml. I will make the required changes very soon. I too believe seeing the progress bars and the convergence plots together would be better. Maybe we could discuss this with others in the meeting and then close this PR.

MarkMageeAstro · 2021-07-09T13:14:03Z

That's great, I would consider a ~10% increase acceptable.

marxwillia · 2021-07-12T14:14:21Z

tardis/montecarlo/montecarlo_numba/base.py

+from tqdm import tqdm
+
+packet_pbar = tq.tqdm(
+    total=860000,


We can't hardcode the number of packets or iterations. These numbers need to be retrieved from the tardis config object.

marxwillia · 2021-07-12T14:14:54Z

tardis/montecarlo/montecarlo_numba/base.py

+    bar_format="{bar}{percentage:3.0f}% packets propagated",
+)
+iteration_pbar = tq.tqdm(
+    total=20,


Retrieve number of iterations from the tardis config object. This is a user specified parameter.

marxwillia · 2021-07-12T14:21:10Z

tardis/montecarlo/montecarlo_numba/base.py

+        with objmode:
+            update_packet_pbar(1)


Try using if (i % 1000 == 0) as a condition. You might need to also change the argument in update_packet_pbar to 1000. Use the tqdm documentation here and click the links to the source code for the update function:

https://tqdm.github.io/docs/tqdm/

marxwillia · 2021-07-22T18:47:33Z

tardis/montecarlo/montecarlo_numba/base.py

+iterations = 20
+packets = 860000


These values need to be read in from the config dict.

jaladh-singhal

The progress bar looks nice and code quality looks ok to me (considering you haven't added docstrings to existing functions that didn't have them already). Rest about code changes - since it deals with numba I'd request @andrewfullard to have a look.

Have you checked how progress bar looks on static html page (in docs)?

wkerzendorf · 2021-08-12T13:52:24Z

this has preliminary approval from me (pending the others approval) - it looks very nice.

jaladh-singhal

Great work @atharva-2001. Everything looks good except mentioning about it in our docs is missing.

AFAIR, you were adding a "Progress bar for Simulation Run" page in "Additional outputs" section explaining options related to progress bar, different modes of display, etc. along with a note in quickstart (?) that progress bar doesn't get displayed in static page but will show up in interactive mode?

review-notebook-app · 2021-08-12T15:39:16Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

jaladh-singhal

Thanks for adding the documentation

jamesgillanders

Great addition to the code!

* Customize bar format * Display bars again after reset when full * Get data from configuration file * Track all packets at once and don't reset the progress bar * Close last container and add docstrings * Add option to enable/disble the progress bar * Display container only in notebooks * Add a progress bar to track iterations * Align bars * Refactor code * Format using black * Add message regarding progress bars in the quickstart notebook * Demonstrate progress bars in the documentation * Change show_progress_bar to show_progress_bars * [build docs]

atharva-2001 marked this pull request as draft July 9, 2021 06:56

atharva-2001 force-pushed the progress_bar branch from 151d490 to e788257 Compare July 9, 2021 06:57

atharva-2001 added feature numba visualization work in progress 🚧 labels Jul 9, 2021

atharva-2001 requested review from jaladh-singhal, jamesgillanders, MarkMageeAstro and marxwillia July 9, 2021 07:00

MarkMageeAstro requested changes Jul 9, 2021

View reviewed changes

marxwillia requested changes Jul 12, 2021

View reviewed changes

marxwillia requested changes Jul 22, 2021

View reviewed changes

atharva-2001 force-pushed the progress_bar branch 5 times, most recently from 9fe7e2f to 4704edc Compare July 29, 2021 06:21

atharva-2001 marked this pull request as ready for review July 29, 2021 07:29

atharva-2001 requested review from marxwillia and MarkMageeAstro July 29, 2021 07:29

atharva-2001 force-pushed the progress_bar branch 3 times, most recently from 7d2d6ae to 4699251 Compare July 29, 2021 08:20

tardis-sn deleted a comment from tardis-bot Jul 29, 2021

jaladh-singhal reviewed Jul 29, 2021

View reviewed changes

atharva-2001 force-pushed the progress_bar branch from 183e49a to f7b1135 Compare August 7, 2021 16:53

jaladh-singhal requested changes Aug 12, 2021

View reviewed changes

jaladh-singhal requested a review from marxwillia August 12, 2021 14:20

atharva-2001 added 12 commits August 12, 2021 21:11

Customize bar format

620433e

Display bars again after reset when full

06a3242

Get data from configuration file

c25e166

Track all packets at once and don't reset the progress bar

fafc74a

Close last container and add docstrings

369ee2e

Add option to enable/disble the progress bar

a4bb0cd

Display container only in notebooks

6f4a475

Add a progress bar to track iterations

edd04d8

Align bars

0417bca

Refactor code

cc98189

Format using black

6d28e39

Add message regarding progress bars in the quickstart notebook

43bd44c

atharva-2001 force-pushed the progress_bar branch from 1cebfdc to 9bd9125 Compare August 12, 2021 15:49

atharva-2001 added 3 commits August 13, 2021 11:09

Demonstrate progress bars in the documentation

f7d06d2

Change show_progress_bar to show_progress_bars

d75e011

[build docs]

54d82ee

atharva-2001 force-pushed the progress_bar branch from e842526 to 54d82ee Compare August 13, 2021 05:40

atharva-2001 requested a review from jaladh-singhal August 13, 2021 07:15

andrewfullard approved these changes Aug 16, 2021

View reviewed changes

jaladh-singhal approved these changes Aug 19, 2021

View reviewed changes

jamesgillanders approved these changes Aug 19, 2021

View reviewed changes

jamesgillanders merged commit 575002c into tardis-sn:master Aug 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding Progress Bars #1703

Adding Progress Bars #1703

atharva-2001 commented Jul 9, 2021 •

edited

Loading

tardis-bot commented Jul 9, 2021

codecov bot commented Jul 9, 2021 •

edited

Loading

MarkMageeAstro left a comment

atharva-2001 commented Jul 9, 2021 •

edited

Loading

MarkMageeAstro commented Jul 9, 2021

marxwillia Jul 12, 2021

marxwillia Jul 12, 2021

marxwillia Jul 12, 2021

marxwillia Jul 22, 2021

jaladh-singhal left a comment

wkerzendorf commented Aug 12, 2021

jaladh-singhal left a comment •

edited

Loading

review-notebook-app bot commented Aug 12, 2021

jaladh-singhal left a comment

jamesgillanders left a comment

Adding Progress Bars #1703

Adding Progress Bars #1703

Conversation

atharva-2001 commented Jul 9, 2021 • edited Loading

tardis-bot commented Jul 9, 2021

codecov bot commented Jul 9, 2021 • edited Loading

Codecov Report

MarkMageeAstro left a comment

Choose a reason for hiding this comment

atharva-2001 commented Jul 9, 2021 • edited Loading

MarkMageeAstro commented Jul 9, 2021

marxwillia Jul 12, 2021

Choose a reason for hiding this comment

marxwillia Jul 12, 2021

Choose a reason for hiding this comment

marxwillia Jul 12, 2021

Choose a reason for hiding this comment

marxwillia Jul 22, 2021

Choose a reason for hiding this comment

jaladh-singhal left a comment

Choose a reason for hiding this comment

wkerzendorf commented Aug 12, 2021

jaladh-singhal left a comment • edited Loading

Choose a reason for hiding this comment

review-notebook-app bot commented Aug 12, 2021

jaladh-singhal left a comment

Choose a reason for hiding this comment

jamesgillanders left a comment

Choose a reason for hiding this comment

atharva-2001 commented Jul 9, 2021 •

edited

Loading

codecov bot commented Jul 9, 2021 •

edited

Loading

atharva-2001 commented Jul 9, 2021 •

edited

Loading

jaladh-singhal left a comment •

edited

Loading