Add no progress result printer to s3 transfer #2747

joguSD · 2017-08-03T20:08:17Z

This seemed relatively easy to implement and a lot of people have been asking for it.
This is basically a copy paste of the OnlyShowErrorsResultPrinter except that it still prints success messages. Just like the --only-show-errors flag you can now also pass --no-progress to omit the progress rendering. Not sure if I hit all the tests we need for this.

Implements: #519

jamesls

Had some small feedback.

jamesls · 2017-09-19T20:30:49Z

awscli/customizations/s3/subcommands.py

@@ -369,6 +369,11 @@
                        'output is suppressed.')}


+NO_PROGRESS = {'name': 'no-progress', 'action': 'store_true',


The no-* params typically have a store_false action with the destination being the param without the no prefix (progress in this case). Also makes it less confusing if we ever add an explicit true version of this arg (--progress).

jamesls · 2017-09-19T20:31:35Z

awscli/customizations/s3/subcommands.py

@@ -369,6 +369,11 @@
                        'output is suppressed.')}


+NO_PROGRESS = {'name': 'no-progress', 'action': 'store_true',
+                    'help_text': (
+                        'File transfer progress is not displayed.')}


It's probably worth documenting that this arg only matter is the other two options aren't provided (quiet/only_show_errors)

codecov-io · 2017-09-21T17:00:57Z

Codecov Report

Merging #2747 into develop will increase coverage by <.01%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop    #2747      +/-   ##
===========================================
+ Coverage    95.58%   95.59%   +<.01%     
===========================================
  Files          151      151              
  Lines        11848    11860      +12     
===========================================
+ Hits         11325    11337      +12     
  Misses         523      523

Impacted Files	Coverage Δ
awscli/customizations/s3/results.py	`98.55% <100%> (+0.01%)`	⬆️
awscli/customizations/s3/subcommands.py	`97.16% <100%> (ø)`	⬆️
awscli/customizations/s3/s3handler.py	`97.79% <100%> (+0.02%)`	⬆️
awscli/customizations/preview.py	`100% <0%> (ø)`	⬆️
awscli/customizations/cloudformation/deploy.py	`100% <0%> (ø)`	⬆️
awscli/clidocs.py	`98.35% <0%> (ø)`	⬆️
awscli/customizations/argrename.py	`100% <0%> (ø)`	⬆️
awscli/customizations/cloudformation/deployer.py	`96.29% <0%> (+0.24%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7cd785b...cf8c3f3. Read the comment docs.

c0reysc0tt

I really hope this gets merged into an upcoming version soon. I manually made these changes to the three python files affected, and now the log files created by all my s3 cron scripts are useful and readable. As far as I can tell, it's a complete fix to a problem people have been complaining about for years. The --no progress option is documented in the help files the same as any other function and does what it says it does. Thank you joguSD!

kyleknap

Looks good. I just had some very small comments. Also make sure you run the integration tests to make sure nothing is broken.

kyleknap · 2017-10-12T22:57:00Z

awscli/customizations/s3/subcommands.py

@@ -369,6 +369,15 @@
                        'output is suppressed.')}


+NO_PROGRESS = {'name': 'no-progress',
+               'action': 'store_false',
+               'dest': 'progress',


Not sure it is needed but all of the parameters like this have a 'default': True in the dictionary. Not sure if it is needed. Do not think it is. But it would not hurt to be consistent.

The other two flags like this don't have a default either, I think it's fine to have no default.

I was referring to all of the boolean flags that have store_false as the action. I just checked the docs and did it on the command line and its fine. Putting the default kwarg whose value is True is redundant.

kyleknap · 2017-10-12T23:01:06Z

awscli/customizations/s3/subcommands.py

+               'dest': 'progress',
+               'help_text': (
+                   'File transfer progress is not displayed. This flag '
+                   'is only applied when the quiet and only-show-errors '


Does the second sentence even matter? Based on the --quiet and --only-show-errors documentation, it will not show progress anyways. I would prefer to remove it in case we do not cause any confusion in the future if we add a new output parameter and forget to update this documentation.

The second sentence was added in response to some of James' feedback.

Oh its because it is conflicting. Like if --no-progress --quiet is provided which one would win? Yeah I'm fine with keeping it as is.

kyleknap · 2017-10-12T23:02:02Z

tests/integration/customizations/s3/test_plugin.py

+        # Copy file into bucket.
+        p = aws('s3 cp %s s3://%s/ --no-progress' % (foo_txt, bucket_name))
+        self.assertEqual(p.rc, 0)
+        # Ensure success message was printed


It might be good to assert there was no progress statement as well in the standard output.

kyleknap

Looks good. 🚢

joguSD added 3 commits August 3, 2017 12:19

Add no progress result printer to s3 transfer

dd03751

Add unit tests

8b564db

Add integration test

0f9fb95

JordonPhillips added the pr:needs-review This PR needs a review from a Member. label Aug 9, 2017

JordonPhillips requested review from dstufft, jamesls, stealthycoin, JordonPhillips and kyleknap August 9, 2017 16:26

Malvineous mentioned this pull request Sep 3, 2017

Resolve #519 by adding --hide-progress #2811

Closed

rickripe mentioned this pull request Sep 6, 2017

S3 progress verbosity should be optional #519

Closed

stealthycoin added enhancement medium labels Sep 18, 2017

jamesls reviewed Sep 19, 2017

View reviewed changes

jamesls added changes-requested and removed pr:needs-review This PR needs a review from a Member. labels Sep 19, 2017

Use progress destination and store false

42496d9

joguSD added pr:needs-review This PR needs a review from a Member. and removed changes-requested labels Sep 21, 2017

c0reysc0tt approved these changes Sep 28, 2017

View reviewed changes

kyleknap reviewed Oct 12, 2017

View reviewed changes

Assert progress related messages are not displayed

cf8c3f3

kyleknap approved these changes Oct 13, 2017

View reviewed changes

joguSD merged commit 2006dc0 into aws:develop Oct 13, 2017

diehlaws added feature-request A feature should be added or improved. and removed enhancement labels Jan 4, 2019

justnance added enhancement and removed feature-request A feature should be added or improved. labels Apr 15, 2019

TheLastCicada mentioned this pull request Mar 1, 2022

added --no-progress to aws s3 commands to create sane log files.... Chia-Network/ansible-roles#97

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add no progress result printer to s3 transfer #2747

Add no progress result printer to s3 transfer #2747

joguSD commented Aug 3, 2017

jamesls left a comment

jamesls Sep 19, 2017

jamesls Sep 19, 2017

codecov-io commented Sep 21, 2017 •

edited

Loading

c0reysc0tt left a comment •

edited

Loading

kyleknap left a comment

kyleknap Oct 12, 2017

joguSD Oct 13, 2017

kyleknap Oct 13, 2017

kyleknap Oct 12, 2017

joguSD Oct 12, 2017

kyleknap Oct 12, 2017 •

edited

Loading

kyleknap Oct 12, 2017

joguSD Oct 13, 2017

kyleknap left a comment

		@@ -369,6 +369,11 @@
		'output is suppressed.')}


		NO_PROGRESS = {'name': 'no-progress', 'action': 'store_true',

Add no progress result printer to s3 transfer #2747

Add no progress result printer to s3 transfer #2747

Conversation

joguSD commented Aug 3, 2017

jamesls left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Sep 21, 2017 • edited Loading

Codecov Report

c0reysc0tt left a comment • edited Loading

Choose a reason for hiding this comment

kyleknap left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kyleknap Oct 12, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kyleknap left a comment

Choose a reason for hiding this comment

codecov-io commented Sep 21, 2017 •

edited

Loading

c0reysc0tt left a comment •

edited

Loading

kyleknap Oct 12, 2017 •

edited

Loading