Add support for data streams (with known total size)

[ppolewicz]

Hi,

It is easy to integrate tqdm with urllib when downloading a file, but when something is being uploaded, it is not so easy. One needs to intercept .read() or .write() calls and update tqdm manually.

I needed it in b2 command line tool, so I have implemented it there, but maybe a stream wrapper could be useful for other tqdm users? This code is more specific to tqdm than to b2 command line tool.

Do you want to support progress tracking of data streams?

[lrq3000]

Thank you for the link and the code, yes we are always open to new features :) We'll see how we can manage to integrate that inside tqdm.

[ppolewicz]

I guess you can just take my stream_with_progress implementation (except kwargs.update call and the inheritance weirdness), call it tqdm_stream_wrapper and provide examples in the documentation... Or are you thinking about adding that functionality to the existing tqdm class?

[lrq3000]

Indeed I'd like to add it to the base tqdm class, I think this
functionality should be supported by all tqdm subclasses.

2015-12-26 20:26 GMT+01:00 Paweł Polewicz [email protected]:

I guess you can just take my stream_with_progress implementation (except
kwargs.update call and the inheritance weirdness), call it
tqdm_stream_wrapper and provide examples in the documentation... Or are
you thinking about adding that functionality to the existing tqdm class?

—
Reply to this email directly or view it on GitHub
#84 (comment).

[ppolewicz]

I see. Do you want me to implement it and create a pull request, or you want to do it yourself?

[lrq3000]

Well yes, you know best about the needs for this feature, so feel free to
do a PR :) Or if you can just provide a minimal example to test the feature
against, this would be very useful!

2015-12-27 1:56 GMT+01:00 Paweł Polewicz [email protected]:

I see. Do you want me to implement it and create a pull request, or you
want to do it yourself?

—
Reply to this email directly or view it on GitHub
#84 (comment).

[ppolewicz]

I'll do both the example and the implementation, please assign it to me. I am also looking into how to test it and document it. I noticed that arguments of tqdm class are documented in a structured way of some kind - where could I read about the syntax of this format?

[lrq3000]

@ppolewicz Ok perfect, thank you very much!
About the format, I don't know, I just copy the style myself when committing changes :p The readme is in ReStructuredText format, maybe that's the same format we use in _tqdm.py (am I right @casperdcl?).
Seems like I can't assign someone outside the organization linked to this github account :/ But nvm, I de-assigned myself.

[ppolewicz]

Thanks for rst hint, it really helped me.

I got it working with an example, but then during review and testing found that supporting read() and write() was easy. Supporting every possible operation that a file-like object might be expected to handle, on the other hand, is not so easy :-) I will work on this further tomorrow.

[lrq3000]

Thank you ppolewicz, and don't worry to take your time, we're not in a hurry :)

[casperdcl]

Yes, it's RST (and also Markdown) compatible. One thing to note is that documentation in code (anything that might show up when running help()) should have at most 76 character long lines (incl. initial spaces).... this is because python adds some padding of ~4 chars (to make the total 80) when running help().

Secondly, please use two spaces between the variable name and the colon, and use [default: ...] for defaults in documentation where applicable (making it compatible with things like docopt).

Thirdly, just in the RST, add two spaces to the end of the bullet point parameters of tqdm to ensure they show up as a sort of header.

[casperdcl]

But above all don't be worried :) you don't have to do everything, we're happy to fill in ;)

[casperdcl]

Note to self: add #84 (comment) to CONTRIBUTING.md

[casperdcl]

fixed by #844