read of SpooledTemporaryFile returns bytes, mypy assumes str

[maweki]

I think this is correct here and not in the mypy repo (if I'm wrong, I am happy to open a ticket there):

following code

from tempfile import SpooledTemporaryFile
stderrfile = SpooledTemporaryFile()
sterr_output = stderrfile.read().decode()

works fine with python3.7 . Using mypy (also with python 3.7) I get the following error:

$ python3 -m mypy test.py 
test.py:3: error: "str" has no attribute "decode"; maybe "encode"?

Mypy incorrectly assumes that the result of the read is a str but using the interactive interpreter you'll see that the read method actually returns a byte object. The type annotations (https://github.com/python/typeshed/blob/master/stdlib/3/tempfile.pyi) say AnyStr as return type which is too narrow in this case.

[srittau]

Unfortunately, there is no good way to express this in typeshed at this point. I have filed python/typing#680, but I don't foresee that this will be implemented any time soon. The best you could do is to use an explicit type annotation:

from tempfile import SpooledTemporaryFile
stderrfile: SpooledTemporaryFile[bytes] = SpooledTemporaryFile()
sterr_output = stderrfile.read().decode()

[maweki]

@srittau ah, okay. Now I understand the problem. If I open this in w+ mode read returns str and in (the default) w+b mode it returns byte.

Yeah, the question is whether the annotation is correct in giving the restricted type AnyStr or the more unrestricted type Union[str,bytes]. As it turns out, in my case I can just change the mode since I only use string data anyways. But this is a problem in general.

[maweki]

@srittau
The question is whether the annotation is correct in giving the restricted type AnyStr or the less restricted type Union[AnyStr,bytes]. Can you elaborate on that?

If I'd write a function with multiple return types, I would annotate it this way.

[srittau]

A Union return type is actually more restricted than an any type. We try to avoid Union return types, since it forces the user to use isinstance checks, especially in cases like this, where the user will know in the majority of cases what the return type will be.