flickrapi.upload() can't handle format "parsed-json"

[B-Con]

FlickrAPI(..., format='parsed-json') can't handle the response, attempting to parse the response as JSON even though it's not. The documentation says the "parsed formats" are available.

The Flickr upload API seems to only respond to uploads with XML, not JSON, so it seems the JSON formats are not valid at all.

[danielhoherd]

This is still a problem. From a pdb session while debugging:

> /home/dho/.cache/pypoetry/virtualenvs/flickrcronpy-jFmCqsjY-py3.7/lib/python3.7/site-packages/flickrapi/core.py(402)_wrap_in_parser()
-> if parse_format not in rest_parsers:
(Pdb) ll
385         def _wrap_in_parser(self, wrapped_method, parse_format, *args, **kwargs):
386             """Wraps a method call in a parser.
387
388             The parser will be looked up by the ``parse_format`` specifier. If there
389             is a parser and ``kwargs['format']`` is set, it's set to ``rest``, and
390             the response of the method is parsed before it's returned.
391             """
392
393             # Find the parser, and set the format to rest if we're supposed to
394             # parse it.
395             if parse_format in rest_parsers and 'format' in kwargs:
396                 kwargs['format'] = rest_parsers[parse_format][1]
397
398             LOG.debug('Wrapping call %s(self, %s, %s)' % (wrapped_method, args, kwargs))
399             data = wrapped_method(*args, **kwargs)
400
401             # Just return if we have no parser
402  ->         if parse_format not in rest_parsers:
403                 return data
404
405             # Return the parsed data
406             parser = rest_parsers[parse_format][0]
407             return parser(self, data)
(Pdb) parse_format
'parsed-json'
(Pdb) rest_parsers
{'xmlnode': (<function FlickrAPI.parse_xmlnode at 0x7fbd9718f320>, 'rest'), 'parsed-json': (<function FlickrAPI.parse_json at 0x7fbd9718f3b0>, 'json'), 'etree': (<function FlickrAPI.parse_etree at 0x7fbd9718f440>, 'rest')}
(Pdb) data
b'<?xml version="1.0" encoding="utf-8" ?>\n<rsp stat="ok">\n<photoid>49543402303</photoid>\n</rsp>\n'
(Pdb) interact
*interactive*
>>> kwargs['format']
Traceback (most recent call last):
  File "<console>", line 1, in <module>
KeyError: 'format'

So at first it seemed like line 396 was failing to update kwargs:

> /home/dho/.cache/pypoetry/virtualenvs/flickrcronpy-jFmCqsjY-py3.7/lib/python3.7/site-packages/flickrapi/core.py(395)_wrap_in_parser()
-> if parse_format in rest_parsers and 'format' in kwargs:
(Pdb) kwargs
{'timeout': None}
(Pdb) ll
385         def _wrap_in_parser(self, wrapped_method, parse_format, *args, **kwargs):
386             """Wraps a method call in a parser.
387
388             The parser will be looked up by the ``parse_format`` specifier. If there
389             is a parser and ``kwargs['format']`` is set, it's set to ``rest``, and
390             the response of the method is parsed before it's returned.
391             """
392
393             # Find the parser, and set the format to rest if we're supposed to
394             # parse it.
395  ->         if parse_format in rest_parsers and 'format' in kwargs:
396                 kwargs['format'] = rest_parsers[parse_format][1]
397
398             LOG.debug('Wrapping call %s(self, %s, %s)' % (wrapped_method, args, kwargs))
399             data = wrapped_method(*args, **kwargs)
400
401             # Just return if we have no parser
402             if parse_format not in rest_parsers:
403                 return data
404
405             # Return the parsed data
406             parser = rest_parsers[parse_format][0]
407             return parser(self, data)
(Pdb) kwargs
{'timeout': None}
(Pdb) n
> /home/dho/.cache/pypoetry/virtualenvs/flickrcronpy-jFmCqsjY-py3.7/lib/python3.7/site-packages/flickrapi/core.py(398)_wrap_in_parser()
-> LOG.debug('Wrapping call %s(self, %s, %s)' % (wrapped_method, args, kwargs))
(Pdb) kwargs
{'timeout': None}
(Pdb) l
393             # Find the parser, and set the format to rest if we're supposed to
394             # parse it.
395             if parse_format in rest_parsers and 'format' in kwargs:
396                 kwargs['format'] = rest_parsers[parse_format][1]
397
398  ->         LOG.debug('Wrapping call %s(self, %s, %s)' % (wrapped_method, args, kwargs))
399             data = wrapped_method(*args, **kwargs)
400
401             # Just return if we have no parser
402             if parse_format not in rest_parsers:
403                 return data
(Pdb) kwargs
{'timeout': None}

I tried this with etree and parsed-json and saw the same problem for both, but when I entered interactive I was able to update kwargs. Digging a bit more, I found that 'format' is always deleted from upload requests on line 427:

> /home/dho/.cache/pypoetry/virtualenvs/flickrcronpy-jFmCqsjY-py3.7/lib/python3.7/site-packages/flickrapi/core.py(421)_extract_upload_response_format()
-> if response_format not in rest_parsers and response_format != 'rest':
(Pdb) ll
409         def _extract_upload_response_format(self, kwargs):
410             """Returns the response format given in kwargs['format'], or
411             the default format if there is no such key.
412
413             If kwargs contains 'format', it is removed from kwargs.
414
415             If the format isn't compatible with Flickr's upload response
416             type, a FlickrError exception is raised.
417             """
418
419             # Figure out the response format
420             response_format = kwargs.get('format', self.default_format)
421  ->         if response_format not in rest_parsers and response_format != 'rest':
422                 raise FlickrError('Format %s not supported for uploading '
423                                   'photos' % response_format)
424
425             # The format shouldn't be used in the request to Flickr.
426             if 'format' in kwargs:
427                 del kwargs['format']
428
429             return response_format

So, from what I see, this looks like upload responses will never work with anything but the default response type.

This seems to correlate to the Flickr API Upload documentation which states "This response is formatted in the REST API response style." And indeed, I see no error when I use:

flickr.upload(filename=photo_filename, format="rest")

[jim-easterbrook]

The Python flickrapi documentation also says:

format
    The response format. This must be either rest or one of the parsed formats etree / xmlnode.

https://stuvel.eu/flickrapi-doc/4-uploading.html