Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twitter JSONDecodeError #2212

Closed
Zoodee opened this issue Jan 21, 2022 · 10 comments
Closed

Twitter JSONDecodeError #2212

Zoodee opened this issue Jan 21, 2022 · 10 comments

Comments

@Zoodee
Copy link

Zoodee commented Jan 21, 2022

Seeing the attached error message when trying to download some urls from twitter.
Error message.txt

Downloading twitter URLs ending in "/media" still work (https://twitter.com/ZoodeeCarving/media) but main account links do not (https://twitter.com/ZoodeeCarving/)

I've tried with my VPN on and off, with login credentials attached and not attached to the command, and with the standalone Gallery-dl exe and the installed version. Same error consistently.

Don't know if this is relevant to this issue but gallery-dl looks like it isnt finding my config file and is using the default config, even though it used to work for me. My config is in C:\Users\USERNAME\AppData\Roaming\gallery-dl
(The config is actually a json file on my PC not a txt but I couldn't upload that here. It's the default example config except retries is set to 2, timeout is set to 10, and twitter retweets is set to true)
config.txt

Thanks, and sorry in advance if the issue is entirely on my end.

@Mayurifag
Copy link

I have the same problem — it just stopped work today in my daily script, which worked fine yesterday for twitter.

@ghost
Copy link

ghost commented Jan 21, 2022

I've been having this issue, as well. I think things stopped working, as soon as Twitter did their NFT update thing. I hope this can be fixed, soon.

@xingduan
Copy link

same problem, endpoint /2/timeline/profile/{}.json is rate limited, the response body is empty, causing decode error

mikf added a commit that referenced this issue Jan 21, 2022
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.

Only Twitter's search doesn't have a GraphQL interface yet.
mikf added a commit that referenced this issue Jan 21, 2022
@mikf
Copy link
Owner

mikf commented Jan 21, 2022

Should now be fixed with cb43f77, 54ef874, and dffa440, but it would be a miracle if there weren't any bugs in there. Let me know if you encounter a crash or if things behave differently than before.

@mikf mikf added the fixed label Jan 21, 2022
@Zoodee
Copy link
Author

Zoodee commented Jan 22, 2022

The fix works again fine for me so far, havent noticed any bugs. Many thanks.

@ermnick
Copy link

ermnick commented Jan 25, 2022

I get this error when it encounters a quote tweet where the quoted tweet is deleted:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/gallery_dl/job.py", line 79, in run
    for msg in extractor:
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/gallery_dl/extractor/twitter.py", line 62, in items
    for tweet in self.tweets():
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/gallery_dl/extractor/twitter.py", line 1140, in _pagination_tweets
    quoted["core"]["user_results"]["result"]
KeyError: 'core'

@Zoodee
Copy link
Author

Zoodee commented Jan 25, 2022

Seeing this error now on some downloads.
error.txt
sometimes getting KeyError - 'entryId' and sometimes getting KeyError - 'core'

@Zoodee Zoodee reopened this Jan 25, 2022
mikf added a commit that referenced this issue Jan 25, 2022
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
@mikf
Copy link
Owner

mikf commented Jan 25, 2022

@Zoodee KeyError - 'entryId' was due to pinned Tweets (#2216 (comment)) and got fixed in 5ed26e1. KeyError - 'core' and some more potential errors were fixed in 2bf554a.

@Zoodee
Copy link
Author

Zoodee commented Jan 25, 2022

I really need to make sure I'm always on the latest version before I post issues, my bad

@Zoodee Zoodee closed this as completed Jan 25, 2022
@Mayurifag
Copy link

Yea everything works fine, it seems, if one uses version from master branch via pip, not the outdated one from your fancy linux repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants