DeviantArt "Watchers-Only" images being downloaded in blurred form

[sbobbo]

So it seems that some artists on Deviantart can set certain images as "Watchers-only". They appear as blurred versions of the images unless you are logged in and are watching them.

When downloading them with gallery-dl, no matter what I do, I am getting blurred images. I have refreshed my oauth token, and I have my client-id and client-secret set. All other images work just fine. Obviously, I am set to watch the users that I am running into this with.

A user where I am encountering this is https://www.deviantart.com/prywinko

Am I doing something wrong or is this a bug in the program?

[sbobbo]

I also have a cookies.txt file set up and that has not helped at all either. I am not sure if that's my problem, but I don't see any relevant errors...

`D:\Downloads\Online\Downloaders\gallery-dl>gallery-dl https://www.deviantart.com/prywinko --cookies=./cookies.txt -v
[gallery-dl][debug] Version 1.14.5
[gallery-dl][debug] Python 3.8.5 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.24.0 - urllib3 1.25.10
[gallery-dl][debug] Starting DownloadJob for 'https://www.deviantart.com/prywinko'
[deviantart][debug] Using DeviantartUserExtractor for 'https://www.deviantart.com/prywinko'
[deviantart][debug] Using DeviantartGalleryExtractor for 'https://www.deviantart.com/prywinko/gallery'
[deviantart][debug] Using custom API credentials (client-id 12160)
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/user/profile/prywinko HTTP/1.1" 200 3007
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/all?username=Prywinko&offset=0&limit=24&mature_content=true HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/metadata?deviationids%5B0%5D=D749004A-77DB-5136-08D2-9976F01D5EDB&deviationids%5B1%5D=AC016CA4-4659-509E-771D-D9E1801C8BB6&deviationids%5B2%5D=231A2849-51A1-F27E-4C66-55B219A15C05&deviationids%5B3%5D=CD170CFC-9C37-491C-9E6D-07711580B4A7&deviationids%5B4%5D=5A5202C9-8AB2-5621-BB17-33EE0AEE2B40&deviationids%5B5%5D=CE05A378-2092-E400-CDF3-2CCF28248E16&deviationids%5B6%5D=6A6C53D7-FE59-B449-3E97-CA8ABE6B3F1D&deviationids%5B7%5D=CAFC4453-DD48-4404-EC86-CFB9ABB6FD35&deviationids%5B8%5D=BC6B8A76-553E-FB47-2566-F3DA17DD3019&deviationids%5B9%5D=FDE7DCA4-5D01-9B43-0A93-EFEE0DE4BC8D&deviationids%5B10%5D=015D6C93-FC7A-ACF0-8112-D5F92191F776&deviationids%5B11%5D=AD62094E-5D01-BBF8-A3F7-84DE2406CBDC&deviationids%5B12%5D=4D34D32D-B89F-D4B2-3932-B12F6499511A&deviationids%5B13%5D=0F831B96-7412-FECB-C32F-B7EC1922E92A&deviationids%5B14%5D=DE98FDA8-E701-DD5F-C3D5-F811346F00F4&deviationids%5B15%5D=36C8F77C-7D39-D807-3D4E-04132582C3F2&deviationids%5B16%5D=7160D9E1-FC0B-C92D-D230-3D8E504BA963&deviationids%5B17%5D=20E2718D-05FD-50AD-84DC-71F06A408AAB&deviationids%5B18%5D=58C1BFCE-F1E4-DBB3-78B9-A111B57650E5&deviationids%5B19%5D=7AD3D203-D92D-B566-1242-B9CDE9FFE91A&deviationids%5B20%5D=A031B03D-1C12-F687-F290-AD3E944EB2B7&deviationids%5B21%5D=8F0E0B41-03DC-E8EC-B652-F85864631A5A&deviationids%5B22%5D=CAB9DCDB-003E-F258-EA30-D77DE292CE6D&deviationids%5B23%5D=1EA11C6C-FB11-A3DB-DDE9-58BECF549724&mature_content=true HTTP/1.1" 200 None
[deviantart][debug] Using download archive './archive.sqlite3'
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/D749004A-77DB-5136-08D2-9976F01D5EDB?mature_content=true HTTP/1.1" 200 551
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): api-da.wixmp.com:443
[urllib3.connectionpool][debug] https://api-da.wixmp.com:443 "GET /_api/download/file?downloadToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsImV4cCI6MTYwMDEwMzQ5NywiaWF0IjoxNjAwMTAyODg3LCJqdGkiOiI1ZjVmYTFmMTg5ZjkyIiwib2JqIjpudWxsLCJhdWQiOlsidXJuOnNlcnZpY2U6ZmlsZS5kb3dubG9hZCJdLCJwYXlsb2FkIjp7InBhdGgiOiJcL2ZcL2NmMjk2YmE1LTBkNWEtNGFjNi1iNTM0LTlhZDNkZjlkZTU4ZlwvZGU0enR3ZC02N2M5YWNhZS03NWM5LTQ1N2QtYjkzNC0xYjhjNjZiNzhkYmYuanBnIn19.cUiALAz2kQCkLU8E22fFGLdJBy9qkXkn3qNtwqAgi80 HTTP/1.1" 200 None

• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-09-12-September - Witcher month-854916637.jpg
  [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/AC016CA4-4659-509E-771D-D9E1801C8BB6?mature_content=true HTTP/1.1" 200 544
  [urllib3.connectionpool][debug] https://api-da.wixmp.com:443 "GET /_api/download/file?downloadToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsImV4cCI6MTYwMDEwMzQ5OCwiaWF0IjoxNjAwMTAyODg4LCJqdGkiOiI1ZjVmYTFmMjQ4NjM3Iiwib2JqIjpudWxsLCJhdWQiOlsidXJuOnNlcnZpY2U6ZmlsZS5kb3dubG9hZCJdLCJwYXlsb2FkIjp7InBhdGgiOiJcL2ZcL2NmMjk2YmE1LTBkNWEtNGFjNi1iNTM0LTlhZDNkZjlkZTU4ZlwvZGU0d25peS1iMmJlNzdiZC0wNThhLTRkMWUtYmUwZC0wZDY0MGQ1MTJmMzcuanBnIn19.YnqvL0mxpna1GuLupsKSU8OHeYMoKktImTi4mT_0hkM HTTP/1.1" 200 None
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-09-10-Heilog Saga. Page 14-854768410.jpg
  [urllib3.connectionpool][debug] Starting new HTTPS connection (1): images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com:443
  [urllib3.connectionpool][debug] https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com:443 "GET /f/cf296ba5-0d5a-4ac6-b534-9ad3df9de58f/de4o9or-725576ee-743e-447a-b995-f27de752853a.jpg/v1/fill/w_1000,h_1500,q_100,strp,blur_30/ciri__lingerie_version__by_prywinko_de4o9or-fullview.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOiIsImlzcyI6InVybjphcHA6Iiwib2JqIjpbW3siaGVpZ2h0IjoiPD0xNTAwIiwicGF0aCI6IlwvZlwvY2YyOTZiYTUtMGQ1YS00YWM2LWI1MzQtOWFkM2RmOWRlNThmXC9kZTRvOW9yLTcyNTU3NmVlLTc0M2UtNDQ3YS1iOTk1LWYyN2RlNzUyODUzYS5qcGciLCJ3aWR0aCI6Ijw9MTAwMCIsImJsdXIiOiIzMCJ9XV0sImF1ZCI6WyJ1cm46c2VydmljZTppbWFnZS5vcGVyYXRpb25zIl19.SYdNM_MDssqV8AYm9rNbahH4RbJu9SpZv4kpWeG8WhM HTTP/1.1" 200 302494
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-09-06-Ciri (lingerie version)-854377227.jpg
  [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/CD170CFC-9C37-491C-9E6D-07711580B4A7?mature_content=true HTTP/1.1" 200 551
  [urllib3.connectionpool][debug] https://api-da.wixmp.com:443 "GET /_api/download/file?downloadToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsImV4cCI6MTYwMDEwMzQ5OCwiaWF0IjoxNjAwMTAyODg4LCJqdGkiOiI1ZjVmYTFmMmU1YmI3Iiwib2JqIjpudWxsLCJhdWQiOlsidXJuOnNlcnZpY2U6ZmlsZS5kb3dubG9hZCJdLCJwYXlsb2FkIjp7InBhdGgiOiJcL2ZcL2NmMjk2YmE1LTBkNWEtNGFjNi1iNTM0LTlhZDNkZjlkZTU4ZlwvZGU0bTRodi05MTcwOTAyNi02ZTkxLTRkMjMtYWQyYi0wZDBiNzk4YTU0NWYuanBnIn19.6lnSyuEtIFv8G-MvT1CATvotU7ecI8B4QBIDXYPutNE HTTP/1.1" 200 None
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-09-05-Cirilla Fiona Elen Riannon-854277187.jpg
  [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/5A5202C9-8AB2-5621-BB17-33EE0AEE2B40?mature_content=true HTTP/1.1" 200 562
  [urllib3.connectionpool][debug] https://api-da.wixmp.com:443 "GET /_api/download/file?downloadToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsImV4cCI6MTYwMDEwMzQ5OSwiaWF0IjoxNjAwMTAyODg5LCJqdGkiOiI1ZjVmYTFmMzVhYWQzIiwib2JqIjpudWxsLCJhdWQiOlsidXJuOnNlcnZpY2U6ZmlsZS5kb3dubG9hZCJdLCJwYXlsb2FkIjp7InBhdGgiOiJcL2ZcL2NmMjk2YmE1LTBkNWEtNGFjNi1iNTM0LTlhZDNkZjlkZTU4ZlwvZGU0YWZ2cC0xZGVlNDVmYS0zZWNkLTRiODktOTEyNi1hMGVhNDA0NmQyNWEuanBnIn19.X1EuITdaard3_sna0Cx-PzXvB_dSXLfbMaEj2P5unUU HTTP/1.1" 200 None
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-08-31-Last day to enter for August bundle!-853732069.jpg
  [urllib3.connectionpool][debug] https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com:443 "GET /f/cf296ba5-0d5a-4ac6-b534-9ad3df9de58f/de48hgr-0f65a39d-c255-4121-a826-2af7da511511.png/v1/fill/w_872,h_859,q_100,strp,blur_30/ciri_wip_by_prywinko_de48hgr-fullview.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOiIsImlzcyI6InVybjphcHA6Iiwib2JqIjpbW3siaGVpZ2h0IjoiPD04NTkiLCJwYXRoIjoiXC9mXC9jZjI5NmJhNS0wZDVhLTRhYzYtYjUzNC05YWQzZGY5ZGU1OGZcL2RlNDhoZ3ItMGY2NWEzOWQtYzI1NS00MTIxLWE4MjYtMmFmN2RhNTExNTExLnBuZyIsIndpZHRoIjoiPD04NzIiLCJibHVyIjoiMzAifV1dLCJhdWQiOlsidXJuOnNlcnZpY2U6aW1hZ2Uub3BlcmF0aW9ucyJdfQ.kTkHsYEoTn0FXLaejsqK3f1I8r65rS6zy2hCFo6zDjM HTTP/1.1" 200 151288
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-08-30-Ciri wip-853640811.jpg
  [urllib3.connectionpool][debug] https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com:443 "GET /f/cf296ba5-0d5a-4ac6-b534-9ad3df9de58f/de483io-f7f77124-b4ba-4a82-85d2-a868fc228f32.jpg/v1/fill/w_1000,h_848,q_100,strp,blur_30/shamir__preview_by_prywinko_de483io-fullview.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOiIsImlzcyI6InVybjphcHA6Iiwib2JqIjpbW3siaGVpZ2h0IjoiPD04NDgiLCJwYXRoIjoiXC9mXC9jZjI5NmJhNS0wZDVhLTRhYzYtYjUzNC05YWQzZGY5ZGU1OGZcL2RlNDgzaW8tZjdmNzcxMjQtYjRiYS00YTgyLTg1ZDItYTg2OGZjMjI4ZjMyLmpwZyIsIndpZHRoIjoiPD0xMDAwIiwiYmx1ciI6IjMwIn1dXSwiYXVkIjpbInVybjpzZXJ2aWNlOmltYWdlLm9wZXJhdGlvbnMiXX0.A-zqoBJyLNV8UyN8D_Ifq0TgYTkCAxFz1TvDCqcsdgo HTTP/1.1" 200 175131
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-08-30-Shamir preview-853622736.jpg
  [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/CAFC4453-DD48-4404-EC86-CFB9ABB6FD35?mature_content=true HTTP/1.1" 200 562
  [urllib3.connectionpool][debug] https://api-da.wixmp.com:443 "GET /_api/download/file?downloadToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsImV4cCI6MTYwMDEwMzQ5OSwiaWF0IjoxNjAwMTAyODg5LCJqdGkiOiI1ZjVmYTFmM2VkODRiIiwib2JqIjpudWxsLCJhdWQiOlsidXJuOnNlcnZpY2U6ZmlsZS5kb3dubG9hZCJdLCJwYXlsb2FkIjp7InBhdGgiOiJcL2ZcL2NmMjk2YmE1LTBkNWEtNGFjNi1iNTM0LTlhZDNkZjlkZTU4ZlwvZGU0ODF1bS0xYzk5MmRkNS02MjQyLTQ5MjgtYmYwMC1lNjkzOTU2ZTA5ZGYuanBnIn19.TpdK0AnFlGACVkzzWSA5RPrH1KV-J1dPL3iS-KQa6no HTTP/1.1" 200 None
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-08-30-Special Thanks from Heilog and Beans-853620574.jpg
  [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/BC6B8A76-553E-FB47-2566-F3DA17DD3019?mature_content=true HTTP/1.1" 200 535
  [urllib3.connectionpool][debug] https://api-da.wixmp.com:443 "GET /_api/download/file?downloadToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsImV4cCI6MTYwMDEwMzUwMCwiaWF0IjoxNjAwMTAyODkwLCJqdGkiOiI1ZjVmYTFmNDVjYWQ1Iiwib2JqIjpudWxsLCJhdWQiOlsidXJuOnNlcnZpY2U6ZmlsZS5kb3dubG9hZCJdLCJwYXlsb2FkIjp7InBhdGgiOiJcL2ZcL2NmMjk2YmE1LTBkNWEtNGFjNi1iNTM0LTlhZDNkZjlkZTU4ZlwvZGU0NmJsai0xMGJmZjk1NC00ZTk0LTQ3YzktYTM0Yi1hZGMzY2NmMzk1YzMuanBnIn19.wfLDpa5hPP7I0CMvfMOKEx_J6GbbUfIRt9gDCORyECg HTTP/1.1" 200 None
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-08-29-Shamir-853539895.jpg
  [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/FDE7DCA4-5D01-9B43-0A93-EFEE0DE4BC8D?mature_content=true HTTP/1.1" 200 552
  [urllib3.connectionpool][debug] https://api-da.wixmp.com:443 "GET /_api/download/file?downloadToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsImV4cCI6MTYwMDEwMzUwMCwiaWF0IjoxNjAwMTAyODkwLCJqdGkiOiI1ZjVmYTFmNGJkNzhlIiwib2JqIjpudWxsLCJhdWQiOlsidXJuOnNlcnZpY2U6ZmlsZS5kb3dubG9hZCJdLCJwYXlsb2FkIjp7InBhdGgiOiJcL2ZcL2NmMjk2YmE1LTBkNWEtNGFjNi1iNTM0LTlhZDNkZjlkZTU4ZlwvZGU0NGFpMC1mMDI1M2NjYi0zM2QyLTQwZTYtYmIyYi1lOWY5YzlhZGEyOGQuanBnIn19.TlkcXx4fn2UFlORopBO-NsoU065b2hWLxHnHHdYqHLI HTTP/1.1" 200 None
• .\gallery-dl\deviantart\X_da-Prywinko\Prywinko-2020-08-28-Shamir teaser(Coming Soon!)-853445160.jpg
  [urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/015D6C93-FC7A-ACF0-8112-D5F92191F776?mature_content=true HTTP/1.1" 200 56`

and my config with a few values replaced:

"deviantart":
{
"client-id": "MYIDHERE",
"client-secret": "MYSECRETHERE",
"refresh-token": "cache",
"archive-format": "{index}",
"include": "gallery,scraps",
"cookies": "./cookies.txt",
"cookies-update": true,
"extra": true,
"flat": true,
"folders": false,
"journals": "html",
"mature": true,
"metadata": true,
"original": true,
"quality": 100,
"skip": "false",
"wait-min": 0,
"filename": "{author[username]}-{date:%Y-%m-%d}-{title}-{index}.{extension}",
"directory": ["{category}", "X_da-{author[username]}"]
},

[biznizz]

Yeah, I've seen these things popping up lately in DA, so they seem fairly new.

I don't think it's a bug, but it's possible the extractor does not currently have settings on how to extract these "watcher only" posts, and can only assume any way to do so would involve DA cookies.

Only mikf can confirm this theory, tho.

[mikf]

This does appear to be a relatively new feature:
https://www.deviantart.com/team/journal/Create-Exclusive-Content-with-Premium-Galleries-847238532

Once this is implemented, it should be possible to access them with a refresh-token for an account that is allowed to see them (no cookies).

[mikf]

It should now be possible to download those images in unblurred form with 136df52

It'll print a warning message whenever "premium" content can't be accessed.
and blurred images shouldn't get downloaded anymore.

[sbobbo]

Excellent, thank you for addressing this so quickly!

I don't suppose you tested whether the images will be "redownloaded" if we have an archive and skip=true? I don't know whether that would possible or would require more work, but it would help tremendously. Having to manually redo those artists recent work would be quite the pain in the ass.

[mikf]

You should be able to overwrite the blurred images with --filter and --no-skip:

$ gallery-dl --filter "'premium_folder_data' in locals()" --no-skip https://www.deviantart.com/prywinko