Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[webtoons] Add a new extractor for webtoons.com #761

Merged
merged 1 commit into from
May 18, 2020
Merged

[webtoons] Add a new extractor for webtoons.com #761

merged 1 commit into from
May 18, 2020

Conversation

iamleot
Copy link
Contributor

@iamleot iamleot commented May 16, 2020

The webtoons extractor can extract episode and entire comic (all
episodes) from webtoons.com.

All the logic of the extractors should be trivial except for a couple
of kludges needed:

  • `ageGatePass' cookie is always set to avoid possible redirect and stop of
    extraction, especially in the comic extractor
  • The image URLs returned by the episode extractor could not be fetched
    directly and the `Referer:' HTTP header needs to be passed to fetch them

Close #593.

The webtoons extractor can extract episode and entire comic (all
episodes) from webtoons.com.

All the logic of the extractors should be trivial except for a couple
of kludges needed:

 - `ageGatePass' cookie is always set to avoid possible redirect and stop of
    extraction, especially in the comic extractor
 - The image URLs returned by the episode extractor could not be fetched
   directly and the `Referer:' HTTP header needs to be passed to fetch them

Close #593.
@mikf mikf merged commit 39cd389 into mikf:master May 18, 2020
@iamleot
Copy link
Contributor Author

iamleot commented May 18, 2020 via email

mikf added a commit that referenced this pull request May 18, 2020
- fix episode listings for french comics
- allow input URLs without explicit scheme
- add 'lang'/'language' metadata
- use str.format() instead of '+' to assemble URLs
@mikf
Copy link
Owner

mikf commented May 18, 2020

And a "Thank You" back to you.

There where some issues with French comic listings and input URLs without a scheme, which I've only spotted after merging. I've fixed those myself in 0378d07. Hope you don't mind.

@iamleot
Copy link
Contributor Author

iamleot commented May 18, 2020

That's much better and simpler, thanks!

@doyleslee
Copy link

Is it possible to sort each episode into a folder, rather than downloading every image into 1 folder? I've been playing around with the config but can't seem to get it to work.

@iamleot
Copy link
Contributor Author

iamleot commented Jul 17, 2020 via email

@doyleslee
Copy link

doyleslee commented Jul 17, 2020

Hm when I try that on my pc (windows) I get the error: unrecognized arguments: URL="https://www.webtoons.com/en/fantasy/tower-of-god/list?title_no=95". I copied it exactly, not sure whats going on here.

Also, what do you mean by the conf(5) main page? Do you mean this "https://github.com/mikf/gallery-dl/blob/master/docs/gallery-dl-example.conf" ?

Sorry if these seem like obvious questions and thank you for helping, I'm not that experienced with python or cmd

@iamleot
Copy link
Contributor Author

iamleot commented Jul 17, 2020

@doyleslee Probably the quoting of arguments (e.g.: using " instead of ') is different in Windows (and I'm completely unfamiliar with it so I couldn't help, sorry!). I'm sure others will help to shed some light!

Regarding gallery-dl.conf(5) man page I meant the man page that should be derived from here:

https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst

But probably this is not present on Windows as well, at least not in the form of a man page.

Here a possible configuration that permits to not pass the -o options before:

{
	"extractor": {
		"webtoons": {
			"directory": ["{category}", "{comic}", "{episode}"],
			"filename": "{num:>02}.{extension}"
		}
	}
}

Saved as gallery-dl.conf you can then use it via, e.g. gallery-dl -c gallery-dl.conf "https://www.webtoons.com/en/comedy/safely-endangered/list?title_no=352"

@doyleslee
Copy link

That configuration worked perfectly - thank you!

@iamleot
Copy link
Contributor Author

iamleot commented Jul 17, 2020

Nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Request] Webtoons.com - Manhwa or Webtoon downloading
3 participants