Add auto downloader #36
Comments
|
I don't know what FMD/FMD2 is, could you elaborate a bit more what exactly you have in mind? It sounds to me like you want to have a library where you can add novels and it checks every so often for new chapters. I'm a bit confused about the auto downloader part in the title. Is it supposed to automatically replace the previously created EPUB with an updated one? |
|
I am sorry for being unclear. So for example i would look up a series and download all the available chapters, then I would click a button to track the series and like you said, every so often it would check for a new chapter. It does not need to replace, just make a new file that ideally would show its different like chapters: chapter 1 is before chapter 2. That way you would not have to go and manually sort them so you can read correctly. Also that wuxiaworld site did not work i dont know if you can fix that? |
|
I don't know how viable that approach would be. Same reason I thought about adding a library function but scraping the idea later. What you're describing would be best implemented as a self contained reader which tracks already read chapters and adds new ones when they become available. I'm not sure how that would work with NG and the EPUBs it creates, they're pretty bad for small 'updates'. (A new 'main' file which replaces the old or new smaller ones). Which site? Wuxiaworld.com or Wuxiaworld.site |
|
Wuxiaworld.com And i was wondering why do you bundle the way you do? all the other downloader for different thing such as comics and manga do it by a per chapter basis because it would run into the same issue you talked about have to re make it every time. If you gave the option to download section instead of what you do now it would fix the issue. The manga downloader HakuNeko has a feature that when you click on the series it gives a list of all the chapters and also the file you already have downloaded which would be nice to have if you let people file multiple files for each chapter instead of all in one. https://github.com/dazedcat19/FMD2 is the link for the manga downloader and tracker, I quite like how it does things and maybe it will give you a better idea of what I am talking about. In my case I don't need a reader, Komga comic server does that for me. In the end it is your choice and there are other things to work on just understand that if you are able to set up tracking/auto downloading it would be huge |
|
The point of NG is to be able to read a webnovel conveniently on your eReader of choice. They all work with EPUBs so it's the perfect format to use for this. It's just not ideal for when you are caught up with a running novel. I think people only download new chapters in larger chunks when they have amassed a certain size of backlog and not short single chapters, I personally don't see the point of that. The only application I can think of would be to download new chapters of various different novels and put them together in a single file to get a comfortable reading experience. |
|
As someone who shares a different view then your view on what people want,
it is not true for everyone. Not everyone wants to wait until there is a
large (what large is being subjective) amount of chapters unread
before downloading again. Wuxiaworld and pretty much most other book sites
run on two parts: the library they already have and the chapters coming out
on some regular basis. Wuxiaworld specifically makes money because people
pay for the latest chapters and don't want to wait for a pile of them to
get together. I just want the option to auto download new chapters as they
show up, this option can be completely up to a person if they want to use
it or not. So instead of looking up the site again, searching for the
series, then picking out the new chapters all manually, everytime, for
every series, just let a series have multiple files for those that choose
to do so and don't have an issue at all with there being multiple files.
I just need you to see that your program can be easier to use and more
useful then it already is even if you don't see the merit in it. For TV
shows it's Sonarr, for comics Mylar3, Manga FMD2, all of those things you
just pick a series and when the new thing comes out it automatically grabs
them because it would take a huge amount of more work to do it by hand. Its
my view that in order to enable this on your program would be harder than
those I listed but even saying that its still worth it. In the end its your
choice but please consider it because as far as I found your the only
person who is really doing this
…On Mon, Jul 6, 2020 at 3:18 PM Flameish ***@***.***> wrote:
The point of NG is to be able to read a webnovel conveniently on your
eReader of choice. They all work with EPUBs so it's the perfect format to
use for this. It's just not ideal for when you are caught up with a running
novel. I think people only download new chapters in larger chunks when they
have amassed a certain size of backlog and not short single chapters, I
personally don't see the point of that. The only application I can think of
would be to download new chapters of various different novels and put them
together in a single file to get a comfortable reading experience.
An indicator to show the last downloaded chapter of a novel has merit
through.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTUGYAU7AFPATQSFQM3R2IPO7ANCNFSM4OQQUCWA>
.
|
|
Okay. How about this: |
|
So to make sure I understand: Last week I bookmarked a series so now I go
to the favorites tab and click check for updates. It will then check if
there are any new chapters and if there are if those chapters amount to 5
or more (using your example for before) that '5' can also be raised or
lowered based on what the user wants. It does in fact have enough chapters
so the program downloads the new chapters then combines them with the old
file of the original chapters you already had to create a new file. But a
user also has the option on favorites tab to just enable auto updates and
every so often (say once a day) the program will scan the series
automatically and if there are ANY chapters new or whatever custom amount
the user wants will download and create the new file. Also you have the
option of an email notification every so often about what series were
updated and what was added. Is my understanding correct?
…On Mon, Jul 6, 2020 at 4:27 PM Flameish ***@***.***> wrote:
Okay. How about this:
On the automatic tab, after you have fetched the novel info, you get the
option to save/favourite it. This will add it to the library (a new tab).
For each novel you get the basic meta info and the chapter number of the
last downloaded chapter along with the number of the newest online. It will
check the web page in regular intervals for updates. There will be a button
to download all elapsed chapters to get caught up again (in a single file)
and another selection to set up an auto download. Along with a save
directory, you can specify how often you want it to create a file. For
example, you only want a new one when 5 chapters were released. Maybe even
make it send an email notification(with optional attachment) when it
creates one.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTSFKW3VSGVJCSHCHO3R2IXS7ANCNFSM4OQQUCWA>
.
|
|
The 2nd half is correct.
|
|
And those novels will be listed in another tab?
…On Mon, Jul 6, 2020, 5:41 PM Flameish ***@***.***> wrote:
The 2nd half is correct.
- The library lists the info of all added novels and polls the
websites for updates in the background every so often.
- You get a 'manual' button to download all new chapters of a novel,
from the previous last downloaded to the newest released combined in 1 file.
- Then you can also (optional) set the novel to auto downloading and
tell it to get every 5 chapters, it will create a new file with just those
5 new chapters and sends you a notification or the whole file per email or
something. Over the next week the author releases another 5 new ones, now
NG creates a new file with just those 5.
Like you mentioned, this option is for people who want the newest
chapters right away.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTV4LMAX5E2XI5RTCYLR2JAJXANCNFSM4OQQUCWA>
.
|
|
For the gui I was thinking along the lines of this with a popup window to setup up the auto downloading.
|
|
So that mock up is that what people see once they put in the site url or
when there looking at there favorites tab?
…On Mon, Jul 6, 2020, 6:21 PM Flameish ***@***.***> wrote:
For the gui I was thinking along the lines of this with a popup window to
setup up the auto downloading.
[image: Untitled]
<https://user-images.githubusercontent.com/47372220/86655850-530e0600-bfe7-11ea-980e-fc2b7319eb79.png>
I would plan to support CLI too.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTT76WM2M42KYEH3HJDR2JE53ANCNFSM4OQQUCWA>
.
|
|
This would be the library/favorites tab yes. I would probably add a new 'star' button on the automatic grabbing tab to add that novel. |
|
I okay, so first you search for a series on a site then add it to your
favorites. Then you can just check a box for it download the latest chapter
instead of having to re search for it each time. Now that's a lot easier,
how hard will it be to code the auto download and track system
Also have any idea about the wuxiaworld issue?
…On Mon, Jul 6, 2020, 6:27 PM Flameish ***@***.***> wrote:
This would be the library/favorites tab yes. I would probably add a new
star button on the automatic grabbing tab to add that novel.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTRZ5OYGCEDCKMI3DK3R2JFWDANCNFSM4OQQUCWA>
.
|
|
Should be pretty easy. I did quite a bit of code refactoring for the CLI update which I can now use much easier for this and I had a polling system in place before but removed it. No idea about the email part but I think it will be fun to figure out. I don't want to make any promises but it should be done in a few days depending if I find the time. Wuxiaworld.com works perfectly fine for me. Can you post some console log etc? |
|
Seems like the wuxiaworld issue was on my end. The cloudfare defence
blocked my vpvn, I switched locations till I found one that worked.
About what you were saying, did you mean the auto downloading will take a
few days or the email system
…On Mon, Jul 6, 2020, 6:41 PM Flameish ***@***.***> wrote:
Should be pretty easy. I did quite a bit of code refactoring for the CLI
update which I can now use much easier for this and I had a polling system
in place before but removed it. Now about the email part but I think it
will be fun to figure out. I don't wanna make any promises but it should be
done in a few days depending if I find the time.
Wuxiaworld.com works perfectly fine for me. Can you post some console log
etc?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTRLUTQAXFO2YB54F6TR2JHJ3ANCNFSM4OQQUCWA>
.
|
|
Both will be part of the library "system". |
|
Alright then, I got a VM set up and looking forward to hearing when the
system is updated to test it out
…On Tue, Jul 7, 2020, 5:02 AM Flameish ***@***.***> wrote:
Both will be part of the library "system".
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTRHYWMUMQAB5RTZ3FTR2LQEDANCNFSM4OQQUCWA>
.
|
|
Check out v2.8.0 |
|
Starting to use it now, I assume with the default threshold of 0 that means
as soon as there is a new chapter it will download but where do I go to
change how often a series is scanned for new chapters
…On Wed, Jul 8, 2020, 8:05 PM Flameish ***@***.***> wrote:
Check out v2.8.0
<https://github.com/Flameish/Novel-Grabber/releases/tag/2.8.0>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTVKLSGCTAR54UULNWDR2UCU5ANCNFSM4OQQUCWA>
.
|
|
Can there be an option to auto create a folder for a series with the same
name you choose for the epub
…On Wed, Jul 8, 2020 at 8:05 PM Flameish ***@***.***> wrote:
Check out v2.8.0
<https://github.com/Flameish/Novel-Grabber/releases/tag/2.8.0>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTVKLSGCTAR54UULNWDR2UCU5ANCNFSM4OQQUCWA>
.
|
|
Threshold 1 means it there is a new chapter. 0 means you're up-to-date. |
|
Then there is an issue, it goes to the correct path to send the books but
it does not create a folder. So instead of main folder/new folder it just
dumps them in main folder. Also the naming either needs spaces or dotes
between each word
…On Thu, Jul 9, 2020, 3:10 AM Flameish ***@***.***> wrote:
Threshold 1 means it there is a new chapter. 0 means you're up-to-date.
It does creat a folder for the file but the whole naming convention needs
a bit more fine tuning. I will probably add a library settings tab to
change stuff like this. Same goes for the scan frequency. (its 20 minutes
at the moment)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTULF2C7B4U5B5S22KDR2VUPXANCNFSM4OQQUCWA>
.
|
|
I'm taking a look at the save location, it should be: ../ Novel-Grabber/<title>/file.epub |
|
Can you set up it so once you start downloading a series it auto creates a
folder for it?
…On Thu, Jul 9, 2020, 5:53 AM Flameish ***@***.***> wrote:
I'm taking a look at the save location, it should be: ../
Novel-Grabber/<title>/file.epub
I had to remove the spaces in the filename to be able to send them per
email. Gonna fix it for non-auto grabbing to have spaces again.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTTN7TJRVXVGQ7M3DRTR2WHSJANCNFSM4OQQUCWA>
.
|
|
Isn't this what you mean ../ Novel-Grabber/<title>/file.epub ? Or do you mean each epub will be put into its own subdirectory again? |
|
Its own. You set a folder location and each time a new series is downloded
another folder inside the original one is created
…On Thu, Jul 9, 2020, 10:11 AM Flameish ***@***.***> wrote:
Isn't this what you mean ../ Novel-Grabber/<title>/file.epub ? Or do you
mean each epub will be put into its own subdirectory again?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTRVRTDCDQR5TVZOMCLR2XFXRANCNFSM4OQQUCWA>
.
|
|
I'm a bit confused. Are you talking about different novels getting their own folder (because they do) or the auto-downloaded files OF a novel get their own sub directories inside its "series" folder? Currently the structure looks like this:
And this is how it would look like if I understand you correctly:
How would the 2nd be better than the first? |
|
My issue is the program does not currently create folders
…On Thu, Jul 9, 2020, 10:42 AM Flameish ***@***.***> wrote:
I'm a bit confused. Are you talking about different novels get their own
folder (because they do) or the auto-downloaded files OF a novel get their
own sub directories inside its "series" folder?
Currently the structure looks like this:
[image: Screenshot from 2020-07-09 16-37-10]
<https://user-images.githubusercontent.com/47372220/87053643-811b6200-c202-11ea-82ec-820704d94544.png>
[image: Screenshot from 2020-07-09 16-37-29]
<https://user-images.githubusercontent.com/47372220/87053652-824c8f00-c202-11ea-8ea6-0f50725bc150.png>
And this is how it would look like if I understand you correctly:
[image: Screenshot from 2020-07-09 16-39-39]
<https://user-images.githubusercontent.com/47372220/87054050-f8e98c80-c202-11ea-9575-94981b3096ce.png>
[image: Screenshot from 2020-07-09 16-40-08]
<https://user-images.githubusercontent.com/47372220/87054059-fab35000-c202-11ea-95f8-f8b97837ca64.png>
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTQ3SLFQYDHAQKYQYTLR2XJNJANCNFSM4OQQUCWA>
.
|
|
Will be fixed in the next release. First screenshots are from 2.8.1 |
|
Good to hear, is it possible to change the output from epub to PDF?
…On Thu, Jul 9, 2020, 11:19 AM Flameish ***@***.***> wrote:
Will be fixed in the next release. First screenshots are from 2.8.1
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTRK4Q5K2PDOAQ7JAUDR2XNXVANCNFSM4OQQUCWA>
.
|
|
I would need to take a look at available pdf libraries. Not sure how compatible they are with HTML(some even broken #23) |
|
Hello, I'd like some clarifications, the saving of files in each folder, is this for all downloads? Or only those items on the the favorite folder and tagged for auto-updates? |
|
Only for favourites tagged for auto-grabbing right now. |
|
Thank you. I'd like to request to keep it that way as I do not see the benefit of creating different folders for single epubs. |
|
It makes it easier for files organization. But as long as there is a box
you can click giving you an option to create a folder for a series or not
that will make sure we both get what we are looking for
…On Thu, Jul 9, 2020, 2:09 PM zpereyra ***@***.***> wrote:
Thank you. I'd like to request to keep it that way as I do not see the
benefit of creating different folders for single epubs.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTUFFXUYD474TSKHL4DR2YBUPANCNFSM4OQQUCWA>
.
|
|
I am pertaining for those of us who do not download mangas, comics or even series or follow/download novels while they are on-going. That is why I requested to just keep the creation of file folders to those who put it in their config. For book library organization or file format changes I have been a user of calibre for years. A lot of novel compilers use it within their program. In fact, for the things you have asked for, I can think of 2 available programs that is doing it and works in cli or within calibre as a plugin. The reason I am here in NG is because of its ability in java enabled sites and configuring the grabber on your own for different websites. The ease of use and simplicity of the program is a big plus for me. And I really appreciate the author's efforts in integrating the cli. |
|
Can we get a tab or some area where we can see what has been downloaded or
is that already there?
…On Thu, Jul 9, 2020, 10:50 PM zpereyra ***@***.***> wrote:
I am pertaining for those of us who do not download mangas, comics or even
series or follow/download novels while they are on-going. That is why I
requested to just keep the creation of file folders to those who put it in
their config.
For book library organization or file format changes I have been a user of
calibre for years. A lot of novel compilers use it within their program. In
fact, for the things you have asked for, I can think of 2 available
programs that is doing it and works in cli or within calibre as a plugin.
The reason I am here in NG is because of its ability in java enabled sites
and configuring the grabber on your own for different websites. The ease of
use and simplicity of the program is a big plus for me. And I really
appreciate the author's efforts in integrating the cli.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTSAQCEZNSWMLHRMBX3R2Z6YHANCNFSM4OQQUCWA>
.
|
|
There is no history right now. Only if you enabled Email notifications I guess. But thats a good idea. Will add it. |
|
If I would like a website added do I tell you here or do I create a new
ticket like when I asked for a PDF option?
…On Fri, Jul 10, 2020, 9:25 AM Flameish ***@***.***> wrote:
There is no history right now. Only if you enabled Email notifications I
guess. But thats a good idea. Will add it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTSXXVZLMX3YIOFX7NTR24JDHANCNFSM4OQQUCWA>
.
|
|
Make a new issue or send me an email directly. Be advised that the website has to have 10+ novels on it. |
|
Not a problem, seems to have a few hundred
…On Fri, Jul 10, 2020, 9:34 AM Flameish ***@***.***> wrote:
Make a new issue or send me an email directly. Be advised that the website
has to have 10+ novels on it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#36 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKDLZTUNQDWR34IRZMMPL73R24KFVANCNFSM4OQQUCWA>
.
|
|
I am still having trouble with both downloading and folder creation. There were about a weeks worth of chapters that had come out that did not download until I manually checked a book series, then it seemed the rest updated. Also new chapters are not going into folders, here is what I see: https://imgur.com/gallery/0rjD1On As for an example for the paths provided: Z:\Chinese\Overgeared |
|
hiiii, how can i download the program? i've been trying to download but i doesn't work, i don't know what i am doing wrong, can someone explain me? |
|
I'm not sure how you found this specific issue. (It's about a library feature to bookmark novels). Please read the Readme for instructions or open a new issue if you still have questions. |
Like FMD/FMD2 you search for a series then add it a list where every so often it will look for new chapters
The text was updated successfully, but these errors were encountered: