Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert not working after upgrading to Calibre-web 0.6.22 #3100

Closed
aab3r opened this issue Jul 9, 2024 · 13 comments
Closed

Convert not working after upgrading to Calibre-web 0.6.22 #3100

aab3r opened this issue Jul 9, 2024 · 13 comments

Comments

@aab3r
Copy link

aab3r commented Jul 9, 2024

Describe the bug/problem

Convert does not work after upgrading to Calibre-web 0.6.22.

Path to Calibre Binaries is correct (/opt/calibre) and indeed, converting from calibre-server works fine.

To Reproduce

Steps to reproduce the behavior:
Converting a books always throws an error.

Logfile


[2024-07-08 14:22:11,048]  INFO {cps.editbooks:320} converting: book id: 1977 from: EPUB to: MOBI
[2024-07-08 14:22:11,053]  INFO {cps.tasks.convert:159} Book id 1977 - target format of .mobi does not exist. Moving forward with convert.
[2024-07-08 14:22:11,738]  INFO {cps.tasks.convert:206} ebook converter failed with error while converting book
[2024-07-08 14:22:11,738] ERROR {cps.tasks.convert:210} Calibre failed with error: ValueError: Empty file: /tmp/calibre_web/metadata_44ed7c43-2e44-4795-be09-2f499330e114.opf

Environment (please complete the following information):

  • OS: Debian bookworm
  • Python version: 3.11.2-1+b1
  • Calibre-Web version: 0.6.22
  • Docker container: None
  • Special Hardware: None
  • Browser: Any
@aab3r
Copy link
Author

aab3r commented Jul 9, 2024

So, if I stop calibre-server, everything works fine... I don't get it, I've always had calibre-server and calibre-web working together without issues...

Is there a change in this version that makes them not compatible?

@OzzieIsaacs
Copy link
Collaborator

Could you please set the log level to debug and check it again.

Is there a change in this version that makes them not compatible?

maybe. Seems you have activated embed metadata on conversion/download… maybe this interferes somehow

@aab3r
Copy link
Author

aab3r commented Jul 10, 2024

Could you please set the log level to debug and check it again.

I'm afraid I don't know how to do that, but I'm willing to follow instructions. I can see there's a "Download debug package" in Administration, but it gives the same info as normal log files.

maybe. Seems you have activated embed metadata on conversion/download… maybe this interferes somehow

Again, if I've done that, I don't know how I've done it. If you could tell me how to deactivate that feature, I can try again and share the logs.

Thanks four your help!

@OzzieIsaacs
Copy link
Collaborator

Debug log: admin section -> basic configuration -> logfile configuration -> Log Level -> DEBUG
activated embed metadata admin section -> basic configuration -> Feature Configuration -> Embed Metadata to Ebook File on Download/Conversion/e-mail -> switch to "not selected"

@aab3r
Copy link
Author

aab3r commented Jul 10, 2024

Seems you have activated embed metadata on conversion/download… maybe this interferes somehow

Yes! That was it. As soon as I disable that option, converting books works fine again.

Anyway, in case it is useful, these are the debug logs with "embed metadata..." enabled:

[2024-07-10 20:12:53,486]  INFO {cps.editbooks:320} converting: book id: 1923 from: EPUB to: MOBI
[2024-07-10 20:12:53,488] DEBUG {cps.services.worker:91} Add Task for user: (user) - Convert Book 1923
[2024-07-10 20:12:53,491]  INFO {cps.tasks.convert:159} Book id 1923 - target format of .mobi does not exist. Moving forward with convert.
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312} Traceback (most recent call last):
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "runpy.py", line 198, in _run_module_as_main
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "runpy.py", line 88, in _run_code
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "site.py", line 47, in <module>
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "site.py", line 43, in main
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "calibre/ebooks/conversion/cli.py", line 422, in main
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "calibre/ebooks/conversion/plumber.py", line 1066, in run
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "calibre/ebooks/conversion/plumber.py", line 1010, in setup_options
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "calibre/ebooks/conversion/plumber.py", line 969, in read_user_metadata
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "calibre/ebooks/metadata/opf2.py", line 636, in __init__
[2024-07-10 20:12:54,245] DEBUG {cps.tasks.convert:312}   File "calibre/ebooks/metadata/utils.py", line 44, in parse_opf
[2024-07-10 20:12:54,246] DEBUG {cps.tasks.convert:312} ValueError: Empty file: /tmp/calibre_web/metadata_4e234b0c-0885-425b-9055-119c8d11f85f.opf
[2024-07-10 20:12:54,246]  INFO {cps.tasks.convert:206} ebook converter failed with error while converting book
[2024-07-10 20:12:54,246] ERROR {cps.tasks.convert:210} Calibre failed with error: ValueError: Empty file: /tmp/calibre_web/metadata_4e234b0c-0885-425b-9055-119c8d11f85f.opf

Thanks a lot for your help!!

@OzzieIsaacs
Copy link
Collaborator

OzzieIsaacs commented Jul 11, 2024

Calibre is locking the database, so there is no workaround. The code now directly outputs the calibre error message:

Another calibre program such as calibre-server or the main calibre program is running. Having multiple programs that can make changes to a calibre library running at the same time is a bad idea. calibredb can connect directly to a running calibre Content server, to make changes through it, instead. See the documentation of the --with-library option for details.

@beedaddy
Copy link
Contributor

Calibre is looking the database, so there is no workaround. The code now directly outputs the calibre error message:

Another calibre program such as calibre-server or the main calibre program is running. Having multiple programs that can make changes to a calibre library running at the same time is a bad idea. calibredb can connect directly to a running calibre Content server, to make changes through it, instead. See the documentation of the --with-library option for details.

I also suffer from the problem running calibre-server in parallel. But does the message mean, that using the --with-library option together with the URL of the running calibre-server should work?

@OzzieIsaacs
Copy link
Collaborator

You need to deactivate the embed metadata option, otherwise there is nothing you can do

@beedaddy
Copy link
Contributor

You need to deactivate the embed metadata option, otherwise there is nothing you can do

I understand that this is currently the case. But, if I understand it correctly, calibredb allows to connect directly to a running calibre Content server if one gives the URL of that calibre server instance. So, currently this is not possible with calibre-web, but it might be a future enhancement? For example, one could configure the URL, which is used as a fallback in the case that the database is locked.

@OzzieIsaacs
Copy link
Collaborator

Okay, now I got it, I‘ll check

@beedaddy
Copy link
Contributor

A very quick and very dirty test seems to work in my case:

--- embed_helper.py.240716	2024-07-16 09:04:17.511906420 +0200
+++ embed_helper.py	2024-07-16 09:04:56.515297154 +0200
@@ -37,7 +37,7 @@
             my_env['CALIBRE_OVERRIDE_DATABASE_PATH'] = os.path.join(config.config_calibre_dir, "metadata.db")
             library_path = config.config_calibre_split_dir
         else:
-            library_path = config.config_calibre_dir
+            library_path = "http://localhost:8080"
         opf_command = [calibredb_binarypath, 'export', '--dont-write-opf', '--with-library', library_path,
                        '--to-dir', tmp_dir, '--formats', book_format, "--template", "{}".format(temp_file_name),
                        str(book_id)]

@OzzieIsaacs
Copy link
Collaborator

I thought a while on it and came to the conclusion that I will not change it. As you mentioned it's an easy fix for you, but for me there are a lot of more use cases to consider:
Sometimes available and sometimes not available servers, servers with activated user login activated, reverse proxys in between, things we can not imagine...
"Https encrypted connections" -> This will attract pen testers to verify that the certificates are really valid, ...
It would need additional configuration settings (at least server ip and port address), this will be confusing for some users (as always), if they need a running calibre server.
So my suggestion is: do the change yourself, I will not do it, as I'm not conviced that this is a common usecase and looks to me like cause a lot f trouble for a long time (I had this with other easy looking features in the past)

@aab3r
Copy link
Author

aab3r commented Jul 24, 2024

Just in case it is useful for anyone reading this, I've found a workaround.

My use case: Calibre-web serves its contents to 10 friends who, from time to time, want to upload a bunch of files. As doing it through Calibre-web, one by one, would be tedious, I had Calibre-server running alongside, which imports all the uploaded files by my friends once a day.

What I've done: besides disabling "embed metadata on conversion/download" (thanks), now Calibre-web is running 24/7 and Calibre-server is disabled.

Then, I've created a cron job that, at 1:00 AM, stops cps.service, starts calibre-server, sleeps for 5 seconds, and then imports all the files from the designed folder (a nextcloud share). After that, the files are deleted from the shared folder, calibre-server is stopped and calibre-web is started again. Something like this:

00 1 * * * /usr/bin/systemctl stop cps.service && /usr/bin/systemctl start calibre-server.service && sleep 5 && /usr/bin/sudo -u calibre /opt/calibre/calibredb add /var/lib/nextcloud/shared_folder/ -r --automerge ignore --with-library "http://192.168.1.1:8080#Library" --username username --password password && rm -rf /var/lib/nextcloud/shared_folder/*.* && /usr/bin/systemctl stop calibre-server.service && /usr/bin/systemctl start cps.service

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants