add batch func

[zengzzzzz]

please review it, thanks.
1 not use the self.new_epub , so del it
2 add batch func with multiprocessing

[yihong0618]

@zengzzzzz thanks will take a look tonight

[yihong0618]

Looks very good and clean for now, will test in my env tonight.

Thanks a lot.

[yihong0618]

@zengzzzzz
可能报这个错我找了一本书
下载链接：https://cloudflare-ipfs.com//ipfs/bafk2bzacednqyfwpaydgs73roibihglile7toce4iiwcjvudjxfvmkejcu7ks?filename=%E6%AA%B8%E6%AA%AC_%E6%A2%B6%E4%BA%95%20%E5%9F%BA%E6%AC%A1%E9%83%8E%20%5B%E6%A2%B6%E4%BA%95%20%E5%9F%BA%E6%AC%A1%E9%83%8E%5D.epub

[yihong0618]

有个地方有一点错误，我在 main 上改了哈，你可以先合并一下。

[zengzzzzz]

有个地方有一点错误，我在 main 上改了哈，你可以先合并一下。

好的。麻烦了哈。谢谢。

[zengzzzzz]

我应该没有权限合并分支。麻烦你帮忙合一下哈。

[yihong0618]

@zengzzzzz 我 fix 了，你拉下代码看看我 comment 那个 herf 问题。

[zengzzzzz]

@zengzzzzz 我 fix 了，你拉下代码看看我 comment 那个 herf 问题。

ok

[zengzzzzz]

@yihong0618 I fix the herf error， please check it again，thanks。

[yihong0618]

@zengzzzzz I feel that some restrictions should be added, now running in my computer will be directly stuck

[yihong0618]

using test_books/lemo.epub to test find some problem.

1. the whole terminal stuck(I am using kitty)
2. seems the same speed with not batch
3. error found
   

[yihong0618]

IMO we'd better use the <p1> + <p2> + <p3> -> chatgpt api -> <p1_t> <pt_2> <p3_t> batch way that every user can go well.

[zengzzzzz]

IMO we'd better use the <p1> + <p2> + <p3> -> chatgpt api -> <p1_t> <pt_2> <p3_t> batch way that every user can go well.

I will try this way
1 will add restrictions (eg: max workers) to avoid stuck
2 tested on my machine, it is faster, but its speed depends on the response of the api
3 this seems to be an error generated by openai, I will verify it

[zengzzzzz]

IMO we'd better use the <p1> + <p2> + <p3> -> chatgpt api -> <p1_t> <pt_2> <p3_t> batch way that every user can go well.

you are right . If we want everyone to use it well, we should try the method you mentioned.

[zengzzzzz]

IMO we'd better use the <p1> + <p2> + <p3> -> chatgpt api -> <p1_t> <pt_2> <p3_t> batch way that every user can go well.

I think the method you mentioned has these advantages
1 fewer tokens, less cost
2 not easy to get stuck
3 everyone can go well

[zengzzzzz]

I implemented the batch translate method, but there are still some flaws in the format, so I decided to use it in my own project. If you are interested or needed, you can refer to the following implementations:

def translate_book(self):
        new_book = epub.EpubBook()
        new_book.metadata = self.origin_book.metadata
        new_book.spine = self.origin_book.spine
        new_book.toc = self.origin_book.toc
        batch_p = []
        batch_count = 0
        for i in self.origin_book.get_items():
            if i.get_type() == 9:
                soup = bs(i.content, "html.parser")
                p_list = soup.findAll("p")
                for p in p_list:
                    if p.text and not p.text.isdigit():
                        batch_p.append(p)
                        batch_count += 1
                        if batch_count == self.batch_size:
                            translated_batch = self.translate_model.translate([p.text for p in batch_p])
                            for j, c_p in enumerate(batch_p):
                                c_p.string = c_p.text + translated_batch[j]
                            batch_p = []
                            batch_count = 0
                    # Process any remaining paragraphs in the last batch
                if batch_p:
                    translated_batch = self.translate_model.translate([p.text for p in batch_p])
                    for j, c_p in enumerate(batch_p): 
                        c_p.string = c_p.text + translated_batch[j]
                    batch_p = []
                    batch_count = 0
                i.content = soup.prettify().encode()
            new_book.add_item(i)
        name = self.epub_name.split(".")[0]
        epub.write_epub(f"{name}_translated.epub", new_book, {})

[yihong0618]

thanks will keep this pr if I use your code I will commit on this.
Thanks again

[zengzzzzz]

thanks will keep this pr if I use your code I will commit on this. Thanks again

thank you, bro

[DennySORA]

我覺得，這部分有些能改成 async 的模式，再加上你的這個。

畢竟目前大多數的瓶頸都是在 openAI API 跟 network 上。

[zengzzzzz]

我覺得，這部分有些能改成 async 的模式，再加上你的這個。

畢竟目前大多數的瓶頸都是在 openAI API 跟 network 上。

you are probably right . a reason for not using async or multprocessing etc is the qps limitation of the open ai api ( 20 per minute) , which makes its performance improvement not obvious.

[tianshanghong]

you are probably right . a reason for not using async or multprocessing etc is the qps limitation of the open ai api ( 20 per minute) , which makes its performance improvement not obvious.

Looks that the rate limit from OpenAI may not be a problem now. For Pay-as-you-go-user (after 48 hours), OpenAI allows 3500 requests per minute and 90,000 tokens per minute.

I created a new PR #62 to support batch processing with asyncio.