Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow sending 50m+ files to telegram channels using MTProto #37

Closed
wants to merge 1 commit into from
Closed

Allow sending 50m+ files to telegram channels using MTProto #37

wants to merge 1 commit into from

Conversation

paskal
Copy link
Collaborator

@paskal paskal commented Jun 17, 2021

This PR switches the Telegram connector from Bot API to the Core API, resolves #23 and likely closes #31.

Before and after:
image
image

@paskal paskal changed the title unblock sending 50m+ files to telegram channel Allow sending 50m+ files to telegram channels Jun 17, 2021
@paskal paskal marked this pull request as ready for review June 18, 2021 23:26
@ernado
Copy link

ernado commented Jul 5, 2021

Hey @paskal, please consider using gotd/td as Telegram client, should be better maintained and probably more ergonomic. For example, uploader for big files is already implemented (same for entities formatting).

@paskal
Copy link
Collaborator Author

paskal commented Jul 11, 2021

I'll check it, but it's unlikely I will change the used library. The existing one already works except for HTML parsing, which I need to fix on this app side and which is unlikely solved in the library you mentioned.

@ernado
Copy link

ernado commented Jul 11, 2021

Got it, thank you for reply.

I've added comparison that can help with trade-offs of picking one or other implementation.

@paskal
Copy link
Collaborator Author

paskal commented Jul 11, 2021

I'll take a closer look into your library now, you got my attention.

@paskal
Copy link
Collaborator Author

paskal commented Jul 11, 2021

@ernado @tdakkota I see that switch to that library would require complete code rewrite and is not trivial because the only real example is gotd/bot source code. I still need to fix the incomplete HTML parsing logic I have for the currently used library; the message module doesn't seem to solve that problem for me.

If you're interested in using that library for this particular use case, I propose you take my code and rewrite it for your library and send an MR to this repo; however, I personally do not want to spend another day on a rewrite of a working code for the sake of rewrite itself.

@ernado
Copy link

ernado commented Jul 11, 2021

Ok, I'd better improve gotd examples and documentation.

Anyway it seems like switching from current library to gotd is a matter of time unless someone is going to spend several months fixing issues, introducing missing features and maintaining.

@paskal
Copy link
Collaborator Author

paskal commented Jul 11, 2021

The trick is, you don't need to fix something which is not broken, and you don't need to introduce missing features if you already have all the required features on the table. I don't see where the necessity to "spend several months fixing issues" in feed-master comes from.

I am running a feed-master from this branch with the same config as one in production for two weeks by now, and it works as reliable as the current master. The only known problems are coming not from the library but from the way I parse the incoming HTML from the RSS feed. gotd/td doesn't solve this for me as well. For the file upload in feed-master, both libraries have feature parity.

@ernado
Copy link

ernado commented Jul 11, 2021

Current library uses MTProto v1 that is deprecated and will be eventually disabled by Telegram in the future, so things can just stop working randomly, not considering that it is like using software that has open CVE.

I'm also going to quote xelaj/mtproto comment here:
https://github.com/xelaj/mtproto/blob/216789b95a5d644ebbdd1acb7eb46ff61647960a/internal/aes_ige/ige_cipher.go#L160-L161

ЭТО ЕБАНАЯ МАГИЧЕСКАЯ ФУНКЦИЯ ОНА НАХУЙ РАБОТАЕТ ПРОСТО БЛЯТЬ НЕ ТРОГАЙ ШАКАЛ ЕБАНЫЙ

If you are ok with this, I'm done here.

@umputun
Copy link
Owner

umputun commented Jul 11, 2021

let me clarify a little bit from my perspective. Sending 50+ Mb files is a nice to have feature, however, this is not that essential for the particular application. I think the time and efforts @paskal already invested into this task were a little bit too much for the end goal, but I can hardly see anyone "spend several months" to improve it or adopt any other library regardless of the ergonomics.

If v1 will be disabled by tg one day - well, so be it. Feed master can always do what it always did with the bot API, i.e. upload under 50m files only and put external links for the bigger file. And, who knows, maybe as tg kill support of v1 they will increase the upload limits.

@ernado
Copy link

ernado commented Jul 13, 2021

Alternatively, you can also host your own local bot api server (can be used instead of https://api.telegram.org), it is official and should be enough to send 50+ Mb files.

@paskal paskal marked this pull request as draft July 14, 2021 22:04
@paskal paskal changed the title Allow sending 50m+ files to telegram channels Allow sending 50m+ files to telegram channels using MTProto Jul 23, 2021
@paskal
Copy link
Collaborator Author

paskal commented Jul 25, 2021

Rebased on top of current master just to show how much more complex it is rather than using the self-hosted bot API server we choose to go with. Closing this PR.

@paskal paskal closed this Jul 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow >50M audio upload to telegram
3 participants