charset 1251 problem

[sakirsa]

Hi. I encountered a problem with the encoding. Cyrillic does not display, how to collect data in the encoding 1251. tryed to use iconv, but Fatal error: Uncaught TypeError: iconv(): Argument #3 ($string) must be of type string, Embed\Extractor given

[oscarotero]

Can I have an URL to reproduce this?

[sakirsa]

https://4pda.to/2022/12/04/406834/sostoyalsya_reliz_clown_of_duty_parodii_na_call_of_duty/

[sakirsa]

https://feedster.ru/emb.php to check if cloudflare blocks

[parrycarry]

Oooo, I too would like to see a solution for this. The Cyrillic is completely voided when the page isn't UTF-8.

[oscarotero]

Embed uses this parser to convert the HTML to DOMDocuments and deal with encoding issues.

The function is used here. I've forced the encoding in the second argument and it works fine:

$this->document = !empty($html) ? Parser::parse($html, "windows-1251") : new DOMDocument();

So probably the automatic encoding detection of the parser doesn't work fine. Not sure how to fix it and I don't have much time to dig into this.

If anyone wants to work on a PR to fix it, it's very appreciated.

[parrycarry]

I haven't thoroughly tested this... but $detected = $encoding ?? mb_detect_encoding($html, NULL, true); seems to fix the problem.... The Windows-1251 Cyrillic shows up as well as UTF-8 Cyrillic no problem...