deline/lines misinterprets UTF-8 character as newline

[Siskin-Bot]

Submitted by: Kaj

In the Syllable websites, we have a Slovenian version. One character that is also in the native name of the language: Slovenščina

the č is interpreted as a newline by deline/lines and functions that use it, such as read/lines, so that the text is corrupted and broken up. č is UTF-8 C48D in hex. This is a common character in Slovenian, so this is a major problem in Slovenian text. I haven't tested further, so there may also be other characters with the same problem. I've also had other problems with DELINE, but haven't analysed those yet.

>> deline/lines "Slovenščina"
== [
    "Slovenš"
    "ina"
]

---

^{Imported from: CureCode [ Version: alpha 110 Type: Bug Platform: Linux x86 libc6 Category: Native Reproduce: Always Fixed-in:none ]
Imported from: metaeducation#1794}

Comments:

Rebolbot commented on Dec 12, 2010:

Submitted by: BrianH

Note: This problem does not occur on Windows (at least on XP with a Unicode console font).

This should be easy to fix, so I lowered the severity. It is important to fix so I raised the priority. There are other DELINE bugs that also need fixing.

---

Rebolbot commented on Jan 7, 2011:

Submitted by: JerryTsai

read/lines has this issue too, when the text file contains certain Chinese characters, and is in utf-8.

---

Rebolbot commented on Jan 26, 2015:

Submitted by: abolka

In the core-tests suite.

---

Rebolbot added on Jan 12, 2016

---