fix rename error if using 'LF' eol in windows

[drzunny]

Hello, DonJayamanne

I found the rename feature goes wrong when i edit a py file with LF eol on Windows(7/8/10).

but if I convert the file EOL to CRLF, everything is ok.

Some editor / IDE will keep both LF and CRLF in one file. So the numbers of character (CR) are not necessarily equal to the line numbers

Here is the effect of this patch:

[gandhis1]

Very useful - is this related to #579?

[drzunny]

@gandhis1 I'm not sure. If you are using Windows and editing python file with LF eol, try to convert the file to CRLF and use rename(F2) again, if rename correctly, maybe it's related to your issue #579 😄

[DonJayamanne]

@drzunny
You are loading the entire document as a string (document.getText()), contrary to your comments.

// In order to prevent the one-time loading of large files from taking up too much memory

You might want to use the VS Code api to read text from the document.

[drzunny]

OK, i will update my PR later, thanks for your suggestion

[DonJayamanne]

Also, if this is specific to windows, shouldn't we check the value of IS_WINDOWS and then invoke the functino getWindowsLineEncodingCount?

[drzunny]

Yes, it specific to Windows, but in this method (getOffsetAt), IS_WINDOWS has already checked before invoke the function getWindowsLineEndingCount.

[DonJayamanne]

@drzunny please could you point me to the rope code that uses LF instead of CRLF

[drzunny]

@DonJayamanne

i check the source code and find the reason in fscommands.py

def file_data_to_unicode(data, encoding=None):
    result = _decode_data(data, encoding)
    if '\r' in result:    # <-------------------- here
        result = result.replace('\r\n', '\n').replace('\r', '\n')
    return result

The filepath will be read as a File object (rope/base/resources.py), in its read method:

    def read(self):
        data = self.read_bytes()
        try:
            return fscommands.file_data_to_unicode(data)
        except UnicodeDecodeError as e:
            raise exceptions.ModuleDecodeError(self.path, e.reason)

it read binary data, and convert to unicode AND REPLACE THE \r.