Implement line-ending recognition and normalization #353
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a companion PR to lapce/lapce#3047
This implement basic line-ending support.
Fixes #3029 because I was focusing on that issue and I decided to just implement line-ending changing while I was at it.
The cause of that issue is that x-rope does not consider Carriage Return (
\r
) as a newline character, but cosmic-text does (and it panics if there is more than one paragraph for our text layouts). CRs in terminals are meant to go to the start of the current line, but old MacOS used it as a line-ending character; as well, other editors like VSCode/GEdit/... treat it as a newline character. We imitate other editors which simply replace a lone\r
character with the line-ending of the file.However, on loading content the editor does not normalize the line-endings, because for large files that could be a lot of replacements. So there is no standardized in-memory newline that we use, which I believe is fine as I don't believe anything cares about that (especially since that was the editor's behavior for the last ~while). The only normalization is with
normalize_limited
which just gets rid of lone CRs since cosmic-text can't handle it.The primary alternative solution to replacing all lone CRs would be to modify
Rope
to consider it as a newline character, but at a glance that appears nontrivial and adds extra work to many common operations. Another alternative would be to treat\r
as a whitespace character and modify cosmic-text to treat it as such. That option is inconsistent with other editors treating it as a newline, however, so I dismissed it as an option.