Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return to ground when we flush the last char #2823

Merged
merged 4 commits into from
Oct 4, 2019

Conversation

zadjii-msft
Copy link
Member

Summary of the Pull Request

The InputStateMachineEngine was incorrectly not returning to the ground state after flushing the last sequence. That means that something like alt+backspace would leave us in the Escape state, not the ground state. This makes sure we return to ground.

Additionally removes the "Parser.UnitTests-common.vcxproj" file, which was originally used for a theoretical time when we only open-sourced the parser. It's unnecessary now, and we can get rid of it.

Also includes a small patch to bcz.cmd, to make sure bx works with projects with a space in their name.

PR Checklist

  The InputStateMachineEngine was incorrectly not returning to the ground state
  after flushing the last sequence. That means that something like alt+backspace
  would leave us in the Escape state, not the ground state. This makes sure we
  return to ground.

  Fixes #2746.

  Additionally removes the "Parser.UnitTests-common.vcxproj" file, which was
  originally used for a theoretical time when we only open-sourced the parser.
  It's unnecessary now, and we can get rid of it.

  Also includes a small patch to bcz.cmd, to make sure bx works with projects
  with a space in their name.
@zadjii-msft zadjii-msft added Issue-Bug It either shouldn't be doing this or needs an investigation. Product-Conhost For issues in the Console codebase labels Sep 20, 2019
Copy link
Contributor

@DHowett-MSFT DHowett-MSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it always correct to return to ground here? Are the individual _ActionXxxDispatches responsible for doing it? If so, why duplicate it here? If not, should they be?
Can you conceive of a case where this is wrong?

@zadjii-msft
Copy link
Member Author

@DHowett-MSFT None of the Action* methods modify the state, they only ever call the appropriate methods on the engine with the appropriate args. Typically, the Event* methods are the ones that actually modify the state. This block is basically just the collected Dispatch parts of each of those states, and it should definitely have this. I can't think of a case where when we flush like this, we'd want to leave it in the not-ground state. The whole point of the flushing thing is to be able to handle weird partial state VT input sequences, like \x1b\x7f or \x1b\x1b, that would normally not actually get dispatched on their own and would pollute the parser state like this.

src/terminal/parser/stateMachine.cpp Outdated Show resolved Hide resolved
Co-Authored-By: Dustin L. Howett (MSFT) <[email protected]>
Copy link
Member

@miniksa miniksa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify why manually executing these things for the input machine is required? I traced around to your comments on InputStateMachineEngine::FlushAtEndOfString() and I'm still unclear why this is a special custom case here.

@zadjii-msft
Copy link
Member Author

Can you clarify why manually executing these things for the input machine is required? I traced around to your comments on InputStateMachineEngine::FlushAtEndOfString() and I'm still unclear why this is a special custom case here.

Sure thing.

One of the "weird things" in VT input is the case of something like alt+[. In VT, that's encoded as \x1b[. However, that's also the start of a CSI, and could be the start of a longer sequence, there's no way to know for sure. For an alt+[ keypress, the parser originally would just sit in the CsiEntry state after processing it, which would pollute the following keypress (e.g. alt+[, A would be processed like \x1b[A, which is wrong).

Fortunately, for VT input, each keystroke comes in as an individual write operation. So, if at the end of processing a string for the InputEngine, we find that we're not in the Ground state, that implies that we've processed some input, but not dispatched it yet. This block at the end of ProcessString will then re-process the undispatched string, but it will ensure that it dispatches on the last character of the string. For our previous \x1b[ scenario, that means we'll make sure to call _ActionEscDispatch('[')., which will properly decode the string as alt+[.

@DHowett-MSFT
Copy link
Contributor

How will this impact partial writing of VT sequences (\x1b[1 ... ;1H)? Was that something we supported before?

@zadjii-msft
Copy link
Member Author

@DHowett-MSFT Fortunately, that still works for the OutputStateMachineEngine. Each engine implements FlushAtEndOfString differently, so the OutputEngine lets the parser stay buffered, while this change only applies to the InputEngine

@DHowett-MSFT
Copy link
Contributor

Hmm. hmm. I was thinking that this might be fine since it's on Input (my bad: i forgot this wasn't output), but consider the case where SSH is buffering and the user holds down right arrow a whole bunch. Maybe it gets a sequence split across a packet?

@zadjii-msft
Copy link
Member Author

That's something that was weighed in this decision originally. Unfortunately there's no way to make both scenarios work, and the ssh scenario is frankly less common. There's just no other way to differentiate between part of an arrow key press and alt+[.

@zadjii-msft zadjii-msft added the Needs-Second It's a PR that needs another sign-off label Sep 24, 2019
@miniksa
Copy link
Member

miniksa commented Oct 3, 2019

Can you clarify why manually executing these things for the input machine is required? I traced around to your comments on InputStateMachineEngine::FlushAtEndOfString() and I'm still unclear why this is a special custom case here.

Sure thing.

One of the "weird things" in VT input is the case of something like alt+[. In VT, that's encoded as \x1b[. However, that's also the start of a CSI, and could be the start of a longer sequence, there's no way to know for sure. For an alt+[ keypress, the parser originally would just sit in the CsiEntry state after processing it, which would pollute the following keypress (e.g. alt+[, A would be processed like \x1b[A, which is wrong).

Fortunately, for VT input, each keystroke comes in as an individual write operation. So, if at the end of processing a string for the InputEngine, we find that we're not in the Ground state, that implies that we've processed some input, but not dispatched it yet. This block at the end of ProcessString will then re-process the undispatched string, but it will ensure that it dispatches on the last character of the string. For our previous \x1b[ scenario, that means we'll make sure to call _ActionEscDispatch('[')., which will properly decode the string as alt+[.

Can you leave some portion of this clarification in the code right there as a comment?

It's way clearer when you explain the Alt+[ causing it to sit in the wrong state than it is by simple inspection.

@miniksa
Copy link
Member

miniksa commented Oct 4, 2019

add the comment @miniksa wanted

IT IS BEAUTIFUL. Thanks.

Copy link
Member

@miniksa miniksa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great. Thanks.

@zadjii-msft zadjii-msft merged commit 53c81a0 into master Oct 4, 2019
DHowett-MSFT pushed a commit that referenced this pull request Oct 17, 2019
The InputStateMachineEngine was incorrectly not returning to the ground
state after flushing the last sequence. That means that something like
alt+backspace would leave us in the Escape state, not the ground state.
This makes sure we return to ground.

Additionally removes the "Parser.UnitTests-common.vcxproj" file, which
was originally used for a theoretical time when we only open-sourced the
parser. It's unnecessary now, and we can get rid of it.

Also includes a small patch to bcz.cmd, to make sure bx works with
projects with a space in their name.

(cherry picked from commit 53c81a0)
DHowett-MSFT pushed a commit that referenced this pull request Oct 17, 2019
- Fix snapping to the cursor in "Terminal Scrolling" mode (GH-2705)

fixes GH-1222

  PSReadline calls SetConsoleCursorPosition on each character. We try to snap.

  However, we'd only ever do this with the visible viewport, the viewport
  defined by `SCREEN_INFORMATION::_viewport`. When there's a virtual viewport in
  Terminal Scrolling mode, we actually need to snap the virtual viewport, so
  that this behavior looks more regular.

(cherry picked from commit 6f4b98a)
- Passthrough BEL in conpty (GH-2990)

fixes GH-2952.

(cherry picked from commit 6831120)
- make filling chars (and, thus, erase line/char) unset wrap (GH-2831)

EraseInLine calls `FillConsoleOutputCharacterW()`. In filling the row with
chars, we were setting the wrap flag. We need to specifically not do this on
ANY _FILL_ operation. Now a fill operation UNSETS the wrap flag if we fill to
the end of the line.

Originally, we had a boolean `setWrap` that would mean...
- **true**: if writing to the end of the row, SET the wrap value to true
- **false**: if writing to the end of the row, DON'T CHANGE the wrap value

,- current wrap value
| ,- fill last cell?
| | ,- new wrap value
| | | ,- comments
|-|-|-|
|1|0|0| THIS CASE WAS UNHANDLED

To handle that special case (1-0-0), we need to UNSET the wrap. So now, we have
~setWrap~ `wrap` mean the following:
- **true**: if writing to the end of the row, SET the wrap value to TRUE
- **false**: if writing to the end of the row, SET the wrap value to FALSE
- **nullopt**: leave the wrap value as it is

Closes GH-1126

(cherry picked from commit 4dd9f9c)
- When reverse indexing, preserve RGB/256 attributes (GH-2987)

(cherry picked from commit 847d6b5)
- do not allow CUU and CUD to move "across" margins. (GH-2996)

If we're moving the cursor up, its vertical movement should be stopped
at the top margin.
Similarly, this applies to moving down and the bottom margin.

Fixes GH-2929.

(cherry picked from commit 0ce08af)
- Prevent the v1 propsheet from zeroing colors, causing black text on black background. (GH-2651)

  fixes GH-2319

(cherry picked from commit b97db63)
- Render the cursor state to VT (GH-2829)

(cherry picked from commit a9f3849)
- Don't put NUL in the buffer in VT mode (GH-3015)

(cherry picked from commit a82d6b8)
- Return to ground when we flush the last char (GH-2823)

The InputStateMachineEngine was incorrectly not returning to the ground
state after flushing the last sequence. That means that something like
alt+backspace would leave us in the Escape state, not the ground state.
This makes sure we return to ground.

Additionally removes the "Parser.UnitTests-common.vcxproj" file, which
was originally used for a theoretical time when we only open-sourced the
parser. It's unnecessary now, and we can get rid of it.

Also includes a small patch to bcz.cmd, to make sure bx works with
projects with a space in their name.

(cherry picked from commit 53c81a0)
- Add support for passing through extended text attributes, like… (GH-2917)
 ...

Related work items: #23678919, #23678920, #23731910, #23731911, #23731912, #23731913, #23731914, #23731915, #23731916, #23731917, #23731918
@ghost
Copy link

ghost commented Oct 23, 2019

🎉Windows Terminal Preview v0.6.2951.0 has been released which incorporates this pull request.:tada:

Handy links:

@zadjii-msft zadjii-msft deleted the dev/migrie/b/2746-return-to-ground branch January 31, 2020 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Bug It either shouldn't be doing this or needs an investigation. Needs-Second It's a PR that needs another sign-off Product-Conhost For issues in the Console codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Alt-Backspace followed by Enter does not behave properly
4 participants