Demonstrate Benefits of Incremental Updates for Small Code Change

[wdanilo]

This task is automatically imported from the old Task Issue Board and it was originally created by jaroslavtulach.
Original issue is here.

---

Why

As Enso user
**I want ** changes in my IDE to be processed quickly
**So that ** smaller changes take smaller time to be processed.

Acceptance Criteria

Scenario: 
Given change of a number in the source for another number
When the numbers have the same amount of digits
Then the internal engine structures (IR and Nodes) shall be updated without reparsing the whole source

Notes:

The Enso IDE (unlike plain text editor) is operating on the structure of the program, not on the letters. Changing content of a node results in replacing an AST subtree with another AST structure. Changing node with a single number value for another number doesn't change anything on the structure of the program. Just, it changes the value the program should see at given place. However, in-spite of that, currently the whole program is being re-parsed; the previous IR data are thrown away; the existing Truffle nodes are thrown away too. That's certainly ineffective and not proportional to the size of the of the user change. Small changes shall require small time to be processed.

We can do it. At least for replacement of 4 by 5 (or another change having same digits to make the initial test case simple). We have to locate the appropriate IR representing the number and just update it. We have to do the same for Truffe nodes - precisely select the one representing the number and change just it (appropriately invalidating the affected compiled code caches). There already is RuntimeServerTest testing such a scenario. We just have to update it to fail if full parsing happens and then work on making sure we can keep all other semantics untouched without re-parsing everything.

Tasks:

• Update/copy the RuntimeServerTest to check whether full parsing happens
• Detect "small" change and find where it happens
• Rather than scheduling full parse, update the internal Truffle AST
• Fix whatever is broken until execution produces the right result
• Make sure expected invariants (Source locations, text, etc.) are met
• Update IR to match reality or somehow mark it as outdated

Comments:

Jaroslav Tulach reports a new STANDUP for yesterday (2022-05-30):

Progress: Meetings to plan "incremental" updates - recorded as https://www.pivotaltracker.com/n/projects/2539304/stories/182323784. Review and a way to execute benchmarks and analyze their graphs in IGV: https://github.com/enso-org/enso/pull/3492/files#r884499400 It should be finished by 2022-06-08.

Next Day: Investigate incremental (proporitional to user change) updates (Enso Bot - May 31, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for today (2022-05-31):

Progress: Trying to get a feel for Edward's sbt problems: #3464 (comment); Review of Hubert's annotation processor magic: #3471; Few comments on Dmitry's background pool: #3493; debugging RuntimeServerTest & planning performance work for me and Marcin It should be finished by 2022-06-08.

Next Day: Continue debuging of proportional to user change updates of Enso programs (Enso Bot - May 31, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-06-01):

Progress: Writing IncrementalUpdatesTest cecd735; debugging up to ExecutionService; then thru text-buffer code; discussion with Marcin & Hubert - transferring IGV knownledge - providing IGV&Enso integration It should be finished by 2022-06-08.

Next Day: Continue debuging of proportional to user change updates of Enso programs (Enso Bot - Jun 2, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-06-02):

Progress: Debugging EnsureCompiledJob, Module, ModuleScope. Way to recognize "simple edit". Way to apply "simple edit". Improving "falsification" of the IncrementalUpdatesTest. It should be finished by 2022-06-08.

Next Day: Continue implementing proportional to user change updates of Enso programs (Enso Bot - Jun 3, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-06-03):

Progress: Test with NodeCountingInstrument; Implementation; Request for review #3508; Discussions; Debuggging in real IDE - hack to recognize "simple edit" in the IDE; meanwhile reviews & proposal to fix sbt incremental build: #3509 It should be finished by 2022-06-08.

Next Day: Prepare a demo/measurement of proportional to user change updates of Enso programs (Enso Bot - Jun 4, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-06-04):

Progress: bunch of debugging; two attempts to rewrite source manipulation in Module; finally done as 6303000 It should be finished by 2022-06-08.

Next Day: Prepare a demo/measurement of proportional to user change updates of Enso programs (Enso Bot - Jun 5, 2022)

---

The _EnsureCompiledJob_ can take `3ms` - it used to take `30ms` or more. Parsing and compilation has been dominant in #182234656 - with the changes in the [PR-3508](https://github.com//pull/3508) this has been almost eliminated. (jaroslavtulach - Jun 6, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-06-06):

Progress: Two hours on Sunday: Reviewing Marcin's #3511; Working on AdaptiveRope: 1a87a6d; Monday: Catching up on review of PR-23; Discussion about language and polyglot; Meeting with Dmitry about performance; measuring performance of single edit; demo time of single edit & IGV Enso extension; reporting user story: https://www.pivotaltracker.com/n/projects/2539304/stories/182388312 It should be finished by 2022-06-08.

Next Day: Towards supporting less trivial edits shifting positions (Enso Bot - Jun 7, 2022)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2022-06-07):

Progress: Generalizing incremental updates with Patchable.Tag: 31648ad; Planning future "incremental" work with Wojciech and Marcin; Expanding the tests It should be finished by 2022-06-08.

Next Day: Implementing "position manager" for incremental updates (Enso Bot - Jun 8, 2022)

---

The exploration tasks are done with [28a162a](https://github.com//pull/3508/commits/28a162a9bd0648824022aded6cbf353eb583e9bc). We conclude these _incremental (proportional to user change) updates of the AST for simple edits_ have a potential to significantly improve Enso IDE user experience. Let's productize them in #182412796 (jaroslavtulach - Jun 8, 2022)

---