Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use RCA based code generation for all scenarios #1523

Merged
merged 4 commits into from
May 21, 2024
Merged

Use RCA based code generation for all scenarios #1523

merged 4 commits into from
May 21, 2024

Conversation

idavis
Copy link
Collaborator

@idavis idavis commented May 14, 2024 •
edited
Loading

new QSC option:

  -p, --profile <PROFILE>
          Target QIR profile for code generation

          Possible values:
          - unrestricted: This is the default profile, which allows all operations
          - base:         This profile restricts the set of operations to those that are supported by the Base profile
          - adaptive-ri:  This profile restricts the set of operations to those that are supported by the AdaptiveRI profile

@idavis idavis self-assigned this May 14, 2024
@github-actions GitHub Actions
Copy link

Benchmark for d09ed47

Click to view benchmark
Test Base PR %
Array append evaluation 341.8±2.90µs 343.1±2.14µs +0.38%
Array literal evaluation 171.2±1.96µs 170.5±0.97µs -0.41%
Array update evaluation 427.3±10.96µs 428.2±4.35µs +0.21%
Core + Standard library compilation 20.7±1.59ms 20.3±1.53ms -1.93%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.1±0.04ms +2.00%
Large file parity evaluation 34.4±0.14ms 34.4±0.32ms 0.00%
Large input file compilation 13.7±0.96ms 13.8±0.97ms +0.73%
Large input file compilation (interpreter) 50.8±2.35ms 52.0±3.19ms +2.36%
Large nested iteration 33.5±0.37ms 33.6±0.15ms +0.30%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1635.1±152.62µs 1642.5±195.13µs +0.45%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.4±0.20ms 8.5±0.45ms +1.19%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1486.9±149.86µs 1504.6±215.66µs +1.19%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 29.0±0.61ms 29.2±0.59ms +0.69%
Teleport evaluation 87.6±3.72µs 88.0±3.85µs +0.46%

@github-actions GitHub Actions
Copy link

Benchmark for 70916c3

Click to view benchmark
Test Base PR %
Array append evaluation 346.9±11.16µs 377.5±1.31µs +8.82%
Array literal evaluation 189.8±1.36µs 192.3±0.75µs +1.32%
Array update evaluation 429.5±4.83µs 445.0±2.90µs +3.61%
Core + Standard library compilation 17.6±0.55ms 17.9±0.84ms +1.70%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.1±0.13ms +2.00%
Large file parity evaluation 33.9±0.15ms 33.8±0.31ms -0.29%
Large input file compilation 12.8±0.61ms 11.8±0.27ms -7.81%
Large input file compilation (interpreter) 47.4±2.12ms 50.6±1.36ms +6.75%
Large nested iteration 34.2±1.68ms 35.4±0.47ms +3.51%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1587.8±89.94µs 1544.6±97.97µs -2.72%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.3±0.15ms 8.1±0.21ms -2.41%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1430.8±60.78µs 1399.4±61.97µs -2.19%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.1±0.28ms 28.0±0.40ms -0.36%
Teleport evaluation 87.4±3.73µs 88.1±3.74µs +0.80%

@idavis idavis mentioned this pull request May 17, 2024
Closed
64 tasks
@idavis idavis force-pushed the iadavis/use-rca-for-ls-errors-in-preview branch from ac76e76 to d5f92ac Compare May 20, 2024 16:54
@github-actions GitHub Actions
Copy link

Benchmark for 99b6716

Click to view benchmark
Test Base PR %
Array append evaluation 333.8±1.42µs 327.4±7.09µs -1.92%
Array literal evaluation 189.7±1.57µs 168.9±1.35µs -10.96%
Array update evaluation 411.3±1.42µs 405.4±3.58µs -1.43%
Core + Standard library compilation 22.8±1.46ms 22.1±0.62ms -3.07%
Deutsch-Jozsa evaluation 5.0±0.04ms 5.2±0.19ms +4.00%
Large file parity evaluation 34.2±0.09ms 34.5±0.09ms +0.88%
Large input file compilation 15.7±0.56ms 16.1±1.03ms +2.55%
Large input file compilation (interpreter) 58.4±2.46ms 53.3±1.72ms -8.73%
Large nested iteration 33.0±0.19ms 32.5±0.64ms -1.52%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1689.8±177.16µs 1693.2±214.29µs +0.20%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.8±0.19ms 8.7±0.14ms -1.14%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1542.9±202.77µs 1556.9±315.98µs +0.91%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 29.5±0.28ms 29.3±0.29ms -0.68%
Teleport evaluation 88.2±4.19µs 90.9±8.32µs +3.06%

@idavis idavis force-pushed the iadavis/use-rca-for-ls-errors-in-preview branch from d5f92ac to 8adb760 Compare May 20, 2024 17:49
@github-actions GitHub Actions
Copy link

Benchmark for 28b0da9

Click to view benchmark
Test Base PR %
Array append evaluation 324.2±1.20µs 331.1±1.67µs +2.13%
Array literal evaluation 170.7±0.80µs 193.2±6.63µs +13.18%
Array update evaluation 404.7±0.97µs 412.5±3.07µs +1.93%
Core + Standard library compilation 20.2±0.74ms 19.7±0.96ms -2.48%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.1±0.05ms +2.00%
Large file parity evaluation 34.0±0.36ms 34.2±0.11ms +0.59%
Large input file compilation 13.3±0.51ms 13.3±0.55ms 0.00%
Large input file compilation (interpreter) 50.0±1.25ms 50.2±1.46ms +0.40%
Large nested iteration 32.3±0.20ms 32.6±0.24ms +0.93%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1591.2±86.26µs 1585.8±78.15µs -0.34%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.9±0.09ms 7.9±0.13ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1430.2±55.01µs 1432.9±61.90µs +0.19%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.0±0.82ms 28.1±0.49ms +0.36%
Teleport evaluation 88.6±9.05µs 90.5±8.39µs +2.14%

@idavis idavis force-pushed the iadavis/use-rca-for-ls-errors-in-preview branch from 8adb760 to eca726a Compare May 20, 2024 21:39
@idavis idavis changed the title Stop using baseprofchk for preview code gen Use RCA based code generation for all scenarios May 20, 2024
@github-actions GitHub Actions
Copy link

Benchmark for 96a96c3

Click to view benchmark
Test Base PR %
Array append evaluation 335.8±4.34µs 336.6±4.03µs +0.24%
Array literal evaluation 189.7±1.09µs 191.9±3.77µs +1.16%
Array update evaluation 412.3±3.67µs 414.4±8.81µs +0.51%
Core + Standard library compilation 20.5±0.87ms 21.2±1.00ms +3.41%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.0±0.22ms 0.00%
Large file parity evaluation 34.1±0.78ms 34.1±0.06ms 0.00%
Large input file compilation 13.6±0.35ms 13.4±0.43ms -1.47%
Large input file compilation (interpreter) 50.2±1.65ms 51.0±3.22ms +1.59%
Large nested iteration 32.9±0.51ms 32.8±0.26ms -0.30%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1590.4±99.55µs 1595.4±82.06µs +0.31%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.0±0.11ms 7.9±0.10ms -1.25%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1444.4±78.92µs 1448.6±109.42µs +0.29%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.0±0.27ms 28.3±0.33ms +1.07%
Teleport evaluation 87.4±3.78µs 88.2±3.64µs +0.92%

@idavis idavis marked this pull request as ready for review May 20, 2024 22:02
swernli
swernli reviewed May 20, 2024
View reviewed changes
Copy link
Collaborator

@swernli swernli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One file that is missing cleanup is the npm API, which still has getQir and newGetQir. Otherwise, changes look good so far!

vscode/src/qirGeneration.ts Show resolved Hide resolved
vscode/src/qirGeneration.ts Outdated Show resolved Hide resolved
vscode/src/qirGeneration.ts Show resolved Hide resolved
language_service/src/state/tests.rs Outdated Show resolved Hide resolved
compiler/qsc/src/codegen.rs Outdated Show resolved Hide resolved
compiler/qsc/src/codegen.rs Show resolved Hide resolved
compiler/qsc/src/interpret/tests.rs Show resolved Hide resolved
@github-actions GitHub Actions
Copy link

Benchmark for 8082d25

Click to view benchmark
Test Base PR %
Array append evaluation 336.1±4.52µs 326.6±4.24µs -2.83%
Array literal evaluation 189.8±0.78µs 170.1±8.62µs -10.38%
Array update evaluation 411.6±2.96µs 405.2±5.19µs -1.55%
Core + Standard library compilation 24.1±2.56ms 20.5±0.94ms -14.94%
Deutsch-Jozsa evaluation 5.1±0.08ms 5.1±0.06ms 0.00%
Large file parity evaluation 34.2±0.10ms 34.4±0.33ms +0.58%
Large input file compilation 15.6±1.20ms 13.7±0.40ms -12.18%
Large input file compilation (interpreter) 53.4±2.84ms 51.0±1.66ms -4.49%
Large nested iteration 33.0±0.86ms 32.3±0.21ms -2.12%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1638.2±122.61µs 1661.1±231.34µs +1.40%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.6±0.37ms 8.5±0.31ms -1.16%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1513.3±250.87µs 1485.6±109.37µs -1.83%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.9±0.63ms 28.3±0.46ms -2.08%
Teleport evaluation 88.3±3.61µs 91.3±4.02µs +3.40%

@github-actions GitHub Actions
Copy link

Benchmark for dd69774

Click to view benchmark
Test Base PR %
Array append evaluation 328.1±2.68µs 326.8±4.25µs -0.40%
Array literal evaluation 173.9±1.20µs 168.4±1.44µs -3.16%
Array update evaluation 414.7±2.99µs 405.7±4.12µs -2.17%
Core + Standard library compilation 19.0±0.35ms 18.7±0.36ms -1.58%
Deutsch-Jozsa evaluation 5.0±0.05ms 5.0±0.05ms 0.00%
Large file parity evaluation 33.9±0.21ms 34.2±0.48ms +0.88%
Large input file compilation 12.4±0.20ms 12.3±0.24ms -0.81%
Large input file compilation (interpreter) 49.2±1.66ms 46.3±1.08ms -5.89%
Large nested iteration 33.3±0.61ms 32.1±0.16ms -3.60%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1554.6±29.43µs 1564.9±48.36µs +0.66%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.7±0.10ms 7.7±0.08ms 0.00%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1418.0±35.11µs 1420.9±44.17µs +0.20%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.4±0.33ms 27.5±0.70ms +0.36%
Teleport evaluation 88.1±3.66µs 89.4±5.95µs +1.48%

@idavis idavis requested a review from minestarks May 21, 2024 15:07
@idavis idavis mentioned this pull request May 21, 2024
Open
10 tasks
@idavis idavis requested a review from ivanbasov as a code owner May 21, 2024 16:09
@idavis idavis enabled auto-merge May 21, 2024 16:11
@github-actions GitHub Actions
Copy link

Benchmark for 6745f5f

Click to view benchmark
Test Base PR %
Array append evaluation 332.0±1.79µs 331.3±2.20µs -0.21%
Array literal evaluation 170.7±1.30µs 170.9±4.26µs +0.12%
Array update evaluation 410.0±1.85µs 409.4±3.95µs -0.15%
Core + Standard library compilation 18.8±0.31ms 19.1±0.74ms +1.60%
Deutsch-Jozsa evaluation 5.1±0.06ms 5.1±0.05ms 0.00%
Large file parity evaluation 34.1±0.19ms 34.2±0.23ms +0.29%
Large input file compilation 12.9±0.66ms 13.0±0.54ms +0.78%
Large input file compilation (interpreter) 47.7±1.32ms 45.9±2.13ms -3.77%
Large nested iteration 32.9±0.22ms 32.7±0.51ms -0.61%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1580.7±68.31µs 1570.1±55.93µs -0.67%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.0±0.10ms 7.7±0.04ms -3.75%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1442.3±115.51µs 1422.6±70.15µs -1.37%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 27.4±0.16ms 27.7±0.45ms +1.09%
Teleport evaluation 90.3±3.80µs 89.4±3.96µs -1.00%

@idavis idavis added this pull request to the merge queue May 21, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 21, 2024
@idavis idavis added this pull request to the merge queue May 21, 2024
Merged via the queue into main with commit 308c578 May 21, 2024
16 checks passed
@idavis idavis deleted the iadavis/use-rca-for-ls-errors-in-preview branch May 21, 2024 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@swernli swernli swernli approved these changes

@minestarks minestarks minestarks approved these changes

@billti billti Awaiting requested review from billti billti is a code owner

@cesarzc cesarzc Awaiting requested review from cesarzc cesarzc is a code owner

@sezna sezna Awaiting requested review from sezna sezna is a code owner

@ScottCarda-MS ScottCarda-MS Awaiting requested review from ScottCarda-MS ScottCarda-MS is a code owner

@ivanbasov ivanbasov Awaiting requested review from ivanbasov ivanbasov is a code owner

Assignees

@idavis idavis

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@idavis @swernli @minestarks