Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarking tests #12603

Merged
merged 11 commits into from
Jul 19, 2024
Merged

Add benchmarking tests #12603

merged 11 commits into from
Jul 19, 2024

Conversation

Procatv
Copy link
Contributor

@Procatv Procatv commented Jun 18, 2024

Summary

Adding to qiskit asv based on bench_press, specifically qiskit_gym tests

Making changes to the qiskit benchmarking suite by adding more tests with different types of circuits. Not yet finished implementing and would appreciate feedback regarding how I could better improve the benchmarking tests, and any other params I could keep track of.

Details and comments

Fixes #12595

@qiskit-bot qiskit-bot added the Community PR PRs from contributors that are not 'members' of the Qiskit repo label Jun 18, 2024
@coveralls
Copy link

coveralls commented Jun 18, 2024

Pull Request Test Coverage Report for Build 9569038976

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 5 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-0.003%) to 89.75%

Files with Coverage Reduction New Missed Lines %
qiskit/transpiler/passes/synthesis/unitary_synthesis.py 2 88.2%
crates/qasm2/src/lex.rs 3 92.11%
Totals Coverage Status
Change from base Build 9562946226: -0.003%
Covered Lines: 63415
Relevant Lines: 70657

💛 - Coveralls

@Procatv Procatv changed the title Feature/update bench Adding benchmarking tests Jun 18, 2024
@Procatv Procatv changed the title Adding benchmarking tests Add benchmarking tests Jun 18, 2024
@Procatv Procatv marked this pull request as ready for review June 23, 2024 21:03
@Procatv Procatv requested a review from a team as a code owner June 23, 2024 21:03
@qiskit-bot
Copy link
Collaborator

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the following people are relevant to this code:

  • @Qiskit/terra-core

@coveralls
Copy link

coveralls commented Jun 23, 2024

Pull Request Test Coverage Report for Build 9636385838

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 811 unchanged lines in 54 files lost coverage.
  • Overall coverage increased (+0.03%) to 89.785%

Files with Coverage Reduction New Missed Lines %
qiskit/converters/circuit_to_gate.py 1 97.44%
qiskit/dagcircuit/dagdepnode.py 1 76.81%
qiskit/compiler/assembler.py 1 96.13%
qiskit/converters/circuit_to_instruction.py 1 98.21%
qiskit/assembler/assemble_schedules.py 1 98.35%
qiskit/qasm2/export.py 1 98.5%
qiskit/pulse/configuration.py 2 96.08%
qiskit/transpiler/passes/synthesis/unitary_synthesis.py 2 88.2%
qiskit/circuit/library/hamiltonian_gate.py 2 83.33%
qiskit/circuit/library/standard_gates/xx_plus_yy.py 2 95.74%
Totals Coverage Status
Change from base Build 9562946226: 0.03%
Covered Lines: 63662
Relevant Lines: 70905

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jun 24, 2024

Pull Request Test Coverage Report for Build 9648917691

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.03%) to 89.794%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 2 92.88%
Totals Coverage Status
Change from base Build 9648699798: 0.03%
Covered Lines: 63754
Relevant Lines: 71000

💛 - Coveralls

@Procatv Procatv force-pushed the feature/update_bench branch from c596bde to a38b49f Compare June 24, 2024 17:10
@coveralls
Copy link

coveralls commented Jun 24, 2024

Pull Request Test Coverage Report for Build 9649517337

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.02%) to 89.78%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 2 91.86%
Totals Coverage Status
Change from base Build 9648699798: 0.02%
Covered Lines: 63744
Relevant Lines: 71000

💛 - Coveralls

@ElePT ElePT added the Intern PR PR submitted by IBM Quantum interns label Jul 2, 2024
@ShellyGarion
Copy link
Member

it's also worth adding random cliffords synthesis benchmarks for various numbers of qubits:

from qiskit.quantum_info import *
cliff = random_clifford(num_qubits)
qc = cliff.to_circuit()

@sbrandhsn sbrandhsn self-assigned this Jul 12, 2024
@coveralls
Copy link

coveralls commented Jul 17, 2024

Pull Request Test Coverage Report for Build 9994484740

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 963 unchanged lines in 54 files lost coverage.
  • Overall coverage increased (+0.2%) to 89.928%

Files with Coverage Reduction New Missed Lines %
qiskit/circuit/library/standard_gates/r.py 1 97.73%
crates/accelerate/src/synthesis/permutation/utils.rs 1 98.88%
qiskit/synthesis/evolution/lie_trotter.py 1 96.77%
qiskit/circuit/library/standard_gates/rxx.py 1 97.56%
qiskit/transpiler/passes/routing/commuting_2q_gate_routing/pauli_2q_evolution_commutation.py 1 97.92%
qiskit/circuit/library/standard_gates/rzx.py 1 97.56%
qiskit/circuit/library/standard_gates/u.py 1 95.92%
crates/accelerate/src/isometry.rs 1 99.65%
crates/accelerate/src/sampled_exp_val.rs 1 88.71%
qiskit/synthesis/evolution/product_formula.py 1 98.64%
Totals Coverage Status
Change from base Build 9648699798: 0.2%
Covered Lines: 65813
Relevant Lines: 73184

💛 - Coveralls

@sbrandhsn sbrandhsn added the Changelog: New Feature Include in the "Added" section of the changelog label Jul 19, 2024
@sbrandhsn
Copy link
Contributor

sbrandhsn commented Jul 19, 2024

This looks great to me, thanks! :-) Can you also write a quick release note as described at https://github.com/Qiskit/qiskit/blob/main/CONTRIBUTING.md#release-notes ?

As this is not directly user-facing, I think it is fine to proceed without a release note. :-)

@sbrandhsn sbrandhsn removed the Changelog: New Feature Include in the "Added" section of the changelog label Jul 19, 2024
@sbrandhsn sbrandhsn enabled auto-merge July 19, 2024 15:18
@sbrandhsn sbrandhsn added this pull request to the merge queue Jul 19, 2024
Merged via the queue into Qiskit:main with commit ec1f42c Jul 19, 2024
15 checks passed
ElePT pushed a commit to mtreinish/qiskit-core that referenced this pull request Jul 24, 2024
* added circuit functions

* updated circuits functions

* added qasm files

* added benchmarking metrics

* cleaning up circuits

* updated tests

* updated tests

* formatting

* removed unused import

* clifford synthesis circ test

* lint test
@ElePT ElePT added the Changelog: None Do not include in changelog label Jul 30, 2024
Procatv added a commit to Procatv/qiskit-terra-catherines that referenced this pull request Aug 1, 2024
* added circuit functions

* updated circuits functions

* added qasm files

* added benchmarking metrics

* cleaning up circuits

* updated tests

* updated tests

* formatting

* removed unused import

* clifford synthesis circ test

* lint test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: None Do not include in changelog Community PR PRs from contributors that are not 'members' of the Qiskit repo Intern PR PR submitted by IBM Quantum interns
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add new benchmarking metrics and tests
6 participants