Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some form of regression test to measure performance of load times #932

Closed
DonJayamanne opened this issue Mar 2, 2018 · 2 comments · Fixed by #1813
Closed

Add some form of regression test to measure performance of load times #932

DonJayamanne opened this issue Mar 2, 2018 · 2 comments · Fixed by #1813
Assignees
Labels
debt Covers everything internal: CI, testing, refactoring of the codebase, etc.
Milestone

Comments

@DonJayamanne
Copy link

DonJayamanne commented Mar 2, 2018

We have a unit test or pre-release (manual) task to ensure load times of the extension have not increased (at least not drastically) in each new version.

Three occasions we've identified slow load times of extension:

@DonJayamanne DonJayamanne added this to the March 2018 milestone Mar 2, 2018
@DonJayamanne DonJayamanne added debt Covers everything internal: CI, testing, refactoring of the codebase, etc. needs decision labels Mar 2, 2018
@MikhailArkhipov
Copy link

May be hard on CI due to random times there. However, local-only test may work as local runs are relatively consistent - at least on comparable machines.

@brettcannon
Copy link
Member

So we can record the number in the test results, but we don't have to succeed/fail based on it (yes that can lead to slow skew, but we are more worried about hard performance loss).

@DonJayamanne DonJayamanne modified the milestones: April 2018, May 2018 Apr 20, 2018
@lock lock bot locked as resolved and limited conversation to collaborators Jul 10, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
debt Covers everything internal: CI, testing, refactoring of the codebase, etc.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants