Skip to content

Commit

Permalink
update Test Runner to handle API errs for non-private test limits (#4025
Browse files Browse the repository at this point in the history 
)

* update Test Runner to handle API errs for non-private test limits

* update record errors to factor in private vs regular tests

- manually reconcile and update all snapshots —______________—


Co-authored-by: Brian Mann <[email protected]>
  • Loading branch information
@jennifer-shehane @brian-mann
jennifer-shehane and brian-mann committed May 17, 2019
1 parent 77c62e0 commit cb51d5e
Show file tree
Hide file tree
Showing 4 changed files with 320 additions and 23 deletions.
197 changes: 184 additions & 13 deletions packages/server/__snapshots__/7_record_spec.coffee.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1537,7 +1537,107 @@ StatusCodeError: 402
`

exports['e2e record api interaction warnings create run warnings grace period - over limit warns when over private test recordings 1'] = `

exports['e2e record api interaction errors create run 402 - free plan exceeds monthly tests errors and exits when on free plan and over recorded tests limit 1'] = `
You've exceeded the limit of test recordings under your free plan this month. The limit is 500 test recordings.
To continue recording tests this month you must upgrade your account. Please visit your billing to upgrade to another billing plan.
https://on.cypress.io/dashboard/organizations/org-id-1234/billing
`

exports['e2e record api interaction errors create run 402 - grouping feature not available in plan errors and exits when attempting parallel run when not available in plan 1'] = `
Grouping is not included under your current billing plan.
To run your tests with groups, please visit your billing and upgrade to another plan with grouping.
https://on.cypress.io/dashboard/organizations/org-id-1234/billing
`


exports['e2e record api interaction warnings create run warnings grace period - grouping feature warns when using parallel feature 1'] = `
Grouping is not included under your free plan.
Your plan is now in a grace period, which means your tests will still run with groups until 2999-12-31. Please upgrade your plan to continue running your tests with groups in the future.
https://on.cypress.io/dashboard/organizations/org-id-1234/billing
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 1.2.3 │
│ Browser: FooBrowser 88 │
│ Specs: 1 found (record_pass_spec.coffee) │
│ Searched: cypress/integration/record_pass* │
│ Params: Group: false, Parallel: false │
│ Run URL: https://dashboard.cypress.io/#/projects/cjvoj7/runs/12 │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: record_pass_spec.coffee... (1 of 1)
Estimated: 8 seconds
record pass
✓ passes
- is pending
1 passing
1 pending
(Results)
┌───────────────────────────────────────┐
│ Tests: 2 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 1 │
│ Skipped: 0 │
│ Screenshots: 1 │
│ Video: true │
│ Duration: X seconds │
│ Estimated: 8 seconds │
│ Spec Ran: record_pass_spec.coffee │
└───────────────────────────────────────┘
(Screenshots)
- /foo/bar/.projects/e2e/cypress/screenshots/record_pass_spec.coffee/yay it passes.png (202x1002)
(Uploading Results)
- Done Uploading (1/1) /foo/bar/.projects/e2e/cypress/screenshots/record_pass_spec.coffee/yay it passes.png
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ record_pass_spec.coffee XX:XX 2 1 - 1 - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
All specs passed! XX:XX 2 1 - 1 -
───────────────────────────────────────────────────────────────────────────────────────────────────────
Recorded Run: https://dashboard.cypress.io/#/projects/cjvoj7/runs/12
`

exports['e2e record api interaction warnings create run warnings grace period - over private tests limit warns when over private test recordings 1'] = `
You've exceeded the limit of private test recordings under your free plan this month. The limit is 500 private test recordings.
Your plan is now in a grace period, which means your tests will still be recorded until 2999-12-31. Please upgrade your plan to continue recording tests on the Cypress Dashboard in the future.
Expand Down Expand Up @@ -1617,10 +1717,10 @@ https://on.cypress.io/dashboard/organizations/org-id-1234/billing
`

exports['e2e record api interaction warnings create run warnings grace period - parallel feature warns when using parallel feature 1'] = `
Parallelization is not included under your free plan.
exports['e2e record api interaction warnings create run warnings grace period - over tests limit warns when over test recordings 1'] = `
You've exceeded the limit of test recordings under your free plan this month. The limit is 500 test recordings.
Your plan is now in a grace period, which means your tests will still run in parallel until 2999-12-31. Please upgrade your plan to continue running your tests in parallel in the future.
Your plan is now in a grace period, which means your tests will still be recorded until 2999-12-31. Please upgrade your plan to continue recording tests on the Cypress Dashboard in the future.
https://on.cypress.io/dashboard/organizations/org-id-1234/billing
Expand Down Expand Up @@ -1697,10 +1797,10 @@ https://on.cypress.io/dashboard/organizations/org-id-1234/billing
`

exports['e2e record api interaction warnings create run warnings paid plan - over limit warns when over private test recordings 1'] = `
You've exceeded the limit of private test recordings under your current billing plan this month. The limit is 500 private test recordings.
exports['e2e record api interaction warnings create run warnings grace period - parallel feature warns when using parallel feature 1'] = `
Parallelization is not included under your free plan.
To upgrade your account, please visit your billing to upgrade to another billing plan.
Your plan is now in a grace period, which means your tests will still run in parallel until 2999-12-31. Please upgrade your plan to continue running your tests in parallel in the future.
https://on.cypress.io/dashboard/organizations/org-id-1234/billing
Expand Down Expand Up @@ -1777,19 +1877,90 @@ https://on.cypress.io/dashboard/organizations/org-id-1234/billing
`

exports['e2e record api interaction errors create run 402 - grouping feature not available in plan errors and exits when attempting parallel run when not available in plan 1'] = `
Grouping is not included under your current billing plan.
exports['e2e record api interaction warnings create run warnings paid plan - over private tests limit warns when over private test recordings 1'] = `
You've exceeded the limit of private test recordings under your current billing plan this month. The limit is 500 private test recordings.
To run your tests with groups, please visit your billing and upgrade to another plan with grouping.
To upgrade your account, please visit your billing to upgrade to another billing plan.
https://on.cypress.io/dashboard/organizations/org-id-1234/billing
====================================================================================================
(Run Starting)
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Cypress: 1.2.3 │
│ Browser: FooBrowser 88 │
│ Specs: 1 found (record_pass_spec.coffee) │
│ Searched: cypress/integration/record_pass* │
│ Params: Group: false, Parallel: false │
│ Run URL: https://dashboard.cypress.io/#/projects/cjvoj7/runs/12 │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
────────────────────────────────────────────────────────────────────────────────────────────────────
Running: record_pass_spec.coffee... (1 of 1)
Estimated: 8 seconds
record pass
✓ passes
- is pending
1 passing
1 pending
(Results)
┌───────────────────────────────────────┐
│ Tests: 2 │
│ Passing: 1 │
│ Failing: 0 │
│ Pending: 1 │
│ Skipped: 0 │
│ Screenshots: 1 │
│ Video: true │
│ Duration: X seconds │
│ Estimated: 8 seconds │
│ Spec Ran: record_pass_spec.coffee │
└───────────────────────────────────────┘
(Screenshots)
- /foo/bar/.projects/e2e/cypress/screenshots/record_pass_spec.coffee/yay it passes.png (202x1002)
(Uploading Results)
- Done Uploading (1/1) /foo/bar/.projects/e2e/cypress/screenshots/record_pass_spec.coffee/yay it passes.png
====================================================================================================
(Run Finished)
Spec Tests Passing Failing Pending Skipped
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ✔ record_pass_spec.coffee XX:XX 2 1 - 1 - │
└────────────────────────────────────────────────────────────────────────────────────────────────┘
All specs passed! XX:XX 2 1 - 1 -
───────────────────────────────────────────────────────────────────────────────────────────────────────
Recorded Run: https://dashboard.cypress.io/#/projects/cjvoj7/runs/12
`

exports['e2e record api interaction warnings create run warnings grace period - grouping feature warns when using parallel feature 1'] = `
Grouping is not included under your free plan.
exports['e2e record api interaction warnings create run warnings paid plan - over tests limit warns when over test recordings 1'] = `
You've exceeded the limit of test recordings under your current billing plan this month. The limit is 500 test recordings.
Your plan is now in a grace period, which means your tests will still run with groups until 2999-12-31. Please upgrade your plan to continue running your tests with groups in the future.
To upgrade your account, please visit your billing to upgrade to another billing plan.
https://on.cypress.io/dashboard/organizations/org-id-1234/billing
Expand Down
30 changes: 27 additions & 3 deletions packages/server/lib/errors.coffee
View file
Original file line number Diff line number Diff line change
Expand Up @@ -690,23 +690,47 @@ getMsgByType = (type, arg1 = {}, arg2) ->
"""
when "FREE_PLAN_EXCEEDS_MONTHLY_PRIVATE_TESTS"
"""
You've exceeded the limit of private test recordings under your free plan this month. #{arg1.usedMessage}
You've exceeded the limit of private test recordings under your free plan this month. #{arg1.usedTestsMessage}
To continue recording tests this month you must upgrade your account. Please visit your billing to upgrade to another billing plan.
#{arg1.link}
"""
when "FREE_PLAN_IN_GRACE_PERIOD_EXCEEDS_MONTHLY_PRIVATE_TESTS"
"""
You've exceeded the limit of private test recordings under your free plan this month. #{arg1.usedMessage}
You've exceeded the limit of private test recordings under your free plan this month. #{arg1.usedTestsMessage}
Your plan is now in a grace period, which means your tests will still be recorded until #{arg1.gracePeriodMessage}. Please upgrade your plan to continue recording tests on the Cypress Dashboard in the future.
#{arg1.link}
"""
when "PAID_PLAN_EXCEEDS_MONTHLY_PRIVATE_TESTS"
"""
You've exceeded the limit of private test recordings under your current billing plan this month. #{arg1.usedMessage}
You've exceeded the limit of private test recordings under your current billing plan this month. #{arg1.usedTestsMessage}
To upgrade your account, please visit your billing to upgrade to another billing plan.
#{arg1.link}
"""
when "FREE_PLAN_EXCEEDS_MONTHLY_TESTS"
"""
You've exceeded the limit of test recordings under your free plan this month. #{arg1.usedTestsMessage}
To continue recording tests this month you must upgrade your account. Please visit your billing to upgrade to another billing plan.
#{arg1.link}
"""
when "FREE_PLAN_IN_GRACE_PERIOD_EXCEEDS_MONTHLY_TESTS"
"""
You've exceeded the limit of test recordings under your free plan this month. #{arg1.usedTestsMessage}
Your plan is now in a grace period, which means your tests will still be recorded until #{arg1.gracePeriodMessage}. Please upgrade your plan to continue recording tests on the Cypress Dashboard in the future.
#{arg1.link}
"""
when "PAID_PLAN_EXCEEDS_MONTHLY_TESTS"
"""
You've exceeded the limit of test recordings under your current billing plan this month. #{arg1.usedTestsMessage}
To upgrade your account, please visit your billing to upgrade to another billing plan.
Expand Down
26 changes: 21 additions & 5 deletions packages/server/lib/modes/record.coffee
View file
Original file line number Diff line number Diff line change
Expand Up @@ -231,9 +231,9 @@ getCommitFromGitOrCi = (git) ->
defaultBranch: null
})

usedMessage = (limit) ->
usedTestsMessage = (limit, phrase) ->
if _.isFinite(limit)
"The limit is #{chalk.blue(limit)} private test recordings."
"The limit is #{chalk.blue(limit)} #{phrase} recordings."
else
""

Expand Down Expand Up @@ -307,7 +307,13 @@ createRun = (options = {}) ->
switch warning.code
when "FREE_PLAN_IN_GRACE_PERIOD_EXCEEDS_MONTHLY_PRIVATE_TESTS"
errors.warning("FREE_PLAN_IN_GRACE_PERIOD_EXCEEDS_MONTHLY_PRIVATE_TESTS", {
usedMessage: usedMessage(warning.limit)
usedTestsMessage: usedTestsMessage(warning.limit, "private test")
gracePeriodMessage: gracePeriodMessage(warning.gracePeriodEnds)
link: billingLink(warning.orgId)
})
when "FREE_PLAN_IN_GRACE_PERIOD_EXCEEDS_MONTHLY_TESTS"
errors.warning("FREE_PLAN_IN_GRACE_PERIOD_EXCEEDS_MONTHLY_TESTS", {
usedTestsMessage: usedTestsMessage(warning.limit, "test")
gracePeriodMessage: gracePeriodMessage(warning.gracePeriodEnds)
link: billingLink(warning.orgId)
})
Expand All @@ -318,7 +324,12 @@ createRun = (options = {}) ->
})
when "PAID_PLAN_EXCEEDS_MONTHLY_PRIVATE_TESTS"
errors.warning("PAID_PLAN_EXCEEDS_MONTHLY_PRIVATE_TESTS", {
usedMessage: usedMessage(warning.limit)
usedTestsMessage: usedTestsMessage(warning.limit, "private test")
link: billingLink(warning.orgId)
})
when "PAID_PLAN_EXCEEDS_MONTHLY_TESTS"
errors.warning("PAID_PLAN_EXCEEDS_MONTHLY_TESTS", {
usedTestsMessage: usedTestsMessage(warning.limit, "test")
link: billingLink(warning.orgId)
})
when "PLAN_IN_GRACE_PERIOD_RUN_GROUPING_FEATURE_USED"
Expand All @@ -345,7 +356,12 @@ createRun = (options = {}) ->
switch code
when "FREE_PLAN_EXCEEDS_MONTHLY_PRIVATE_TESTS"
errors.throw("FREE_PLAN_EXCEEDS_MONTHLY_PRIVATE_TESTS", {
usedMessage: usedMessage(limit)
usedTestsMessage: usedTestsMessage(limit, "private test")
link: billingLink(orgId)
})
when "FREE_PLAN_EXCEEDS_MONTHLY_TESTS"
errors.throw("FREE_PLAN_EXCEEDS_MONTHLY_TESTS", {
usedTestsMessage: usedTestsMessage(limit, "test")
link: billingLink(orgId)
})
when "PARALLEL_FEATURE_NOT_AVAILABLE_IN_PLAN"
Expand Down
Loading

0 comments on commit cb51d5e

Please sign in to comment.