Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Public API for CT Framework Definitions #25780

Merged
merged 32 commits into from
Feb 21, 2023
Merged

Conversation

lmiller1990
Copy link
Contributor

@lmiller1990 lmiller1990 commented Feb 13, 2023

Closes #25638

Note: No merge yet- still a known issue, see end of the PR description.

Code was reviewed separately in

A second pass is always good, though! 🙏

Screenshots

(1) it adds a community definition with relevant tag in launchpad

1

(2) Known bug in dep detection - not introduced here - see #25782

2

(3) it works!

3

Test Plan

  1. make Solid.js app: npx degit solidjs/templates/js my-app
  2. cd my-app
  3. Install deps and my simple Solid.js definition: npm install cypress-ct-lachlan-solid-js

Two options, either

  1. clone this branch, yarn install so you have the latest everything, yarn cypress:open at monorepo root for global mode, choose my-app (the one you just made)
  2. grab a binary: b2f6343#comments

Then...

  1. Go through onboarding - Solid should show up. Use Vite, the template in step 1 is a Vite one.
  2. It won't properly detect that you installed Solid.js. Known issue, unrelated to this PR, see Launchpad CT onboarding dependencies screen not always reliable #25782. Just hit Skip
  3. Should work like any other CT lib, example test you can use:
  4. wasn't that easy!! almost the same as a first party onboarding :D
import App from '../../src/App'

describe('App', () => {
  it('playground', () => {
    cy.mount(<App />)
    cy.get('a').contains('Learn Solid')
  })
})

Known issues/TODO before merge

  • Not working with binary. Definitions fail to resolve with Ignoring TypeError: Invalid host defined options. Seems to be V8 related according to this tweet, which is the only info I can find about this error. Working on it. Edit seems to be fixed b2f6343#comments
  • Fix flaky test (launchpad test for adding Qwik, a third party definition)

Fast follows

These are all out of scope for this PR - but we will do those next sprint.

lmiller1990 and others added 3 commits February 13, 2023 10:00
* chore: refactoring and types

* rework source of frameworks

* revert rename

* fix tests

* fix more tests

* types

* update code

* use same public API internally

* rename interfaces

* rename

* work on dev server api

* fix types

* fix test

* attempt to support getDevServerConfig

* tests

* add function to define framework [skip ci]

* rework a lot of types

* fix test

* update tests and types

* refactor

* revert changes

* lint

* fix test

* revert

* remove

* add "community" label [skip ci]

* refactor

* types

* lint

* fix bug

* update function name

* address feedback

* improve types with Pick

* refactor using type guard

* correct label

---------

Co-authored-by: Zachary Williams <[email protected]>
* chore: refactoring and types

* rework source of frameworks

* revert rename

* fix tests

* fix more tests

* types

* update code

* use same public API internally

* rename interfaces

* rename

* work on dev server api

* fix types

* fix test

* attempt to support getDevServerConfig

* tests

* add function to define framework [skip ci]

* rework a lot of types

* fix test

* update tests and types

* refactor

* revert changes

* lint

* fix test

* revert

* remove

* add "community" label [skip ci]

* refactor

* types

* lint

* fix bug

* update function name

* address feedback

* feat: scan for 3rd party ct plugins

* add e2e test

* unit tests [run ci]

* tweak resolution

* rebase, address comments

* fix windows paths

* remove .gitignore

* fix test

---------

Co-authored-by: Lachlan Miller <[email protected]>
@cypress
Copy link

cypress bot commented Feb 13, 2023

30 flaky tests on run #44269 ↗︎

0 26852 1281 0 Flakiness 30

Details:

remove worklfow
Project: cypress Commit: 75733be311
Status: Passed Duration: 19:26 💡
Started: Feb 21, 2023 6:08 AM Ended: Feb 21, 2023 6:28 AM
Flakiness  e2e/origin/navigation.cy.ts • 1 flaky test • 5x-driver-electron

View Output Video

Test
delayed navigation > errors > redirects to an unexpected cross-origin and calls another command in the cy.origin command
Flakiness  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-electron

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry
Flakiness  commands/net_stubbing.cy.ts • 1 flaky test • 5x-driver-firefox

View Output Video

Test
network stubbing > intercepting request > can delay and throttle a StaticResponse
Flakiness  e2e/origin/commands/assertions.cy.ts • 1 flaky test • 5x-driver-firefox

View Output Video

Test
cy.origin assertions > #consoleProps > .should() and .and()
Flakiness  cypress/cypress.cy.js • 3 flaky tests • 5x-driver-firefox

View Output Video

Test
... > correctly returns currentRetry
... > correctly returns currentRetry
... > correctly returns currentRetry

The first 5 flaky specs are shown, see all 18 specs in Cypress Cloud.

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

}

const officialFrameworks = CT_FRAMEWORKS.map((framework) => resolveComponentFrameworkDefinition(framework))
const thirdParty = await detectThirdPartyCTFrameworks(this.ctx.currentProject)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a race condition, if this is slow, the UI is wrong

image

https://app.circleci.com/pipelines/github/cypress-io/cypress/49442/workflows/64f996ae-9b7e-4e74-b7d7-a091da0047c6/jobs/2054201/tests#failed-test-0

I will fix it

cy.skipWelcome()

cy.contains('Component Testing').click()
cy.contains('button', 'Qwik').should('be.visible')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a race condition causing CI to fail here. Introduced here, since getting frameworks is now an async operation, since we glob in node_modules.

We need to fix this before merging.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried reproducing this and could not get it to flake. I tried locally as well as the circle runner (running 20+ times). I have an idea that might fix this. Maybe we can pair later on this

@lmiller1990 lmiller1990 requested a review from a team February 13, 2023 11:34
@astone123
Copy link
Contributor

Any reason the Vite icon looks different between community and first-party frameworks?
Screenshot 2023-02-13 at 11 10 19 AM
Screenshot 2023-02-13 at 11 10 34 AM

@astone123
Copy link
Contributor

I'm also not seeing the mount log in the command log... it looks like in your adapter you call Cypress.log to print this out but it's not showing up for some reason

@ZachJW34
Copy link
Contributor

@astone123 Looked into the Vite logo issue. Kind of convoluted but this is what is happening:

  • we use vite-svg-loader to load svgs which has an optimization step. Something like <linear-gradient id="my-unique-id"> gets transformed to <linear-gradient id="a"> (minification)
  • We load an external icon that has <linear-gradient id="a"> defined, one of the gradients gets overwritten.

The solidjs svg icon we are loading is stomping on the vite logo since they both end up with the same id for the gradient. A fix for this is to make the external svg have unique ids, so rather than a it would be solidjs-a. We could also try and fix this within our Vite config by modifying the options we pass to vite-svg-loader. I tried this but ended up breaking most of our icons 😅

I wouldn't say this is a blocker though

Copy link
Contributor

@ZachJW34 ZachJW34 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! We should hold off on merging this feature branch till after the release

@marktnoonan
Copy link
Contributor

Yeah this was fine for me with no issues that weren't already known. Very happy to see it. Will hold off on approval till we see if there's more changes coming for the race condition #25780 (comment)

@lmiller1990 lmiller1990 changed the title feat: Public API for CT Framework Definitions [NO MERGE YET] feat: Public API for CT Framework Definitions Feb 13, 2023
@astone123
Copy link
Contributor

Any idea why we're not seeing the log message in the command log when the component mounts? I was messing around and wrote an adapter for Preact and my log wasn't showing up either. Looks like this is the same thing that we do for first-class integrations, but it works there.

@lmiller1990
Copy link
Contributor Author

lmiller1990 commented Feb 14, 2023

@astone123 Maybe you need a default options.log ?? true? Looking at your code, options defaults to {}. so when you do if (Cypress.log) it will always be false.

Eg maybe: astone123/cypress-ct-test-preact#1

Official adapters do log !== false (see here for example). I think you might need to do something similar.

Copy link
Contributor

@jordanpowell88 jordanpowell88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lmiller1990 Is the defineComponentFramework() function not suppose to include the icon property? I noticed this when trying to use it to help with the qwik implementation

@lmiller1990
Copy link
Contributor Author

@lmiller1990 Is the defineComponentFramework() function not suppose to include the icon property? I noticed this when trying to use it to help with the qwik implementation

@jordanpowell88 Yep, just noticed this myself. Fixed 598be2d

@lmiller1990 lmiller1990 changed the title [NO MERGE YET] feat: Public API for CT Framework Definitions feat: Public API for CT Framework Definitions Feb 21, 2023
@lmiller1990
Copy link
Contributor Author

lmiller1990 commented Feb 21, 2023

This is looking good, did final smoke test on windows, all good. Known CI failures are unrelated and fixed by: #25889

I am going to admin merge this as CI was green and the only change since was to resolve conflicts in circleci/workflows.yml.

@lmiller1990 lmiller1990 merged commit 1d3aab9 into develop Feb 21, 2023
@lmiller1990 lmiller1990 deleted the feature/ct-public-api branch February 21, 2023 07:43
mjhenkes added a commit that referenced this pull request Feb 21, 2023
* fix: update newProject ref when switching between organizations in SelectCloudProjectModal (#25730)

* chore: debug page tooltip distance and artifact border (#25727)

* misc: debug page tooltip distance and artifact border

* add changelog entry

* fix CT test

* fix: Improve error handling around calls to `this.next` in middleware (#25702)

* chore: update changelog validation example (#25742)

* misc: improve debug loading text wrap responsiveness (#25703)

* misc: Increase max failures in IATR badge to 99 (#25737)

* chore: exclude collaborator issues/PRs from triage project (#25769)

* feat: add --auto-cancel-after-failures flag (#25237)

Co-authored-by: Emily Rohrbough <[email protected]>
Co-authored-by: Matt Schile <[email protected]>
Co-authored-by: Ryan Pei <[email protected]>
Co-authored-by: Emily Rohrbough <[email protected]>

* chore: Update v8 snapshot cache (#25592)

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* Update update_v8_snapshot_cache.yml

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

---------

Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>
Co-authored-by: cypress-bot[bot] <47117332+cypress-bot[bot]@users.noreply.github.com>

* fix: implement new graphql fields for spec counts (#25757)

Co-authored-by: Stokes Player <[email protected]>
Co-authored-by: Mike Plummer <[email protected]>

* feat: Bundle cy.origin() dependencies at runtime (#25626)

Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>

* chore: remove zenhub from release process (#25701)

Co-authored-by: Matt Schile <[email protected]>

* feat: add Cypress.Commands.overwriteQuery (#25674)

* feat: add Cypress.Commands.overwriteQuery

Co-authored-by: Emily Rohrbough <[email protected]>
Co-authored-by: Zach Bloomquist <[email protected]>

* fix: spawn child process with process.env in macOS arm64 (#25753)

Co-authored-by: Matt Schile <[email protected]>
Co-authored-by: Emily Rohrbough <[email protected]>
Co-authored-by: Zach Bloomquist <[email protected]>

* chore: lint system tests in CI (#25673)

* fix: Suppress filesystem errors during glob search (#25774)

* chore: issue with ts-loader missing in binary and problematic esbuild norewrite construct (#25797)

* chore: update changelog linting (#25809)

* docs(guides): add more detail to code-signing (#25794)

Co-authored-by: Emily Rohrbough <[email protected]>

* chore: update workflows.yml to include the v8 snapshot update branch (#25784)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>

* chore: internal request preflight (#25772)

---------

Co-authored-by: Emily Rohrbough <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>
Co-authored-by: Matt Henkes <[email protected]>
Co-authored-by: Zach Bloomquist <[email protected]>

* chore: bump for 12.6.0 release (#25812)

* chore: release @cypress/webpack-batteries-included-preprocessor-v2.4.0

[skip ci]

* chore: release @cypress/webpack-preprocessor-v5.17.0

[skip ci]

* test: skip flaky GitDataSource test (#25825)

* chore: making our add-to-triage-board workflow reusable within the Cypress-io org (#25820)

* chore: Making our add to triage workflow callable from other projects inside the Cypress-io org in Github

* chore: updated cypress-example-kitchensink version (#25828)

* fix: duplicate and expired cookies (#25761)

* chore: add regression tests for duplicate cookies and bad expiry times

* avoid prepending domain with dot for cookies that are set with the server side jar. This is to avoid the cookie being duplicated if it is set or overridden in a different context (request that can actually set the cookie or via document.domain)

* feat: use cookie.toString() in the cookie patch to more accurately set cookies on the document, which should include other properties besides key=value

* fix: add logic to handle expired cookies in the document.cookie patch, as well as in CDP

* chore: build binary for cookie fixes for users to test

* chore: change name of fixture to something more accurate

* chore: comment why we are using the toughCookie toString method in the patch

* [run ci]

* chore: add changelog entry

* [run ci]

* fix: revert back to key=value when getting document.cookie as those are the only values are displayed (oversight on my end)

* [run ci]

* chore: make compatible with cypress.require

* fix: add tests for hostOnly/non hostOnly cookies to make sure property gets sent up to automation client correctly. No longer need custom cookie prop to determine destination

* [run ci]

* fix: stale unit test

* chore: adjust comments

* [run ci]

* fix: bad domain logic

* [run ci]

* chore: remove irrelevant comment

* [run ci]

* fix: adjust cookie login text to spec hostOnly cookie within the cookie patch. This should yield the same behavior as we are bound to same origin within the spec bridge

* [run ci]

* [run ci]

* fix: allow for cookies on request of same key to take precedence over cookies in the jar, regardless of how many hierachy cookies exist in the jar

* chore: fix cookie misc tests for cy.origin (dont run cy.origin)

* [run ci]

* chore: skip misc cookie tests in webkit as headless behavior doesn't clear cookies between tests correctly

* Revert "fix: allow for cookies on request of same key to take precedence over cookies in the jar, regardless of how many hierachy cookies exist in the jar"

This reverts commit 17de188.

* [run ci]

* chore: split changelog entry into two parts

* chore: update logic to remove else statement and add comments

* [run ci]

* chore: readd windows snapshot branch in workflows

* [run ci]

* chore: fix workflows from bad merge

* [run ci]

* Revert "chore: split changelog entry into two parts"

This reverts commit 4352ef5.

* [run ci]

* fix: Fix type definitions for cy.reload() (#25779)

Co-authored-by: Emily Rohrbough <[email protected]>

* misc: Debug header updates (#25823)

* fix: allow running tests outside Vite project root folder (#25801)

* fix: allow running tests outside Vite project root folder

* update snapshots

* add changelog entry

---------

Co-authored-by: Lachlan Miller <[email protected]>

* fix: mount component in [data-cy-root] (#25807)

* fix(angular): mount component in [data-cy-root]

* fix e2e test

* add changelog entry

* changelog [skip ci]

* changelog

---------

Co-authored-by: Lachlan Miller <[email protected]>

* chore: updating add to triage baord github action to use org secret (#25868)

* chore: updating add to triage board github action to use org secret

* chore: release @cypress/angular-v2.0.2

[skip ci]

* chore: release @cypress/vite-dev-server-v5.0.3

[skip ci]

* chore: Update v8 snapshot cache (#25822)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>

* feat: support host only cookies (#25853)

* feat: allow setCookie API to take a hostOnly option

* chore: add jsdoc/typescript description to render to users

* chore: add changelog entry

* [run ci]

* chore: fix types

* chore: fix cookie login tests

* chore: update e2e cookie system tests

* [run ci]

* chore: fix cookie command tests. localhost cookies are calculated as hostOnly, which is consistent with how cypress works today

* chore: fix system tests for cookies.

* [run ci]

* chore: fix system tests

* chore: skip hostOnly assertions in webkit (for now)

* [run ci]

* chore: add property definitions to setCookieOptions

* [run ci]

* chore: add comments to hostOnly prop in firefox when setting a cookie

* fix(webpack-dev-server): touch component-index during onSpecsChange to avoid writing to app file (#25861)

* testing: try disabling uTimesSync and see what happens

* build binaries [run ci]

* fix: touch component index file instead of browser.js

* build binaries [run ci]

* update test

* fix test

* add test for custom HTML file in config

* use existing component index in webpack-dev-server unit tests

---------

Co-authored-by: Lachlan Miller <[email protected]>

* chore: release @cypress/webpack-dev-server-v3.2.4

[skip ci]

* chore: improve types for server automation cookie client (#25836)

* chore: improve types for automation cookies

* [run ci]

* fix: the cookie_behavior tests by syncing cookies immediately if … (#25855)

* fix: fix the cookie_behavior tests by syncing cookies immediately if the application is already stable

* chore: add changelog entry

* [run ci]

* chore: address comments from code review

* feat: Public API for CT Framework Definitions (#25780)

* chore: rework component onboarding in launchpad (#25713)

* chore: refactoring and types

* rework source of frameworks

* revert rename

* fix tests

* fix more tests

* types

* update code

* use same public API internally

* rename interfaces

* rename

* work on dev server api

* fix types

* fix test

* attempt to support getDevServerConfig

* tests

* add function to define framework [skip ci]

* rework a lot of types

* fix test

* update tests and types

* refactor

* revert changes

* lint

* fix test

* revert

* remove

* add "community" label [skip ci]

* refactor

* types

* lint

* fix bug

* update function name

* address feedback

* improve types with Pick

* refactor using type guard

* correct label

---------

Co-authored-by: Zachary Williams <[email protected]>

* chore: typing error

* feat: scan for 3rd party ct plugins (#25749)

* chore: refactoring and types

* rework source of frameworks

* revert rename

* fix tests

* fix more tests

* types

* update code

* use same public API internally

* rename interfaces

* rename

* work on dev server api

* fix types

* fix test

* attempt to support getDevServerConfig

* tests

* add function to define framework [skip ci]

* rework a lot of types

* fix test

* update tests and types

* refactor

* revert changes

* lint

* fix test

* revert

* remove

* add "community" label [skip ci]

* refactor

* types

* lint

* fix bug

* update function name

* address feedback

* feat: scan for 3rd party ct plugins

* add e2e test

* unit tests [run ci]

* tweak resolution

* rebase, address comments

* fix windows paths

* remove .gitignore

* fix test

---------

Co-authored-by: Lachlan Miller <[email protected]>

* lint config

* spacing

* try fix race cond

* fix import error

* build binary

* try update snapshot

* try using require

* support namespaced definitions (#25804)

* remove category

* add icon prop

* support esm -> cjs compiled typescript

* fix test

* misc: add CTA footer to launchpad framework dropdown (#25831)

* remove test project dependencies

* rebase

* windows

* windows again

* add changelog entry

* changelog

* revert workflow

* remove worklfow

---------

Co-authored-by: Zachary Williams <[email protected]>
Co-authored-by: Adam Stone-Lord <[email protected]>

* chore: release @cypress/webpack-dev-server-v3.3.0

[skip ci]

* fix: Add missing error message when `req.continue` is used incorrectly (#25884)

---------

Co-authored-by: Adam Stone-Lord <[email protected]>
Co-authored-by: Zachary Williams <[email protected]>
Co-authored-by: Mike Plummer <[email protected]>
Co-authored-by: Matt Schile <[email protected]>
Co-authored-by: Alejandro Estrada <[email protected]>
Co-authored-by: Emily Rohrbough <[email protected]>
Co-authored-by: Ryan Pei <[email protected]>
Co-authored-by: Emily Rohrbough <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: cypress-bot[bot] <2f0651858c6e38e0+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>
Co-authored-by: cypress-bot[bot] <47117332+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Mark Noonan <[email protected]>
Co-authored-by: Stokes Player <[email protected]>
Co-authored-by: Chris Breiding <[email protected]>
Co-authored-by: Zach Bloomquist <[email protected]>
Co-authored-by: willmsC <[email protected]>
Co-authored-by: Zach Bloomquist <[email protected]>
Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Tim Griesser <[email protected]>
Co-authored-by: Matt Henkes <[email protected]>
Co-authored-by: semantic-release-bot <[email protected]>
Co-authored-by: Ben M <[email protected]>
Co-authored-by: Bill Glesias <[email protected]>
Co-authored-by: Podles <[email protected]>
Co-authored-by: Paolo Caleffi <[email protected]>
Co-authored-by: Lachlan Miller <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Public API for Component Testing Onboarding
5 participants