Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cypress snapshotting causes attributeChangedCallback to be triggered on custom elements #7187

Closed
benjaminknauer opened this issue Apr 30, 2020 · 5 comments · Fixed by #8080 · May be fixed by Omrisnyk/npm-lockfiles#145 or Omrisnyk/npm-lockfiles#164
Assignees
Labels
type: unexpected behavior User expected result, but got another

Comments

@benjaminknauer
Copy link

benjaminknauer commented Apr 30, 2020

The documentation advertises that all websites can be tested with cypress. Unfortunately, I cannot confirm this for websites that use custom elements (https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_custom_elements).

During the checks for elements with Cypress, the custom elements are apparently cloned or newly created. This means that the constructor of the custom element and possibly the attributeChangedCallback is called.

The functionality of the application is unfortunately changed by the Cypress tests! If, for example, a global variable is written in these lifecycle methods, a fetch request is executed, or the like, this happens several times in the test.

Here is a cypress project through which this behavior can be reproduced:
https://gitlab.com/benjaminknauer/cypress-custom-elements-bug

The project runs tests against this page, which contains a custom element that prints calls of the lifecycle methods to the console: https://benjaminknauer.gitlab.io/cypress-custom-elements-bug/

Best regards
Benjamin

Current behavior:

Custom element lifecycle methods are called several times.
The execution of the tests influences the application.

image

Desired behavior:

Custom element lifecycle methods should be called just once. The execution of the application should not be affected by the tests.

image

Test code to reproduce

Project with cypress tests
https://gitlab.com/benjaminknauer/cypress-custom-elements-bug

context("customElement - broken", () => {
    it("customElement", () => {
        // when
        cy.visit("https://benjaminknauer.gitlab.io/cypress-custom-elements-bug");

        // then
        cy.get(".nonExistentElement").should("be.visible");
    });
});

Site with a custom element to run test against:
https://benjaminknauer.gitlab.io/cypress-custom-elements-bug/

Versions

Cypress 4.5.0 / Chrome / Fedora 30

@jennifer-shehane
Copy link
Member

jennifer-shehane commented May 4, 2020

You can recreate this by running the following test:

it("customElement", () => {
  cy.visit("https://benjaminknauer.gitlab.io/cypress-custom-elements-bug");
  cy.get(".nonExistentElement").should("be.visible");
});

attributeChangedCallback Invoked each time one of the custom element's attributes is added, removed, or changed. Which attributes to notice change for is specified in a static get observedAttributes method

I am fairly confident that this is related to this line of code in Cypress where we are calling clone on the body of the application under test to create snapshots for review in the Test Runner. This seems to have been a part of the code for at least a few years. f294f6a

If I comment out this line - the attributeChangedCallback is no longer logged a bunch of times:

https://github.com/cypress-io/cypress/blob/develop/packages/driver/src/cy/snapshots.js#L150

I'm not sure why clone causes the attributeChangedCallback to be called exactly.

Workaround

Turn off snapshotting of tests by setting numTestsKeptInMemory: 0 in your configuration. This will bypass the code that's calling snapshotting altogether.

@jennifer-shehane jennifer-shehane changed the title Cypress tests affect the execution of applications that use custom elements Cypress snapshotting causes attributeChangedCallback to be triggered on custom elements May 4, 2020
@cypress-bot cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label May 4, 2020
@jennifer-shehane jennifer-shehane added type: unexpected behavior User expected result, but got another stage: needs investigating Someone from Cypress needs to look at this and removed stage: needs investigating Someone from Cypress needs to look at this labels May 4, 2020
@benjaminknauer
Copy link
Author

Please note that the constructor method is also called several times. When a custom element instance is created and if the custom element has a DOM attribute, the attributeChangedCallback is executed and the value changes from null to the value set in the DOM. This is normal behavior for custom elements.

@goofballLogic
Copy link

goofballLogic commented May 20, 2020

in our case, the custom element triggers an XHR so the problem is particularly bad

@cypress-bot cypress-bot bot added stage: needs review The PR code is done & tested, needs review stage: work in progress and removed stage: needs investigating Someone from Cypress needs to look at this stage: needs review The PR code is done & tested, needs review labels Jul 24, 2020
@panzarino panzarino self-assigned this Jul 26, 2020
@cypress-bot cypress-bot bot added stage: needs review The PR code is done & tested, needs review and removed stage: work in progress labels Jul 27, 2020
@cypress-bot cypress-bot bot added stage: work in progress stage: needs review The PR code is done & tested, needs review and removed stage: needs review The PR code is done & tested, needs review stage: work in progress labels Jul 27, 2020
@cypress-bot cypress-bot bot added stage: pending release and removed stage: needs review The PR code is done & tested, needs review labels Jul 28, 2020
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Jul 28, 2020

The code for this is done in cypress-io/cypress#8080, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@cypress-bot
Copy link
Contributor

cypress-bot bot commented Aug 3, 2020

Released in 4.12.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v4.12.0, please open a new issue.

@cypress-bot cypress-bot bot locked as resolved and limited conversation to collaborators Aug 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.