Cypress snapshotting causes attributeChangedCallback to be triggered on custom elements

[benjaminknauer]

The documentation advertises that all websites can be tested with cypress. Unfortunately, I cannot confirm this for websites that use custom elements (https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_custom_elements).

During the checks for elements with Cypress, the custom elements are apparently cloned or newly created. This means that the constructor of the custom element and possibly the attributeChangedCallback is called.

The functionality of the application is unfortunately changed by the Cypress tests! If, for example, a global variable is written in these lifecycle methods, a fetch request is executed, or the like, this happens several times in the test.

Here is a cypress project through which this behavior can be reproduced:
https://gitlab.com/benjaminknauer/cypress-custom-elements-bug

The project runs tests against this page, which contains a custom element that prints calls of the lifecycle methods to the console: https://benjaminknauer.gitlab.io/cypress-custom-elements-bug/

Best regards
Benjamin

Current behavior:

Custom element lifecycle methods are called several times.
The execution of the tests influences the application.

Desired behavior:

Custom element lifecycle methods should be called just once. The execution of the application should not be affected by the tests.

Test code to reproduce

Project with cypress tests
https://gitlab.com/benjaminknauer/cypress-custom-elements-bug

context("customElement - broken", () => {
    it("customElement", () => {
        // when
        cy.visit("https://benjaminknauer.gitlab.io/cypress-custom-elements-bug");

        // then
        cy.get(".nonExistentElement").should("be.visible");
    });
});

Site with a custom element to run test against:
https://benjaminknauer.gitlab.io/cypress-custom-elements-bug/

Versions

Cypress 4.5.0 / Chrome / Fedora 30

[jennifer-shehane]

You can recreate this by running the following test:

it("customElement", () => {
  cy.visit("https://benjaminknauer.gitlab.io/cypress-custom-elements-bug");
  cy.get(".nonExistentElement").should("be.visible");
});

attributeChangedCallback Invoked each time one of the custom element's attributes is added, removed, or changed. Which attributes to notice change for is specified in a static get observedAttributes method

I am fairly confident that this is related to this line of code in Cypress where we are calling clone on the body of the application under test to create snapshots for review in the Test Runner. This seems to have been a part of the code for at least a few years. f294f6a

If I comment out this line - the attributeChangedCallback is no longer logged a bunch of times:

https://github.com/cypress-io/cypress/blob/develop/packages/driver/src/cy/snapshots.js#L150

I'm not sure why clone causes the attributeChangedCallback to be called exactly.

Workaround

Turn off snapshotting of tests by setting numTestsKeptInMemory: 0 in your configuration. This will bypass the code that's calling snapshotting altogether.

[benjaminknauer]

Please note that the constructor method is also called several times. When a custom element instance is created and if the custom element has a DOM attribute, the attributeChangedCallback is executed and the value changes from null to the value set in the DOM. This is normal behavior for custom elements.

[goofballLogic]

in our case, the custom element triggers an XHR so the problem is particularly bad

[cypress-bot]

The code for this is done in cypress-io/cypress#8080, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

[cypress-bot]

Released in 4.12.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v4.12.0, please open a new issue.