-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide a way for custom commands to retry on failed should() #3109
Comments
@alexkrolick can you provide the code for |
See testing-library/cypress-testing-library#30 - there are a number of custom commands |
@alexkrolick I see, you're asking for plugin authors to have access to the retry mechanism, custom commands using |
Yes exactly |
This workaround works: cy.getByTestId = (id) => cy.get(`[data-test-id="${id}"]`); |
The custom queries we are trying to support aren't wrappers around jQuery attribute selectors; some of them use multiple DOM traversals to match labels to inputs, for example. |
I've been looking at this for a bit now and digging into the source code gave me the command However, this command is not documented and can, therefore, be a bit iffy to work with at the moment. The way I was able to figure out how to use it was again by looking to the source of the default commands found here: https://github.com/cypress-io/cypress/tree/develop/packages/driver/src/cy/commands When you get it working it works great, but expect to spend a lot of time tinkering. It helps to use the same basic structure that's used in default commands. I'm at a point where I got it working perfectly except for the edit: I got it working now! The basic format should look something like the following. I left out logging, options etc for clarity. Cypress.Commands.add('aThing', (element, options={}) => {
/**
* This function is recursively called untill the timeout passes or the upcomming
* assertion passes. Keep this function as fast as possible.
*
* @return {Promise}
*/
function resolveAThing() {
// Resolve a thing
const aThing = $(element).attr('aThing');
// Test the upcomming assertion where aThing is the value used to assert.
return cy.verifyUpcomingAssertions(aThing, options, {
// When the upcoming assertion failes first onFail is called
// onFail: () => {},
// When onFail resolves onRetry is called
onRetry: resolveAThing,
});
}
return resolveAThing();
}); |
You can make an arbitrary function retry and pass along its return value using a small hack that combines Here's an example of a custom command that makes sure all cy.get('h2')
.should(($els) => $els.fn = () => {
return $els.toArray().map(el => el.innerText)
})
.invoke('fn')
.should(headers => {
const sortedHeaders = headers.concat().sort()
expect(headers).deep.eq(sortedHeaders)
}) orbetter yet, add a new command that does this called Cypress.Commands.addAll({
prevSubject: 'optional',
},
{
try: (subject, fn) => {
return cy.wrap({ try: () => fn(subject) }, { log: false })
.invoke('try')
},
}) and use it like: cy.get('h2').try(($els) => {
return $els.toArray().map(el => el.innerText)
})
.should(headers => {
const sortedHeaders = headers.concat().sort()
expect(headers).deep.eq(sortedHeaders)
}) without a cypress parent command:cy.try(() => {
const h2s = cy.state('document').querySelectorAll('h2')
return Array.from(h2s).map(el => el.innerText)
}).should(headers => {
const sortedHeaders = headers.concat().sort()
expect(headers).deep.eq(sortedHeaders)
}) |
It's a nice idea to add a command that is basically Couple of ideas: Name it retry cy.get('foo')
.retry((elem) => {
// ...
}); Name it thenTry cy.get('foo')
.thenTry((elem) => {
// ...
}); Overwrite then cy.get('foo')
.then((elem) => {
// ...
}, {retry: true}); |
What's the status on this? I've got a two-part const getBalance = () => {
return cy.wrap(new Cypress.Promise((resolve, reject) => {
cy.get('h1').children('span').invoke('text').then(whole => {
cy.get('h3').children('span').invoke('text').then(fraction => {
cy.log(`Got balance: ${whole}${fraction}`)
resolve(`${whole}${fraction}`)
})
})
}))
}
describe('Test', () => {
it(`Should wait until balance is non-zero`, () => {
// cy.wait(2000)
getBalance().should('not.equal', '0.00')
})
}) The above runs I can get the above test to pass by uncommenting the My second attempt, inspired by the conversation between @Lakitna and @bkucera above. Cypress.Commands.addAll({ prevSubject: 'optional' }, {
retry: (subject, fn) => {
return cy.wrap({ retry: () => fn(subject) }, { log: false }).invoke('retry')
},
})
describe('Test', () => {
cy.retry(getBalance).should('not.equal', '0.00')
}) The above errors out with |
There is no official stance on this yet. But that won't stop us! I've made a quick implementation of this idea using the retry name and tried to document it a bit so you can alter it. I only tested if it retries, I have no idea what its limitations are. const _ = Cypress._;
const $ = Cypress.$;
/**
* Basically `then`, but will retry if any upcoming assertion fails
* @param {JQuery} subject
* @param {function} fn
* @param {boolean} [options.log=true]
* Log to Cypress bar
*
* @return {*}
*/
Cypress.Commands.add('retry', {prevSubject: 'optional'}, (subject, fn, options={}) => {
_.defaults(options, {
log: true,
});
// Setup logging
const consoleProps = {
'Applied to': $(subject),
};
if (options.log) {
options._log = Cypress.log({
$el: $(subject),
name: 'retry',
message: fn.name,
consoleProps: () => {
return consoleProps;
},
});
}
/**
* This function is recursively called untill timeout or the upcomming
* assertion passes. Keep this function as fast as possible.
*
* @return {Promise}
*/
function resolve() {
const result = fn(subject);
// Update logging
if (options.log) {
consoleProps.Yielded = result;
}
// Test the upcomming assertion where result is the value used to assert.
return cy.verifyUpcomingAssertions(result, options, {
// When the upcoming assertion failes first onFail is called
// onFail: () => {},
// When onFail resolves onRetry is called
onRetry: resolve,
});
}
return resolve();
}); The full extend of my tests: it('retries', function() {
let c = 0;
cy.retry(() => ++c)
.should('equal', 5);
}); |
Thank you @Lakitna 🙏 An(other) example using But I tinkered a bit, got it working & was able to remove the race condition (ie a // cy.resolve(fn).should(blah) will re-run the promise-returning fn until
// the value it resolves to passes the assertion
Cypress.Commands.add('resolve', { prevSubject: 'optional' }, (subject, fn, opts={}) => {
const resolve = () => {
fn(subject).then(res => cy.verifyUpcomingAssertions(res, opts, { onRetry: resolve }))
}
return resolve();
});
// an example function that returns a Cypress.Promise
const getBalance = () => {
return cy.wrap(new Cypress.Promise((resolve, reject) => {
cy.get('h1').children('span').invoke('text').then(whole => {
cy.get('h3').children('span').invoke('text').then(fraction => {
cy.log(`Got balance: ${whole}${fraction}`)
resolve(`${whole}${fraction}`)
})
})
}))
}
describe('Test', () => {
it(`Should wait until balance is non-zero`, () => {
cy.resolve(getBalance).should('not.contain', '0.00')
})
}) |
Behold, the birth of the module I've made the extension on the For more details see the repo at https://github.com/Lakitna/cypress-commands |
Hi, I was using the @Lakitna example to implement my own retry mechanism to assert three different
How can I make the |
For this problem, I ended up creating my own implementation of "retry-ability" (in TypeScript): interface RetryOptions<T> extends Cypress.Timeoutable {
interval: number;
default?: T;
throw: boolean;
}
export function retry<T>(checker: () => T, confirmer: (result: T) => boolean, options?: Partial<RetryOptions<T>>): Cypress.Chainable<T>;
export function retry<T>(checker: () => T, confirmer: (result: T) => boolean, originalOptions?: Partial<RetryOptions<T>>): Cypress.Chainable<T> {
const options: RetryOptions<T> = {
...{ timeout: Cypress.config('defaultCommandTimeout'), interval: 200, throw: true },
...originalOptions
};
return cy.wrap(
new Promise<T>((resolve, reject) => {
const startTime = Date.now();
const result = checker();
if (confirmer(result)) {
resolve(result);
return;
}
const intervalId = setInterval(() => {
const currentTime = Date.now();
const endTime = startTime + options.timeout;
if (currentTime >= endTime) {
if (options.throw) {
reject(new Error(`Timed out while retrying after ${options.timeout}ms`));
} else if ('default' in options) {
resolve(options.default);
} else {
resolve();
}
clearInterval(intervalId);
return;
}
const result = checker();
if (confirmer(result)) {
resolve(result);
clearInterval(intervalId);
return;
}
}, options.interval);
}),
{ log: false, timeout: options.timeout + options.interval }
);
} Usage Example: retry(
() => {
const selector = 'your selector';
return Cypress.$(selector);
},
elements => elements.length > 0,
{ throw: false, default: Cypress.$() }
); |
Here's my naive implementation (Typescript, but can easily be converted to Javascript): const retryableResolver = (getter: (...x: any) => any, ...args) => {
const end = Date.now() + config('defaultCommandTimeout');
const resolve = () => Cypress.Promise
.try(() => getter(...args))
.then(value => Date.now() > end ? value : cy.verifyUpcomingAssertions(value, {}, {onRetry: resolve}));
return resolve();
}; And then use it when adding a command, here's an example that collects texts of passed subject elements and returns them as a string array: const texts = elements => elements.map((i, e) => $(e).text()).get();
Cypress.Commands.add(
'texts',
{prevSubject: 'element'},
elements => retryableResolver(texts, elements)); |
Does anyone have a nice way to effectively replace cy.get with simpler logging, while keeping the element highlighting and all that? I want something like the following, but I want the logging behavior to be synchronous the same way as the real export function getByTestId<E extends HTMLElement>(
testId: string,
options?: Partial<Cypress.Loggable & Cypress.Timeoutable & Cypress.Withinable & Cypress.Shadow>
): Cypress.Chainable<JQuery<E>> {
const props = {
Subject: null as JQuery<E>,
Yielded: null as unknown,
}
const logConf: Partial<Cypress.LogConfig> = {
$el: undefined,
name: 'getByTestId',
displayName: 'get(testid)',
message: testId,
consoleProps: () => props,
}
const log = Cypress.log(logConf)
return cy.get(`[data-testid=${testId}]`, {log: false}) All my attempts to retry either end up with an infinite loop, or failing due to element not existing when it's expected that the element doesn't exist. |
This could work, retrying for 5 times within the interval of 1s
|
I am also unable to get the above solutions to work consistently. I have the following assertion:
And the following code:
@jennifer-shehane, @Lakitna , @bohendo , is there any reason why the above code would fail like so? I would expect since that final assertion failed, it should retry the whole chain, but it appears not to do that. Another shot of the command log: I'm hoping to get a solution without intervals or timeout using Cypress methods. Also, the only way our team has been avoiding this issue has been by adding |
Hello! I just wanted to let you all know that we're adding this in Cypress 12.0.0. It turns out that "ability to retry commands" was pretty central to resolving #7306 , and as part of that effort, we're exposing the This will be going out in a couple of weeks with Cypress 12, but if you want a bit of a preview, here's the PR we have open with cypress-testing-library to update them to use the new API: https://github.com/testing-library/cypress-testing-library/pull/238/files As an easier way to get started, here's a preview of the API docs: https://deploy-preview-4835--cypress-docs.netlify.app/api/cypress-api/custom-queries, and the re-written guide on retry-ability that discusses queries vs. other commands: https://deploy-preview-4835--cypress-docs.netlify.app/guides/core-concepts/retry-ability#Commands-Queries-and-Assertions The docs are still in review, but I'd welcome any comments or questions on them if people want to read it / try out the pre-release builds of Cypress 12 (latest as of now: b9d053e#comments). |
Cypress 12.0.0 is going out today, which should address these needs. If there's still anything lacking once you take a look at |
This was release in 12.0.0 |
Current behavior:
#1210 (comment)
Desired behavior:
Versions
Cypress 3.1.4
Related
The text was updated successfully, but these errors were encountered: