Future of Angular E2E & Plans for Protractor

[kyliau]

TLDR

The Angular team plans to end development of Protractor at the end of 2022 (in conjunction with Angular v15).

Why?

Protractor was created in 2013 when WebDriver APIs were not yet a standard and end-to-end (e2e) tests were hard to write due to lack of support for async / await. To solve this problem, Protractor wraps selenium-webdriver and abstracted asynchronous operations from developers with the use of Control Flow.

Since then, the JavaScript standard and ecosystem advanced considerably, providing modern syntax and much better development tools. Nonetheless, Protractor is not able to leverage such technology without forcing users to rewrite their tests. Meanwhile, robust alternatives have emerged in the web testing space. Developers will see more benefits from adopting a more modern testing tool than from updating to a breaking version of Protractor which does not provide additional functionality or developer ergonomic improvements.

We would like to hear from the community on

• the deprecation timeline
• what we can do to provide reliable integration with third-party solutions
• how users can transition by following migration guidelines
• additional concerns that would ensure a smooth transition

This RFC will close on Friday April 16, 2021.

---

State of Protractor

The Angular team created Protractor in 2013 when WebDriver APIs were not yet a standard and end-to-end (e2e) tests were challenging to set up. The proliferation of callback patterns in JavaScript back then made asynchronous operations difficult to write and manage.

Protractor, echoing the approach taken by the underlying selenium-webdriver, solved this problem by managing promises via Control Flow. Control Flow makes asynchronous calls appear synchronous, thereby avoids the use of Promises entirely.

This strategy worked well for some time, but as the JavaScript standards and toolings evolve, Protractor regressed. Starting from ES2017, async / await makes handling Promises significantly easier because they no longer have to be chained. However, async / await is fundamentally incompatible with Control Flow, and support for Control Flow is dropped from selenium-webdriver v4.0 completely. Protractor, being dependent on selenium-webdriver, is not able to upgrade to the new version without introducing a huge breaking change and forcing users to do a migration for all their tests.

Besides dropping support for Control Flow, making Protractor compatible with ES2017 involves a significant amount of work to overhaul its implementation. Legacy dependencies such as jasminewd2 and Q promise library ought to be removed, which will bring about further breaking changes.

Since Protractor was initially designed to support AngularJS, many of its features like locators and mock modules are specific to AngularJS. These features only work in AngularJS and not Angular. They will no longer be relevant once development on AngularJS ceases by December 31, 2021.

Removing Control Flow and dropping AngularJS-specific features would make Protractor essentially just a wrapper around selenium-webdriver that provides no additional functionality.

Current Landscape for Web Testing

Many testing solutions have appeared since the creation of Protractor, and they offer better stability and much improved support for features like cross-browsers testing. Some even eschew the WebDriver standards completely in favor of using the DevTools protocols directly.

In order to understand the adoption trend of these alternatives in the Angular community, we conducted a survey on e2e testing in January 2021. We received close to 1000 responses and got a lot of feedback from the community. A clear signal that we received is that there is no one-size-fits-all solution for all Angular projects out there. Fewer than 20% of the respondents use Protractor in their project. Some users prioritize the cross-browsers support that the WebDriver standards guarantee, whereas other users prefer the stability and flexibility that DevTools protocol offers. A similar survey focused entirely on enterprise customers yielded comparable results.

Moving Forward

Deprecating Protractor is not a decision that is taken lightly by the team. Inside Google, we support thousands of projects and hundreds of thousands of test targets that depend on Protractor. After evaluating the costs and benefits of updating Protractor vs migration effort imposed on users, we conclude that the best strategy to set users up for success in the long term is to encourage migration to a more modern and framework-agnostic testing platform. The reasoning is as follows:

Removing Control Flow from user code is a gigantic effort because it forces users to update all their tests. Google went through this effort, and it took a major infrastructure team half of their time in 2018. Even then, they could only migrate tests written in TypeScript due to the availability of more robust tooling for source code analysis and transformation.

After going through such a massive endeavour, we realize the “new” Protractor is no better than other existing solutions. The only feature that differentiates Protractor from selenium-webdriver at this point is the ability to automatically wait for the application under test to become stable (waitForAngular feature in Protractor).

Although waitForAngular is useful, it strongly couples the testing platform to the Angular framework. Some teams at Google have found that solutions that do not require knowledge of Angular can perform the tests equally well by using a robust retry strategy. We believe this is how e2e testing should be done going forward, and projects in Google are already converging towards a testing platform that is WebDriver compliant and framework agnostic. This allows our web test team to maintain a single solution for all web applications.

Externally, there are many excellent alternatives available to the open source community, such as:

1. Cypress
2. PlayWright
3. Puppeteer
4. Selenium-webdriver
5. TestCafe
6. WebdriverIO

(The list is sorted in alphabetical order and is non-exhaustive.)

Deprecation

The most important focus for the Angular team is to ensure a smooth transition for Protractor users. We have been engaging with different vendors to ensure

• there is sufficient documentation / tooling to help users migrate to other platforms
• third-party platforms integrate well with Angular CLI, so that CLI configurations are automatically handled by vendor-specific builders

So far, the Angular team has reached out to the teams at Cypress and WebdriverIO to achieve the goals above. We will provide regular updates as more resources become available. If there is a particular platform that you’d like us to reach out to, please let us know in the comments below.

Our proposed deprecation timeline is as follows:

• Angular version 12 (May 2021)
  • Announcement of deprecation
  • Angular CLI will not include Protractor for new projects
  • Show warning when @angular-devkit/build-angular:protractor is used
  • Only PRs that address security issues and fixes for browser releases that break Protractor tests will be merged
• Angular version 15 (end of 2022)
  • End of development on Protractor
• August 2023 end of life of Protractor. For more information about the future of Protractor see our blog post

No decision is final until we assess all the feedback from this RFC.

Extended LTS

For users who require extended long-term support after the end of life of Protractor, we suggest reaching out to our external partners:

• flare.build
• herodevs

If your team / company provides similar services, please let us know!

FAQs

1. What about Component Harnesses?
   Angular CDK's component harnesses include a ProtractorHarnessEnvironment for using harnesses in Protractor. This environment will be deprecated and replaced with a new environment backed by selenium-webdriver. The harness APIs themselves will be unchanged.
2. How will Angular CLI handle ng e2e for new projects starting from version 12?
   The CLI will not include a default e2e builder for new projects, and users will decide which third-party builder to install. This is because picking an e2e platform really depends on the requirements of the project and there is no clear best fit for all Angular projects.
3. In version 15, would all Protractor tests break?
   The Protractor builder will continue to work until version 15. After that, we plan to remove the Protractor builder (@angular-devkit/build-angular:protractor), which means ng e2e will no longer run Protractor. However, we are open to revising the timeline based on user feedback. Please let us know in the comments below.
4. Protractor is useful because it automatically waits for the application to become stable. How can I use that in other platforms?
   Under the hood, Protractor uses Testability provided by @angular/core to check if the app is stable. This feature can be integrated into other platforms in a straightforward way. See the example for selenium-webdriver.
5. Which e2e framework is the lowest effort and cost to migrate to from Protractor?
   In terms of APIs, selenium-webdriver is most similar to Protractor because it is used under the hood. If your tests still use Control Flow, you’ll have to remove it first by adding async / await to the Protractor calls. After that, you can replace the Protractor methods with those in selenium-webdriver. Although they are not 1:1 replacements, they are very close.
6. Where will e2e migration strategies be documented?
   We are consolidating a list of migration strategies doc written by vendors and community members. See Links section below.
7. What other packages will be deprecated along with Protractor at the end of 2022?
   • protractor-cookbook
   • jasminewd
   • webdriver-js-extender
   • blocking-proxy
   • webdriver-manager
8. My team just did a migration to disable Control Flow. What are our options?
   It is possible to create a thin wrapper around selenium-webdriver to provide APIs that are compatible with Protractor's. This is an idea that needs further exploration. Please let us know in the comments below if such tool is useful to you.

Open questions

1. Should the Angular team provide a different deprecation timeline?
2. In addition to migration docs and CLI integration, how can the Angular team support Protractor users in the transition to other platforms?
3. What are the alternatives to deprecating Protractor?

Links

This section is a work in progress. We will continue to consolidate more links and migration guides here.

1. Migrating from Protractor to Cypress
   https://nx.dev/latest/angular/modern-angular/protractor-to-cypress
2. Cypress official doc for Protractor users (placeholder link for now, will be live in the future)
   http://on.cypress.io/protractor-to-cypress

[pavankjadda]

Cypress is great alternative. I have been using it since few months, easy to configure and write tests. May be provide CLI option to add it to the new or existing project?

[mgechev]

@pavankjadda we're working with the Cypress team to ensure smooth integration with the Angular CLI.

[batbrain9392]

We have huge protractor e2e code @ Blueface Comcast. A good documentation for common migration use cases would be highly beneficial.

[yjaaidi]

If you are looking for CDK Harness support on Cypress, here it is: https://github.com/jscutlery/test-utils/tree/main/packages/cypress-harness

[amirrustam]

Hi @batbrain9392. I'm Amir from the Cypress team. We're working on helpful migration guidance. To make this material better for you and rest of the community, I'd love to learn more about your project.

If you're up for it, feel free to email, or schedule a chat.

[chihab]

I am quite convinced Cypress offers the best DX out there but no more opinion on ng e2e is definitely the best opinion.

[jessesanders]

I'm glad to see protractor going away and that there is a good plan on how to deprecate it. Selenium/Webdriver is based on technology almost 20 years old and is far too fragile for today's dynamic pages to fulfill the need for a reliable testing platform. Cypress offers a much more consistent and reliable testing platform where testing edge cases becomes almost trivial.

[LanderBeeuwsaert]

We are using both Cypress and Playwright and have found Playwright to be both easier to use and more robust.
Added benefits are amongst others ability to test Safari/webkit, less flaky tests and possibilities for conditional testing.
Can somebody else that is using both frameworks comment on the advantages they see in Cypress vs Playwright?
Are we missing something?

[vsravuri]

@amirrustam

Thanks for your assistance. I too have lot of code written in Protractor to migrate to Cypress. I will schedule a chat with you help you understand my requirement.

[zbagley]

Looking forward to the e2e upgrade, and have my fingers crossed the transition from Protractor will be fairly seamless. Are the any plans for a CLI update to provide a similar tool to ng update for e2e side of the transition? I know how rough updates were at the start, and really hoping the e2e update will be less painful that the early days and closer to how updates are now.

[amirrustam]

Hi @LanderBeeuwsaert. Amir from Cypress team here. Playwright is also a fantastic tool (and team)!

With regards to Safari, we've added WebKit support to the Cypress roadmap.

Can you elaborate more on your mentioning of "more robust"? I can address any potential concerns for you and the community, and your feedback is always helpful and appreciated for improving the tool.

If you're interested in deeper dive conversation on the topic, feel free to email, or schedule a chat.

[amirrustam]

Sure thing @vsravuri, and I extend the same offer to everyone else here as well.
The Cypress team is here to help as much as we possibly can.

[cnishina]

As for # 8 on the FAQs: "It is possible to create a thin wrapper around selenium-webdriver to provide APIs that are compatible with Protractor's."

I am working on this library. It still needs to go through internal open source approvals. 🤞

[alvipeo]

I tried Cypress and then switched to Playwright. More features and easier to use.

[LanderBeeuwsaert]

Hey @amirrustam , let me first say that Cypress for us has helped us out amazingly as well.
For us however there have been some things that have made it difficult (or even not possible) to use.
No framework is perfect, and it's cool to see the Cypress team working on a lot of these improvements, so I'm sure it's gonna get even better when time goes on.

For reference, for us the dealbreakers have been:

• No Webkit, upcoming, that's super: Safari (WebKit) Support cypress-io/cypress#6422 (comment)
• No parallel browser instances. (this one is huge for us, we develop judging software where 20 judges are interacting in parallel. Being able to simulate this with playwright was a big step for us. Also, we run 8 test suites in parallel now on a single computer (albeit with 16 cores :)), cutting our CI testing run times by 8).
• Flaky issues with chained selectors: Re-query elements that are found 'detached' from the DOM cypress-io/cypress#7306 (comment)
• Unintuitive Awaiting: Await-ing Cypress Chains cypress-io/cypress#1417
• No conditional testing: https://docs.cypress.io/guides/core-concepts/conditional-testing#Welcome-wizard

[nartc]

Thanks so much for putting this (and the whole thought process) together for such a complex problem, @kyliau, and the team!

With AngularCLI dropping scaffolding E2E tests out of the box, it will cause more problems for beginners wanting to start with Angular. We pride ourselves so much in being a complete framework with all this built-in tools and seeing the AngularCLI starting to drop each toolings out of its box (like Linting) hurts a bit.

That said, I do understand the concern about being opinionated about which E2E solution to bake in the CLI when there are gazillion options out in the ecosystem. It would be nice to settle with the most-voted option (looks like Cypress won by a landslide here) and provide the option to opt-out of Cypress.

Another option is the documentations need to be very deliberated about the importance of E2E testing. With AngularCLI not scaffolding an E2E solution, the chance of getting into E2E testing is getting even worse IMO

[adisreyaj]

Thanks for taking this decision which was long overdue. Cypress seems to be the obvious way to go given its support and community. And if there aren't any huge turn downs of using it with Angular, I'd say its a win win.

I guess this would give the Angular team more time to invest in the framework core.

[oscarvaldes]

I've had an amazing experience using Cypress as my go-to e2e framework for the past 3 years for both enterprise and small scale projects. The community and its' integration with https://nx.dev/latest/angular/cypress/overview is just an added bonus. Also with the increased popularity of web components today, something that is framework agnostic is almost a must have. Great job and details describing the decision making behind this! It's very clear a lot of careful thought and analysis was behind this decision.

[LeJeanbono]

If you manage to do some Cypress component testing in browser with angular, fell free to use https://github.com/bahmutov/cypress-angular-unit-test

[MikaStark]

Knowning that Protractor will be deprecated. Is there a way to remove it from angular 11 projects properly in order to get ehead of v12 deprecation ?

Is uninstalling protractor, removing e2e target in angular.json and removing e2e folder enough ?

[christian-bromann]

Hey folks 👋 ,
this is Chris maintaining the WebdriverIO project. First of, big thank you to everyone involved developing Protractor, it has been a great framework and WebdriverIO has adopted a couple of feature ideas from it. We are glad that the Angular team has reached out to us to include WebdriverIO as an transition option. The team is happy to offer a free, open governed and flexible framework, hosted under the OpenJS Foundation alongside projects like Node.js and webpack.

WebdriverIO allows you to leverage both the ability to run true cross browser testing based on WebDriver as well as a lot of debugging and introspection capabilities through the DevTools protocol. With the help of various plugins the framework provides running performance and PWA audits through Google Lighthouse as well as mocking and stubing any kind of browser requests. It comes with a Jest style built in assertion library and tons of reporters, some maintained by the project some by the community.

We are currently actively working on the following items to make the transition for Protractor users as smooth as possible:

• build an Angular CLI schematic (see webdriverio/webdriverio-schematics ✅ )
• setup a codemod to help transform Protractor code into WebdriverIO commands, see @wdio/codemod ✅
• [ ] Add Angular framework stabilizer
  Won't do: @cnishina mentioned that there aren't any Angular specific wait mechanism needed anymore
• Write up docs with a step by step description how to transition over (Document the transition from a Protractor to WebdriverIO framework webdriverio/webdriverio#6678 ✅ )

Feel free to raise an issue in the WebdriverIO project if you see anything else from the Protractor ecosystem you want to see adopted. WebdriverIO, as it is right now, can already being used for any kind of Angular projects but with these additional initiatives we hope to help with the transition even further. Cheers!

Update (16/04/2021): we implemented and battle tested a codemod that transforms ~90% of Protractors API and we continue to extend it as we support the migration process of various organisations. A migration guide was drafted and will be published soon.

Update (19/04/2021): we released a migration guide that explains how a Protractor migration can be run using our codemod. The guide will be extended as we get more feedback from folks running into issues.

Update (20/04/2021): a community member has build an expected condition library that is a drop-in replacement for the Protractor one. We will update the codemod within the next days to simplify the migration. Thanks @elaichenkov!

[StanislavKharchenko]

Thanks @kyliau for news and plans share.
I would like to share my opinion and several points which could be critical and need to be revised.

1. First of all - yes, Control Flow should be removed and this is not thing that should live in 2021 and of course in future.
2. Million users around the world are using Protractor and rely on it. Why? Different reasons:
   • simple of usage within modern async/await
   • selenium-webdriver wrapper (selenium is w3c standard now)
   • own config and runner
   • waitForAngular hook
   • Protractor is own @google and @angular project, which adds more confidence (company with the great name)
     Unfortunately not all users know about deprecation plan and still thinking that Protractor project maintained by Google (see above)
3. Migration will be simple for teams using E2E as part of its own web projects based on Angular (using under angular-cli). But, this will be a very huge pain for companies and teams, which have independent E2E infrastructure with Protractor as dependency.

By this concerns, users should be concentrated on migration to unknown 3d-party framework without any guarantee what will with it in future and not on test cases development and E2E coverage.

My propositions and options:

1. Deprecate legacy Protractor which have Control-Flow and notify users (according to current plan till the end of 2022);
2. Introduce some light version of selenium-webdriver wrapper which will have: Protractor API, runner, configuration to use Jasmine 3.X, default hook for Angular testing (waitForAngular) and, the main, modern selenium 4.0 which is fully w3c.
   Here could be sub options:
   2.1 Implement next gen light version of Protractor or pick-up Protractor 6 where control-flow was removed and it on selenium Most of work was done here, just refresh it and it will help users instead of deep investigation of external 3d frameworks, its documentation, transitions and more-more pains. IMO, this will require less time-investments.
   2.2 Another new lib, like teased by Craig @cnishina, with the further support.
   Actually, such lib will have very-very simple support since it will be fully rely on selenium-webdriver, its API and functionality. From your side need just support config+runner, waitForAngular and dependency upgrades from time to time.

We asking to re-thinking about initial plan and have compromise option.
Thank you in advance!

[aboudard]

I think giving the choice on install is the best option.
It's great that Cypress has this success, but it might not be the best default choice for every team and project.
I can name a situation where it can be tricky : behind a corporate proxy ...

@MikaStark I think you can do exactly that, that's what we do when we enable Cypress.

[DavertMik]

Consider using CodeceptJS https://codecept.io as an alternative.
With it, you can easily switch between engines like WebDriver or Playwright. So one engine gets deprecated and a new one emerges you will need to write 0 code to transition.

[santoshyadavdev]

Great to see the Cypress team is extending support for smooth integration with CLI. Looking forward to having Cypress as default for E2E.

[AlexSkorkin]

Hi,

This is Alex from the DevExpress TestCafe team. TestCafe supports many of the mentioned features out of the box:

[V] Safari support, including Safari mobile
[V] Cross-browser support (Chrome, FF, IE, Edge, etc.)
[V] Latest JavaScript/TypeScript Syntax Support
[V] Run parallel browser instances
[V] Robust and stable test cases
[V] Automatic awaiting mechanisms
and much more.

We'd love to chat and answer your questions if you have any. Feel free to contact us at [email protected].

[alvipeo]

Playwright is owned by Microsoft, and they also own TypeScript. Consider this fact too please. They also own VS Code.

[SerkanSipahi]

I recommend that the Angular team should leave everything that saves time to do better from others. Angular has other strengths and those are the ones that should be further developed and focused on. In this context I think that the Angular Component Team is working on the migration to MDC Web. This brings many advantages. You have more time for more important things. What I'm getting at is that the same is true for e2e testing. You should rely on technologies such as Cypress that have gained industry acceptance and basically build Angular tooling (Harnesses, etc) with the Cypress API.

[StanislavKharchenko]

@SerkanSipahi Yes, and we rely on Protractor which have gained industry acceptance since 2013, it solve business needs and we don't understand why we need to rewrite all code base to another framework.
We know that it was developed by @google and this was a main reason why we choose Protractor. We don't know who is the Cypress in the world (but we know who is Google and Angular), we investigated that it very limited for our needs.

And, also, within the same opinion could say like "khmm we evaluate that React (or something else) good, we will stop development on Angular and you can switch to React".

[GSasu]

Can someone explain to me why is Cypress so supported when it's nothing more than overrated version of Selenium 1 (yes you read that correctly Selenium 1) but done in JS. When the community worked so hard to improve that and move away from it to things like DevTools protocol or WebDriver 3/4, why do we want to go back now to something that was horrible?

Please consider the following table before answering 🤔

PROs	CypressJS	CodeceptJS	Playwright	WebDriverIO
	Quick & Easy to setup	Built by a large community of SDETs	Cross browser support	Very mature community
	Fast to implement & debug	Supports multiple test libraries	Built and maintained by a large company (Microsoft)	WebDriver 7 is very fast
	Relatively fast to execute simple tests	Single syntax	Full docker support	Full API testin support
	Modern tool	Multiple reporters supported	Typescript focused out of the box	Large collections of plugins, examples and helpers
	Considered a standard by FE developers	Active community and updates	Fast running	Very good documentation
	Good for startups	Support for mobile web testing	Multi language support	Parallel run
	Large community	Support for hybrid mobile testing	Bidirectional events	Easy debugging
	API mocking support	Support for native mobile testing	Auto-waiting for elements	Full docker support
	Adopted by a lot of large companies	Full docker support with multiple options	Stubbing and mocking through traffic capturing	Multiple reporters supported
		Easy debugging	Supports iFrames	Multiple runners supported
		Multiple debugging options made available	Supports multi-tabs	Supports multi-tabs
		Parallel runs through multi-process	Supports BDD	Full support of iFrames
		Parallel runs through Worker threads	Reporters support	Supports multi-session browser tests
		Support for multiple browsers	Flakiness detection feature	Supports all browsers
		Large number of plugins and helpers made available by the community	Supports multiple runners (Mocha Jasmin Jest)	Supports BDD
		Full support for iFrames, multi tabs and multi browser sessions	Headless Browser with event-driven architecture	Has mobile support (Native, hybrid and web)
		Easy setup for services like Browserstack/Saucelabs	Partial mobile support	Fully event driven
		Full BDD support	Multi session support	Very stable and reliable
		SMART features (Autowaiting and more)	Supports multi-page and third-party implementations	Many maintainers and very mature community
		Easy to keep using the latest versions of packages	Doesn’t generate any files	Supports Shadow DOM
		Can be built to become a distributable package	Parallelisation support	Supports interacting natively with Angular and React elements
		Support Shadow DOM	Fully event driven	Extended support for DevTools Protocol
		Database support		Front-end performance metrics gathering
		Full API support (REST and GraphQL)		Continuously growing and improving with newer and newer features
		Flakiness detection through ReportPortal.io		Can natively talk to Puppeteer instances if needed
		Stubbing and mocking support		Easy debugging
		Fully event driven		Multiple options for Visual testing
		Multiple options for visual testing		Support for REST and GraphQL testing
		Supports single syntax for all libraries supported (WebdriverIO, Playwright, TestCafe, Puppeteer, Protractor etc)		Multi DB engine support
				Full integration with Browserstack/Saucelabs/ReportPortal/Selenoid
				Easy syntax
				Supports mocking and stubbing

CONs	CypressJS	CodeceptJS	Playwright	WebDriverIO
	It has issues scaling up	Requires getting used to the syntax	Fairly new	Can be slow if not setup correctly
	Features offered for free in other tools, either cost money with Cypress or demands you to build it yourself	Does have a learning curve	API is still evolving so things you learn now may change next version	Slow if WDIO version used is lower than v6
	It has difficulties with uploads and downloads	It does require custom code if you want to have a helper allowing you to test kafka for instance	Small community for now	Does not have auto waits for elements
	They are moving more and more features behind the subscription model	Not a framework but a wrapper	Does not support real devices but supports emulators	Retry mechanism needs to be built (however there are already a ton of examples of reliable stable mechanisms built by the community)
	Limited support for iFrames	Supports only JS/TS	Has no support for IE11 or non-browser platforms
	No support for multi-tabs	Mocha only as a runner	Documentations and community are not as good as the other framework yet.
	Can not drive 2 browsers at the same time	Depending on library chosen and configuration tests can be slow (however they can also be very fast)
	Does not provide support of IE or Safari
	Parallel runs require a machine with significant amount of resources
	Does not support any type of mobile interaction
	No Shadow DOM support
	Can’t navigate to a different domain URL
	API testing is inefficient if needed because Cypress always runs a browser, while API testing does not require one
	Developers need to find a lot of workarounds to issues the tooling has
	Flakiness detection is a paid feature instead of being tied to reporting (like many other frameworks out there)
	Supports only JS/TS

[diemol]

👋 everyone!

Here is Diego, a core member from the Selenium project. We'd like to thank the Protractor team and its contributors for their huge efforts over all these years 🤗 Protractor is one of the main reasons why the JS Selenium WebDriver bindings have been highly used during the last years.

To continue this amazing collaboration, we'd like to invite anyone who is interested to provide a solution like or similar to the one mentioned at FAQ#8 (@cnishina how can we support you?), to let us know how we can help at the Selenium project. Feel free to reach out to us through our Slack/IRC channel, we are here and there to help the community.

[LanderBeeuwsaert]

Reading through all the pros and cons of the different existing technologies, it seems that there is quite some variation in what the preferred technologies is for different teams.
Any technology choice will leave the people that have chosen a different technology disappointed that angular has not chosen their technology as the default one.

My question would be how feasible it is to give the choice to the end-user, similar (but I presume much more complex to pull off) as with the choice between css/scss/… when starting a new angular project.

IMO it would be ideal that angular facilitates the different e2e testing tools, without limiting to a specific one.
Probably this would necessitate some sort of collaboration between the angular team & some of the major tools.
Where angular provides some generic scaffolding that each tool/technology can build upon/integrate with.
This would then by default also future-proof the solution when new tools arise.

A similar concept would be useful for the whole jasmine/jest/… choice for unit testing.

Even without understanding specifics, I can imagine that this would probably be the most time-consuming and complex solution for the angular team; but it would probably also be the one that serves the community the most.

I have no clue of course about feasibility and/or budget considerations, just my 2 cents.
Whatever the choice; thank you for asking the community for their input about this subject.
It is enlightening to read the multiple viewpoints and opinions that have to be taken into account by the team.

[SerkanSipahi]

@StanislavKharchenko

When it comes to that point, you can throw React in the garbage because they often release a new API which is supposed to be the new standard and declare the old one not good. Do you want to rewrite your React app every time? I don't understand the confusion about Protractor which is hardly or not at all supported anymore.

That's your opinion. I respect that. You could do what you want :)
And this is not about changing a framework. It's an e2e test and not replace Angular for React!

And, also, within the same opinion could say like "khmm we evaluate that React (or something else) good, we will stop development on Angular and you can switch to React".

[csvan]

@LanderBeeuwsaert me and others have been saying this all along. Don't force a specific framework on developers, provide bindings for the most popular ones instead. Angular should have never tied itself to an e2e framework in the first place.

[e-oz]

Angular should have never tied itself to an e2e framework in the first place.

Bullshit. I'm thankful to them for this.

Phrases "I strongly recommend" and "you should" look ridiculous from persons who are not even contributors to the Protractor code. Give some respect.

[alvipeo]

I agree that one of the biggest points of Angular is all-in-one package that just works. I mean starting from DI to material, to e2e testing. For me (and my clients) it means predictability and not wasting time looking around for some sh*t to make things work.
Angular team spent time collaborating with MS on TypeScript, they also provided VS Code extensions. I'm sure they can work with MS on Playwright or stay with Protractor.

But they should keep to the idea "all-in-one and it works" (c) :)

[LanderBeeuwsaert]

I don't think that having a default (whatever that default would be) and giving choices that can be easily switched to (for those who have different needs outside of that default); are mutually exclusive.
So I'm not sure the whole discussion about; "angular has to provide an all-in-one package" versus "angular needs to give developers the choice"; is even necessary.

If it would be easy to setup different/extra e2e tools; suppose that you could be able to set them up easily in parallel (we run cypress & playwright for example); it would possibly make the decision/migration/... around protractor also easier.
Giving teams that use protractor the option to code their new tests with a different tool while not having to lose their investment into protactor.
Not ideal maybe, but for a lot of teams probably better than a big migration-at-once approach.

[alvipeo]

"If it would be easy to setup different/extra e2e tools" is never easy ;) Mostly because you depend on something you don't really control.

[LanderBeeuwsaert]

Maybe. Any level of abstraction implies some loss of control.
Alternative is to write everything in assembler.

[csvan]

@e-oz consider your own history and take your own advice.

[csvan]

@GSasu the main driver of Cypress is usability. I can agree that the paywall is a problem, but gains in productivity are significant. It all depends on the problem you are trying to solve. We use Cypress for all our e2e needs (including live monitoring) and it works fantastic. It's not a perfect fit for everyone, but then again neither is WebdriverIO (which is also great).

I don't agree that Cypress is a regression in terms of architecture though. It's not trying to be Selenium or even fit all the same use cases.

[e-oz]

@csvan I'm not saying to people what they "should" or "should not" do. As I understand, Ng team is asking for your opinion here, not for the instructions.

[csvan]

Ng team is asking for your opinion here, not for the instructions.

It is my opinion. You reading it as an instruction is not my problem, especially when the context makes it obvious that was not the case. If you are not sure what people intended, you can always ask.

[GSasu]

@GSasu the main driver of Cypress is usability. I can agree that the paywall is a problem, but gains in productivity are significant. It all depends on the problem you are trying to solve. We use Cypress for all our e2e needs (including live monitoring) and it works fantastic. It's not a perfect fit for everyone, but then again neither is WebdriverIO (which is also great).

I don't agree that Cypress is a regression in terms of architecture though. It's not trying to be Selenium or even fit all the same use cases.

The main driver of a test framework should be stability, reliability, reusability extendability, feature maturity, documentation and community, not usability.... that's something only FE engineers care about in a test tool. SDETs other QA engineers are usually capable of making any test tool "usable". I am happy that you are having a great time with Cypress, there are exceptions to all rules.

You don't have to believe me or take my word on the architecture regression of Cypress (I'm not the holder of absolute truth nor the father of all knowledge), but take the steps I took and then come back with the feedback:

• pull down/fork the cypress repo and go through it
• next look through the deprecated documentation of Selenium 1 (not WebDriver) and compare Cypress feature with let's say Selenium IDE (And all it's add-ons)
• tell me what's different in there other than the language?

If Cypress is so great, we might as well go back to Selenium IDE and accept being called "monkey testers" because all we do is "clicky clicky things" and we are not real engineers right?

If we are so sure of Cypress and we as a community start accepting paying for features that other tools offer for free than I'ld rather pay for a real tool like let's say Ranorex than Cypress. Just a thought 🤔

[alvipeo]

@LanderBeeuwsaert

Maybe. Any level of abstraction implies some loss of control.
Alternative is to write everything in assembler.

Genius! Go on, the whole thread is listening...

[csvan]

@GSasu

Calm down, nobody (me included) is saying that Cypress is absolutely fantastic and the de-facto way to do e2e testing. It isn't. It is, however, extremely good at the niche of e2e testing that it is focused on, and it provides a lot of value for those needing that niche. Other tools do the same things in their respective areas, and everybody should pick whatever helps them deliver value and build sustainable solutions.

I haven't done the digging you have done regarding Cypress vs. Selenium architecture so I can't comment. Even if that is the case, it doesn't change the fact that Cypress is great at what it does, and that is what we need it for.

If Cypress is so great, we might as well go back to Selenium IDE and accept being called "monkey testers" because all we do is "clicky clicky things" and we are not real engineers right?

I don't understand how you are seeing it this way. Our testers are engineers and they love Cypress because it has increased their productivity tremendously. That's value delivered, and that's what we are after. I am really happy for anyone having the same results with another framework, and encourage them to stick with it.

[GSasu]

@csvan I was and still am very calm, don't know what made you think I wasn't 🤔 . The example with the "monkey testers" is because historically speaking that's how we were described as in the past with Selenium 1. SO to me, anything that wants to reproduce this is just plain old bad.
In terms of the "de-facto" you haven't said it, but I don't think you know how large is the amount of engineers that come in and tell SDETs that they want Cypress because that's the industry standard.

I don't and probably never will consider it a standard, it's far from it, but maybe we should all bold out the fact that it's not a standard maybe that way other engineers stop calling it that way.

I wish other tools receive the same marketing that Cypress receives, I'm sure at that point Cypress would drown in it's own pot.

[csvan]

@GSasu I absolutely don't consider Cypress a standard (or think it should be) so no disagreement there :)

[luke-hill]

I think from my perspective (Admittedly a weak one), of attempting to create a new JS framework from all the ones mentioned was that I came from a Ruby / scripting area (This was in a previous job, and I've since gone back to ruby).

What I found when using Cypress and what others may consider the "leading standard", was it was very easy to setup as a novice JS user. But it also didn't let me do the things I wanted to as an Engineer with several years experience. Perhaps this was misconfiguration on my part, but Cypress felt very good and easy if you did things the way they told you. For example just some 1liners that I found almost impossible in Cypress were things like

• Store an element as a value (Not the text, but the web element itself). It wanted me to chain a function onto it.
• Create a scripted like test where I stored value 1, then I got value 2, then I did this. It likes you writing in a very heavy JS way with 5 method chains.
• Integrating with other plugins / trying to mix-n-match with things I like (I quite like the layout of cucumber, I like using external plugins like reporting e.t.c.)

Compare that to my brief experience with wd.io, which whilst it wasn't as nice to setup, and I couldn't get working as well, was permitting for me to to code things and store things the way I wanted. I think Angular sat somewhere inbetween the two when I trialled it.

So for me, given everyone is proclaiming pros and cons, I think the best thing angular can do is just link to every migration guide it gets, but steers 100% clear of saying this is the migration you should do. A few people have said this is good and a few have said this is bad. But when you trade on your name, when you leave your area, the name you pass on will be associated to you.

[JMuhire]

Go with cypress

[mdmintz]

[gkushang]

Why not consider CodeceptJS? https://codecept.io/. It's a nice API with "I" syntax which supports any engines underneath:

Supported Engines out of the box (most popular):

• WebdriverIO
• Playwright
• TestCafe
• Nightmare

If anything changes in the future, just switch the engine underneath without rewriting any tests ❤️

More advantages: Exclusive support for Shadow DOM elements for both WebdriverIO and Playwright. Just write a simple CSS locator without worrying about Shadow/WebComponent implementations

I.click('any-shadow-element') // run against Webdriver.io and Playwright

^^ We have implemented at Salesforce. For more info on ShadowDOM support, take a look at SaleforceEngineeringBlog/WebdriverIO-ShadowDOM

[kyliau]

Hey everyone! Thank you so much for all your feedback. We will review the comments thoroughly, and post an update here once we decide the next steps. We truly appreciate all the inputs!

[kyliau]

Hey everyone, thanks for all the great feedback, the Angular team really appreciates your comments.

To summarize, a few high level takeaways from the RFC are:

1. A lot of users are happy with their existing non-Protractor e2e setup
2. Some users want freedom of choice, whereas some users want Angular to be more opinionated and provide a replacement for Protractor out of the box
3. Some, especially enterprise users, would like to keep Protractor going, if possible (point no. 8 in the FAQ)

For point (2), the stance of Angular is that we want to be opinionated when it comes to the core experience. This means sane defaults in new applications, fast serve/build, and advanced optimizations and best practices enabled out of the box. We strongly believe there are opportunities for the community / open source partners to chime in when it comes to peripheral infrastructure like e2e testing. Leveraging the power of the community will enable our team to scale better, and help us focus on the core features that make Angular unique. In light of this, we have decided not to include Protractor in new projects starting from version 12, and instead work with partners to integrate their solution with Angular CLI via schematic/builder to make sure the ng e2e command continues to work seamlessly. So far we have reached out and heard back from Cypress, TestCafe, and WebdriverIO. We will continue to expand our collaboration with other partners going forward (please reach out to us!).

For point (3), we are exploring the possibility of a shared ownership of the project with other enterprise partners. This effort will keep Protractor going in the form of version 6 (no Control Flow, no legacy deps, no jasminewd). As many users have pointed out, Protractor without Control Flow is still very much useful on its own. Projects that have migrated away from Control Flow will benefit the most from such effort. If your team / company is interested in such collaboration, please get in touch with us at [email protected]. If we garner enough interest, we will work out a proposal with interested parties.

Thank you everyone for taking the time to respond to this RFC!