In the past Selenium was considered the ideal test automation tooling for you, if you needed real browser based tests for your web application. With Selenium you could focus on real user experience (UX) tests for unit and regression testing.
For a while now you may have read a lot about Cypress. Cypress is the new kid on the block and supposedly the new fancy way of web UI application test automation as well as the only and ideal alternative to Selenium. Many Cypress developers and users are predicting the end of Selenium coming soon. We can not read the crystal bowl. Instead we prefer to let the facts decide. Having said that, we have a slightly different opinion compared to most Cypress promoters. Let us explain why by diving deeper into both frameworks.
Table of Contents
- Why Selenium?
- Possiblities of using Selenium
- Why Cypress?
- Possiblities of using Cypress
- Ease of learning
- Browser abstraction
- Operating system
- Programming languages
- The Cypress license
- Available documentation
- Maturity of Cypress
- Using for load and remote testing
- Testing framework and libs
- Basic Architecture
- Synthetic vs native events
- Conclusion for Cypress vs Selenium
1. Why Selenium?
Selenium WebDriver, we hate and we love it, depending on how deep you need to go into the framework for your tests. The more you focus on UX (real end user regression testing), the more you get into the internals of the WebDriver. However, from our experience in about 98% of the cases you'll never face any restrictions considering the abstraction over all possible browsers.
1.1. Possiblities of using Selenium
To consider the possibilities of Selenium, we will get familiarized with the benefits of Selenium.
1.1.1. Ease of learning
It is said that developers have a steep learning curve at the beginning of using Selenium. This is true if the developers are not aware that they are performing remote controlling of a browser. And in simple terms Selenium could be considered as a remote control of a running browser. What increases the complexity of Selenium is the required interprocess communication between your test code, your browser and sometimes by an additional native WebDriver process. The native WebDriver typically gets automatically executed by Selenium and you may have seen it already (e.g. with the processes geckodriver or chromedriver).
1.1.2. Browser abstraction
In many UI testing frameworks, Selenium WebDriver is incorporated to enable test automation on browsers. This means that all major browsers natively support Selenium WebDriver for testing, which makes Selenium quite interesting for developers. Nowadays you can use WebDriver for any major browser, including:
- Chrome (headless mode)
- Internet Explorer
- HtmlUnit (headless mode)
This means you can easily do a multi-browser testing with your Selenium based tests by running the same tests with multiple different WebDrivers. Of couse, mostly the main issues arrise with the browsers which do not focus on standards, such as the IE.
But now you have an easy and fast way to identify browser issues in your application.
1.1.3. Operating system
Selenium supports many operating systems, enabling you to develop once and run the code on multiple operating systems. You can run your tests on:
- iOS (partially)
1.1.4. Programming languages
One major benefit of Selenium is the support for many different languages. You can use nearly any of the major languages with a relevant market share. Some of the supported languages are:
This wide support for programming languages makes Selenium quite interesting for many developer communities using different languages.
1.1.5. The Selenium license
Selenium is open source and released under the Apache 2.0 license.
1.1.6. Available documentation
Wide support of browsers, programming languages and operating systems makes Selenium a very well documented testing framework.
1.1.7. Maturity of Selenium
Selenium was developed in 2004. Selenium 2 was introduced in 2009 and Selenium 3 in 2016. Although Selenium is already rather old considering the year of the first release, the project has been regularly developed and updated.
1.1.8. Using for load and remote testing
Selenium brings in the RemoteWebDriver, which even makes it possible to run remote and load tests with real browsers and real browser behaviour.
1.1.9. Testing framework and libs
Selenium does not force you to use any specific testing framework. You can use JUnit 3, 4, 5, TestNG, Cucumber, Mocha, Phantom, and many more since Selenium is "only" abstracting the browser layer for the developers. This makes Selenium very handy for any tech lead, as you can stay with your preferred programming language and testing framework.
1.1.10. Basic Architecture
Selenium runs outside of the browser and communicates with the browser through the network. You have your test code, which is typically running in one process; often you have a custom driver, which is sometimes natively implemented in Selenium or in a custom addition process like the geckodriver, which runs outside of the browser and outside of the test code. This means you have the indirection of inter-process communications through the network. This communication is well abstracted, but you need to know this to understand odd issues that sometimes arise when using Selenium.
Synthetic vs native events
What are synthetic and what are native events in terms of UI testing?
The benefit of native events is the fact that the real user interaction can be simulated like the user would have really clicked on a button.
But sadly there is also a drawback. Native events are not platform independent. This means the abstraction layer of Selenium needs to cope with all different operating systems, to make platform dependency as irrelevant as possible for the developers.
1.1.11. Synthetic events
Of course synthetic events evict the need of abstracting native events for different operating systems, but that also brings in a new fact - that you cannot be sure if you get the same behaviour as when a real user is firing real (native) events in the browser.
To give you a good example which drawbacks synthetic events have:
Let's assume you need to simulate a mouse move and a click after hovering over the right WebElement, in our example a button. Now assume this button is hidden by a layer on top of the button layer. If you now fire a native mouse click event, the browser would get a native click event and would try to click on the area where the button should be. But since the button is hidden by the layer on top of the button, the button click will not be executed.
Also, consider the TAB-Key behaviour of web applications. When you press the 'tab' key your focus jumps from the current WebElement to the next element. This is done by getting the native TAB-Key event from the operating system and passing it to the browser. With synthetic events the browser does not get a 'tab' key is pressed event and therefore it will not change the focus in your application.
2. Why Cypress?
Cypress is gaining a lot of attention, but what makes Cypress so special during the development?
2.1. Possiblities of using Cypress
What are the benefits of Cypress?
2.1.1. Ease of learning
The Cypress users are of opinion that it is easier to use Cypress compared to Selenium. This means you can easily test your web application with Cypress. You need to install an exe and everythings gets automatically done for you. Also, the developer friendly documentation helps a lot.
2.1.2. Browser abstraction
Even if the developers of Cypress had cross-browser testing in mind, they have not implemented any other browser support apart from Chrome untill February 2019.
This means you can not do real cross-browser testing, which is crucial for environments with non unique browser environments.
2.1.3. Operating system
Just like Selenium, Cypress also supports various operating systems. This means you develop once and run the code on multiple operating systems. You can run the tests on:
2.1.4. Programming languages
2.1.5. The Cypress license
Cypress was also released as open source under the MIT license. However, compared to Selenium not all parts are open source, e.g. the dashboard in Cypress is closed source.
2.1.6. Available documentation
The documentation is good, not as good when compared to Selenium, but you can find all relevant details for development with Cypress.
2.1.7. Maturity of Cypress
Cypress is a very young framework compared to Selenium. It was initially started in 2015 and the first 1.0.0 version was released in 2017. The quality of the issues getting reported and fixed already looks quite mature. Also, the focus on only one browser is reducing the complexity for the Cypress developers compared to Selenium.
2.1.8. Using for load and remote testing
Cypress can be used with Mocha and Chai for testing. Whatever they support, Cypress will also support it. By design neither is implemented to do load testing.
2.1.9. Testing framework and libs
2.1.10. Basic Architecture
Cypress is built on top of the Electron app, and since Cypress runs IN the browser, it allows the developers to do very unique DOM manipulations directly in the browser and during testing. Also, Cypress does not need any network communication, which brings it to often cited "developer friendly test automation".
2.2. Synthetic vs native events
Cypress has plans to support native events, but as of February 2019 there are no specific roadmaps to get native events implemented.
2.2.1. Native events
Not supported as of February 2019.
2.2.2. Synthetic events
3. Conclusion for Cypress vs Selenium