What is Cross Browser Testing?
Which Cross Browser Testing tools should i use?
Which Browsers, OS, Devices and Platforms should i test on?
If you are looking for these answers, go on and read the Blog post.
We have many browsers like Chrome, IE, Opera and Mozilla in the market currently. These browsers also get regular updates resulting in many versions of them. Browsers are updated to make sure they have new features and supports various platforms. More than browsers we have frequent release of new devices and new OS in them. The OS too get regular updates resulting in many version of them.
So how do you make sure that your application works well in all these browsers, OS and devices?
#CrossBrowserTesting is the answer.
Websites need Cross Browser testing to make sure that they are working on all browsers. With so many variations of browser, OS and devices, we need Cross browser testing tools. These tools help ease the whole process of testing and giving a bug free application.
Let us see few reasons why you might need Cross Browser testing tools.
You may agree to the point that every business these days need to have a website. It can be a simple website with standard pages like Services, Products and Contact page.
A complicated web based tool running on a Cloud platform.
In either case, one does not want to lose the business by turning down customers. Unless they are real pain in ass.
One also does not want customer to abandon website because it is not compatible to their devices.
I hope you agree to what i am saying. So what is the common factor in both?
The world has gone beyond the days when there was only One browser called Internet Explorer. With Internet penetration increasing, the no. of people using the Browsers have gone up. We also have many new browsers in the market giving competition to IE. They have actually dethroned IE from its top place.
Let us see some stats on Browsers before we move ahead to make sure you are testing on the right browsers.
List of Browsers and Market Share
Below is the list of most used browsers as of today: (PS: I have combined both Desktop and Mobile Browsers)
- Android Default Browser
- UC Browser
Below are some stats which shows Chrome as the most popular browser with 45.71% market share. Next comes IE and Firefox if you combine both Desktop and Mobile browsers.
If you consider only the Desktop browsers, then share of Chrome increases even more to 53.3%.
What about if you take only Mobile browsers? This is where people are shifting towards right.
Share of Chrome goes to 32.32% in this case. With new entrants like UC Browser (Alibaba backed company) with 13.91% market share.
I wanted to dig a bit deeper and wanted to analyze if the Location plays a role and guess what?
So if most of your users are mobile and are in US then do not ignore your application compatibility to Safari. And if they are in India then do not ignore UC Browser.
So does it make the task of Web based application & website makers difficult?
It is a fact that Consumers are using different browsers and versions.
So Web developers need to make sure they work seamlessly on various browsers.
And also various Devices, Resolutions and Operating systems.
It is also referred to as cross browser compatibility testing. Let us see the definition of Cross browser testing.
What is Cross Browser Testing?
Wikipedia defines Cross browser as
“Cross-browser refers to the ability of a website, web application, HTML construct or client-side script to function in environments that provide its required features and to bow out or degrade gracefully when features are absent or lacking”
In simple terms, Cross Browser testing is a way to test a website on all major browsers. This includes various supported versions on desktop, tablet and mobile devices (android and iOS).
The aim is to ensure consistency across browsers, platforms and devices. One should also make sure that the best user experience gets delivered on all these platforms.
There are various leading cross browser testing tools available in the market. There are both free and commercial cross browser testing tools with different features.
But do you know which cross browser testing tool should you go for?
Do you want to just go for a tool which compares UI, design, layout etc. page by page?
Or something like Ghostlab which mirrors the user experience?
Do you want a Cloud based solution or a desktop based solution?
We will be looking at all these things in detail in this article.
Why Do I Need Cross Browser Testing?
Most people do manual testing on various browsers to check compatibility of their application. This is the easiest way to do this with good features in browsers these days.
Chrome for example offers Device Mode to test your website on various devices. PS: Right click and Select Inspect Element in Chrome to check this feature.
But does manual testing guarantee a complete test coverage? No
What if you have 100s of web pages, Frequent build frequency?
It not only becomes time consuming but almost impossible to do manual testing in such cases.
Businesses are dependent on Websites these days.
Are you looking to generate sales from your website?
Are you having a SaaS based web application where your users need to login online?
Your website might work perfectly in Chrome, but your users might be using IE or Firefox. If the website does not open properly there then they might just quit.
Gartner analyst, Thomas Murphy puts it very correctly that “Most browser compatibility shouldn’t be something you learn in the testing tools but at design by following appropriate design guides and the tools built into page development environments, i.e. developers should deliver pages that render in the right way in every browser.”
Before you choose a tool for Cross browser testing you need to know:
What is it you’re trying to test?
Testing GUI itself? Or is it functional testing?
How deep is your technical knowledge?
Here are some more reasons why cross-browser testing has become a need:
New Browsers and Changing Taste of Consumers:
New browsers keep on launching and old browsers keep updating themselves with new features. Also customers keep on switching the browsers based on the functionality offered.
Please remember that customers never use the same browser for a longer period of time.
Organizations should follow the trend and test their applications available on all the browsers. With many versions of same browser, it is important to automate the whole process.
Believe me, it save lot of time and you do not want to do manual testing on more than 200+ versions.
Maintaining Separate Tests:
You want to write test once and run it across all browsers with ease. No one wants to create separate tests for each browser and take a maintenance overhead.
Testing on Mobile Browsers:
With increase in websites accessed on mobiles, it has become imperative that you test on them. You do not want to miss on customers coming via mobile. Automation helps you test on mobile browsers and resolutions with ease.
Avoid security breaches with Cross browser testing. Make sure vulnerabilities of the browser does not impact your application security.
Make sure your application performs well on all the browsers. Consistency across platforms in important for a world class application.
What types of Testing does Cross Browser Testing Include?
Cross Browser testing should look at all the famous browsers, and various browser versions. The support for various devices like Desktop, Mobile, Tablet, and Console should be tested. Screen Resolution & OS (operating systems) should also not be ignored.
What does a Cross Browser Testing tool do?
There are almost 200+ latest browser versions and platforms on which one needs to test. These are the ones which are mostly used and should be supported.If you combine all the combinations of browsers, OS and devices then it might go to 900+ also. For rapid and automated cross-browser testing, the tool should take care of all scenarios.
A good cross browser testing tool accelerates browser compatibility testing by using reusable automate scripts based on the past test executions. Adapting to the latest browsers and offering cloud-based testing on various combinations of OS/Platforms and Browsers is a plus.
Few of the features of which an ideal Cross Browser Testing tool should have are:·
Scripts Portability: These are nothing but reusable libraries to kick-start automation efforts. These should come with architecture to support easier portability of scripts across environments.
Good Dashboard: Comprehensive real time dashboard to get reports by browser and OS. This is helpful for testers and developers to analyze the behavior of the application.
Real Browsers: No fake browsers. Real test in real browsers and devices.
Combinations and Compatibility: Various combinations of browsers and platforms.
Cloud based Testing: Cloud based solution is ideal. Access is quick for executing tests and scripts on combinations of OS/Platforms and Browsers.
Real time Mobile Emulators: Real devices are better. But official iOS, Android and Opera mobile and tablet emulators are also good option.
Desktop OS: Support for multiple OS and versions like OS X El Capitan, OS X Yosemite. Also Windows 8 and Windows 10 to list a few.
Speed and Security: Faster access and secure connections
Challenges with Cross Browser Testing
The main challenge usually comes for the browser – Internet Explorer. IE is part of major browser lists as we have seen above. Browsers like Firefox, Chrome, Opera behaves in the same way. This lets you test the application on them with ease. The challenge is always with running test in IE.
One also needs to factor in:
- Performance: Loading time of calls to the server
- Automation: How much to automate?
- Right Tools: Which tools to go for?
How to do Cross Browser Testing?
If you’re looking for an open source option, then I suggest using Selenium WebDriver API. Selenium can be used with either C# or Java as well as several other languages. But those two are the most common in my experience.
One can use ChromeDriver.EXE and InternetExplorerDriverServer.EXE to instantiate custom Chrome and IE browsers which can be automated. Selenium natively supports Firefox without needing the use of an external driver. Multiple versions of IE can also be supported. So the top three main browsers get the support with Selenium.You can setup a test lab with different OS on VMs (Virtual machines), and use Selenium Grid as router. Use Jenkins as CI, where you mention the params of OS/Browser, as params. That’s it and you have an in-house cross browser test lab.
Go for selenium webdriver 2.0 to carry out your cross browser testing for web. Appium can be used for automating iOS and Android app testing.
Phantom.js is another option if you want to go for headless browser testing.
https://www.virtualbox.org/ is a free solution to run your VM’s. Install VirtualBox. Then download or install each client OS you need. Head over to modern.ie and download the appropriate VMs [http://www.modern.ie/en-us/virtualization-tools#downloads]. Clone the installed VMs for each specific browser you need if you’ve already installed the OS. Then, when it’s time to test, launch your VMs one at a time and check against your local dev environment.
Too complex? Or too much capital expenditure?
Then, you should explore the use of some Cross browser testing tools which can do this for you. Or take some expert help.
Top 8 Free and Commercial Tools for Cross Browser Testing
Cross Browser testing involves around Functionality testing and Design check usually. Majority of cross browser testing tools can only test your functionality amongst different browsers. If you are concerned about your responsive design, you might need to use other tools.
Below are some cross browser testing tools to get connected to a remote browser of your choice. You can also test on multiple browsers locally or remote sites. There are cloud based solutions as well as desktop applications. I have mentioned both Free and commercial tools to let you choose the best options available.
Top 5 Cloud Based Cross Browser Testing Tools
Type: Free and Paid
It is FREE for an open source project. You need to sign up using Github, Google code or Souceforge and you get unlimited testing minutes, 5 parallel VMs. You will also get PUBLIC ACCESSIBILITY of all your tests.
Otherwise, you can sign up for a free account with 14 days trial. That gives you 90 hours of automated testing, 8 concurrent VMs and unlimited manual testing. With Saucelabs, you can test on an extensive list of 500+ devices/OS/browsers. They also provide tools for tunneling so you can test local code and access to other debugging tools.
Paid plans starts with as low as $12/month and goes up to $149 for small teams. Enterprise plans can be customized for you if you are looking for volume pricing.
Type: Free (For open source projects) and Paid
Browserstack is a browser-based virtualization service and is a great cross browser testing tool. In Browserstack, you get a virtualized browser within your own browser without any setup or installing anything.
BrowserStack also has the tunneling capability that allows you to test your local pages remotely. BrowserStack offers live web based cross browser functional testing and automated testing with selenium. It also comes with browser screenshot capture and a responsive design testing service.
They are again FREE for Open source projects. But unfortunately they have no free tier, they do give you a pretty good trial period though.
They also have a Freelancer Package at $12.50/mo billed annually. $19 month-to-month. This includes 100 minutes of Live. Only 1 user/account.
There LIVE service package starts at $29/user/month and AUTOMATE package at $59/user/month. All plans comes with access to 700+ real desktop browsers, local testing and range of resolutions.
With over 25000 customers, the tool looks very impressive and it is easier to use than Saucelabs.
Type: Free Trial and Paid
‘We keep it real’, that’s the moto of CBT and selling formula apart from other impressive features they offer. That’s how they try to differentiate from Browserstack which uses virtualization. With CrossBrowserTesting.com, you can test your sites on more than 900 browsers. They also give access to more than 40 operating systems, including iOS, Android, Windows, Mac and more. Also you’re testing it in real browsers, running on physical devices.
The features are same like other tools – LIVE testing, Automated screenshots, Comparisons, Selenium Automation.
Free Trial for 7days is on offer. Paid plans starts from $29/user/month with unlimited live testing. Automated testing is limited to number of minutes and changes with plans.
Good product and service support.
Type: Free and Paid
Spoon is a Sandbox tool. Spoon offers Browser Sandbox and Selenium Sandbox for automated testing. Spoon takes care of all the Selenium infrastructure and networking for you with Sandbox.
Browser Sandbox makes cross-browser and backwards compatibility testing easy. Just click any browser to launch it instantly from the web.
It gives easy options if you are switching from Sauce labs, Browserstack and Testing bot.
To use it, Simply launch the browser from Spoon.net and enter your test URL in the navigation bar.
Free options contains 1 Concurrent VM and 24hour Container storage. Paid plans start from $19 per user per month. They include online sandbox tools like Browser Sandbox, Selenium Sandbox, SQL Sandbox.
Type: Free Trial and Paid
TestingBot offers Live, Web-Based Browser Testing on real browsers and Selenium Testing. It allows you to test in over 500 real browser combinations including Internet Explorer 6, 7, 8, 9, 10, 11. Support for mobile tests on iPhone/iPad (iOS 8) and Android (JellyBean, KitKat) with Appium exists.
You will agree with me that it is not easy to setup VMs on your own. TestingBot makes it easy and lets you concentrate on writing and executing tests.
Features are comparable to the other tools. It offers Free 100 minutes of manual/automated testing with FREE Trial.
Paid plans start from $12/month for Unlimited users and 400 minutes of Automated testing.
They too have a Free Open source plan.
Top 3 Desktop Based Cross Browser Testing Tools
You looked at Top 5 Cross browser testing tools on Cloud. Now let us look at some desktop based applications too. If you are not a fan of cloud based applications then these might be the right choice for you.
Type: Free Trial and Paid
Ghostlab is a desktop application and needs to a download. With Ghostlab, you can test any website on various browsers and mobile devices simultaneously. To start, drag the URL into Ghostlab and click the “Play” button. It works on Mirroring principle. Once you start testing in one of the connected browsers or devices, all other browsers and devices will mirror your actions.
It is basically good to mimic the user experience.Comes with 7 day Free Trial (actual usage) and costs $34 for individual license.
Type: Free Trial and Paid
The pricing starts from $15 user/month. But the professional version at $25 is what you should choose. They have 14 – Day Free Trial Version which you can download for Windows.
Keep below requirements in mind for their application: Microsoft Windows XP SP3, Vista, 7 or 8 / Pentium 4 or higher CPU /2 GB Memory / .Net Framework 4/ Microsoft Internet Explorer 7 / Recommended: Microsoft Internet Explorer 11
Type: Free and Paid
MultiBrowser (formerly Multi-Browser Viewer) is a Windows software application. It enables developers to easily test their websites to ensure that correct functionality and rendering across all major web browsers and desktop/mobile devices.
Their selling moto is ‘No emulators, no shimmied user agents, no VMs’. Each standalone browser is encapsulated within a single isolated executable, helping you avoid test lab setup headaches and focus on actual testing.
Though they use Mobile Browser Emulators.
They also offer Responsive design screenshots, Visual studio integration and Record and playback tests.
Free version allows 25 Responsive web design tests per day. Standalone browsers and mobile emulators support is not there in Free version.
Standard and Professional version are at $139 and $229 one time. You can also use Professional version free for 14 days after which you will be back to Free version.
Things to consider before choosing a Cloud based Cross Browser Testing tool – A Cloud based Cross Browser Testing Checklist:
- Speed or Testing Time to Render a Page: How fast does it open on slowest of connections also. It should not lag when dealing with highly interactive websites
- Pricing: Are you being restricted to number of hours of testing? This is being adopted by many of these tools.
- User Interface: Is it easy to shift from testing IE7 on Windows 2005 to testing Opera on a Mac device?
- Customer Support: This can be a major factor if you are looking for continuous help
- Real Devices or Official Emulators: You need to decide which one you want to test on. Though they might give same experience and results, it is a personal choice.
More Free Cross Browser Compatibility Testing tools:
Below is the list of few more tools that can do functional testing, responsive design check and UI testing based on you need. Few are Free to use and have paid versions too.
Broswserling is Simple and easy to use and has a free plan that’s great for individual developers. Free plan includes Limited 3 minute sessions, Internet Explorer 9 only, Windows Vista only and 1024×768 resolution only. So it is great if you want to know how it works but will not test your application on multiple platforms. So you need to go for a Paid plan no matter what for true cross browser testing.
Unfortunately, it also doesn’t offer mobile device testing. It offers tunneling, but it’s less flexible than Sauce Labs’ or BrowserStack’s.
I tried running it for my website and it looks like this:
This allows you to Test Microsoft Edge and versions of IE6 through IE11 using free virtual machines you download and manage locally. It is a dev center built, so you can spend less time testing for the various versions of Internet Explorer and more time building what matters to you on the modern web.
In MS Edge, Compact Inspector watches for patterns of interaction known to cause issues and reports them automatically. This allows us to identify problems quickly without first memorizing a bunch of documentation. Try it out, I’m sure you will like it.
Galen Framework runs well in Selenium Grid. You can set up your tests to run in a cloud like Sauce Labs or BrowserStack so that you can even test your responsive websites on different mobile devices. Galen can run multiple tests in parallel which is also a nice time saver.
Browsershots makes screenshots of your web design in different operating systems and browsers. It is a free open-source online web application providing developers a convenient way to test their website’s browser compatibility in one place.
I Tested it for my website and got screenshots only for few devices. Have a look:
Looking to test Mobile apps in Cross platforms? Have a look at Appium.
APPIUM: A CROSS-BROWSER MOBILE AUTOMATION TOOL
Appium is basically an open source test automation tool to automate native and hybrid mobile apps. It uses JSON wire protocol internally to interact with iOS and Android native apps using the Selenium WebDriver. So it’s good if you are looking to do cross browser testing on mobile platforms.
At present Appium only supports Android and iOS based apps but support for Firefox mobile OS is in pipeline.
Unit Testing Across Browsers
If you are not looking for UI testing, but looking for unit tests across browsers, then you have few Open source projects you can have a look at:
Let’s you Test your code on real browsers and real devices such as phones, tablets or on a headless PhantomJS instance. Also describe your tests with Jasmine, Mocha, QUnit, or write a simple adapter for any framework you like. Its Open source. Also gives Continuous integration with Jenkins. Yeti works with IE 6+, Android 4+, iOS 4+, Chrome, Firefox, and Safari.
Similar to Karma. A server/client model (though it runs mixed mode also) allowing you to have one browser pool to execute tests on. It runs QUnit, jasmine, Mocha, YUI, and DOH tests. Yeti can launch browsers with Selenium and provide JUnit XML results for easy integration with CI systems like Jenkins and Sauce Labs.
Which Cross Browser Testing tool should I Use?
Depending on skill set your testers have, the choice of the tool can vary. I would recommend using selenium with VM setup if you are technical enough to setup things on your own. Browserstack, Saucelabs and CBT are good tools to try. You can choose one among them based on your satisfaction levels. Well written tests are agnostic of browser. This allows you to choose any browser combination offered by these tools to run against the same tests.
In my experience, home grown solutions requires more maintenance and deliver less value. Though they are expensive, but are faster and can be modified based on your need. But the person dependency is high in this case. If the person who sets it up and writes test cases is no longer in your organization, you are in deep trouble.
Are you using any of these tools listed above? If yes, which tools have you found most useful and accurate to test a responsive website?
Recommend more tools in the comments below and we will be happy to include them in the list.
Let us know if you need help in choosing the right tool and implementing Cross Browser Testing at your organization.
FREE BONUS: Download this Entire list as PDF.Also get 11 Paid and Free Cross Browser Testing Tools/Compatibility Tools/Responsive Testing Tools. Easily save it on your computer for quickreference or print it for your Organization. Includes 11 testing tools not found in this post