The current trend in user interfaces is geared towards graphical user interfaces (GUIs). GUIs are very complicated and hence GUI testing is very time consuming. Automation is a requirement for testing any larger graphical user interfaces, but automating GUI tests isn't a forthright task. Automation is not that easy at first.
Testing always under time constraint:
Every software comes to a team with deadlines and delivery dates. And that's the major challenge of automation, There is huge list of stories and tasks including understanding product requirements,writing test cases,writing automated test scripts ,executing them, make them run as a whole may be as a part of daily build,etc which needs to complete within specified time lines.
Proper knowledge of the requirements:
Most of the time business analysts are responsible for communicating with customers for understanding the requirements. What if tester fails to understand that requirements? Will she be able to test the application properly? Testers require good listening and understanding capabilities along with communication skills.
Automation Strategy , Framework picking and Testing tool Selection:
Availability of various tools in market and that too open source is making life of a tester more complicated rather than easier. In the starting decisions like which framework to choose? Which tools will best work here? Which coding language supports that tools? arises and that is the first and foremost task which needs to be done before starting automation.
Testers centering on detecting easy bugs:
There are many organizations and customers in which testers have to provide a list of bugs and at the end of day that can act as a performance indicator for them. Finding easy bugs those don’t require deep understanding and testing but to make a website or a software more robust we need to dig into more complex issues which could be be rare though important,tough to find and remains unnoticeable if not given proper attention. According to me to include each test in automation test cases is next to impossible.But automating the majority gives you the time to focus on the “strange” few scenarios that automated tests cannot see.
Relationship with business analysts:
Developing relations ,by this I mean continuous communication with the developer. So this need a mature and experienced tester who can optimistically handle the situation by her involvement in doing tasks in their own format. The problem become more intense when questions arises like if this bug is actually a bug or a part of functionality itself!
How to handle lack of experience in testers
In Agile testing particularly,the teams are mix of different experienced people all around not only QA's but also developers,so the problem is to come on a common track where every individual can contribute equally. Which thus,results in issues like time management ,inapt , undone work.
Priority of test cases:Which tests to execute first?
Decision like which test cases should be executed and with what priority? Which tests are more important over others? This requires good experience to work under pressure. When and where to start automation? How and when to stop testing? When to pause the testing? GUI is quite fuzzy and brittle,it should target most important and utmost areas covering basics first at ground level like Home page , Log in page ,etc and then moving to the other high level areas
Should we target the automation of 'X' module:
Should automating this application is required ,at all? How much time will it take? At what level automation should be done? Does automation really works? Decision of automation or manual testing will need to address the pros and cons of each process.For every module we should have a discussion over it as automation testing is like development only where QA team has to decide proper development tool, development language, correct designing ,etc. We also have to decide which module should be target to automate and which we should leave.
Simulating real user experience and handling complicated elements and functionalities:
There are multiple issues like handling pop up ,say ,whether you want to proceed , clicking on a selected color of a slice of pie , clicking on a highlighted text,etc are most of the common issues which I have see most of time while adding test steps in an automation framework.
Reuse of Test scripts:
Application development methods are dynamic ,making it difficult to manage the test tools and test scripts. Test script migration or reuse is very essential but difficult task. GUI is an ever changing part of any software ,and updating GUI often break your test cases. Small alteration might break whole test suite.
Robust handling of environment and response slowness:
This is something which cant be resolved easily as wait functions sometimes makes overall run of test suite very very slow. For instance, if there is a web site say IRCTC which sometimes gives quick response and often user have to struggle for getting a ticket booked ,on contrary, Google which is the best and fast search engine on web. So the point is to automate web sites having different response time is a bit tricky and difficult.
Multiple browsers and cross browser support:
This issue is not specifically for automation testing but also valid for manual testing. Testing over multiple browsers is always a head-ache. So as to make automation test out of it,a tester need to add browser compatibility executable in the project , desired capabilities of each browser ,etc. For example:
public void initDriver() { try { if (Property.IsRemoteExecution.equalsIgnoreCase("true")) { String remoteURL; DesiredCapabilities capabilities = new DesiredCapabilities(); if (Property.RemoteURL.toLowerCase().contains("saucelabs")) { // set all saucelab capabilities here. } remoteURL = Property.RemoteURL + "/wd/hub";
URL uri = new URL(remoteURL); if (browserName.equalsIgnoreCase("firefox")) { FirefoxProfile remoteProfile = new FirefoxProfile(); remoteProfile.setPreference( "webdriver_assume_untrusted_issuer", false); remoteProfile.setAcceptUntrustedCertificates(true); remoteProfile.setEnableNativeEvents(false); capabilities.setBrowserName("firefox"); capabilities.setCapability("firefox_profile", remoteProfile .toString().toString()); driver = new RemoteWebDriver(uri, capabilities); } else if (browserName.equalsIgnoreCase("internetexplorer")) { capabilities.setBrowserName("internet explorer"); capabilities.setCapability("ignoreProtectedModeSettings", true); driver = new RemoteWebDriver(uri, capabilities); }
} else { if (browserName.equalsIgnoreCase("firefox")) { FirefoxProfile ffprofile = new FirefoxProfile(); ffprofile.setPreference( "webdriver_assume_untrusted_issuer", "false"); driver = new FirefoxDriver(ffprofile); } else if (browserName.equalsIgnoreCase("internetexplorer")) { DesiredCapabilities iecapabilities = DesiredCapabilities .internetExplorer(); iecapabilities.setCapability("ignoreProtectedModeSettings", "true"); iecapabilities .setCapability( InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, "true"); driver = new InternetExplorerDriver(iecapabilities); } else if (browserName.equalsIgnoreCase("chrome")) { DesiredCapabilities chromeCapabilities = DesiredCapabilities .chrome(); chromeCapabilities.setCapability("chrome.switches", Arrays.asList("--start-maximized")); chromeCapabilities.setCapability("chrome.switches", Arrays.asList("--ignore-certificate-errors")); ChromeDriverService service = new ChromeDriverService.Builder() .usingAnyFreePort() .usingChromeDriverExecutable( new File("chromedriver.exe")).build(); service.start(); driver = new ChromeDriver(service);
} }
Regression testing:
Doing sanity of a developing software is a tedious task in itself and testing it as a whole is irrepressible. Time-to-time tackling new functionalities and ever changing features and backward compatibility checks are some most important things which needs to be tested
Testing the complete application:
How do we avoid breaking the tests when making the next iteration? Have we tested everything? Does this set of test-cases test everything that needs to be tested” and “do some of these test-cases overlap In my opinion its next to impossible if we talk about developing software. There could be thousands of test scenarios. It’s not possible to test each and every combination both in manual as well as in automation testing.
There is a lot to test!
Automation testing is just simply not writing automated test case out of manual test cases.It requires proper framework ,tool , domain knowledge and many other things ,recently I have given a session in which I shared some experience of mine and I will be discussing my two cents via this post. Automating UI tests for the web, however, is not easy. The web browser offers numeral challenges that make automating web UI tests much more laborious than automating “desktop” software UI tests.
What a website should Never have?
Intervene with the capability to scroll.
Permit form re-submissions.
Disable keyboard backing.
Fail to allow feedback.
Disable keyboard navigation.
Move content without user interaction.
Use static location without an alternative plan.
Use pop-up windows.
Get characterized by unexpected traits.
What Automation testing can achieve ?
Save time notably.
One time effort that too initially while setting up environment.
Remarkably lessen load and effort of manual testing resulting ,tester can add her manual efforts in finding new uncovered test scenarios that could be added in the testing framework.
Saves Money and reduces load of tester in last days of delivery.
Consider the cost of your QA team- or if you’re a single developer, the cost of your development time- spent today manually running UI tests. Consider the cost to your business when you ship web applications that have functional problems not caught by your unit tests. Saving even a few of those testing hours or even one of those embarrassing- potentially costly- bugs helps quickly reveal how much money automated web UI testing can save your business.
Reduces Headaches.
This way one can get an idea what mindset we should carry while setting up an automation testing framework and how to deal with the challenges.