Finding right tool for automation testing is very crucial. Wrong selection of tool may cause trouble and can also result of rework and limitations.
How do you choose a tool ?
People usualy go with a most popular tool in the category or where they have some prior experience and compfertability. Often selecting automation tools in this way may cause major rework in the future or can be a reason of criticle compatibility issue. Chosing right automation tools is very criticle and should not be taken lightly.
Things that should really matter for Test Automation
- Scalability
- Compatibility across browsers
- Compatibility across OS
- Ease of creating test scripts
- Easy maintenance
- Reporting mechanism
- Price
Lets see how the same tool or test automation solution might not work in different teams situations.
A tale of 2 teams
Team A
- Cost sensitive client
- Team comfortable with many languages
- Application needs to support IE, Chrome, Firefox, Safari
- Responsive design
- Strict performance benchmarks
- Asynchronous backend systems
Team B
- Client maintains own tech stack preferences
- Team comfortable with Java and JavaScript
- Support needed majorly for in-house Chrome users
- Possibility of training end-users after going live
- Client comfortable with tools support cost
Pugh Matrix can help do the comparison
The Pugh Matrix is a decision-making tool to compare multiple alternatives. The steps that you will take to create your Pugh Matrix are:
- Define your evaluation criteria. What are the most important and desired characteristics of your solution?
- Weight your evaluation criteria as to the relative importance of each.
- Define your different improvement alternatives and optional approaches.
- Select a “BASELINE” from the alternatives, which will typically be your current state.
- For each criteria, rate each alternative as better, same, or worse than your baseline. You can use a simple + for better, s for same, and – for worse.
- Total the count of each.
- Select your best alternative.
- Explore whether an even more optimal solution might exist by creating a hybrid of the best from various alternatives.
PUGH MATRIX FOR TEAM - A 
PUGH MATRIX FOR TEAM - B 
for more details : https://www.isixsigma.com/dictionary/pugh-matrix/
Last but not the least The Paradox of choice
Number of choices increases due to:
- Available list of tools/frameworks - Democratization of source code led to development of tools with similar features but subtle differences
- Team constraints/preferences
- Client requirements
