Building a mobile testing team

by Justin Rohrman

The key questions a manager needs to answer when starting a new mobile testing effort are:

  • How many people do we need?
  • What roles do we need?
  • Where can we find testers?
  • What skills should we look for in our testers?

Even if you already have a testing team, or various roles that perform those tasks, you’ll still need people who understand the significant changes that mobile applications require in the testing process. Mobile applications flip Mike Cohn’s traditional testing pyramid on its head. Testers must put a greater emphasis on manual testing and real-device testing, which requires a larger time investment.

In many cases, this means adding some sort of specialization to your team, or building a mobile testing team from scratch. The following sections should help you find answers to the key questions you need to address when starting a mobile testing effort.

Roles and numbers needed for a mobile testing team

Mobile software testing is a specialty. The perfect team would have experience with all mobile testing situations and technologies, but the perfect team doesn't exist. You also have a limited budget, and you have to work within that.

The best strategy is to build a mobile testing team based on the company's most immediate need, and train for the future. A good software tester can come from anywhere, and you don’t always need a large number of people to build a mobile testing strategy that works effectively for your products.

Companies that can only afford one mobile tester—or perhaps only need one—will want a generalist who can work on a variety of projects. If you already have a testing team, this is a good strategy—start small with one generalist tester who also specializes in mobile testing and can harness the rest of the team for support. That person should understand software testing, mobile platforms, and how to accelerate getting apps to production by using automation. In other cases, your app may be small and new enough that you only need one person to handle mobile testing.

Other companies with high stability requirements for their products might want a larger team dedicated to mobile testing. Organizations that are fully dedicating themselves to mobile products will want to consider having:

  • A front-end/usability specialist
  • A mobile automation specialist
  • A programmer with deep understanding of the different mobile platforms

Where can you find or train some of these specialists? Let’s take a look at where to find testers.

Looking inside the company

Large companies are always in flux. A person might begin his career in the support department fielding calls from customers that have questions or have discovered a bug in their software. From there he might move into a role as a product manager helping customers, a software testing generalist, or a developer.

Moving between departments is a common pattern for software testers. Each time they take on a new role, they get valuable experience and insight into what makes testing important and how to better perform the job. Moving someone from a role outside of testing onto a mobile testing team can be beneficial for larger companies that have people wanting new experiences. However, smaller companies will probably not be able to do this. 

Take a look in other departments. Chances are there is a person or two interested in learning about testing mobile software. Transferring people from other test groups can be a quick route to building a team with the needed skill set immediately. These people come with some understanding of software testing, and hopefully some experience with mobile products too. When that isn’t an option, these are some other roles that can produce very good testers, after some training:

  • Product support personnel
  • Product managers
  • Developers

These roles will bring different perspectives to testing that can often improve the quality of the work by reducing blind spots.

Smaller companies will probably not be able to hire from within. Sometimes you’ll either need to hire more people or add more responsibilities for a current worker.

Looking outside the company

When companies can’t move someone into a mobile testing role from a different department, they will have to look elsewhere. Professional networks and recruiters are the fastest way to find these people.

Recruiters have large databases of people looking for work. If a team needs a tester with experience in iOS 8, the Amazon Fire tablet, and specific Samsung devices, a recruiter will be able to pull up a list of people with those skills on their résumés. Alternatively, a recruiter can cast a much wider net. Some mobile testing teams are okay with more junior testers who have no mobile experience or people coming from other roles who are interested in a new career in testing. Recruiters also do some vetting before the candidate gets to the office for an interview. Here's what a real-world example of a "mobile tester" job advertisement:

The downside that comes along with large candidate databases is the placement fee. Recruiting companies will want a fee for each person that is hired. Another downside is that non-technical recruiters often have trouble vetting candidates’ technical skills, meaning that you’ll have to do a good job of this yourself.

Professional networks are a little better. People working in technology will usually know other technology workers from local meetups and previous jobs. The benefit is that the people interviewing already know something about how the candidate works. Do they understand peculiarities about mobile hardware? Do they understand approaches to testing mobile products? Do they know about iOS and Android usability?

These questions are quick and easy to answer if the candidate comes through a professional reference. Sending a few employees to local meetups and conferences with business cards will usually find some people interested in interviewing.

If a good candidate can't be found through recruiters or professional networks, widen the search criteria to include mobile developers. Software testing, whether it is a generalist role or something specific such as mobile testing, is often a transitional career. Some software companies hire new computer science graduates and put them in a testing role as a way to learn the product and also as a simple introduction to the technical stack.

Mobile software tester can be a good transitional role for teams that are looking to build out their mobile development team. Start a new person exclusively on the mobile testing team. After a few months, start introducing him to small bug fixes. Over the course of his first year, you should ensure that he is getting a well-rounded understanding of the product, and by the end of his first year he should be ready to start more complicated feature work.

If you do find qualified mobile testing candidates through external searches, the next step is to assess their skills.

Skills assessment

Try having candidates test actual software to assess their skills during the interview. Build a small mobile application in-house with some bugs built-in, and install it on a platform that is relevant to what the team is using. Point candidates to the software, give them a specific mission, and let them test for a predetermined time. This gives the interviewer the ability to see how candidates actually work. Do they ask important questions? Can they find bugs the team cares about? How do they communicate their discoveries? How do they communicate work status?

Make a checklist of the mobile testing skills needed and the skills a candidate has. The more overlap there is, the better the candidate is. Here are a few broad categories to help you get started.

Hardware

The software we use every day on laptop and desktop computers is restricted by the hardware. Local storage constrains how much data can be stored. RAM controls how much software can be run at any point in time. Mobile environments are exactly the same, but on a smaller scale. A mobile device has only so much storage, so much RAM, and so much graphic processing ability. A tester with knowledge of hardware constraints might have a different approach to load testing a native mobile app running on iOS 8 versus Android Nougat based on how those resources are allocated.

There are other considerations that only a person experienced with mobile platforms will notice. Does the Wi-Fi connection drop randomly in the developer version of iOS 8 on an iPad Mini? Does the battery get a little too hot on Samsung devices running Android Marshmallow? A mobile tester experienced with platform problems will know the answers to these questions.

Platform

All testers should have, or be able to develop, expertise in mobile operating systems. Each mobile software platform—iOS, Android, Windows Mobile, and so on—is a special environment. The most obvious difference is the user interface design pattern. The menu button, for example, will be located in a different place and have a different icon for each platform.

There are many other platform differences that will influence how testers approach their work. Someone working on an Android platform needs to be aware that the product will be used on a myriad of devices, so there are a lot more unknowns developing for that platform than for iOS. Apple’s platform also has its own quirks—many of the most annoying ones are in the approval process for Apple's App Store, which is more restrictive than the Android store.

Automation

Teams with an automation strategy will need a person who specializes in automation for mobile products. Automation, especially for user interface tests, is a skill that takes time to develop. Tools that build automation scripts by performing a use case on the app are especially helpful and require no coding knowledge. However, building automation teams without some experienced people often results in failed projects and blown budgets.

The most qualified candidates can sometimes be found among people who have worked as both software developers and testers. WebDriver is still the most popular tool for running the UI during automated tests, but it will have to pass through a proxy such as Appium rather than running on the device directly. A mobile tester doesn't necessarily need experience with a specific automation tool.

That said, automating the user interface in mobile apps should generally be the last resort, just as you would expect when testing web apps. Testing at the API layer and coaching developers to design better unit tests will also contribute to a strong mobile automation effort.

But starting with automation underneath the user interface will result in faster and more stable tests that can be run with every build. Skill in one tool will usually translate to the others and rely on the same fundamentals. As with web apps, some mobile testers will specialize in building tooling and automation. The skill set here will directly affect choices for the next topic, team fit.

Here are a few popular free, open-source mobile automation tools that testers might mention on their résumés:

Again, we’re not interested in finding out if they know a particular tool, just that they have experience with modern mobile test automation tools.

Team fit

This delivery cadence will affect the shape of the development team, the skills and culture needed. Start by looking at the existing release cadence, along with the desired release cadence, and what changes will have to happen to reach that desired state. A mobile web team that moves fast might want to release a half-dozen times a day, which means testers working concurrently with the developers, perhaps at the same workstation. An app-based product that needs a few days to process through the store might release every few weeks and have testers working separately. Map the desired cadence to a desired culture and team structure. Then hire, transfer, and/or train testers who fit that team structure and will help make it a reality.

Additional reading

Building a testing team from scratch is hard. Mobile concerns make it even harder to get right. There are a few other resources available that provide more information on how to build testing teams, train new testers, and cultivate their talent.