In a world where users have multiple digital devices and a plethora of programs to choose from, quality is often what sets one app apart from another. Crashes, bugs, unreliable security, slow performance, poor interoperability, and many more issues can have an extremely negative impact on the user experience and, ultimately, on the developer’s bottom line. Today more than ever, with social media and online reviews and ratings sites such as G2.com, optimizing performance through quality assurance (QA) and testing is crucial for upholding a brand’s credibility and increasing market share.
How do you define and measure quality?
Quality is a subjective term that has different connotations to different individuals. A product manager, software engineer, and CEO will likely have three different perspectives on what quality is. One thing we know is that nothing is ever 100 percent perfect, so determining at the outset of a QA program what the threshold or target goals will be at the finish line is paramount. For example, is it quality when:
- The software is known to have 300 bugs, and you fix 250 of them?
- The App Store gives the software a rating of 4 stars out of 5?
- A compliance checklist is fully met, but several bugs are discovered after the product is released?
When measuring quality levels, it is helpful to collect both quantitative as well as qualitative data to gather a more accurate assessment. Quantitative measures in the way of numbers and percentages provide a benchmark and solid evidence of the results achieved. The key is to determine in advance what that metric will be – is 80 percent good enough, or is 90 percent a more appropriate or attainable target goal?
Qualitative measures, though more involved, provide additional insights that can guide quality levels. Putting software into the hands of real users – such as internal staff, through “dogfooding” programs – solicits valuable feedback that may result in design changes that improve product quality. Quality can actually be quantified through user feedback. If a community of users uncovers additional new bugs, then a new percentage of bug fixes can be reported.
When should you measure quality?
Also keep in mind that quality has a life cycle and should be measured differently at varying stages. A product that has already shipped has a different set of problems than a product that is being released for the first time. In the latter case, the focus is on anticipating as many issues as possible and therefore, being proactive and preventative. A product that has evolved over time needs to be examined for defects, or regression tested, while looking for, or anticipating, new issues that may occur as a result of adding new features and functionality.
As times change and technology progresses, user expectations change as well. Staying abreast of new technologies and keeping a pulse on what users want is essential to QA and maintaining performance.