Gathering real-time feedback to discover product defects and improve design
“Dogfooding,” or “‘eating your own dogfood” is the procedure of using your own designs and APIs to evaluate the qualities and capabilities of a product prior to its release. The term is believed to have come from a 1970s Alpo dog food television commercial, in which Lorne Greene pointed out that he fed Alpo to his own dogs. In 1988, the term was introduced at Microsoft to increase internal usage of the company’s product. It continued to spread throughout the company, and in 1991, Dave Cutler insisted on the practice of dogfooding in the development of Windows NT, which led to more immediate fixes based on real-time user feedback.
When switching roles from that of designer to consumer, new discoveries about how a product or service is actually used can better come to light. While dogfooding isn’t the sole method of judging a design’s worthiness, real-time feedback can be invaluable for early defect discovery and design improvement.
Dogfooding can be helpful to designers, developers, marketing pros, and even testers themselves. The following scenarios exemplify when dogfooding can be beneficial:
-In the later stages of product development
-When just a few features are being added and tested
-When the product can be more complete as a documented API or release
-A more obscure form of dogfooding or self-hosting is when developers’ workstations are constantly updated with new builds as they become available; these changes “force” developers to write code to accommodate changes in a system that’s following project guidelines.
The steps to get you there
Q Analysts has developed a process for creating a successful dogfooding program, based on the many programs we have helped design and execute for some of the world’s largest tech companies. Of course, each program will have its own unique requirements, but in general, we believe that following a series of steps will help guide the program to achieve the best results.
1. Define the program structure. As with any project, it is important to articulate goals, strategies, and tactics. Will the program be conducted on a local campus or across multiple offices around the country? Who are the target users? How will they be incented to participate? How will the feedback be captured and recorded?
2. Advertise the program to the appropriate audience. This can be done any number of ways, with email being the most common. Slack or any other internal communication vehicle the company uses may also work well. Sometimes, recruiting early adopters and curious employees is facilitated by the sense of “I’m the first to try this out,” while knowing their feedback will help shape the design of the product. Very often the early beta users are company engineers who have a vested interest in fine-tuning the product. Alternatively, some programs may require more effort to create interest, with a multi-faceted marketing program that includes internal social media promotion, free swag, targeted flyer placement, and live interaction with participants during high-traffic mealtimes.
3. Implement registration procedures. Make it very clear how participants will sign up and provide next steps to support engagement.
4. Distribute the product. When distributing the product to participants, it is imperative to conduct initial installations and training to fully ensure they have what would be a typical user experience. These could be set up as “dogfood sessions,” which include instructions, Q&A, and online updates as needed. The point is to facilitate the user experience and provide a great start with confidence to use the product and report important feedback. When it comes to hardware, keep in mind such steps as flashing software and preparing configurations, inventory tracking, shipping and distribution (potentially worldwide), as well as returns and repairs, which are all a part of the operations process.
5. Communicate results. This can be done in a multitude of ways that include feedback forms, social media, field tests, focus studies, user studies, A/B testing (or split testing, a method of comparing two versions of a webpage or app against each other to determine which one performs better), and email. The feedback should be compiled and reported to influence data-driven decisions in a short period of time. Publish triage actions on defects that were reported to show progress and demonstrate to participants how valuable their feedback is.
6. Ensure that personal assistance is provided to participants at all times. Maintain office hours at locations convenient for participants, set up a “customer service” phone line, or implement an online chat mechanism to make it as easy – and frustration-free – as possible for participants to get needed support.
7. Uphold best practices for privacy and identity protection. Given the concerns around privacy and identify protection, it is paramount to define and be clear about the levels of privacy and configurations put into place. The program should include participants who volunteer or otherwise opt in to provide their feedback, which is usually anonymous and eliminated over time.
Ultimately, the most important outcome in any dogfood program is to learn how a product behaves and performs in the hands of end users. Correlating near real-time information and feedback from participants is used to make important decisions about product features and usability. This information should give stakeholders an immediate view of how the dogfooding program contributes to product quality in the long term.
