Is Your Minimum Viable Product (MVP) Biased?

It’s a terrific idea: ship out a software product with the fewest bells and whistles needed to perform a task. The early real-world feedback on this minimum viable product (MVP) will enable refinements and an even better product sooner than traditional software development. MVP was popularized as part of “Lean Startup” companies and was quickly embraced as a component of Agile software development. But the MVP process embeds troubling equity pitfalls.

That’s because many MVP builders don’t apply an equity lens, but Abt is taking steps to create more equitable initial releases and final products by gathering feedback from diverse audiences. The shift will provide products and their tangible benefits more quickly to those who may need them most.

One mitigation strategy is hiring a diverse development and test team since teams often believe a product is ready when it works for them. The testers should include members of the demographics of the target user groups, ensuring that those with lived experience provide feedback on the product. Another approach is to incorporate expected user profiles with an equity lens into the definition of what it means to be viable. The product must be viable for each user sub-group to be considered an MVP.  A human-centered design (HCD) approach, which focuses on users’ wants, needs, and preferences in every phase of the development process, is critical. So is journey mapping, which visualizes how different personas navigate the system to achieve their goals.

A diverse business analysis team that can translate the results of the HCD and journey mapping exercises into concrete requirements and test cases is also essential. The development roadmap and testing plans should include definitions of viability for each of the solution’s user groups.

Minimum viable is thus not defined with respect to a generic user, which defaults to young or middle-aged, white, male, abled, cisgender, high income, American, urban, and English-speaking. Instead the context for minimum viable is each key user demographic. For example, acceptance criteria for a minimum viable facial recognition product should be defined not as successfully matching 95% of the population overall, but as 95% of men, 95% of women, 95% of African Americans, and 95% of whites.

At Abt, we are also experimenting with using our Configuration Control Board (CCB) to enforce equity goals. The purpose of a CCB is to ensure that all key elements of a product (including security testing, communication, implementation, rollback if needed, etc.) are in place prior to any release. The CCB members include leaders from each of these key disciplines, each of whom reviews the overall change from their specialty area.

Most of the areas the CCB governs are not minimally viable. A product will work (at least for a little while) without backups in place or without enforcing strict security. The CCB already redefines the concept of minimally viable to ensure compliance with certain baseline standards of good practice. We are now including equity among these standards.   

Development teams that seek to release new products must demonstrate that they have identified all key target users and that the product is viable for all of those groups. They do so by completing a form that requires them to specify target audiences. The form lists out equity issues that may arise with respect to supporting those groups. The development team then indicates how the product and testing process addresses the concern.

While the work of building the solution equitably begins in the requirements and design phase, the CCB serves as enforcer, since the CCB will not approve the release until the equity criteria are met. Enlisting an equity expert for the board as either a voting or consulting member to review the equity criteria also is a potentially valuable approach.   

Another way to address the imbalanced representation in the feedback loop from early adopters as a group is to learn from our survey research colleagues and think of the early adopters as a sample group. Both private and public sector organizations releasing MVPs could collect data on early adopters, then weight the feedback from individuals based on their characteristics of interest.

Neeva, a search engine company that Snowflake recently purchased, did this successfully. The company created a waiting list to get early access to its product and found that more than 75% of initial sign-ups were male. When collecting feedback from users, the company weighted the responses to equalize male and female feedback. This enabled Neeva to build its roadmap to support product features—such as a family subscription plan—that were more heavily favored by women than men.

Disaggregating feedback based on key demographic variables can also help to offset skewed results from an unrepresentative early adopter group.  Applying sampling, weighting, and disaggregation approaches to product feedback can be applied to public-sector systems and is a great example of how combining expertise in equity, research methods, and digital work can produce a superior result. If the MVPs are indeed experiments, then sound experimental methods are essential.

Final Thoughts

When people hear the acronym MVP, they usually immediately think of “most valuable player.” Software developers bemusedly inform them that the acronym refers to minimum viable product. Unfortunately, in this case, the joke is on the software teams, since too often the process of defining minimal requirements unintentionally and inequitably anoints certain types of players as more valuable than others. The software team misconstrues itself as the most valuable of all. But with proper awareness and procedures to address inequities, the minimum viable product can become a most valuable product—and viable for all.

¹ Early Adopters is the second of five groups of adopters described in Everett M Rogers 1962 book Diffusion of Innovation. The full list includes “Innovators,” “Early Adopters,“ Early Majority, ”Late Majority,” and “Laggards.”