Moving target of customers’ needs – Unendorsed challenge of building enterprise products

The primary emphasis for building a great product is to discover a right problem and getting married to it. However, customers’ needs and problems are increasingly becoming moving targets. The needs and problems evolve, so do the solutions to address them (i.e. the outcome too evolves) triggering an uncertainty of which needs or problems to address and what outcomes to deliver. It is a challenge that has primarily gone unendorsed or not consciously realized until now. The biggest mistake that Product Managers commit while building new product is to freeze needs and problems at a specific point in time and start building products for those static needs. Nevertheless, Enterprise products address hard problems and building such products consume around 12 – 24 months. With technology evolving much faster than ever, market dynamics changing too fast, and human race undergoing more changes than before, new needs or problems emerge, prevalent needs or problems extinct, and solutions embracing technology advancements to address needs or problems come in various shapes and forms. Primarily, Product Manager cannot taste success building a product for static needs. Product Manager should learn to hit a moving target anticipating how needs or problems change as a function of time in future. Agile methodologies and MVP (Minimum Viable Product) address those fallacies by reducing development cycles and incrementally validating whether the product is addressing right needs or right problems and delivering right outcomes. However, MVP seldom works for enterprise products.

 

Why MVP and agile methodologies could not address those problems of dynamic needs?

Enterprise products affect real businesses of customers, they might either help customers increase revenues, manage costs or streamline operations. The fundamental truth is enterprise products affect business in some form and hence they should be robust and resilient with zero-tolerance for failures while addressing a real business problem that is at the top of customers’ priority list. Therefore, delivering something quick, validating it and incrementally evolving the product does not augur well for enterprise products. Enterprise customers seek a complete solution. There is also a necessity for shrouding development efforts of certain enterprise products in secrecy to introduce an element of surprise and euphoria during launch. Then, how do Product Managers validate enterprise products causing minimal interruption to its targeted customers? Through maintaining a subtle balance between using MVP and relying on customer insights and experiments. Product Manager shall use MVP at the initial stages of product development to validate whether there is a real problem, whether the new product is addressing the real problem, and whether the new product is targeting the right customer segment. The problem with that approach is MVP does not consider time as one of its parameters. While constructing a hypothesis and validating assumptions using MVP, Product Manager has to validate assumptions across time space ensuring whether needs or problems persist across the duration of the product lifecycle and identify how they evolve. The appropriate approach is to understand the causal effect between needs, problems and corresponding factors that cause them. MVP framework for such analysis is:

 

The fundamental premise is to understand what is causing the need while validating the existence of a real problem or need and bundle the findings with customer insights to understand how the need or problem manifests in future. Understanding causal-effect of the need and its corresponding triggers along with customer insights should provide required clarity about the relevance of the need or how the need evolves in future. Customer insights are essential to comprehend what has caused the present to diverge from the past and use that as a reference to anticipate what will cause future to diverge from the present. Building customer insights are one of the key foundational pieces for building great products that enterprise customers want. Product Manager has to build customer insights through experiments and observing customers in their natural habitat, immersing in their business, assimilating their business process, problems and challenges and not just listen to what they say but to read between the lines to understand what they did not say. There is a dire need for restoring the lost art of gathering customer insights. MVP bundled with customer insights and experiments are required for validating any assumptions related to the new product and building a visual map of how customers’ needs or problems evolve and what new outcomes they might desire. Product Manager should learn to hit a moving target anticipating how needs or problems change as a function of time in future. How do you hit a moving target with an arrow, we notice the rate at which the target is moving, how long does it take for an arrow to hit the target and accordingly predict the possible position of the target when the arrow is released. Product Manager should do something similar not just to understand needs and problems of today but also to predict the needs and problems while the new product hits the market. Doing so, the new product will hit the bull’s eye when it hits the market.

Why does moving targets of needs and problems affect enterprise segment alone?

A unique challenge that Product Managers encounter with enterprise products alone and it is not applicable for consumer products is that enterprise customers’ are resistant to frequent product upgrades because of CAPEX (Capital Expense) and OPEX (Operating Expense), they tend to use a product for a longer period of 3-5 years (especially on premise products). On the contrary, business models of most consumer products rely on frequent product upgrades. There is no benefit in stacking all the value in a single product and making customers stick to a consumer product for 3-5 years. The duration for moving target of needs or problems of enterprise customers just shifts from product launch to eclipse the entire duration of product life cycle.

 

Product Manager should anticipate customers’ needs or problems as a function of time across the entire duration of product life cycle. Customer insights combined with extensive knowledge of how markets evolve, how technologies evolve, and how customers’ behaviors change should provide an estimation of how customers’ needs or problems change as a function of time and what new outcomes are possible. For certain enterprise products, addressing customers’ needs or problems as they evolve is not entirely possible without building a product architecture or a platform that can scale. I was once managing an HW product used by ISPs (Internet Service Providers) for defining policies of their internet users. During the launch of the product, internet speeds offered to each user was low. However, the internet speeds offered to each user raised exponentially far exceeding our predictions and the product could not meet the new requirement resulting in the early retirement of the product (or rather the product has to retire prematurely). In the enterprise segment, we tend to face a similar problem with complex SW products as well and refactoring the architecture can help but it will be costlier. Even so, refactoring the entire HW or product architecture takes time and there is a risk of not being able to address evolving needs or problems in a timely manner. When the only thing that is certain about future is uncertainty, a question that stares at every Product Manager is how to anticipate what needs will prevail and what new needs will emerge. In most cases, needs will be static, but the scale required varies. Another question that stares at every Product Manager is how much to scale and when?

 

Please keep watching this space as I revert back to further to continue the conversation on this topic. Meanwhile, I would like to understand whether you agree with above challenges. If so, please share your thoughts in the comments section below or drop me a note at murali.erraguntala@gmail.com. If we are not reaching an agreement on this topic than any further conversations will be futile, looking forward to hearing your thoughts or opinions.

Cross pollination of agile and waterfall

Withstanding all the hype surrounding agile, we did follow waterfall model while we built the new product. Agile does not work for all kinds of products even though waterfall model had its own flaws of not allowing the possibility to modify the requirements after entering development phase. So in order to introduce agility within waterfall model, we integrated both agile and waterfall. This blog talks about our experiences of experimenting with a new methodology combining both agility and waterfall. The below excerpts are from my eBook – ‘Building New Product – Experiences of a Product Manager

Our traditional approach to product development has always been the waterfall model. Nevertheless, while we embarked on our journey to build the new product we built the product upon the foundation of waterfall methodology while introducing agility within it. The ability or willingness to experiment with processes or methodologies should be fundamental for any new product development. The experimentation of processes or methodologies can occur when we know why we are doing what we are doing. There should always be a fit between the product and its development methodology, alike product-market fit. If we follow herd mentality and religiously follow a methodology without realizing why we are doing what we are doing, there will be hardly any scope for candid introspection of whether the product and its development methodology have the right fit.

Agile - WaterfallWe never consciously discussed altering waterfall and introducing agility within it. Program Manager naturally adapted the waterfall model to accommodate agility while we had several assumptions to validate, unknowns to eliminate and risks to mitigate. During the course of product development phase. There were never any serious discussions to brainstorm how to integrate both agile and waterfall development methodologies. The objectives and constraints of product development phase naturally triggered our Program Manager to combine them. Real kudos to our Program Manager to introduce agility within waterfall model. It is easy to follow stage gate process to ensure that we complete one stage and move to other. However, while we can only partially complete a stage and move to next stage. Later reverting to earlier stages based on the outcome of a current stage, it is extremely difficult to track dependencies across stages. Let me brief what we did.

For an enterprise product, the architecture of the new product is crucial. I can split the product into two blocks. Architecture block and product functionality block. Redesigning the architecture of the product is always costly and it is as good as developing a new product. Further, the choice of an HW is heavily dependent on the architecture and choice of product architecture is dependent on the HW. Both are mutually dependent. There is a necessity to ensure that HW and architecture can allow the software (i.e. functionality block) to scale as needs of customers evolve with little or no change to both the elements. Why are we putting so much emphasis on product architecture? I once came across a tweet that states every scale has an expiry date. True to those words, software has to scale in future as customer needs evolve and product architecture should support such scale requirements in future with little or no changes to product architecture. Certain elements of product architecture are rigid and it is not possible to modify it as new needs arise. Therefore, it becomes crucial to building a product architecture with a hindsight of how customer needs evolve in future. I did vehemently establish the fact several times in this eBook that the new product should cater to needs of tomorrow and not just for needs of today. Building MVP is not viable to identify needs of tomorrow while it is a viable option for identifying needs of today. Developing customer insights along with identification of factors that can influence the evolution of technology, the evolution of the market, the evolution of customer needs are crucial for anticipating probable needs of tomorrow. Clearly, for enterprise products where architecture can play a big role in determining the success of the new product and where the design and development of architecture happen upfront, agile is counterproductive. Incrementally building product architecture is a grave mistake. However, Program Manager introduced agility in building functionality blocks that address actual customer needs.

During requirements phase, we did outline all assumptions, unknowns, and dependencies across requirements of both product architecture and product functionality. The first priority was to eliminate all unknowns and validate all assumptions surrounding the design of product architecture. The design was mere theoretical until we start building it. Therefore, there will be many assumptions primarily around the ability of product architecture to meet scale requirements. To validate all those assumptions and eliminate unknowns, we constructed hypotheses. On those lines, we also constructed hypotheses surrounding product functional requirements. The product development happens in an interval of 6-8 weeks called DTHOs. Our 1st priority was to eliminate unknowns and validate all assumptions related to product architecture in earlier DTHOs. Dependent products requirements were refined based on the outcomes of those DTHOs.

 

Hypotheses of Product Architecture
Hypotheses Associated Requirements Dependent Requirements
H1 R1 R11
H2 R2 R12
H3 R3 R13
H4 R4 NA
H5 R5 NA

 

Hypotheses of Product Functionality
Hypotheses Associated Requirements Dependent Requirements
H6 R6 R14
H7 R7 NA
H8 R8 R15
H9 R9 R16
H10 R10 NA

The list of hypotheses surrounding both product architecture and product functionality has provided an indication to Development Manager and Program Manager on two aspects. First, the set of functionalities or requirements (R1…R10) that engineering team should implement during the initial DTHOs for validating hypotheses. Second, the set of functionalities or requirements (R11…R16) that engineering team should delay until validation of hypotheses (H1…H10). Delayed functionalities or requirement are later implemented depending upon pivot or preserve scenario in accordance with the outcome of validating each hypothesis.  PRD has clearly articulated the list of incomplete requirements. As initial DTHOs validated hypotheses (H1…H10) through implementing requirements (R1…R10), I refined all dependent requirements (R11…R16) implemented in subsequent DTHOs. The dependent requirements are mostly functional requirements that are refined after validation of hypotheses related to both the requirements of product functionality and product architecture.

One example of a hypothesis is – Ability of the product to meet throughput scale requirements (Hypotheses – H1). The PRD had a clear requirement for the product to deliver 60 Gbps at an average packet size of 512 bytes while maintaining low latency. The conventional wisdom is to measure and report maximum throughput at an average packet size of 512 bytes. Engineering team did validate the hypothesis by building the entire product but performing sizing through simulation. We identified during hypothesis validation that 60 Gbps is achievable albeit with a higher packet size. As I had indicated earlier, marketing team later did lots of analysis, found out the average packet sizes in customer networks is much higher, and we could conveniently defy the conventional wisdom. We never froze entire requirements. Depending on the hypotheses validation, dependent requirements were refined during the course of the new product development introducing agility within waterfall model.

The biggest criticism of waterfall model is that it always heads in one direction. However, we did introduce agility to shift our thinking back and forth between development, design, and requirements. Requirements (R11…R16) were kept open and refined after corresponding hypotheses are validated. We clearly introduced a loop containing product requirements, product design, and product development until we validated all the hypotheses. However, following a strict waterfall model, we had to build a foundation for product architecture upfront unlike in agile before going in a loop containing requirements, design, and development of product functionality. Such model is required for enterprise software products until we find out a methodology for modularly building product architectures for complex systems. Modular architecture should eliminate rigidness facilitating incremental additions to product architecture without the need for redesigning the entire system.

I really feel bad that some of the Product Managers hide their inefficiencies under the veil of methodologies such as Agile, MVP etc. Those are excellent methodologies drafted with good intentions. Nevertheless, those methodologies are more abused than used. MVP formulated by Eric Ries is an excellent way to incrementally build the product through an incremental process of build, measure and learn. Does it mean that a Product Manager should learn every aspect of product requirements through MVP? When it gets difficult for a Product Manager to comprehend customer requirements, the easy way is to put emphasis on MVP. If we start validating every requirement of the product, it will unnecessarily delay product development. There is always a necessity to strike a balance which I find missing. Further, in the case of enterprise products that have a direct effect on customer’s business, the customers will not be ready to validate it thoroughly. While building complex products for enterprise customers, it is always crucial to do extensive customer research and draft those details in PRD to get the bigger picture of the overall product. User stories do not communicate bigger picture. I can definitely bet if I ask for a PRD, some might even perceive me as a person from the dinosaur age. I am not showing my indignation on everyone. Probably, just a handful of them. My request is to embrace methodologies and process, comprehend them thoroughly and adapt them to suit your new product development. Always find the right fit between the product and its corresponding methodology.