A Product Management Framework for AI Transformation
*Originally published on LinkedIn on Oct 6th, 2023
I've been thinking a lot about how company building and product management in the B2B software space will change in the coming years as a result of significant advancements in AI. A rainy weekend in Vancouver gave me the opportunity to put these thoughts on a page.
The Three Waves:
The first wave of AI adoption has been focused on content processing and basic reasoning. Transcription, summarization, sentiment analysis, and chatbot interfaces add significant value at relatively low cost to any product that works with meaningful amounts of data. Knowledge of how to leverage this new technology effectively should be considered essential for most product managers.
It appears that the second wave of AI adoption will be focused on content creation. For many use-cases, this will also be an excellent way to unlock value. One example is stock photography. As a hobbyist photographer, I have been experimenting with the generative AI image and video creation tools. Although they are still rough around the edges, I'm blown away at what will be possible as these tools become more refined and I've already started noticing AI created images in ad campaigns.
I anticipate that there will be a number of cases where the application of generative AI will be fraught with challenges that result in a roller coaster of adoption. As an example, software that leverages generative AI could be used to write significant quantities of personalized sales prospecting e-mails and that may provide a short-term uptick in leads. However, when every sales team in the world has the capacity to generate thousands of customized e-mails every day, that entire sales motion may become ineffective as open-rates plummet and new tools are put in place to filter out that content. Product managers should become aware of what is possible and be very thoughtful about identifying use-cases where sustainable and ethical long-term value can be created through automated content creation. At the same time, new opportunities will be created to filter through an increasing amount of synthetic noise to find the gems of content that are unique and insightful.
The third wave of AI adoption will be about AI's ability to transform anything into an API that empowers it to complete tasks. This is the wave that I believe will lead to rethinking of entire categories of software. AI will be able to effectively turn channels and interfaces that are currently used by humans into APIs that can be used by software programs to complete tasks. By being able to access a wide range of specialized tools, AI will also be able to complete reasoning tasks that exceed what is possible in an Large Language Model model. Consider what will be possible when an AI based software program can interact with a GUI as easily as it currently interacts with an API. As an example, I uploaded a partial snapshot of the user interface for AutoCAD to ChatGPT and asked it how to make a certain shape. AutoCAD is a notoriously difficult and advanced CAD program but without doing any additional prompting, ChatGPT gave clear and concise instructions (full disclaimer, I haven't double checked if they are correct - but I also didn't provide it with any context about how to use this program.) If the ChatGPT was given control of my mouse and keyboard, it appears it would have been able to complete the task on its own.
Six Step Framework
To start to map out the impact that these innovations will have on product categories, I suggest thinking about six steps that all B2B users go through when solving a problem. Today, all of these steps involve meaningful human work and there are longstanding (and until now, largely valid) assumptions about where to draw the line between what humans will do and what software will do. These assumptions may cause product managers to operate with blinders on; accepting constraints that no longer need to exist.
The Six Steps:
1) Specifying the objective and setting boundary conditions for the desired solution
2) Collecting inputs
3) Applying domain specific expert knowledge to specify the solution
4) Design and analysis
5) Coordinating with third parties
6) Generating an output
A Case Study
To demonstrate how to apply this framework, let's consider a very low-tech use-case of a company that builds wooden decks. For the purpose of this analysis, let's assume that general purpose robots do not exist and that we will only be using AI features that exist today or are on the verge of being possible today.
Step 1: Specifying the objective and setting boundary conditions for the desired solution
Although AI has the potential to help in this stage, I believe this is where the human touch will still be highly valued. A project manager should meet the client at their home. During this meeting, they will establish trust, learn about their family and how they plan to use the deck, get a sense for the surrounding landscaping, work through the trade-offs between budget and size, observe their interior and exterior decorating style, and determine how the customer views trade-offs between up-front cost vs ongoing maintenance.
Step 2: Collecting inputs
There are two types of inputs required for building a deck. The first is measurements in the field. This includes dimensions and soil conditions. Assuming general purpose robots don't exist, those specifications would be collected by the project manager. The other type of input is generally available online but not through an API. This includes the current version of the building code used by the municipality, wood span tables, specifications from parts manufacturers, climate information regarding the frost depth, property survey information to determine the property lines, and material price and availability from distributors. In the past, all of these inputs had to be collected manually through a combination of phone calls and website visits. With AI, all of that information will be collected automatically by an AI agent that can browse the web and read and interpret web pages, PDF files, tables, and images.
Step 3: Applying domain specific expert knowledge to specify the solution
Across a very wide range of fields, this is a step that computers have struggled to help with in the past. In this stage, the project manager (or perhaps a designer, draftsperson, or engineer,) would have interpreted the information collected in the prior stage to make key decisions. This means they would read the building code and then make the key design decisions. How many footings are needed? How thick does the beam need to be? What will be the rise and run on the staircase and how many steps will be required? What types of fasteners will be used? AI allows us to transform this step. Large Language Models are generally not capable of performing this type of reasoning out-of-the-box but if they are provided with appropriate domain specific context and situation specific variables, they can chain together appropriate conclusions. In this case, the AI tools would have access to the written building code and other resources to be able to create the specifications for the deck.
Step 4: Design and analysis
In the next step, a user who is trained on a complex CAD program would draw out the design. Innovation in this space has largely focused on making tools that are more powerful (e.g. parametric design features) or easier to use (e.g. SketchUp). AI could change things completely because it can use the software in the same way that a skilled user can, even if there is no API. Although a single person may only be able to become proficient in a handful of advanced software tools, AI agents will be able to read the user manual of any software program in an instant and then interact with it as if it were a skilled human user by taking over control of the keyboard and mouse. In this example, AI will take the specifications from the prior step and then draw the full deck in CAD. It will be able to identify and resolve conflicts, optimize the design to minimize material waste, and produce a bill of materials and cut-list.
Step 5: Coordinating with third parties
In industries such as travel and finance, significant progress has been made on facilitating communication between software systems within and across organizations through APIs. That said, across all industries, there is still an immense amount of time spent by humans manually communicating between parties, filling out forms, scheduling appointments, checking inventory, placing orders over the phone, requesting details, etc. AI technologies (including the ability to write and read e-mails and interpret and generate voice) mean that all existing traditional manual communication channels can effectively be used as an API by software programs. In the blink of an eye, the number of interconnections that are possible between software systems and between software systems and the outside world has increased significantly. In the deck example, AI can take the next steps to submit the building permit, phone the gas company to ensure there are no gas lines in the construction zone, schedule equipment rentals, negotiate with product distributors, hire subcontractors, and purchase the required materials.
6) Generating an output
I am leaving this step in the example for completeness. In a number of cases, AI will be able to take the final step of generating the output. Examples could include allocating money to an investment portfolio, generating and submitting a report, or 3D printing a part. In our example of building a wooden deck, this is where AI will step aside and the carpenters will pick the project back up (until general purpose construction robots are available.)
As an exercise, I'd encourage you to choose any type of problem that business users face and to map out the six steps and how AI may transform those steps. How does life change in the coming years for a tax accountant, a doctor, or a sports coach?
What are the implications for Product Managers?
Let's assume for the moment that everything I have described above will come to fruition in the next five years. An interesting question for product managers is how to capture the new value. It is not obvious how the solution space will evolve. In the example, will existing CAD programs bolt on AI capabilities to move up and downstream? Will firms specialize in being virtual experts across a wide range of domains? Will material manufacturers work to make their information AI friendly so that it is easier for AI tools to integrate them into designs? Will standards agencies re-write codes in a way that is more clear to ensure more consistent interpretation by AI tools? Will companies that do not even exist today replace longstanding incumbents?
I would recommend that product managers do the following to prepare:
1) Look at your user needs through a much wider lens that is anchored on outcomes and does not make any assumptions about the role the human user will play in achieving the outcomes
The boundaries that currently exist around your solution will fall to the wayside. Instead of saying "The user needs to draw a rectangle," say "The user needs a drawing of a rectangle" and be open to the possibility that AI may play an increasingly active role.
2) Officially add AI as a user persona
Whether you like it or not, AI tools will very likely be interfacing with your product. As for all major user groups of your product, you should have a persona for those AI tools to see your product through their "eyes."
3) Dive deep into issues surrounding ethics, privacy, uncertainty, and security
By their nature, many AI tools or combinations of AI tools are opaque, non-deterministic, prone to generating incorrect data, and liable to take potentially unpredictable actions if appropriate guardrails are not in place. Responsible application of AI will require product managers to become deeply versed in these issues.
4) Play
Many AI tools today are free or cheap. Set aside time to try them out and experiment with them. Challenge the tools to see where they break and where they excel. As you learn more, study how they work behind the scenes and see if you can improve them for your specific situation.
Conclusion
As product managers, we have seen a number of tech fads come and go. I'm confident that AI won't be one of them. It's here to stay and it has the potential to allow us to deliver a lot of new value to our users. I expect that the coming years may be chaotic as entire categories of software are redefined. I'd love to hear what you are doing. How are you thinking of AI as a part of your product management process?