A CIO and CTO Technology Guide To Generative AI - McKinsey
A CIO and CTO Technology Guide To Generative AI - McKinsey
DOWNLOADS
H
ardly a day goes by without some new business-busting
development related to generative AI surfacing in the media. The
excitement is well deserved— McKinsey research estimates that
generative AI could add the equivalent of $2.6 trillion to $4.4 trillion of
value annually.[ 1 ]
CIOs and chief technology officers (CTOs) have a critical role in capturing
that value, but it’s worth remembering we’ve seen this movie before. New
technologies emerged—the internet, mobile, social media—that set off a
melee of experiments and pilots, though significant business value often
proved harder to come by. Many of the lessons learned from those
developments still apply, especially when it comes to getting past the
pilot stage to reach scale. For the CIO and CTO, the generative AI boom
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 1/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
2. Reimagine the business and identify use cases that build value
through improved productivity, growth, and new business models.
Develop a “financial AI” (FinAI) capability that can estimate the true
costs and returns of generative AI.
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 2/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
Instead, CIOs and CTOs should work with risk leaders to balance the real
need for risk mitigation with the importance of building generative AI
skills in the business. This requires establishing the company’s posture
regarding generative AI by building consensus around the levels of risk
with which the business is comfortable and how generative AI fits into the
business’s overall strategy. This step allows the business to quickly
determine company-wide policies and guidelines.
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 3/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
CIOs and CTOs should be the antidote to the “death by use case” frenzy
that we already see in many companies. They can be most helpful by
working with the CEO, CFO, and other business leaders to think through
how generative AI challenges existing business models, opens doors to
new ones, and creates new sources of value. With a deep understanding
of the technical possibilities, the CIO and CTO should identify the most
valuable opportunities and issues across the company that can benefit
from generative AI—and those that can’t. In some cases, generative AI is
not the best option.
Providing this level of counsel requires tech leaders to work with the
business to develop a FinAI capability to estimate the true costs and
returns on generative AI initiatives. Cost calculations can be particularly
complex because the unit economics must account for multiple model
and vendor costs, model interactions (where a query might require input
from multiple models, each with its own fee), ongoing usage fees, and
human oversight costs.
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 5/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
IT operations (ITOps): CIOs and CTOs will need to review their ITOps
productivity efforts to determine how generative AI can accelerate
processes. Generative AI’s capabilities are particularly helpful in
automating such tasks as password resets, status requests, or basic
diagnostics through self-serve agents; accelerating triage and
resolution through improved routing; surfacing useful context, such as
topic or priority, and generating suggested responses; improving
observability through analysis of vast streams of logs to identify events
that truly require attention; and developing documentation, such as
standard operating procedures, incident postmortems, or performance
reports.
models
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 6/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
The CIO and CTO can think through the implications of these options as
three archetypes:
There are two common approaches for integrating data with generative
AI models in this archetype. One is to “bring the model to the data,”
where the model is hosted on the organization’s infrastructure, either
on-premises or in the cloud environment. Cohere, for example, deploys
foundation models on clients’ cloud infrastructure, reducing the need
for data transfers. The other approach is to “bring data to the model,”
where an organization can aggregate its data and deploy a copy of the
large model on cloud infrastructure. Both approaches achieve the goal
of providing access to the foundation models, and choosing between
them will come down to the organization’s workload footprint.
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 7/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
Each archetype has its own costs that tech leaders will need to consider
(Exhibit 1). While new developments, such as efficient model training
approaches and lower graphics processing unit (GPU) compute costs
over time, are driving costs down, the inherent complexity of the Maker
archetype means that few organizations will adopt it in the short term.
Instead, most will turn to some combination of Taker, to quickly access a
commodity service, and Shaper, to build a proprietary capability on top of
foundation models.
Exhibit 1
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 8/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
Exhibit 2
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 9/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
For the Taker archetype, this level of coordination isn’t necessary. But for
companies looking to scale the advantages of generative AI as Shapers
or Makers, CIOs and CTOs need to upgrade their technology architecture.
The prime goal is to integrate generative AI models into internal systems
and enterprise applications and to build pipelines to various data sources.
Ultimately, it’s the maturity of the business’s enterprise technology
architecture that allows it to integrate and scale its generative AI
capabilities.
There are five key elements that need to be incorporated into the
technology architecture to integrate generative AI effectively (Exhibit 3):
Model hub, which contains trained and approved models that can be
provisioned on demand and acts as a repository for model checkpoints,
weights, and parameters.
Exhibit 3
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 11/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
In evolving the architecture, CIOs and CTOs will need to navigate a rapidly
growing ecosystem of generative AI providers and tooling. Cloud
providers provide extensive access to at-scale hardware and foundation
models, as well as a proliferating set of services. MLOps and model hub
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 12/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
In this context, CIOs, CTOs, and chief data officers need to work closely
together to do the following:
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 13/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
team
CIOs and CTOs need to ensure that the platform team is staffed with
people who have the right skills. This team requires a senior technical
leader who acts as the general manager. Key roles include software
engineers to integrate generative AI models into existing systems,
applications, and tools; data engineers to build pipelines that connect
models to various systems of record and data sources; data scientists to
select models and engineer prompts; MLOps engineers to manage
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 14/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
Realistically, the platform team will need to work initially on a narrow set
of priority use cases, gradually expanding the scope of their work as they
build reusable capabilities and learn what works best. Technology leaders
should work closely with business leads to evaluate which business cases
to fund and support.
Our latest empirical research using the generative AI tool GitHub Copilot,
for example, helped software engineers write code 35 to 45 percent
faster.[ 5 ] The benefits, however, varied. Highly skilled developers saw
gains of up to 50 to 80 percent, while junior developers experienced a 7
to 10 percent decline in speed. That’s because the output of the
generative AI tools requires engineers to critique, validate, and improve
the code, which inexperienced software engineers struggle to do.
Conversely, in less technical roles, such as customer service, generative
AI helps low-skill workers significantly, with productivity increasing by 14
percent and staff turnover dropping as well, according to one study.[ 6 ]
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 15/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
Beyond training up tech talent, the CIO and CTO can play an important
role in building generative AI skills among nontech talent as well. Besides
understanding how to use generative AI tools for such basic tasks as
email generation and task management, people across the business will
need to become comfortable using an array of capabilities to improve
performance and outputs. The CIO and CTO can help adapt academy
models to provide this training and corresponding certifications.
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 16/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
practices
1. “The economic potential of generative AI: The next productivity frontier,” McKinsey, June 14,
2023.
2. “The economic potential of generative AI: The next productivity frontier,” McKinsey, June 14,
2023.
3. Begum Karaci Deniz, Martin Harrysson, Alharith Hussin, and Shivam Srivastava, “Unleashing
developer productivity with generative AI,” McKinsey, June 27, 2023.
4. Vishal Dalal, Krish Krishnakanthan, Björn Münstermann, and Rob Patenge, “Tech debt:
Reclaiming tech equity,” McKinsey, October 6, 2020.
5. “Unleashing developer productivity with generative AI,” June 27, 2023.
6. Erik Brynjolfsson, Danielle Li, and Lindsey R. Raymond, Generative AI at work, National
Bureau of Economic Research (NBER) working paper, number 31161, April 2023.
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 18/19
28/9/23, 21:55 A CIO and CTO technology guide to generative AI | McKinsey
Search Openings
https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/technologys-generational-moment-with-generative-ai-a-cio-and-cto-guide#/ 19/19