Computing, cognition

and the future of knowing

How humans and machines are forging
a new age of understanding

Dr. John E. Kelly III

Senior Vice President,
IBM Research and Solutions Portfolio
 It’s not surprising that the Those systems have been deterministic; cognitive
systems are probabilistic. They generate not just
Here at IBM, we have been The history of computing
public’s imagination has answers to numerical problems, but hypotheses, working on the foundations of and the rise of cognitive
been ignited by Artificial reasoned arguments and recommendations about cognitive computing technology
Intelligence since the term more complex — and meaningful — bodies of data. for decades, combining more
To understand the future of cognitive computing,
was first coined in 1955. What’s more, cognitive systems can make sense than a dozen disciplines of it’s important to place it in historical context. To date,
In the ensuing 60 years, of the 80 percent of the world’s data that computer advanced computer science with there have been two distinct eras of computing — the
scientists call “unstructured.” This enables them Tabulating Era and the Programming Era — and IBM
we have been alternately to keep pace with the volume, complexity and 100 years of business expertise.
has played a central role in the development of both.
captivated by its promise, unpredictability of information and systems in the Now we are seeing first hand its We believe cognitive computing is the third and most
wary of its potential for modern world. potential to transform businesses, transformational phase in computing’s evolution.
(Image 1)
abuse and frustrated by None of this involves either sentience or autonomy governments and society.
its slow development. on the part of machines. Rather, it consists of
We have seen it turn big data from obstacle to
The Tabulating Era (1900s — 1940s)
augmenting the human ability to understand — and The birth of computing consisted of single-purpose
But like so many advanced technologies that were act upon — the complex systems of our society. This opportunity, help physicians make early diagnoses
mechanical systems that counted, using punched
conceived before their time, Artificial Intelligence augmented intelligence is the necessary next step for childhood disease and suggest creative
cards to input and store data, and to eventually
has come to be widely misunderstood —co-opted in our ability to harness technology in the pursuit of solutions for building smarter cities. And we believe
instruct the machine what to do (albeit in a primitive
by Hollywood, mischaracterized by the media, knowledge, to further our expertise and to improve that this technology represents our best — perhaps
way). These tabulation machines were essentially
portrayed as everything from savior to scourge the human condition. That is why it represents not our only — chance to help tackle some of the
calculators that supported the scaling of both business
of humanity. Those of us engaged in serious just a new technology, but the dawn of a new era of most enduring systemic issues facing our planet,
and society, helping us to organize, understand, and
information science and in its application in the technology, business and society: the Cognitive Era. from cancer and climate change to an increasingly
manage everything from population growth to the
real world of business and society understand complex global economy.
advancement of a global economy.
the enormous potential of intelligent systems. The success of cognitive computing will not be
The future of such technology — which we believe measured by Turing tests or a computer’s ability
will be cognitive, not “artificial”— has very different to mimic humans. It will be measured in more
characteristics from those generally attributed practical ways, like return on investment, new market
to AI, spawning different kinds of technological, opportunities, diseases cured and lives saved.
scientific and societal challenges and opportunities,
with different requirements for governance, policy
and management.

Cognitive computing refers to systems that learn

at scale, reason with purpose and interact with
humans naturally. Rather than being explicitly
programmed, they learn and reason from their
interactions with us and from their experiences with
their environment. They are made possible by
advances in a number of scientific fields over the
past half-century, and are different in important ways
from the information systems that preceded them. Image 1

The Tabulating Era The Programming Era The Cognitive Era

(1900s – 1940s) (1950s – present) (2011 – )
2 3
 The Programming Era (1950s — present) Preliminary analyses indicate that the symbiotic realized by combining the data analytics and “Teams of human plus machine dominated even the
The shift from mechanical tabulators to electronic partnership will perform intellectual operations much statistical reasoning of machines with uniquely strongest computers. Human strategic guidance
systems began during World War II, driven by more effectively than man alone can perform them.” 1 human qualities, such as self-directed goals, combined with the tactical acuity of a computer was
military and scientific needs. Following the war, common sense and ethical values. overwhelming. We [people] could concentrate on
digital “computers” evolved rapidly and moved into — J.C.R. Licklider, “Man-Computer Symbiosis,” strategic planning instead of spending so much time
businesses and governments. They performed March 1960 This is what Watson was built to do, and is in fact on calculations. Human creativity was even more
if/then logical operations and loops, with instructions already doing. Banks are analyzing customer paramount under these conditions.” 4
Licklider knew that cognitive computing would be requests and financial data to surface insights to
coded in software. Originally built around vacuum a necessary and natural evolution of programmable
tubes, they were given a huge boost by the invention help them make investment recommendations. — Garry Kasparov
computing, even if he didn’t yet know how it would Companies in heavily regulated industries are
of the transistor and the microprocessor, which be accomplished. Fifty years later, massively
came to demonstrate “Moore’s Law,” doubling in querying the system to keep up with ever-changing
parallel computing and the accumulation of oceans legislation and standards of compliance. And
capacity and speed every 18 months for six
decades. Everything we now know as a computing
of structured and unstructured data would lay the
groundwork for cognitive computing.
oncologists are testing ways in which cognitive
systems can help interpret cancer patients’ clinical
The technical path
device — from the mainframe to the personal
computer, to the smartphone and tablet — is a information and identify individualized, evidence- forward and the science
based treatment options that leverage specialists’
programmable computer.
experience and research. of what’s possible
The Cognitive Era (2011 — ) The world’s first What is the experience of this like for the
The potential for something beyond programmable
systems was foreseen as far back as 1960, when cognitive system professionals involved? World-renowned oncologist While Licklider helped inform a philosophical
Dr. Larry Norton of the Memorial Sloan Kettering approach to cognitive computing, there was little he
computing pioneer J.C.R. Licklider wrote his
Cancer Center, which is working with Watson could do to articulate the technical path forward.
seminal paper “Man-Computer Symbiosis.” Much
In February 2011, the world was introduced to to help physicians personalize cancer treatments, That path is still being defined, constantly adjusting
of modern computing is based on Licklider’s
Watson, IBM’s cognitive computing system that says, “Computer science is going to evolve rapidly, as the world outside our computer labs evolves.
research and insights:
defeated Ken Jennings and Brad Rutter at Jeopardy!. and medicine will evolve with it. This is coevolution. In particular, we are keenly aware of how data is
“Man-computer symbiosis is an expected It was the first widely seen demonstration of We’ll help each other. I envision situations where shaping the future. Gartner estimates that the
development in cooperative interaction between cognitive computing, and it marked the end of the myself [sic], the patient, the computer, my nurse world’s information will grow by 800 percent in the
men and electronic computers. It will involve so-called AI winter. The programmable systems and my graduate fellow are all in the examination next five years, and that 80 percent of that data
very close coupling between the human and the that had revolutionized life over the previous six room interacting with one another.” 2 will be unstructured. It includes everything humans
electronic members of the partnership. The main decades could not have made sense of the messy, have recorded in language (from textbooks to
aims are: unstructured data required to play Jeopardy!. We first saw hints of this symbiosis after Watson’s poems), captured in images (CAT scans to family
Watson’s ability to answer subtle, complex, chess-playing predecessor, Deep Blue, defeated photos) and recorded in sounds. It is the data
1. to let computers facilitate formulative thinking pun-laden questions made clear that a new era World Chess Champion Garry Kasparov in 1997. hidden in aromas, tastes, textures and vibrations.
as they now facilitate the solution of formulated of computing was at hand. Following that demonstration, Kasparov would go It comes from our own activities, and from a planet
problems, and on to participate in new “freestyle” chess leagues, being pervasively instrumented.
Since Jeopardy!, Watson has tackled increasingly in which players are free to use any computer
2. to enable men and computers to cooperate complex data sets and developed understanding, programs they like. In these leagues, some players
in making decisions and controlling complex reasoning and learning that go far beyond deciphering. compete unassisted. Some rely entirely on computer
situations without inflexible dependence on Indeed, the goal of cognitive computing is to illuminate programs. But those players who combine computer
predetermined programs… aspects of our world that were previously invisible — input with their intuition and competitive instincts
patterns and insight in unstructured data, in are the most successful. 3
particular — allowing us to make more informed
decisions about more consequential matters.
The true potential of the Cognitive Era will be

4 5
 In a global economy and society where value As a result, Watson’s confidence level was incredibly When Watson played Jeopardy!, it did one thing —  history, loyalty program patterns, electronic
increasingly comes from information, knowledge low: 14 percent. If this had been a regular Jeopardy! natural language Q&A, based on five technologies. medical records and data from wearables — 
and services, this data represents the most clue, where contestants must buzz in, rather than Today, Q&A is only one of many Watson capabilities and add to that picture details that have been
abundant, valuable and complex raw material in one in “Final Jeopardy,” that confidence level would available as an application programming interface. difficult or impossible to detect: tone, sentiment,
the world. And until now, we have not had the have caused Watson not to press its buzzer. Watson Since then, we have developed more than two emotional state, environmental conditions and the
means to mine it effectively. knew what it didn’t know as indicated by the five dozen new APIs, powered by 50 different cognitive strength and nature of a person’s relationships.
question marks shown in Image 2. technologies. This is a critical distinction between the They reason through the sum total of all this
Programmable systems are based on rules that technical approach to cognitive computing and other structured and unstructured data to find what really
shepherd data through a series of predetermined current approaches to Artificial Intelligence. Cognitive matters in engaging a person. By continuously
processes to arrive at outcomes. While they are computing is not a single discipline of computer learning, these engagements deliver greater
powerful and complex, they are deterministic — ­ science. It is the combination of multiple academic and greater value, and become more natural,
thriving on structured data, but incapable of fields, from hardware architecture to algorithmic anticipatory and emotionally appropriate.
processing qualitative or unpredictable input. strategy to process design to industry expertise.
This rigidity limits their usefulness in addressing 2. They scale and elevate expertise:
many aspects of a complex, emergent world, Many products and services Every industry’s and profession’s knowledge is
where ambiguity and uncertainty abound. expanding at a rate faster than any professional
that we use every day — from can keep up with — journals, new protocols, new
Cognitive systems are probabilistic, meaning search-engine advertising legislation, new practices and entirely new fields.
they are designed to adapt and make sense of the
complexity and unpredictability of unstructured
applications to facial recognition A clear example is found in healthcare, where it
is estimated that in 1950, it took 50 years to
information. They can “read” text, “see” images on social media sites to double the world’s medical knowledge; by 1980,
and “hear” natural speech. And they interpret that “smart” cars, phones and seven years; and in 2015, less than three years.
information, organize it and offer explanations of
what it means, along with the rationale for their
electric grids — are beginning Meanwhile, each person will generate one million
gigabytes of health-related data in his or her
conclusions. They do not offer definitive answers. to demonstrate aspects of lifetime, the equivalent of about 300 million books.
In fact, they do not “know” the answer. Rather, Artificial Intelligence.
they are designed to weigh information and ideas Cognitive systems are designed to help
from multiple sources, to reason, and then offer Most consist of purpose-built, narrowly focused organizations keep pace, serving as a companion
Image 2
hypotheses for consideration. A cognitive system applications, specific to a particular service. They for professionals to enhance their performance.
assigns a confidence level to each potential insight use a few of the core capabilities of cognitive Because these systems master the language of
or answer. However, cognitive systems computing. Some use text mining. Others use image professions — the language of medicine, or sales,
A mistake Watson made during Jeopardy! illustrates
can learn from their mistakes. recognition with machine learning. Most are limited or cuisine — they can both understand and
to the application for which they were conceived.
this. At the end of the first day of competition, Large-scale machine learning teach complex expertise. This reduces the time
required for a professional to become an expert.
the category for “Final Jeopardy” was “US Cities,” is the process by which Cognitive systems, in contrast, combine five
In addition, because these systems are taught
and the clue was, “Its largest airport was named core capabilities:
for a World War II hero; its second largest, for a
cognitive systems improve by leading practitioners — whether in customer
World War II battle.” The answer is Chicago (O’Hare with training and use. 1. They create deeper human engagement: service, oncology diagnosis, case law or any other
Cognitive systems create more fully human field — they make available to broad populations
and Midway). Watson guessed, “What is Toronto?????”
After ingesting a corpus of knowledge, curated by interactions with people — based on the mode, the know-how of the best.
There are many reasons why Watson was confused
experts on any given subject, cognitive systems are form and quality each person prefers. They take
by this question, including its grammatical structure,
trained by being fed a series of question-and-answer advantage of the data available today to create
the presence of a city in Illinois named Toronto
pairs. This machine “knowledge” is then enhanced a fine-grained picture of individuals — such as
and the Toronto Blue Jays playing baseball in the
as humans interact with the system, providing geolocation data, web interactions, transaction
American League.
feedback on the accuracy of the system’s responses.

6 7
 3. They infuse products and services
with cognition:
If cognitive computing is to fulfill its true promise,
the underlying platform must be broad and flexible
schools can create longitudinal student records
and personalized education plans.
Implications and
Cognition enables new classes of products and enough to be applied by any company in any
In one of the most promising near-term applications
obligations for
services to sense, reason and learn about their industry. And it must be able to be applied across
users and the world around them. This allows for industries. To do that requires a holistic approach to of cognitive computing, IBM is working with more the advance of
than a dozen leading cancer institutes to accelerate
continuous improvement and adaptation, and for
augmentation of their capabilities to deliver uses
research and development, with the goal of creating
a robust platform with a range of capabilities to the ability of clinicians to identify and personalize cognitive science
not previously imagined. We see this happening support diverse applications from an ecosystem treatment options for patients. The program seeks
already with cars, medical devices, appliances and of developers. to reduce from weeks to minutes the time it takes to
The Cognitive Era is the next step in the application
even toys. The Internet of Things is dramatically translate DNA insights, understand an individual’s
of science to understand nature and improve the
expanding the universe of digital products and This platform must encompass machine learning, genetic profile and gather relevant information from
human condition. In that sense, it is a new chapter of
services — and where code and data go, reasoning, natural language processing, speech medical literature. The resulting analysis allows
a familiar story, and the controversy surrounding
cognition can now follow. and vision, human-computer interaction, dialog and doctors to target the specific cancer-causing
Artificial Intelligence is merely the latest example of
narrative generation and more. Many of these genetic mutations in each patient. Watson completes
the age-old debate between those who believe in
4. They enable cognitive processes capabilities require specialized infrastructure that the genetic material and medical literature review
progress and those who fear it. Within the scientific
and operations: leverages high-performance computing, specialized process in only a few minutes, producing a report
community — as opposed to the media and popular
Cognition also transforms how a company hardware architectures and even new computing and data visualization of the patient’s case and
entertainment — the verdict is in. There is broad
operates. Business processes infused with paradigms. Each grows from its own scientific or evidence-based insights on potential drugs that may
agreement on the importance of pursuing a cognitive
cognitive capabilities capitalize on the academic field. But these technologies must be be relevant to an individual patient’s unique DNA
future, along with recognition of the need to develop
phenomenon of data, from internal and external developed in concert, with hardware, software, cloud profile. The clinician can then evaluate the evidence
the technology responsibly.
sources. This gives them heightened awareness platforms and applications that are built expressly to determine whether a targeted therapy may be
of workflows, context and environment, leading to work together in support of cognitive solutions. more effective than standard care for the patient. “Technology creates possibilities and potential,
to continuous learning, better forecasting and but ultimately, the future we get will depend on the
increased operational effectiveness — along with With Watson’s rapid evolution, we are getting a
choices we make. Technology is not destiny.
decision-making at the speed of today’s data. glimpse of what is possible. One example is a
We shape our destiny.” 5
This is good news in a world where, for example, cognitive medical imaging application that analyzes
an average billion-dollar company spends X-rays, MRIs and ultrasound images. It processes — Erik Brynjolfsson, MIT
almost 1,000 person hours per week managing the natural language of medical journals, textbooks
its suppliers. and articles. It uses machine learning to correct and
improve its understanding. And it develops deep Specifically, we must continue to shape the effect of
5. They enhance exploration and discovery: knowledge representations and reasoning that can cognitive computing on work and employment. Like
Ultimately, the most powerful tool that cognitive help surface possible diagnoses. To do this requires all technology, cognitive computing will change the
businesses will possess is far better “headlights” specialized graphic processing units to support nature of work done by people. It will help us perform
into an increasingly volatile and complex future. the large scale of data and human domain expertise some tasks faster and more accurately. It will make
Such headlights are becoming more important to guide its learning and interpret its results. many processes cheaper and more efficient. It will
as leaders in all industries are compelled to place also do some things better than humans, which has
big bets — on drug development, on complex The power of this new model can be applied to any been the case since the dawn of civilization. What
financial modeling, on materials science innovation, domain. Oil and gas companies can combine has always happened is that higher value is found in
on launching a startup. By applying cognitive seismic imaging data with analyses of hundreds of new skills, and humans and our institutions adapt
technologies to vast amounts of data, leaders thousands of papers and reports, current events, and evolve. There is no reason to believe it will be
can uncover patterns, opportunities and economic data and weather forecasts to create risk different this time. Indeed, given the exponential
actionable hypotheses that would be virtually and reward scenarios for exploratory drilling. Or, by growth in knowledge, discovery and opportunity
impossible to discover using traditional research analyzing test scores, attendance and information opened up by a Cognitive Era, there is every reason
or programmable systems alone. about student behavior on digital learning platforms,

8 9
 population wants to buy a product; not knowing
where to find critical natural resources; not knowing
where the risks lie in every investor’s portfolio.

“The greatest bar to wise action and the greatest

source of fear is ignorance. A tiny candle gives
misleading light and throws huge and ominous
shadows. The sun at noon gives great light
and throws no shadows. It is time to get this whole
problem of men and machines under a blazing
noonday beam. Computers will never rob man
of his initiative or replace the need for his creative
thinking. By freeing man from the more menial
or repetitive forms of thinking, computers will
actually increase the opportunities for the full use
of human reason.” 6

— Thomas Watson Jr., former Chairman and CEO

to believe that the work of humans will become

ever-more interesting, challenging and valuable.
Paving the way for At IBM, we believe that many of the perennial
problems of the world can be solved — and we

Equally important is the need for societal controls

the next generation believe that cognitive computing is the tool that will
help us accomplish this ambitious goal.
and safeguards. Again, such concerns are not of human cognition
unique to intelligent systems. Questions about The much-hyped drama of “man vs. machine”
security — both individual and institutional — attach is a distraction from that, and it rests on an exciting
themselves to every transformational technology, In the end, all technology but misguided fiction. Cognitive systems are not
from automobiles to pharmaceuticals to mobile our competitor, nor will they become so. Neither the
phones. These issues are already urgent, and
revolutions are propelled not science nor the economics support such fears.
will remain so as cognitive technologies develop. just by discovery, but also Cognitive systems, as they actually exist, are a tool
They are fueled especially by today’s radical by business and societal to deepen the relationship that really matters — the
democratization of technology — driven by the relationship between humans and the world.
rapid spread of networks and the cloud, and
need. We pursue these new
the accompanying reduction in costs. possibilities not because we In so doing, we will pave the way for the next
generation of human cognition, in which we think
We believe that the answer lies not in attempting
can, but because we must. and reason in new and powerful ways. It’s true that
to limit that democratization, but rather in embracing As with every revolutionary technology, our initial cognitive systems are machines that are inspired
it while designing cognitive systems with privacy, understanding will be limited — both by the world’s by the human brain. But it’s also true that these
security and human control integrated into their fabric. complexity and by our own deeply ingrained biases machines will inspire the human brain, increase our
and heuristics. However, for all these limitations, capacity for reason and rewire the ways in which
progress is imperative. Indeed, we pay a significant we learn. In the 21st century, knowing all the answers
price for not knowing: not knowing what’s wrong won’t distinguish someone’s intelligence — rather,
with a patient; not knowing which part of the the ability to ask better questions will be the mark
of true genius.

10 11
1. Licklider, J.C.R. Man-Computer Symbiosis. IRE Transactions on © Copyright IBM Corporation 2015
Human Factors in Electronics. 1960. http://groups.csail.mit.edu/
medg/people/psz/Licklider.html IBM Global Services
Route 100
2. Kelly, John E. and Steve Hamm. Smart Machines: IBM’s Watson
and the Era of Cognitive Computing. New York. Columbia Somers, NY 10589
University Press, 2014. USA.

3. Kelly, Kevin. The Three Breakthroughs That Have Finally Produced in the United States of America
Unleashed AI on the World. Wired Magazine. October, 2014. October 2015
4. McAfee, Andrew. Did Garry Kasparov Stumble into a New All Rights Reserved
Business Process Model? Harvard Business Review. February,
2010. https://hbr.org/2010/02/like-a-lot-of-people/ IBM, the IBM logo and ibm.com are trademarks or registered
trademarks of International Business Machines Corporation in
5. Brynjolfsson, Erik and Andrew McAfee. The Second Machine the United States, other countries, or both. If these and other IBM
Age: Work, progress, and prosperity in a time of brilliant
trademarked terms are marked on their first occurrence in this
machines. New York. W.W. Norton & Company, 2014.
information with a trademark symbol (® or ™), these symbols
6. Watson, Thomas Jr. IBM Archives. https://www-03.ibm.com/ indicate US registered or common law trademarks owned by IBM
ibm/history/exhibits/watsonjr/watsonjr_quoted.html at the time this information was published. Such trademarks may
also be registered or common law trademarks in other countries.
A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at ibm.com/legal/
copytrade.shtml Other company, product and service names
may be trademarks or service marks of others.

References in this publication to IBM products and services do

not imply that IBM intends to make them available in all countries
in which IBM operates.

Please Recycle