SlideShare a Scribd company logo
Lessons Learned
Original Statement
The U.S. needs a way, given
a representative simulation, to
rapidly explore a strategy for
possible novel uses of
existing platforms and
weapons.
David Song
B.S. CS
Yujing Zhang
M.A. East Asian Studies
Shashank Rammoorthy
M.S. CS
Jonathan Sepulveda
M.S. Materials Science
Project Wargames
30+
interviews
Current Statement
Strategic wargames stand to
benefit from a stronger integration
of AI+ML but are struggling to find
adoption and usage. How can this
be addressed?
1
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
Exploring the Problem
2
Lessons Learned 3
Lessons Learned
But what is a wargame?
4
1. Physics-based simulators
2. TTXs for campaigns
3. High-level diplomacy wargames
Lessons Learned
Then we played with wargames and talked to experts...
5
Lessons Learned 6
So we went ‘back to the drawing board’...
Picking one case study of interest, we
focused on the Cross-Straits scenario…
● We visited talks on Taiwan’s
Economic Security to understand
variables often overlooked
● The sad reality is, wargame players
often have very little time to
anticipate or prepare for wargames,
and are not concerned with non-
combatant variables
● The most important element of
wargames ought to be to teach
people to think creatively
Lessons Learned
We realized wargaming was not easy
Then we started learning and asking…
● What are CoAs?
● Differences between wargaming vs.
M&S?
● Why does the DoD want innovation
within such traditional processes?
● What hasn’t been attempted
already?
7
Excerpt from Battle! Practical Wargaming by Charles Grant
Lessons Learned
Problems we found with wargaming
8
We immediately found out that there are…
● Many moving parts & elements:
researchers → designers →
wargamers → Red and Blue teams
● Little data is shared amongst
designers and team members
● There are multiple types of
wargames
● Most games remain TTXs
● Too difficult to build de novo a
robust, AI-capable Red adversary
That left us thinking:
● Virtual wargames are often used for
training, but seldom as formal
proceedings; why?
● Should we seek to replace
wargamers altogether with AI+ML
‘magic’?
Lessons Learned
What some of our interviewees told us…
9
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"There are a lot of reasons for
why AI/ML hasn't been used in
wargaming in the DoD…technical
illiteracy within the DoD…policies
are inflexible, and systems are
difficult to change"
Lessons Learned
What some of our interviewees told us…
10
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"We should never use new
wargames, but rather well-studied
wargames and already-studied
ones in predicting our outcomes &
enhancing our adjudications”
Lessons Learned
What some of our interviewees told us…
11
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"Wargames need data to eat from.
You need a volume of data that
AI+ML can learn from and be helpful;
where is that data coming from a
wargame? Wargames don’t generate
simulation data in copious amounts”
Lessons Learned
What some of our interviewees told us…
12
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"We put too much emphasis on the
Red...players are generally not from
Russia or China, and so you struggle to
extrapolate specific country foreign
policy decisions versus general human
reactions to international crises.”
Many different objections to
AI+ML!
Lessons Learned
We got out of the building, went to an NPS Conference…
13
● Existing AI+ML capabilities have to be
better leveraged
● AI+ML does not need to make drastic
technical advances to be feasible for
wargaming uses
● There’s no readily accessible central
repository of past wargame data
outcomes to help future teams
● Ideally, forecast decision spaces as
opposed to having AI+ML play moves
Lessons Learned
It’s really hard to build Alphastar
with the simulations so far
14
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
15
Pivot: Let’s help humans
play wargames better with AI
tools instead of replacing
humans completely with AI
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
16
Identifying
Recommendations
Lessons Learned 17
1. Digitize wargames to increase iterations
"Accelerating and making
wargames more efficient will
come down to a series of
technologies that could very
well be unclassified in
development”
Major defense startup
company
Lessons Learned 18
2. Create AI tools for digitized wargames
“Not many people in the U.S.
have access to the decision-
making teams you find in Red
teams [the PLA and the CCP].”
Elizabeth Bartels | RAND
Lessons Learned 19
3. Implement organizational changes
We read-up on traditional approaches to
building wargames, noting its strengths..
● We found more support that cross-talk
between wargamers & designers and
players is important
● Academics do not differ that sharply
from the assessment of wargame players
insofar as AI+ML integration is concerned
● Information asymmetry predominates
within wargames–not an attribute of UCC
commanders
● Need more collaborations among
wargamers, designers, combat &
command
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
20
Next steps
1. Validate our recommendations
2. Shadow a wargame
3. Continue working with our sponsor

More Related Content

PDF
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
PDF
Antifragile
PPTX
Zero to One - Book Summary Report
PPTX
Why transformation efforts fail
PDF
Defining the Agile Mindset
PPT
Power Point Karaoke Slides - Love Relationships
PPT
Resilience: The Key to Leadership Success
PDF
Agile innovation and Thinking Like a Startup
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
Antifragile
Zero to One - Book Summary Report
Why transformation efforts fail
Defining the Agile Mindset
Power Point Karaoke Slides - Love Relationships
Resilience: The Key to Leadership Success
Agile innovation and Thinking Like a Startup

Similar to Team Wargames - 2022 Technology, Innovation & Great Power Competition (20)

PDF
Artificial Intelligence - Opportunities and Challenges for Military Modeling ...
PPTX
Military simulator a case study
PPTX
Military simulator a case study
PPTX
Introduction To My Graduation Project
PPTX
1st Seminar- Intelligent Agent for Medium-Level Artificial Intelligence in Re...
PPTX
Introduction To My Graduation Project
PDF
Researchers eye embedding artificial intelligence (AI) into war games simulat...
PDF
Artificial Intelligence in the Military: An Overview of the Capabilities, App...
PPT
LearningKit.ppt
PDF
Why AI Is Shaping our games - Johanna Pirker, 2019
PDF
Maze Base
PDF
Why AI is shaping our games
PDF
This was a triumph: Evolving intelligent bots for videogames. And for Science.
PPTX
Timeline Presentation
PPTX
First Seminar Presentation
PDF
Dissertation defense
PPT
Project on ai gaming
PDF
Simulation & Training Perspectives for the Advancement of Armoured Vehicle We...
PPTX
Algorithms of Armageddon: What Happens When We Insert AI Into Our Military ...
PPTX
Matrix Game Simulation Presentation - VCOW 2025 v1.1.pptx
Artificial Intelligence - Opportunities and Challenges for Military Modeling ...
Military simulator a case study
Military simulator a case study
Introduction To My Graduation Project
1st Seminar- Intelligent Agent for Medium-Level Artificial Intelligence in Re...
Introduction To My Graduation Project
Researchers eye embedding artificial intelligence (AI) into war games simulat...
Artificial Intelligence in the Military: An Overview of the Capabilities, App...
LearningKit.ppt
Why AI Is Shaping our games - Johanna Pirker, 2019
Maze Base
Why AI is shaping our games
This was a triumph: Evolving intelligent bots for videogames. And for Science.
Timeline Presentation
First Seminar Presentation
Dissertation defense
Project on ai gaming
Simulation & Training Perspectives for the Advancement of Armoured Vehicle We...
Algorithms of Armageddon: What Happens When We Insert AI Into Our Military ...
Matrix Game Simulation Presentation - VCOW 2025 v1.1.pptx
Ad

More from Stanford University (20)

PPTX
Team Networks - 2022 Technology, Innovation & Great Power Competition
PPTX
Team LiOn Batteries - 2022 Technology, Innovation & Great Power Competition
PPTX
Team Quantum - 2022 Technology, Innovation & Great Power Competition
PPTX
Team Disinformation - 2022 Technology, Innovation & Great Power Competition
PPTX
Team Acquistion - 2022 Technology, Innovation & Great Power Competition
PPTX
Team Climate Change - 2022 Technology, Innovation & Great Power Competition
PPTX
Altuna Engr245 2022 Lessons Learned
PPTX
Invisa Engr245 2022 Lessons Learned
PPTX
ānanda Engr245 2022 Lessons Learned
PPTX
Gordian Knot Center Roundtable w/Depty SecDef
PDF
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
PDF
Team Catena - 2021 Technology, Innovation & Great Power Competition
PPTX
Team Apollo - 2021 Technology, Innovation & Great Power Competition
PPTX
Team Drone - 2021 Technology, Innovation & Great Power Competition
PPTX
Team Short Circuit - 2021 Technology, Innovation & Great Power Competition
PPTX
Team Aurora - 2021 Technology, Innovation & Great Power Competition
PPTX
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...
PPTX
Lecture 8 - Technology, Innovation and Great Power Competition - Cyber
PPTX
Lecture 7 - Technology, Innovation and Great Power Competition - Space
PPTX
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...
Team Networks - 2022 Technology, Innovation & Great Power Competition
Team LiOn Batteries - 2022 Technology, Innovation & Great Power Competition
Team Quantum - 2022 Technology, Innovation & Great Power Competition
Team Disinformation - 2022 Technology, Innovation & Great Power Competition
Team Acquistion - 2022 Technology, Innovation & Great Power Competition
Team Climate Change - 2022 Technology, Innovation & Great Power Competition
Altuna Engr245 2022 Lessons Learned
Invisa Engr245 2022 Lessons Learned
ānanda Engr245 2022 Lessons Learned
Gordian Knot Center Roundtable w/Depty SecDef
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
Team Catena - 2021 Technology, Innovation & Great Power Competition
Team Apollo - 2021 Technology, Innovation & Great Power Competition
Team Drone - 2021 Technology, Innovation & Great Power Competition
Team Short Circuit - 2021 Technology, Innovation & Great Power Competition
Team Aurora - 2021 Technology, Innovation & Great Power Competition
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...
Lecture 8 - Technology, Innovation and Great Power Competition - Cyber
Lecture 7 - Technology, Innovation and Great Power Competition - Space
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...
Ad

Recently uploaded (20)

PDF
Complications of Minimal Access Surgery at WLH
PDF
Supply Chain Operations Speaking Notes -ICLT Program
PDF
1_English_Language_Set_2.pdf probationary
PDF
advance database management system book.pdf
PDF
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
PDF
LDMMIA Reiki Yoga Finals Review Spring Summer
PDF
Computing-Curriculum for Schools in Ghana
PPTX
Chinmaya Tiranga Azadi Quiz (Class 7-8 )
PDF
A systematic review of self-coping strategies used by university students to ...
PPTX
Orientation - ARALprogram of Deped to the Parents.pptx
PPTX
A powerpoint presentation on the Revised K-10 Science Shaping Paper
PDF
Weekly quiz Compilation Jan -July 25.pdf
PPTX
Cell Types and Its function , kingdom of life
PPTX
Final Presentation General Medicine 03-08-2024.pptx
PDF
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
PDF
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
PDF
Trump Administration's workforce development strategy
PDF
SOIL: Factor, Horizon, Process, Classification, Degradation, Conservation
PPTX
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
PPTX
Digestion and Absorption of Carbohydrates, Proteina and Fats
Complications of Minimal Access Surgery at WLH
Supply Chain Operations Speaking Notes -ICLT Program
1_English_Language_Set_2.pdf probationary
advance database management system book.pdf
GENETICS IN BIOLOGY IN SECONDARY LEVEL FORM 3
LDMMIA Reiki Yoga Finals Review Spring Summer
Computing-Curriculum for Schools in Ghana
Chinmaya Tiranga Azadi Quiz (Class 7-8 )
A systematic review of self-coping strategies used by university students to ...
Orientation - ARALprogram of Deped to the Parents.pptx
A powerpoint presentation on the Revised K-10 Science Shaping Paper
Weekly quiz Compilation Jan -July 25.pdf
Cell Types and Its function , kingdom of life
Final Presentation General Medicine 03-08-2024.pptx
OBE - B.A.(HON'S) IN INTERIOR ARCHITECTURE -Ar.MOHIUDDIN.pdf
ChatGPT for Dummies - Pam Baker Ccesa007.pdf
Trump Administration's workforce development strategy
SOIL: Factor, Horizon, Process, Classification, Degradation, Conservation
Tissue processing ( HISTOPATHOLOGICAL TECHNIQUE
Digestion and Absorption of Carbohydrates, Proteina and Fats

Team Wargames - 2022 Technology, Innovation & Great Power Competition

  • 1. Lessons Learned Original Statement The U.S. needs a way, given a representative simulation, to rapidly explore a strategy for possible novel uses of existing platforms and weapons. David Song B.S. CS Yujing Zhang M.A. East Asian Studies Shashank Rammoorthy M.S. CS Jonathan Sepulveda M.S. Materials Science Project Wargames 30+ interviews Current Statement Strategic wargames stand to benefit from a stronger integration of AI+ML but are struggling to find adoption and usage. How can this be addressed? 1
  • 2. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future Exploring the Problem 2
  • 4. Lessons Learned But what is a wargame? 4 1. Physics-based simulators 2. TTXs for campaigns 3. High-level diplomacy wargames
  • 5. Lessons Learned Then we played with wargames and talked to experts... 5
  • 6. Lessons Learned 6 So we went ‘back to the drawing board’... Picking one case study of interest, we focused on the Cross-Straits scenario… ● We visited talks on Taiwan’s Economic Security to understand variables often overlooked ● The sad reality is, wargame players often have very little time to anticipate or prepare for wargames, and are not concerned with non- combatant variables ● The most important element of wargames ought to be to teach people to think creatively
  • 7. Lessons Learned We realized wargaming was not easy Then we started learning and asking… ● What are CoAs? ● Differences between wargaming vs. M&S? ● Why does the DoD want innovation within such traditional processes? ● What hasn’t been attempted already? 7 Excerpt from Battle! Practical Wargaming by Charles Grant
  • 8. Lessons Learned Problems we found with wargaming 8 We immediately found out that there are… ● Many moving parts & elements: researchers → designers → wargamers → Red and Blue teams ● Little data is shared amongst designers and team members ● There are multiple types of wargames ● Most games remain TTXs ● Too difficult to build de novo a robust, AI-capable Red adversary That left us thinking: ● Virtual wargames are often used for training, but seldom as formal proceedings; why? ● Should we seek to replace wargamers altogether with AI+ML ‘magic’?
  • 9. Lessons Learned What some of our interviewees told us… 9 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "There are a lot of reasons for why AI/ML hasn't been used in wargaming in the DoD…technical illiteracy within the DoD…policies are inflexible, and systems are difficult to change"
  • 10. Lessons Learned What some of our interviewees told us… 10 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "We should never use new wargames, but rather well-studied wargames and already-studied ones in predicting our outcomes & enhancing our adjudications”
  • 11. Lessons Learned What some of our interviewees told us… 11 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "Wargames need data to eat from. You need a volume of data that AI+ML can learn from and be helpful; where is that data coming from a wargame? Wargames don’t generate simulation data in copious amounts”
  • 12. Lessons Learned What some of our interviewees told us… 12 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "We put too much emphasis on the Red...players are generally not from Russia or China, and so you struggle to extrapolate specific country foreign policy decisions versus general human reactions to international crises.” Many different objections to AI+ML!
  • 13. Lessons Learned We got out of the building, went to an NPS Conference… 13 ● Existing AI+ML capabilities have to be better leveraged ● AI+ML does not need to make drastic technical advances to be feasible for wargaming uses ● There’s no readily accessible central repository of past wargame data outcomes to help future teams ● Ideally, forecast decision spaces as opposed to having AI+ML play moves
  • 14. Lessons Learned It’s really hard to build Alphastar with the simulations so far 14
  • 15. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future 15 Pivot: Let’s help humans play wargames better with AI tools instead of replacing humans completely with AI
  • 16. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future 16 Identifying Recommendations
  • 17. Lessons Learned 17 1. Digitize wargames to increase iterations "Accelerating and making wargames more efficient will come down to a series of technologies that could very well be unclassified in development” Major defense startup company
  • 18. Lessons Learned 18 2. Create AI tools for digitized wargames “Not many people in the U.S. have access to the decision- making teams you find in Red teams [the PLA and the CCP].” Elizabeth Bartels | RAND
  • 19. Lessons Learned 19 3. Implement organizational changes We read-up on traditional approaches to building wargames, noting its strengths.. ● We found more support that cross-talk between wargamers & designers and players is important ● Academics do not differ that sharply from the assessment of wargame players insofar as AI+ML integration is concerned ● Information asymmetry predominates within wargames–not an attribute of UCC commanders ● Need more collaborations among wargamers, designers, combat & command
  • 20. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future 20 Next steps 1. Validate our recommendations 2. Shadow a wargame 3. Continue working with our sponsor