Artificial Intelligence

( and Stupidity )
in the Computer Game Industry

Lars Lidén
[email protected]

2001
 Valve
Seattle, WA (2000)

Presto Studios
San Diego, CA (1998)
Introduction
• The term A.I. is used broadly in computer game industry
• Games come in many genres and the techniques
employed by the different genres vary widely
• Will be discussing A.I. for non-player characters (NPCs)
in a first-person or third-person shooter
• Fully rendered 3D environment
• Player controls a single character
Introduction
Won’t be discussing use of AI for:
• Board games (Chess, Go)
• Strategy games (Civilization / Age of Empires)
• Camera control in 3D games
• Simulation games (The Sims, Sim City)
• Racing games
Example:

Half-Life 2
Components of an AI System
Decision Making
Tactical Analysis
Artificial Stupidity
Components of an AI System
Decision Making
Tactical Analysis
Artificial Stupidity
Components of an AI System
 Animation

 Movement

 Behavior
Components of an AI System
 Animation

Responsible for controlling NPC body

• Movement

• Behavior
AI Components: Animation
• Animation Selection
• Parameterize Animation
• Selected Joint Control
• Inverse Kinematics
AI Components: Animation
• NPC models built by artists
Use tools such as “3D Studio Max” or “Maya”

• Models are constructed from bones

• Bones are connected by articulated joints

• The skeletal system is covered by a mesh of textured

polygons
AI Components: Animation
Example:
Texturing and Lighting a Mesh
Bones and Articulators
AI Components: Animation
Animation sequences are generated by defining how
joints should articulate through time

Walking sequence:
AI Components: Animation
Animation sequences for a model are either:

• Hand generated by a computer animator

• Recorded from real human (or animal) movements

and applied to a skeletal system (“motion capture”)
AI Components: Animation
Motion Capture:
Tom Molet (EGCAS ’96)
AI Components: Animation
Animation sequences tend to be:

Motion primitives:
• Run, Walk, Jump, Side-step, Climb

Transitions
• Start_Walk, Run_To_Jump, Jump_Land
AI Components: Animation
Some animation sequences only take control of
part of the body:

• wave_hello
• hand_signal_stop
• swing_ice_axe
AI Components: Animation
First step in A.I. Is to select which animation
sequence or sequences should be applied to a model

Many influences:
• Desired behavior chosen by decision system
• What animation is currently playing
• The current velocity and direction of the NPC
• The terrain the NPC is standing on
AI Components: Animation
Second step is to parameterize animations
Speed up or slow down animation
• Slow walk, fast walk
• Accelerate / decelerate stop and start of run
• Slow run as approach sharp turn
Blend between animations
• walk-to-run
• 70% normal walk + 30% limp
Layer animations
• Mix hand_wave on top of walk animation
AI Components: Animation
Next might add selected Joint Control
Take control of particular joints
• Either:
• Ignore joint motion in pre-generated animation
• Blend with pre-generated joint motion
Used for:
• Head Turning
• Looking at a particular object or location
• Arm aiming
• Point gun at a location
AI Components: Animation
And finally, add inverse kinematics
• Algorithmically determine the joint configuration
required for an end-effecter (hand or foot) to reach a
particular location
• Used for:
• Keep the feet on the ground on uneven terrain or when
walking up stairs
• Reaching hand out to open a door, pick up and object
AI Components: Animation
• Animation Selection

• Parameterization of Animations

• Selected Joint Control

• Inverse Kinematics
Components of an AI System
• Animation
Responsible for controlling NPC body

 Movement
Responsible for controlling NPC movement

• Behavior
Movement: Pathfinding
• The primary component of a movement system is
pathfinding.
• Find a route for an NPC to travel from one location in the
world to another.
• Route depends on:
• NPC’s size
• Will NPC’s body fit in a given location?
• NPC’s navigation ability
• Walk, Jump, Climb, Swim
Movement: Pathfinding Tools
• Waypoint
• Position in a map that is used for navigation
• Usually placed in world manually by a level designer
• Link
• Connection between two waypoints
• Often annotated with the required navigation type (Jump, Swim, Climb)
• For a given NPC, two waypoints are linked when:
• The NPC has room enough to move from one node to another without
colliding with the world geometry
• The NPC has the required navigation ability
• Node Graph
• Data structure holding all waypoints and links
• Either generated manually by a level designer or automatically by the
computer and annotated by a level designer
Movement: Node Graph
Movement: Route Generation
• A* is the preferred pathfinding algorithm for quickly
finding a short path between two waypoints
• Often modified for other route types:
• Fastest Route
Shortest route is not often the fastest as it may take an NPC longer to
climb a ladder or swim between two waypoints than to run or walk
• Safest Route
For tactical reasons an NPC may choose to prefer a longer less
exposed route than a shorter or faster route with no cover
Components of an AI System
• Animation
Responsible for controlling NPC body

• Movement
Responsible for controlling NPC movement

 Behavior
Responsible for controlling NPC decision making
AI Components: Behavior
Responsible for deciding what behavior an NPC
should express

Each NPC has a think cycle:

1. Sense Environment
2. Select Behavior
3. Express behavior
AI Components: Behavior
1. Sense Environment
• What to I see?
• What can I hear / smell?
• Includes internal states:
• How tired am I?
• What’s my health?

2. Select Behavior
• Finite State Machine
• Decision Tree

3. Express Behavior
• Select appropriate animation sequences
Components of an AI System
Decision Making
Tactical Analysis
Artificial Stupidity
Decision Making:
• Module that does behavior selection

• Example of one such system

• Overview
Many of the details and features have been omitted
Decision Making:
Making
System consists of three types of objects:

1. Tasks

2. Schedules

3. Conditions
Decision Making:
Making
TASKS
Simple things for a NPC to do, such as:
• Turn to face a location or an object
(TASK_FACE_FRIEND)
• Find a path to a location
(TASK_FIND_PATH_TO_ENEMY)
• Move along a path
(TASK_WALK_PATH)
• Stop moving
(TASK_STOP_MOVING)
Decision Making:
Making
Tasks can take arguments:

• Wait for a while

(TASK_WAIT) < Time to wait >

• Play a particular animation

(TASK_PLAY_ANIMATION) < Animation Name >

• Say Something
(TASK_SAY_PHRASE) < Phrase >
Decision Making:
Making
SCHEDULES

• Named lists of tasks

• Selected by logic system
• Performed by NPC until:
• All tasks have finished
• One task has failed
• Schedule has been Interrupted
Decision Making:
Making
Examples:
• SCHEDULE_GET_WEAPON
TASK_FIND_WEAPON
TASK_FIND_PATH
TASK_RUN_PATH
TASK_PICKUP_WEAPON

• SCHEDULE_FLANK_ATTACK
TASK_FIND_FLANK_POSITION
TASK_FIND_PATH
TASK_RUN_PATH
TASK_RANGE_ATTACK
Decision Making:
CONDITONS
Predicates that are set every time an NPC thinks

Conditions serve two purposes:

1. Schedule Selection

2. Schedule Interruption
Decision Making:
Examples:
• See an enemy
(CONDITON_SEE_ENEMY)
• Hear danger
(CONDITON_HEAR_DANGER)
• Took heavy damage
(CONDITION_HEAVY_DAMAGE)
• Enemy blocked by something
(CONDITION_ENEMY_OCCLUDED)
Decision Making: Conditions
Schedule selection is rule based:
• If (CONDITION_HEAR_DANGER) and not (CONDITION_HAVE_WEAPON)
select schedule (SCHEDULE_GET_WEAPON)

• If (CONDITION_HAVE_WEAPON) and (CONDITION_OUT_OF_AMMO)

select schedule (SCHEDULE_RELOAD_WEAPON)
Decision Making: Conditions
Interrupt conditions

• Schedules also contain interrupt conditions

• Same set of predicates used for schedule selection

• If interrupt condition is listed in the schedule, and the

condition is true, the schedule will be interrupted
Decision Making: Conditions
Example:

• SCHEDULE_GET_WEAPON
TASK_FIND_WEAPON
TASK_FIND_PATH
TASK_RUN_PATH
TASK_PICKUP_WEAPON

CONDITION_HEAVY_DAMAGE
CONDITION_ENEMY_DEAD
Decision Making: Conditions
Task Failure
• Sometimes a task can’t be competed

• For example:
• (TASK_FIND_WEAPON) when no weapon available
• (TASK_FIND_PATH) when no way to get to destination

• In such cases the schedule will exit pre-maturely

Decision Making: Conditions
Fail Schedules:
• Can specify a fail schedule:

SCHEDULE_GET_WEAPON
TASK_SET_FAIL_SCHEDULE SCHEDULE_TAKE_COVER
TASK_FIND_WEAPON
TASK_FIND_PATH
TASK_RUN_PATH
TASK_PICKUP_WEAPON

• Take cover schedule chosen when any of the tasks fail

Decision Making:
NPC Think Cycle:
• Update predicate values (conditions)

• If any conditions interrupt the current schedule, select a new schedule

• Perform next task in schedule list

• If task fails, go to fail schedule (if defined) or select a new schedule

• If all tasks have been completed, select a new schedule

Components of an AI System
Decision Making
Tactical Analysis
Artificial Stupidity
Tactical Analysis
• Level designers place waypoints in the environment
for navigation
• Node graph contains node connectivity information
for a map
• Waypoints can also be evaluated for their visibility
• Information can be used to make tactical decisions
Waypoint Analysis

A World With 6 Nodes and 2 Enemies

Waypoint Analysis
• Limited CPU time
• Decisions must be made quickly (as few CPU
cycles as possible)
• Data must stored efficiently
• Store visibility data in a “bit-string” class

Va = visibility from node “a”

Waypoint Analysis

Visibility Matrix
Waypoint Analysis
Danger Nodes
• Determined by “OR”ing the j k
visibility of all enemy’s (k) V  V j
nearest nodes j 0

Safe Nodes
• Is its inverse
V
Waypoint Analysis

Safe and Dangerous Nodes

Waypoint Analysis

Safe and Dangerous Nodes

Finding a Safe Attack Position
While attacking a selected enemy, an NPC
shouldn’t expose itself to other enemies

A good attack position will:

• Provide line-of-site (LOS) to the selected enemy
• Provide cover from all other enemies
Finding a Safe Attack Position
To find such locations, first find all nodes which
have LOS to the selected enemy

Call selected enemy “a”

Va
Finding a Safe Attack Position

Nodes are visible to selected enemy

Finding a Safe Attack Position
Next determine the set of nodes that are visible to
all other enemies

j k
Va  V j , j  a
j 0
Finding a Safe Attack Position

Nodes are visible to other enemies

Finding a Safe Attack Position
The set of good attack positions is the set of nodes
with LOS to the enemy intersected with the inverse
of the set of nodes with LOS to all other enemies

V 'a  Va Va
Finding a Safe Attack Position

Safe nodes to shoot from

Static Waypoint Evaluation
• Unless cheating is employed, NPCs don’t have full
knowledge of the world.
• May not know where all their enemies are located
• Find a good location to wait in for attack
• Not all positions are created equal
Static Waypoint Evaluation
To find a good set up position:
• Establish the exposure of all waypoints in a map

• Process can be done off line, before game is even

started
Static Waypoint Evaluation

Evaluate Each Node For Visibility

Static Waypoint Evaluation
A good location is one which:
1. Has high exposure (visibility)
• Easy to locate enemies
• Easy to establish LOS to attack and enemy

2. Has areas of low exposure nearby

• Can hide easily
• Can run for cover easily
Static Waypoint Evaluation

Good “Sniping” Positions

Pinch Points
Observation of human players reveals that
experienced players anticipate the actions of their
opponents (Laird 2000)
For example, if an enemy enters a room with only a single
exit an experienced player will wait just outside the exit
setting up an ambush

Such “pinch points” can be pre-calculated by

analyzing the node graph
Pinch Points
To find pinch points:

For each node, N in the node graph with only two neighbors:
Temporarily eliminate node, N, from the graph, call its neighbors as A & B.
If both A & B are connected to large regions, N is not a pinch point, try another N.
Attempt to find a path between A& B.
If path exists, N is not a pinch point, try another N.
Call the node connected to the larger region, O (for outside).
Call the node connected to the smaller region, I (for inside).

Let’s do that again step-by-step:

Pinch Points

• For each node, N in the node graph with only two neighbors:
Pinch Points

• Temporarily eliminate node, N, from the graph, call its neighbors as A & B.
Pinch Points

• If both A & B are connected to large regions, N is not a pinch point, try another N.
Pinch Points

• Attempt to find a path between A& B, if exists try another N.

Pinch Points

• Call the node connected to the larger region, O (for outside).

• Call the node connected to the smaller region, I (for inside).
Pinch Points
Once a pinch point has been located a good
ambush location is one which:

• Has a line of site to the waypoint outside the pinch

location “O”

• Can’t be seen from the pinch location “N”

Pinch Points
• Nodes that have a line of site to pinch location “O”
VO
• Can’t be seen from the pinch location “N”
VN
• Good ambush locations is their intersection:
VP  VO VN
Pinch Points

I = Inside Node
N = Pinch Point
0 = Outside Node

Pinch Point
Pinch Points
Another Example:
Pinch Points
Result:
Pinch Points
Slightly altered version to find pinch points at the end of
hallways:

For each node, N in the node graph with only two neighbors:
• Temporarily eliminate node, N, from the graph, call its neighbors as A & B.
• If both A & B are connected to large regions, N is not a pinch point, try another N.
• If O’s neighbor has only one other neighbor in addition to N.
• Move N to O.
• Move O to the other neighbor of the old O
• Repeat until O has only one neighbor.
• Attempt to find a path between A& B.
• If path exists, N is not a pinch point, try another N.
• Call the node connected to the larger region, O (for outside).
• Call the node connected to the smaller region, I (for inside).
Pinch Points

• If O’s neighbor only has one other neighbor in addition to N

Pinch Points

• Move N to O, Move O to other neighbor of old O

• Repeat till O has only one neighbor
Pinch Points

• Move N to O, Move O to other neighbor of old O

• Repeat till O has only one neighbor
Pinch Points

• Calculate good ambush locations:

VP  VO VN
Pinch Points

I = Inside Node
N = Pinch Point
0 = Outside Node

Hall Pinch
Pinch Points
Final Example:
Pinch Points

• For each node, N in the node graph with only two neighbors
Pinch Points

• Attempt to find a path between A& B.

• If path exists, N is not a pinch point, try another N
Pinch Points
If NPCs organize into squads regions with multiple pinch
points can be employed:

For each node, N1 in the node graph with only two neighbors:
• Temporarily eliminate node, N1, from the graph, call its neighbors as A & B.
• If A & B are connected to large regions, N1 is not a pinch point, try another N1
• Attempt to find a path between A& B.
• While generating the path if a node with only two neighbors is found,
• Temporarily eliminate it and call it N2.
• Attempt to find a path between A& B.
• If path exists, not a pinch point, try another N1
• Call the nodes connected to the smaller regions, I1 and I2 (for inside).
• Call the nodes connected to the larger regions, O1 and O2 (for outside).
Pinch Points

• While generating the path if a node with only two neighbors is found
• Temporarily eliminate it and call it N2
Pinch Points

• Attempt to find a path between A & B

• If path exists N1 is not a pinch point, try another N1
Pinch Points

• Call the nodes connected to the smaller regions, I1 and I2 (for inside).
• Call the nodes connected to the larger regions, O1 and O2 (for outside).
Pinch Points

• Calculate good ambush locations:

VP1  VO1 VN1 VN 2

VP2  VO2 VN1 VN 2

Pinch Points

I = Inside Node
N = Pinch Point
0 = Outside Node

Double Pinch
Tactical Analysis: Review
Using the node graph to evaluate map locations:
• Finding safe and dangerous locations
• Finding places from which to attack
• Finding location to set up sniper positions
• Finding pinch points
Components of an AI System
Decision Making
Tactical Analysis
Artificial Stupidity
ARTIFICIAL STUPIDITY
When NPCs Should Make Mistakes
Intelligence != Fun
• What makes a game entertaining and fun does
not necessarily correspond to making characters
smarter

• The player is, after all, supposed to win

• 11 Ways to be stupid
1. Don’t Cheat
• Sounds easy, but many games “cheat” by
making NPCs omniscient
• Know where enemies are even without seeing them
• Know where to find weapons or ammo
• Players usually eventually detect cheating or at
least get the feeling that the NPC’s behavior
seems somehow “unnatural”
2. Always miss the player the first time
• It’s not fun to suddenly and unexpectedly take
damage
• Player may feel cheated, particularly if attacked
with a weapon that kills the player or does a lot
of damage
• By missing the player the first time, it gives the
player a second to react and still keeps the
tension high
3. Have horrible aim (wide cone)
• Having abundant gun fire in the air keeps the
player on the move and the tension high
• However, the player is supposed to win
• By giving NPC bad aim, one can have abundant
gun fire without being too hard on the player
• “Half-Life” used a wide spread on NPC weapons
(as much at 40 degrees)
4. Never shoot when first see the player
• When a player first walks into an area and is
spotted by an enemy, the enemy should never
attack right away
• A secondary activity, such as running for cover
or finding a good shooting location is more
desirable
• Gives player time to react
5. Warn the Player
• Before attacking the player, warn the player that
you are about to do so
• Make a sound (beep/click)
• Play a quick animation
• Say “Gotcha!”, “Take this”
• This is particularly important when attacking
from behind
6. Attack “kung-fu” style
• Player is usually playing the role of “Rambo” (i.e.
one man taking on an army)
• Although many NPCs may be in a position to
attack the player, only a couple should do so at
a time
• The remaining NPCs should look busy,
reloading, changing positions, etc.
7. Tell the player what you are doing
• Interpreting the actions of NPCs can often be
subtle
• Complex behaviors are often missed by the
player. (Lot’s of work for nothing)
• NPCs should tell the player what they are doing
• “flanking!” “cover me!” “retreat!”
• Players with often intuit intelligence behavior that
isn’t really there
8. Intentionally be vulnerable
• Players learn to capitalize on opponent’s weaknesses.
• Rather than allowing the player to discover unintentional
weaknesses in the AI, vulnerability should be designed into an
NPC’s behavior.
• Stop moving before attacking
• Pause and prepare weapon before attacking
• Act surprised and slow to react when attacked from behind
• Planned vulnerability makes the characters seem more realistic
• Unintentional mistakes break the realism (seems like fighting a
computer program)
9. Don’t be perfect
• Human players make mistakes
• When NPCs behave perfectly they seem
unnatural
• If an NPC knows how to avoid trip mines, run
into then occasionally
• When reloading, sometimes fumble with the gun
10. Pull back last minute
Trick:
• Push the player to the limit
• Attack vigorously until the player is near death
• Then pull back. Enemy becomes easier to kill
• Makes player feel like they really accomplished
something
11. React To Mistakes
• Mistakes in AI are inevitable

• Unhandled, they make make the AI look dumb

• By recognizing mistakes and reacting to them

intelligently they can be turned into features
11. React To Mistakes
• Example 1:
• Occasionally when an NPC throws a grenade, it bounces off of
another object and lands back at the NPCs feet
• (Note that the player occasionally makes this mistake too!)

• Looks dumb as the NPC blows himself up

• If the NPC reacts, however, the mistake turns into a feature:

• NPC body and facial expression can show surprise, fear
• NPC can say “Oh Shoot!” or “Doh!”
11. React To Mistakes
• Example 2:
• Player throws a grenade at a group of NPCs. As they are
crowded together not all of them are able to find a path to get
away

• Looks dumb if the NPCs that can’t get away, shuffle around
trying to get out

• If we detect that the problem has arisen, can have the trapped
NPC’s react
• Crouch down and put hands over head
Components of an AI System
Decision Making
Tactical Analysis
Artificial Stupidity
Conclusions
 Some Unsolved Problems:
Automatic generation of waypoints
• Currently waypoints placed by hand in levels by level
designers
• Placement is time consuming
• Requires editing every time map is changed
• Error prone
• No reliable system for automatically placing waypoints
 Some Unsolved Problems:
Generation of dynamic waypoints
• Pre-generated waypoints only work for static maps
• Can deal with some moving objects, by turning
node connections on/off or re-testing connections in
real-time
• Maps that are generated dynamically or maps that
change significantly during run time are problematic
Conclusions:
• In the computer game industry the term A.I. is used to
cover a broad range of technologies including animation,
movement, navigation and behavior.
• What is required to make a game entertaining does not
always coincide with making the game’s characters
“smarter”.
• It is world where the illusion of intelligence is far more
important then actual intelligence