5 - Artificial Intelligence for Games
Game AI creates intelligent behaviors for non-player characters (NPCs), enemies, and other game entities. This section covers key AI algorithms and techniques used in game development, with implementations in C#.
Introduction to Game AI
Game AI differs from academic AI in several important ways:
- Goal: Game AI aims to create entertaining, believable behaviors rather than optimal solutions.
- Resource Constraints: Game AI must run in real-time with limited CPU and memory resources.
- Illusion of Intelligence: Often focuses on creating the appearance of intelligence rather than true intelligence.
- Predictability: Sometimes needs to be predictable enough for players to learn and counter.
Common applications of AI in games include:
- Enemy and NPC behavior
- Pathfinding and navigation
- Decision making
- Procedural content generation
- Dynamic difficulty adjustment
- Player modeling
Finite State Machines
Finite State Machines (FSMs) are one of the most common AI techniques in games, providing a simple way to model character behavior.
Basic FSM Implementation
/// <summary>
/// Represents the possible states of an enemy in a finite state machine.
/// </summary>
public enum EnemyState
{
/// <summary>
/// The enemy is idle, not moving or taking any action.
/// </summary>
Idle,
/// <summary>
/// The enemy is patrolling along predefined points.
/// </summary>
Patrol,
/// <summary>
/// The enemy is chasing the player.
/// </summary>
Chase,
/// <summary>
/// The enemy is attacking the player.
/// </summary>
Attack,
/// <summary>
/// The enemy is fleeing from the player.
/// </summary>
Flee
}
/// <summary>
/// Implements a Finite State Machine (FSM) for enemy AI behavior.
/// This class demonstrates how to create a simple but effective AI using states and transitions.
/// </summary>
public class EnemyFSM
{
/// <summary>
/// The current state of the enemy.
/// </summary>
private EnemyState currentState;
/// <summary>
/// Maps each state to its corresponding action method.
/// </summary>
private readonly Dictionary<EnemyState, Action> stateActions;
/// <summary>
/// Maps each state to its possible transitions, where each transition has a condition function.
/// </summary>
private readonly Dictionary<EnemyState, Dictionary<EnemyState, Func<bool>>> transitions;
/// <summary>
/// Gets or sets the position of the enemy in the game world.
/// </summary>
public Vector2 Position { get; set; }
/// <summary>
/// Gets or sets the position of the player in the game world.
/// </summary>
public Vector2 PlayerPosition { get; set; }
/// <summary>
/// Gets or sets the range within which the enemy can detect the player.
/// </summary>
public float DetectionRange { get; set; } = 5.0f;
/// <summary>
/// Gets or sets the range within which the enemy can attack the player.
/// </summary>
public float AttackRange { get; set; } = 1.5f;
/// <summary>
/// Gets or sets the current health of the enemy.
/// </summary>
public float Health { get; set; } = 100.0f;
/// <summary>
/// Gets or sets the maximum health of the enemy.
/// </summary>
public float MaxHealth { get; set; } = 100.0f;
/// <summary>
/// Gets or sets the health threshold below which the enemy will flee.
/// </summary>
public float FleeThreshold { get; set; } = 30.0f;
/// <summary>
/// Gets or sets the movement speed when patrolling.
/// </summary>
public float PatrolSpeed { get; set; } = 0.05f;
/// <summary>
/// Gets or sets the movement speed when chasing.
/// </summary>
public float ChaseSpeed { get; set; } = 0.1f;
/// <summary>
/// Gets or sets the movement speed when fleeing.
/// </summary>
public float FleeSpeed { get; set; } = 0.15f;
/// <summary>
/// Gets the current state of the enemy.
/// </summary>
public EnemyState CurrentState => currentState;
// Patrol-specific properties
private readonly Vector2[] patrolPoints;
private int currentPatrolIndex;
private float patrolWaitTime;
private float patrolTimer;
// Attack-specific properties
private float attackCooldown = 1.0f;
private float attackTimer = 0.0f;
private bool canAttack = true;
/// <summary>
/// Initializes a new instance of the EnemyFSM class.
/// </summary>
/// <param name="startPosition">The initial position of the enemy.</param>
/// <param name="patrolPoints">An array of points defining the enemy's patrol path.</param>
/// <exception cref="ArgumentNullException">Thrown when patrolPoints is null.</exception>
/// <exception cref="ArgumentException">Thrown when patrolPoints is empty.</exception>
public EnemyFSM(Vector2 startPosition, Vector2[] patrolPoints)
{
if (patrolPoints == null)
throw new ArgumentNullException(nameof(patrolPoints), "Patrol points cannot be null.");
if (patrolPoints.Length == 0)
throw new ArgumentException("At least one patrol point is required.", nameof(patrolPoints));
Position = startPosition;
this.patrolPoints = patrolPoints;
currentPatrolIndex = 0;
patrolWaitTime = 2.0f;
patrolTimer = 0.0f;
// Initialize with Idle state
currentState = EnemyState.Idle;
// Set up state actions
stateActions = new Dictionary<EnemyState, Action>
{
{ EnemyState.Idle, Idle },
{ EnemyState.Patrol, Patrol },
{ EnemyState.Chase, Chase },
{ EnemyState.Attack, Attack },
{ EnemyState.Flee, Flee }
};
// Set up state transitions
transitions = new Dictionary<EnemyState, Dictionary<EnemyState, Func<bool>>>();
// Transitions from Idle
transitions[EnemyState.Idle] = new Dictionary<EnemyState, Func<bool>>
{
// Transition to Patrol when the wait time is over
{ EnemyState.Patrol, () => patrolTimer >= patrolWaitTime },
// Transition to Chase if the player is detected
{ EnemyState.Chase, () => IsPlayerDetected() }
};
// Transitions from Patrol
transitions[EnemyState.Patrol] = new Dictionary<EnemyState, Func<bool>>
{
// Transition to Idle when reaching a patrol point
{ EnemyState.Idle, () => HasReachedPatrolPoint() },
// Transition to Chase if the player is detected
{ EnemyState.Chase, () => IsPlayerDetected() }
};
// Transitions from Chase
transitions[EnemyState.Chase] = new Dictionary<EnemyState, Func<bool>>
{
// Transition to Attack if the player is in attack range
{ EnemyState.Attack, () => IsPlayerInAttackRange() },
// Transition to Patrol if the player is no longer detected
{ EnemyState.Patrol, () => !IsPlayerDetected() },
// Transition to Flee if health is low
{ EnemyState.Flee, () => Health <= FleeThreshold }
};
// Transitions from Attack
transitions[EnemyState.Attack] = new Dictionary<EnemyState, Func<bool>>
{
// Transition to Chase if the player moves out of attack range
{ EnemyState.Chase, () => !IsPlayerInAttackRange() },
// Transition to Flee if health is low
{ EnemyState.Flee, () => Health <= FleeThreshold }
};
// Transitions from Flee
transitions[EnemyState.Flee] = new Dictionary<EnemyState, Func<bool>>
{
// Transition to Idle if the player is far away and health is above the threshold
{ EnemyState.Idle, () => IsPlayerFarAway() && Health > FleeThreshold }
};
}
/// <summary>
/// Updates the enemy's state and behavior.
/// </summary>
/// <param name="deltaTime">The time elapsed since the last update, in seconds.</param>
public void Update(float deltaTime)
{
// Check for state transitions
CheckStateTransitions();
// Execute current state action
ExecuteCurrentStateAction();
// Update timers
UpdateTimers(deltaTime);
}
/// <summary>
/// Checks for and applies state transitions based on current conditions.
/// </summary>
private void CheckStateTransitions()
{
if (transitions.ContainsKey(currentState))
{
foreach (var transition in transitions[currentState])
{
if (transition.Value())
{
// State is changing
EnemyState oldState = currentState;
currentState = transition.Key;
// Log the state change (in a real game, you might want to trigger animations or effects)
Console.WriteLine($"Enemy state changed from {oldState} to {currentState}");
// Only apply the first valid transition
break;
}
}
}
}
/// <summary>
/// Executes the action associated with the current state.
/// </summary>
private void ExecuteCurrentStateAction()
{
if (stateActions.ContainsKey(currentState))
{
stateActions[currentState]();
}
else
{
// This should never happen if the FSM is set up correctly
Console.WriteLine($"Error: No action defined for state {currentState}");
}
}
/// <summary>
/// Updates all timers used by the FSM.
/// </summary>
/// <param name="deltaTime">The time elapsed since the last update, in seconds.</param>
private void UpdateTimers(float deltaTime)
{
// Update patrol timer if in Idle state
if (currentState == EnemyState.Idle)
{
patrolTimer += deltaTime;
}
// Update attack cooldown timer
if (!canAttack)
{
attackTimer += deltaTime;
if (attackTimer >= attackCooldown)
{
canAttack = true;
attackTimer = 0.0f;
}
}
}
/// <summary>
/// Behavior for the Idle state.
/// In this state, the enemy stands still and waits.
/// </summary>
private void Idle()
{
// In the Idle state, the enemy doesn't move or take any action
// This could be expanded to include idle animations or looking around
}
/// <summary>
/// Behavior for the Patrol state.
/// In this state, the enemy moves between predefined patrol points.
/// </summary>
private void Patrol()
{
// Get the current patrol point
Vector2 target = patrolPoints[currentPatrolIndex];
// Calculate direction to the patrol point
Vector2 direction = (target - Position).normalized;
// Move towards the target at patrol speed
Position += direction * PatrolSpeed;
// Check if we've reached the patrol point (within a small threshold)
if (Vector2.Distance(Position, target) < 0.1f)
{
// Move to the next patrol point
currentPatrolIndex = (currentPatrolIndex + 1) % patrolPoints.Length;
// Reset the patrol timer for the next idle period
patrolTimer = 0.0f;
}
}
/// <summary>
/// Behavior for the Chase state.
/// In this state, the enemy moves towards the player.
/// </summary>
private void Chase()
{
// Calculate direction to the player
Vector2 direction = (PlayerPosition - Position).normalized;
// Move towards the player at chase speed
Position += direction * ChaseSpeed;
}
/// <summary>
/// Behavior for the Attack state.
/// In this state, the enemy attacks the player if within range.
/// </summary>
private void Attack()
{
if (canAttack)
{
// Perform attack
Console.WriteLine("Enemy attacks the player for 10 damage!");
// In a real game, you would apply damage to the player here
// player.TakeDamage(10);
// Start attack cooldown
canAttack = false;
attackTimer = 0.0f;
}
// Optionally, the enemy could move slightly during attacks
// For example, to maintain optimal attack distance
float currentDistance = Vector2.Distance(Position, PlayerPosition);
float optimalDistance = AttackRange * 0.8f; // Stay at 80% of max attack range
if (MathF.Abs(currentDistance - optimalDistance) > 0.1f)
{
Vector2 direction;
if (currentDistance > optimalDistance)
{
// Move closer
direction = (PlayerPosition - Position).normalized;
Position += direction * PatrolSpeed * 0.5f; // Move slower during attack
}
else
{
// Move away slightly
direction = (Position - PlayerPosition).normalized;
Position += direction * PatrolSpeed * 0.3f; // Move even slower when backing up
}
}
}
/// <summary>
/// Behavior for the Flee state.
/// In this state, the enemy moves away from the player.
/// </summary>
private void Flee()
{
// Calculate direction away from the player
Vector2 direction = (Position - PlayerPosition).normalized;
// Move away from the player at flee speed
Position += direction * FleeSpeed;
// In a real game, the enemy might also try to find cover or healing items
}
/// <summary>
/// Determines whether the player is within the enemy's detection range.
/// </summary>
/// <returns>True if the player is detected; otherwise, false.</returns>
private bool IsPlayerDetected()
{
return Vector2.Distance(Position, PlayerPosition) <= DetectionRange;
}
/// <summary>
/// Determines whether the player is within the enemy's attack range.
/// </summary>
/// <returns>True if the player is in attack range; otherwise, false.</returns>
private bool IsPlayerInAttackRange()
{
return Vector2.Distance(Position, PlayerPosition) <= AttackRange;
}
/// <summary>
/// Determines whether the enemy has reached the current patrol point.
/// </summary>
/// <returns>True if the enemy has reached the patrol point; otherwise, false.</returns>
private bool HasReachedPatrolPoint()
{
return Vector2.Distance(Position, patrolPoints[currentPatrolIndex]) < 0.1f;
}
/// <summary>
/// Determines whether the player is far enough away to be considered "far away".
/// </summary>
/// <returns>True if the player is far away; otherwise, false.</returns>
private bool IsPlayerFarAway()
{
return Vector2.Distance(Position, PlayerPosition) > DetectionRange * 1.5f;
}
/// <summary>
/// Takes damage and potentially transitions to the Flee state if health drops below the threshold.
/// </summary>
/// <param name="amount">The amount of damage to take.</param>
public void TakeDamage(float amount)
{
Health -= amount;
// Ensure health doesn't go below 0
Health = MathF.Max(0, Health);
// Log the damage (in a real game, you might trigger visual effects)
Console.WriteLine($"Enemy took {amount} damage. Health: {Health}/{MaxHealth}");
// The state transition to Flee will happen on the next Update
}
/// <summary>
/// Heals the enemy by the specified amount.
/// </summary>
/// <param name="amount">The amount of health to restore.</param>
public void Heal(float amount)
{
Health += amount;
// Ensure health doesn't exceed maximum
Health = MathF.Min(Health, MaxHealth);
// Log the healing (in a real game, you might trigger visual effects)
Console.WriteLine($"Enemy healed for {amount}. Health: {Health}/{MaxHealth}");
}
}
/// <summary>
/// Represents a 2D vector with x and y components.
/// This struct is used for positions, directions, and velocities in 2D space.
/// </summary>
public struct Vector2
{
/// <summary>
/// The x component of the vector.
/// </summary>
public float x;
/// <summary>
/// The y component of the vector.
/// </summary>
public float y;
/// <summary>
/// Gets a vector with components (0, 0).
/// </summary>
public static Vector2 zero => new Vector2(0, 0);
/// <summary>
/// Gets a vector with components (1, 1).
/// </summary>
public static Vector2 one => new Vector2(1, 1);
/// <summary>
/// Gets a vector with components (1, 0).
/// </summary>
public static Vector2 right => new Vector2(1, 0);
/// <summary>
/// Gets a vector with components (-1, 0).
/// </summary>
public static Vector2 left => new Vector2(-1, 0);
/// <summary>
/// Gets a vector with components (0, 1).
/// </summary>
public static Vector2 up => new Vector2(0, 1);
/// <summary>
/// Gets a vector with components (0, -1).
/// </summary>
public static Vector2 down => new Vector2(0, -1);
/// <summary>
/// Initializes a new instance of the Vector2 struct.
/// </summary>
/// <param name="x">The x component of the vector.</param>
/// <param name="y">The y component of the vector.</param>
public Vector2(float x, float y)
{
this.x = x;
this.y = y;
}
/// <summary>
/// Gets the magnitude (length) of the vector.
/// </summary>
public float magnitude => MathF.Sqrt(x * x + y * y);
/// <summary>
/// Gets the squared magnitude of the vector.
/// This is faster to compute than magnitude and is useful for comparisons.
/// </summary>
public float sqrMagnitude => x * x + y * y;
/// <summary>
/// Gets a normalized version of this vector (a vector with the same direction but magnitude 1).
/// </summary>
public Vector2 normalized
{
get
{
float mag = magnitude;
if (mag > float.Epsilon)
return new Vector2(x / mag, y / mag);
return zero;
}
}
/// <summary>
/// Normalizes this vector in place (makes it have magnitude 1 while preserving direction).
/// </summary>
public void Normalize()
{
float mag = magnitude;
if (mag > float.Epsilon)
{
x /= mag;
y /= mag;
}
else
{
x = 0;
y = 0;
}
}
/// <summary>
/// Calculates the dot product of two vectors.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>The dot product of the vectors.</returns>
public static float Dot(Vector2 a, Vector2 b)
{
return a.x * b.x + a.y * b.y;
}
/// <summary>
/// Calculates the 2D cross product of two vectors.
/// In 2D, this returns a scalar representing the z component of the 3D cross product.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>The z component of the 3D cross product.</returns>
public static float Cross(Vector2 a, Vector2 b)
{
return a.x * b.y - a.y * b.x;
}
/// <summary>
/// Calculates the distance between two vectors.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>The distance between the vectors.</returns>
public static float Distance(Vector2 a, Vector2 b)
{
float dx = a.x - b.x;
float dy = a.y - b.y;
return MathF.Sqrt(dx * dx + dy * dy);
}
/// <summary>
/// Calculates the squared distance between two vectors.
/// This is faster than Distance and is useful for comparisons.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>The squared distance between the vectors.</returns>
public static float SqrDistance(Vector2 a, Vector2 b)
{
float dx = a.x - b.x;
float dy = a.y - b.y;
return dx * dx + dy * dy;
}
/// <summary>
/// Linearly interpolates between two vectors.
/// </summary>
/// <param name="a">The starting vector.</param>
/// <param name="b">The ending vector.</param>
/// <param name="t">The interpolation factor (0-1).</param>
/// <returns>The interpolated vector.</returns>
public static Vector2 Lerp(Vector2 a, Vector2 b, float t)
{
// Clamp t to the range [0, 1]
t = t < 0 ? 0 : (t > 1 ? 1 : t);
return new Vector2(
a.x + (b.x - a.x) * t,
a.y + (b.y - a.y) * t
);
}
/// <summary>
/// Linearly interpolates between two vectors without clamping the interpolation factor.
/// </summary>
/// <param name="a">The starting vector.</param>
/// <param name="b">The ending vector.</param>
/// <param name="t">The interpolation factor.</param>
/// <returns>The interpolated vector.</returns>
public static Vector2 LerpUnclamped(Vector2 a, Vector2 b, float t)
{
return new Vector2(
a.x + (b.x - a.x) * t,
a.y + (b.y - a.y) * t
);
}
/// <summary>
/// Returns a vector with the minimum components of two vectors.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>A vector with the minimum components.</returns>
public static Vector2 Min(Vector2 a, Vector2 b)
{
return new Vector2(
MathF.Min(a.x, b.x),
MathF.Min(a.y, b.y)
);
}
/// <summary>
/// Returns a vector with the maximum components of two vectors.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>A vector with the maximum components.</returns>
public static Vector2 Max(Vector2 a, Vector2 b)
{
return new Vector2(
MathF.Max(a.x, b.x),
MathF.Max(a.y, b.y)
);
}
/// <summary>
/// Adds two vectors.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>The sum of the vectors.</returns>
public static Vector2 operator +(Vector2 a, Vector2 b) => new Vector2(a.x + b.x, a.y + b.y);
/// <summary>
/// Subtracts the second vector from the first.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>The difference of the vectors.</returns>
public static Vector2 operator -(Vector2 a, Vector2 b) => new Vector2(a.x - b.x, a.y - b.y);
/// <summary>
/// Negates a vector.
/// </summary>
/// <param name="a">The vector to negate.</param>
/// <returns>The negated vector.</returns>
public static Vector2 operator -(Vector2 a) => new Vector2(-a.x, -a.y);
/// <summary>
/// Multiplies a vector by a scalar.
/// </summary>
/// <param name="a">The vector.</param>
/// <param name="b">The scalar.</param>
/// <returns>The scaled vector.</returns>
public static Vector2 operator *(Vector2 a, float b) => new Vector2(a.x * b, a.y * b);
/// <summary>
/// Multiplies a scalar by a vector.
/// </summary>
/// <param name="a">The scalar.</param>
/// <param name="b">The vector.</param>
/// <returns>The scaled vector.</returns>
public static Vector2 operator *(float a, Vector2 b) => new Vector2(a * b.x, a * b.y);
/// <summary>
/// Divides a vector by a scalar.
/// </summary>
/// <param name="a">The vector.</param>
/// <param name="b">The scalar.</param>
/// <returns>The scaled vector.</returns>
public static Vector2 operator /(Vector2 a, float b)
{
if (b == 0)
throw new DivideByZeroException("Cannot divide a vector by zero.");
return new Vector2(a.x / b, a.y / b);
}
/// <summary>
/// Determines whether two vectors are equal.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>True if the vectors are equal; otherwise, false.</returns>
public static bool operator ==(Vector2 a, Vector2 b)
{
return a.x == b.x && a.y == b.y;
}
/// <summary>
/// Determines whether two vectors are not equal.
/// </summary>
/// <param name="a">The first vector.</param>
/// <param name="b">The second vector.</param>
/// <returns>True if the vectors are not equal; otherwise, false.</returns>
public static bool operator !=(Vector2 a, Vector2 b)
{
return a.x != b.x || a.y != b.y;
}
/// <summary>
/// Determines whether this vector is equal to another object.
/// </summary>
/// <param name="obj">The object to compare with.</param>
/// <returns>True if the objects are equal; otherwise, false.</returns>
public override bool Equals(object obj)
{
if (obj is Vector2 other)
{
return this == other;
}
return false;
}
/// <summary>
/// Returns a hash code for this vector.
/// </summary>
/// <returns>A hash code for this vector.</returns>
public override int GetHashCode()
{
return HashCode.Combine(x, y);
}
/// <summary>
/// Returns a string representation of this vector.
/// </summary>
/// <returns>A string representation of this vector.</returns>
public override string ToString()
{
return $"({x}, {y})";
}
}
Hierarchical FSM
public enum MainState
{
Combat,
NonCombat
}
public enum CombatState
{
Attack,
Defend,
Flee
}
public enum NonCombatState
{
Idle,
Patrol,
Investigate
}
public class HierarchicalFSM
{
private MainState currentMainState;
private CombatState currentCombatState;
private NonCombatState currentNonCombatState;
// Game-specific properties
public Vector2 Position { get; set; }
public Vector2 PlayerPosition { get; set; }
public float DetectionRange { get; set; } = 5.0f;
public float AttackRange { get; set; } = 1.5f;
public float Health { get; set; } = 100.0f;
public float MaxHealth { get; set; } = 100.0f;
public HierarchicalFSM(Vector2 startPosition)
{
Position = startPosition;
currentMainState = MainState.NonCombat;
currentNonCombatState = NonCombatState.Idle;
}
public void Update(float deltaTime)
{
// Update main state
UpdateMainState();
// Update sub-state based on main state
switch (currentMainState)
{
case MainState.Combat:
UpdateCombatState();
ExecuteCombatState();
break;
case MainState.NonCombat:
UpdateNonCombatState();
ExecuteNonCombatState();
break;
}
}
private void UpdateMainState()
{
switch (currentMainState)
{
case MainState.Combat:
// Transition to NonCombat if player is far away
if (Vector2.Distance(Position, PlayerPosition) > DetectionRange * 1.5f)
{
currentMainState = MainState.NonCombat;
currentNonCombatState = NonCombatState.Investigate;
}
break;
case MainState.NonCombat:
// Transition to Combat if player is detected
if (Vector2.Distance(Position, PlayerPosition) <= DetectionRange)
{
currentMainState = MainState.Combat;
currentCombatState = CombatState.Attack;
}
break;
}
}
private void UpdateCombatState()
{
switch (currentCombatState)
{
case CombatState.Attack:
// Transition to Defend if health is low
if (Health < MaxHealth * 0.5f)
{
currentCombatState = CombatState.Defend;
}
// Transition to Flee if health is very low
else if (Health < MaxHealth * 0.2f)
{
currentCombatState = CombatState.Flee;
}
break;
case CombatState.Defend:
// Transition to Attack if health is high enough
if (Health > MaxHealth * 0.7f)
{
currentCombatState = CombatState.Attack;
}
// Transition to Flee if health is very low
else if (Health < MaxHealth * 0.2f)
{
currentCombatState = CombatState.Flee;
}
break;
case CombatState.Flee:
// Transition to Defend if health is high enough
if (Health > MaxHealth * 0.4f)
{
currentCombatState = CombatState.Defend;
}
break;
}
}
private void UpdateNonCombatState()
{
switch (currentNonCombatState)
{
case NonCombatState.Idle:
// Transition to Patrol after some time
// (In a real implementation, you would use a timer here)
currentNonCombatState = NonCombatState.Patrol;
break;
case NonCombatState.Patrol:
// Transition to Investigate if something suspicious is detected
// (In a real implementation, you would check for suspicious events)
break;
case NonCombatState.Investigate:
// Transition to Idle if nothing is found
// (In a real implementation, you would check if investigation is complete)
currentNonCombatState = NonCombatState.Idle;
break;
}
}
private void ExecuteCombatState()
{
switch (currentCombatState)
{
case CombatState.Attack:
// Move towards player and attack
if (Vector2.Distance(Position, PlayerPosition) > AttackRange)
{
// Move towards player
Vector2 direction = (PlayerPosition - Position).normalized;
Position += direction * 0.1f;
}
else
{
// Attack
Console.WriteLine("Enemy attacks!");
}
break;
case CombatState.Defend:
// Keep distance and defend
float optimalDistance = AttackRange * 1.2f;
float currentDistance = Vector2.Distance(Position, PlayerPosition);
if (currentDistance < optimalDistance - 0.1f)
{
// Move away from player
Vector2 direction = (Position - PlayerPosition).normalized;
Position += direction * 0.05f;
}
else if (currentDistance > optimalDistance + 0.1f)
{
// Move towards player
Vector2 direction = (PlayerPosition - Position).normalized;
Position += direction * 0.05f;
}
// Defend
Console.WriteLine("Enemy defends!");
break;
case CombatState.Flee:
// Move away from player
Vector2 fleeDirection = (Position - PlayerPosition).normalized;
Position += fleeDirection * 0.15f;
break;
}
}
private void ExecuteNonCombatState()
{
switch (currentNonCombatState)
{
case NonCombatState.Idle:
// Do nothing
break;
case NonCombatState.Patrol:
// Move along patrol path
// (In a real implementation, you would follow a patrol path)
break;
case NonCombatState.Investigate:
// Move towards suspicious location
// (In a real implementation, you would move towards a specific point)
break;
}
}
}
Behavior Trees
Behavior Trees provide a more flexible and modular approach to AI decision making than FSMs.
Basic Behavior Tree Implementation
public enum NodeStatus
{
Success,
Failure,
Running
}
public abstract class BehaviorNode
{
public abstract NodeStatus Execute();
}
public class Sequence : BehaviorNode
{
private List<BehaviorNode> children = new List<BehaviorNode>();
private int currentChild = 0;
public Sequence(params BehaviorNode[] nodes)
{
children.AddRange(nodes);
}
public override NodeStatus Execute()
{
// If we've run all children, reset and return success
if (currentChild >= children.Count)
{
currentChild = 0;
return NodeStatus.Success;
}
// Execute the current child
NodeStatus status = children[currentChild].Execute();
// Process the result
switch (status)
{
case NodeStatus.Success:
// Move to the next child
currentChild++;
// If we've run all children, reset and return success
if (currentChild >= children.Count)
{
currentChild = 0;
return NodeStatus.Success;
}
// Otherwise, we're still running
return NodeStatus.Running;
case NodeStatus.Failure:
// If any child fails, the sequence fails
currentChild = 0;
return NodeStatus.Failure;
case NodeStatus.Running:
// If a child is still running, the sequence is still running
return NodeStatus.Running;
default:
return NodeStatus.Failure;
}
}
}
public class Selector : BehaviorNode
{
private List<BehaviorNode> children = new List<BehaviorNode>();
private int currentChild = 0;
public Selector(params BehaviorNode[] nodes)
{
children.AddRange(nodes);
}
public override NodeStatus Execute()
{
// If we've run all children, reset and return failure
if (currentChild >= children.Count)
{
currentChild = 0;
return NodeStatus.Failure;
}
// Execute the current child
NodeStatus status = children[currentChild].Execute();
// Process the result
switch (status)
{
case NodeStatus.Success:
// If any child succeeds, the selector succeeds
currentChild = 0;
return NodeStatus.Success;
case NodeStatus.Failure:
// Move to the next child
currentChild++;
// If we've run all children, reset and return failure
if (currentChild >= children.Count)
{
currentChild = 0;
return NodeStatus.Failure;
}
// Otherwise, we're still running
return NodeStatus.Running;
case NodeStatus.Running:
// If a child is still running, the selector is still running
return NodeStatus.Running;
default:
return NodeStatus.Failure;
}
}
}
public class Inverter : BehaviorNode
{
private BehaviorNode child;
public Inverter(BehaviorNode child)
{
this.child = child;
}
public override NodeStatus Execute()
{
NodeStatus status = child.Execute();
switch (status)
{
case NodeStatus.Success:
return NodeStatus.Failure;
case NodeStatus.Failure:
return NodeStatus.Success;
case NodeStatus.Running:
return NodeStatus.Running;
default:
return NodeStatus.Failure;
}
}
}
public class Repeater : BehaviorNode
{
private BehaviorNode child;
private int maxRepeats;
private int currentRepeats = 0;
public Repeater(BehaviorNode child, int maxRepeats = -1)
{
this.child = child;
this.maxRepeats = maxRepeats;
}
public override NodeStatus Execute()
{
// If we've reached the maximum number of repeats, reset and return success
if (maxRepeats > 0 && currentRepeats >= maxRepeats)
{
currentRepeats = 0;
return NodeStatus.Success;
}
// Execute the child
NodeStatus status = child.Execute();
// If the child is still running, we're still running
if (status == NodeStatus.Running)
return NodeStatus.Running;
// If the child has completed (success or failure), increment the repeat counter
if (status == NodeStatus.Success || status == NodeStatus.Failure)
{
currentRepeats++;
// If we've reached the maximum number of repeats, reset and return success
if (maxRepeats > 0 && currentRepeats >= maxRepeats)
{
currentRepeats = 0;
return NodeStatus.Success;
}
}
// Always return running unless we've reached the maximum repeats
return NodeStatus.Running;
}
}
// Example action nodes for an enemy AI
public class IsPlayerInSightRange : BehaviorNode
{
private Enemy enemy;
public IsPlayerInSightRange(Enemy enemy)
{
this.enemy = enemy;
}
public override NodeStatus Execute()
{
if (Vector2.Distance(enemy.Position, enemy.PlayerPosition) <= enemy.SightRange)
return NodeStatus.Success;
else
return NodeStatus.Failure;
}
}
public class IsPlayerInAttackRange : BehaviorNode
{
private Enemy enemy;
public IsPlayerInAttackRange(Enemy enemy)
{
this.enemy = enemy;
}
public override NodeStatus Execute()
{
if (Vector2.Distance(enemy.Position, enemy.PlayerPosition) <= enemy.AttackRange)
return NodeStatus.Success;
else
return NodeStatus.Failure;
}
}
public class IsHealthLow : BehaviorNode
{
private Enemy enemy;
private float threshold;
public IsHealthLow(Enemy enemy, float threshold = 0.3f)
{
this.enemy = enemy;
this.threshold = threshold;
}
public override NodeStatus Execute()
{
if (enemy.Health / enemy.MaxHealth <= threshold)
return NodeStatus.Success;
else
return NodeStatus.Failure;
}
}
public class ChasePlayer : BehaviorNode
{
private Enemy enemy;
private float speed;
public ChasePlayer(Enemy enemy, float speed = 0.1f)
{
this.enemy = enemy;
this.speed = speed;
}
public override NodeStatus Execute()
{
Vector2 direction = (enemy.PlayerPosition - enemy.Position).normalized;
enemy.Position += direction * speed;
return NodeStatus.Success;
}
}
public class AttackPlayer : BehaviorNode
{
private Enemy enemy;
private float cooldown;
private float timer = 0;
public AttackPlayer(Enemy enemy, float cooldown = 1.0f)
{
this.enemy = enemy;
this.cooldown = cooldown;
}
public override NodeStatus Execute()
{
if (timer <= 0)
{
Console.WriteLine("Enemy attacks!");
enemy.AttackPlayer();
timer = cooldown;
return NodeStatus.Success;
}
else
{
timer -= 0.1f; // Assuming this is called every 0.1 seconds
return NodeStatus.Running;
}
}
}
public class Flee : BehaviorNode
{
private Enemy enemy;
private float speed;
public Flee(Enemy enemy, float speed = 0.15f)
{
this.enemy = enemy;
this.speed = speed;
}
public override NodeStatus Execute()
{
Vector2 direction = (enemy.Position - enemy.PlayerPosition).normalized;
enemy.Position += direction * speed;
return NodeStatus.Success;
}
}
public class Patrol : BehaviorNode
{
private Enemy enemy;
private Vector2[] patrolPoints;
private int currentPoint = 0;
private float speed;
public Patrol(Enemy enemy, Vector2[] patrolPoints, float speed = 0.05f)
{
this.enemy = enemy;
this.patrolPoints = patrolPoints;
this.speed = speed;
}
public override NodeStatus Execute()
{
// If we've reached the current patrol point, move to the next one
if (Vector2.Distance(enemy.Position, patrolPoints[currentPoint]) < 0.1f)
{
currentPoint = (currentPoint + 1) % patrolPoints.Length;
}
// Move towards the current patrol point
Vector2 direction = (patrolPoints[currentPoint] - enemy.Position).normalized;
enemy.Position += direction * speed;
return NodeStatus.Success;
}
}
// Example enemy class
public class Enemy
{
public Vector2 Position { get; set; }
public Vector2 PlayerPosition { get; set; }
public float SightRange { get; set; } = 5.0f;
public float AttackRange { get; set; } = 1.5f;
public float Health { get; set; } = 100.0f;
public float MaxHealth { get; set; } = 100.0f;
private BehaviorNode behaviorTree;
public Enemy(Vector2 position, Vector2[] patrolPoints)
{
Position = position;
// Create the behavior tree
behaviorTree = new Selector(
// If health is low, flee
new Sequence(
new IsHealthLow(this),
new Flee(this)
),
// If player is in attack range, attack
new Sequence(
new IsPlayerInAttackRange(this),
new AttackPlayer(this)
),
// If player is in sight range, chase
new Sequence(
new IsPlayerInSightRange(this),
new ChasePlayer(this)
),
// Otherwise, patrol
new Patrol(this, patrolPoints)
);
}
public void Update()
{
behaviorTree.Execute();
}
public void AttackPlayer()
{
// In a real game, this would deal damage to the player
}
}
Utility-Based AI
Utility-based AI makes decisions by evaluating the utility (usefulness) of different actions.
Basic Utility AI Implementation
public class UtilityAI
{
private List<UtilityAction> actions = new List<UtilityAction>();
private UtilityAction currentAction;
public UtilityAI()
{
}
public void AddAction(UtilityAction action)
{
actions.Add(action);
}
public void Update()
{
// Find the action with the highest utility
float highestUtility = 0;
UtilityAction bestAction = null;
foreach (var action in actions)
{
float utility = action.CalculateUtility();
if (utility > highestUtility)
{
highestUtility = utility;
bestAction = action;
}
}
// If we found a valid action and it's different from the current one, switch to it
if (bestAction != null && bestAction != currentAction)
{
if (currentAction != null)
currentAction.Exit();
currentAction = bestAction;
currentAction.Enter();
}
// Execute the current action
if (currentAction != null)
currentAction.Execute();
}
}
public abstract class UtilityAction
{
protected List<UtilityConsideration> considerations = new List<UtilityConsideration>();
public void AddConsideration(UtilityConsideration consideration)
{
considerations.Add(consideration);
}
public float CalculateUtility()
{
if (considerations.Count == 0)
return 0;
float utility = 1.0f;
// Multiply all consideration scores together
foreach (var consideration in considerations)
{
utility *= consideration.CalculateScore();
}
return utility;
}
public virtual void Enter() { }
public abstract void Execute();
public virtual void Exit() { }
}
public abstract class UtilityConsideration
{
public abstract float CalculateScore();
}
// Example implementation for an NPC in a game
public class NPC
{
public Vector2 Position { get; set; }
public Vector2 PlayerPosition { get; set; }
public float Health { get; set; } = 100.0f;
public float MaxHealth { get; set; } = 100.0f;
public float Hunger { get; set; } = 0.0f;
public float MaxHunger { get; set; } = 100.0f;
public float Energy { get; set; } = 100.0f;
public float MaxEnergy { get; set; } = 100.0f;
private UtilityAI ai;
public NPC(Vector2 position)
{
Position = position;
// Create the utility AI
ai = new UtilityAI();
// Add actions with considerations
var attackAction = new AttackAction(this);
attackAction.AddConsideration(new PlayerProximityConsideration(this, true));
attackAction.AddConsideration(new HealthConsideration(this, true));
ai.AddAction(attackAction);
var fleeAction = new FleeAction(this);
fleeAction.AddConsideration(new PlayerProximityConsideration(this, false));
fleeAction.AddConsideration(new HealthConsideration(this, false));
ai.AddAction(fleeAction);
var eatAction = new EatAction(this);
eatAction.AddConsideration(new HungerConsideration(this));
ai.AddAction(eatAction);
var sleepAction = new SleepAction(this);
sleepAction.AddConsideration(new EnergyConsideration(this));
ai.AddAction(sleepAction);
var idleAction = new IdleAction(this);
idleAction.AddConsideration(new IdleConsideration());
ai.AddAction(idleAction);
}
public void Update()
{
// Update vital stats
Hunger += 0.1f;
Energy -= 0.05f;
// Clamp values
Hunger = Math.Clamp(Hunger, 0, MaxHunger);
Energy = Math.Clamp(Energy, 0, MaxEnergy);
// Update AI
ai.Update();
}
}
// Example actions
public class AttackAction : UtilityAction
{
private NPC npc;
public AttackAction(NPC npc)
{
this.npc = npc;
}
public override void Enter()
{
Console.WriteLine("Starting to attack!");
}
public override void Execute()
{
Console.WriteLine("Attacking player!");
// Move towards player
Vector2 direction = (npc.PlayerPosition - npc.Position).normalized;
npc.Position += direction * 0.1f;
// Attack logic would go here
}
public override void Exit()
{
Console.WriteLine("Stopping attack.");
}
}
public class FleeAction : UtilityAction
{
private NPC npc;
public FleeAction(NPC npc)
{
this.npc = npc;
}
public override void Enter()
{
Console.WriteLine("Starting to flee!");
}
public override void Execute()
{
Console.WriteLine("Fleeing from player!");
// Move away from player
Vector2 direction = (npc.Position - npc.PlayerPosition).normalized;
npc.Position += direction * 0.15f;
}
public override void Exit()
{
Console.WriteLine("Stopping flee.");
}
}
public class EatAction : UtilityAction
{
private NPC npc;
public EatAction(NPC npc)
{
this.npc = npc;
}
public override void Enter()
{
Console.WriteLine("Starting to eat!");
}
public override void Execute()
{
Console.WriteLine("Eating food!");
// Reduce hunger
npc.Hunger -= 5.0f;
if (npc.Hunger < 0)
npc.Hunger = 0;
}
public override void Exit()
{
Console.WriteLine("Finished eating.");
}
}
public class SleepAction : UtilityAction
{
private NPC npc;
public SleepAction(NPC npc)
{
this.npc = npc;
}
public override void Enter()
{
Console.WriteLine("Going to sleep!");
}
public override void Execute()
{
Console.WriteLine("Sleeping...");
// Restore energy
npc.Energy += 2.0f;
if (npc.Energy > npc.MaxEnergy)
npc.Energy = npc.MaxEnergy;
}
public override void Exit()
{
Console.WriteLine("Waking up.");
}
}
public class IdleAction : UtilityAction
{
private NPC npc;
public IdleAction(NPC npc)
{
this.npc = npc;
}
public override void Enter()
{
Console.WriteLine("Starting to idle.");
}
public override void Execute()
{
Console.WriteLine("Idling...");
// Do nothing
}
public override void Exit()
{
Console.WriteLine("Stopping idle.");
}
}
// Example considerations
public class PlayerProximityConsideration : UtilityConsideration
{
private NPC npc;
private bool preferClose;
public PlayerProximityConsideration(NPC npc, bool preferClose)
{
this.npc = npc;
this.preferClose = preferClose;
}
public override float CalculateScore()
{
float distance = Vector2.Distance(npc.Position, npc.PlayerPosition);
float normalizedDistance = Math.Clamp(distance / 10.0f, 0, 1);
// If we prefer being close to the player, invert the score
if (preferClose)
return 1.0f - normalizedDistance;
else
return normalizedDistance;
}
}
public class HealthConsideration : UtilityConsideration
{
private NPC npc;
private bool preferHighHealth;
public HealthConsideration(NPC npc, bool preferHighHealth)
{
this.npc = npc;
this.preferHighHealth = preferHighHealth;
}
public override float CalculateScore()
{
float healthRatio = npc.Health / npc.MaxHealth;
// If we prefer high health, return the health ratio
if (preferHighHealth)
return healthRatio;
else
return 1.0f - healthRatio;
}
}
public class HungerConsideration : UtilityConsideration
{
private NPC npc;
public HungerConsideration(NPC npc)
{
this.npc = npc;
}
public override float CalculateScore()
{
float hungerRatio = npc.Hunger / npc.MaxHunger;
// The hungrier we are, the higher the score
return hungerRatio;
}
}
public class EnergyConsideration : UtilityConsideration
{
private NPC npc;
public EnergyConsideration(NPC npc)
{
this.npc = npc;
}
public override float CalculateScore()
{
float energyRatio = npc.Energy / npc.MaxEnergy;
// The more tired we are, the higher the score
return 1.0f - energyRatio;
}
}
public class IdleConsideration : UtilityConsideration
{
public override float CalculateScore()
{
// Always return a low score, so idle is only chosen when nothing else is suitable
return 0.1f;
}
}
Goal-Oriented Action Planning (GOAP)
GOAP is a planning system that allows AI to determine a sequence of actions to achieve a goal.
Basic GOAP Implementation
public class WorldState
{
private Dictionary<string, bool> state = new Dictionary<string, bool>();
public WorldState()
{
}
public WorldState(WorldState other)
{
foreach (var kvp in other.state)
{
state[kvp.Key] = kvp.Value;
}
}
public bool GetState(string key)
{
if (state.ContainsKey(key))
return state[key];
return false;
}
public void SetState(string key, bool value)
{
state[key] = value;
}
public bool MeetsConditions(Dictionary<string, bool> conditions)
{
foreach (var condition in conditions)
{
if (!state.ContainsKey(condition.Key) || state[condition.Key] != condition.Value)
return false;
}
return true;
}
}
public class GOAPAction
{
public string Name { get; private set; }
public float Cost { get; private set; }
private Dictionary<string, bool> preconditions = new Dictionary<string, bool>();
private Dictionary<string, bool> effects = new Dictionary<string, bool>();
public GOAPAction(string name, float cost)
{
Name = name;
Cost = cost;
}
public void AddPrecondition(string key, bool value)
{
preconditions[key] = value;
}
public void AddEffect(string key, bool value)
{
effects[key] = value;
}
public bool CheckPreconditions(WorldState state)
{
return state.MeetsConditions(preconditions);
}
public WorldState ApplyEffects(WorldState state)
{
WorldState newState = new WorldState(state);
foreach (var effect in effects)
{
newState.SetState(effect.Key, effect.Value);
}
return newState;
}
public Dictionary<string, bool> GetPreconditions()
{
return preconditions;
}
public Dictionary<string, bool> GetEffects()
{
return effects;
}
public virtual bool Perform()
{
Console.WriteLine($"Performing action: {Name}");
return true;
}
}
public class GOAPPlanner
{
private List<GOAPAction> availableActions = new List<GOAPAction>();
public void AddAction(GOAPAction action)
{
availableActions.Add(action);
}
public List<GOAPAction> Plan(WorldState currentState, Dictionary<string, bool> goal)
{
// Create a goal state
WorldState goalState = new WorldState();
foreach (var kvp in goal)
{
goalState.SetState(kvp.Key, kvp.Value);
}
// Create a priority queue for A* search
var openList = new List<Node>();
var closedList = new HashSet<string>();
// Start with the current state
openList.Add(new Node(null, 0, currentState, null));
while (openList.Count > 0)
{
// Sort the open list by f = g + h
openList.Sort((a, b) => a.F.CompareTo(b.F));
// Get the node with the lowest cost
Node currentNode = openList[0];
openList.RemoveAt(0);
// If the current state meets the goal conditions, we've found a plan
if (currentNode.State.MeetsConditions(goal))
{
// Reconstruct the plan
List<GOAPAction> plan = new List<GOAPAction>();
while (currentNode.Action != null)
{
plan.Add(currentNode.Action);
currentNode = currentNode.Parent;
}
// Reverse the plan to get the correct order
plan.Reverse();
return plan;
}
// Add the current node to the closed list
closedList.Add(GetStateHash(currentNode.State));
// Try each available action
foreach (var action in availableActions)
{
// Skip actions that can't be performed in the current state
if (!action.CheckPreconditions(currentNode.State))
continue;
// Apply the action to get the new state
WorldState newState = action.ApplyEffects(currentNode.State);
// Skip states we've already visited
string newStateHash = GetStateHash(newState);
if (closedList.Contains(newStateHash))
continue;
// Calculate the cost
float g = currentNode.G + action.Cost;
float h = CalculateHeuristic(newState, goal);
float f = g + h;
// Check if this state is already in the open list with a lower cost
bool skip = false;
foreach (var node in openList)
{
if (GetStateHash(node.State) == newStateHash && node.F <= f)
{
skip = true;
break;
}
}
if (skip)
continue;
// Add the new node to the open list
openList.Add(new Node(currentNode, g, newState, action));
}
}
// If we get here, no plan was found
return new List<GOAPAction>();
}
private float CalculateHeuristic(WorldState state, Dictionary<string, bool> goal)
{
// Simple heuristic: count the number of unsatisfied goal conditions
int count = 0;
foreach (var condition in goal)
{
if (!state.GetState(condition.Key) == condition.Value)
count++;
}
return count;
}
private string GetStateHash(WorldState state)
{
// This is a simplified hash function for the state
// In a real implementation, you would want a more robust hash
string hash = "";
foreach (var action in availableActions)
{
foreach (var effect in action.GetEffects())
{
hash += $"{effect.Key}:{state.GetState(effect.Key)},";
}
}
return hash;
}
private class Node
{
public Node Parent { get; }
public float G { get; } // Cost so far
public float H { get; } // Heuristic cost to goal
public float F => G + H; // Total cost
public WorldState State { get; }
public GOAPAction Action { get; }
public Node(Node parent, float g, WorldState state, GOAPAction action)
{
Parent = parent;
G = g;
State = state;
Action = action;
}
}
}
// Example usage for an NPC in a game
public class GOAPAgent
{
private WorldState currentState = new WorldState();
private Dictionary<string, bool> goal = new Dictionary<string, bool>();
private GOAPPlanner planner = new GOAPPlanner();
private List<GOAPAction> currentPlan = new List<GOAPAction>();
private int currentActionIndex = 0;
public GOAPAgent()
{
// Initialize the current state
currentState.SetState("hasWeapon", false);
currentState.SetState("hasAmmo", false);
currentState.SetState("enemyVisible", true);
currentState.SetState("enemyDead", false);
currentState.SetState("atCover", false);
// Set the goal
goal["enemyDead"] = true;
// Create actions
var getWeapon = new GOAPAction("GetWeapon", 1);
getWeapon.AddPrecondition("hasWeapon", false);
getWeapon.AddEffect("hasWeapon", true);
var getAmmo = new GOAPAction("GetAmmo", 1);
getAmmo.AddPrecondition("hasAmmo", false);
getAmmo.AddEffect("hasAmmo", true);
var takeCover = new GOAPAction("TakeCover", 2);
takeCover.AddPrecondition("atCover", false);
takeCover.AddEffect("atCover", true);
var shootEnemy = new GOAPAction("ShootEnemy", 3);
shootEnemy.AddPrecondition("hasWeapon", true);
shootEnemy.AddPrecondition("hasAmmo", true);
shootEnemy.AddPrecondition("enemyVisible", true);
shootEnemy.AddEffect("enemyDead", true);
var flankEnemy = new GOAPAction("FlankEnemy", 4);
flankEnemy.AddPrecondition("enemyVisible", false);
flankEnemy.AddEffect("enemyVisible", true);
// Add actions to the planner
planner.AddAction(getWeapon);
planner.AddAction(getAmmo);
planner.AddAction(takeCover);
planner.AddAction(shootEnemy);
planner.AddAction(flankEnemy);
// Create the initial plan
currentPlan = planner.Plan(currentState, goal);
// Print the plan
Console.WriteLine("Initial Plan:");
foreach (var action in currentPlan)
{
Console.WriteLine($"- {action.Name}");
}
}
public void Update()
{
// If we have no plan or have completed the current plan, create a new one
if (currentPlan.Count == 0 || currentActionIndex >= currentPlan.Count)
{
currentPlan = planner.Plan(currentState, goal);
currentActionIndex = 0;
if (currentPlan.Count == 0)
{
Console.WriteLine("No plan found!");
return;
}
Console.WriteLine("New Plan:");
foreach (var action in currentPlan)
{
Console.WriteLine($"- {action.Name}");
}
}
// Execute the current action
GOAPAction currentAction = currentPlan[currentActionIndex];
if (currentAction.Perform())
{
// Update the world state with the action's effects
currentState = currentAction.ApplyEffects(currentState);
// Move to the next action
currentActionIndex++;
// Check if we've completed the plan
if (currentActionIndex >= currentPlan.Count)
{
Console.WriteLine("Plan completed!");
// Check if the goal has been achieved
if (currentState.MeetsConditions(goal))
{
Console.WriteLine("Goal achieved!");
}
else
{
Console.WriteLine("Goal not achieved, replanning...");
currentPlan = planner.Plan(currentState, goal);
currentActionIndex = 0;
}
}
}
else
{
// If the action failed, replan
Console.WriteLine($"Action {currentAction.Name} failed, replanning...");
currentPlan = planner.Plan(currentState, goal);
currentActionIndex = 0;
}
}
}
Machine Learning for Game AI
Machine learning can be used to create more adaptive and realistic AI behaviors.
Reinforcement Learning Example
public class QLearningAgent
{
private Dictionary<string, Dictionary<string, float>> qTable = new Dictionary<string, Dictionary<string, float>>();
private List<string> actions;
private float learningRate;
private float discountFactor;
private float explorationRate;
private Random random = new Random();
public QLearningAgent(List<string> actions, float learningRate = 0.1f, float discountFactor = 0.9f, float explorationRate = 0.1f)
{
this.actions = actions;
this.learningRate = learningRate;
this.discountFactor = discountFactor;
this.explorationRate = explorationRate;
}
public string ChooseAction(string state)
{
// Ensure the state exists in the Q-table
if (!qTable.ContainsKey(state))
{
qTable[state] = new Dictionary<string, float>();
foreach (var action in actions)
{
qTable[state][action] = 0;
}
}
// Exploration: choose a random action
if (random.NextDouble() < explorationRate)
{
return actions[random.Next(actions.Count)];
}
// Exploitation: choose the best action
return GetBestAction(state);
}
public void Learn(string state, string action, float reward, string nextState)
{
// Ensure the states exist in the Q-table
if (!qTable.ContainsKey(state))
{
qTable[state] = new Dictionary<string, float>();
foreach (var a in actions)
{
qTable[state][a] = 0;
}
}
if (!qTable.ContainsKey(nextState))
{
qTable[nextState] = new Dictionary<string, float>();
foreach (var a in actions)
{
qTable[nextState][a] = 0;
}
}
// Get the current Q-value
float currentQ = qTable[state][action];
// Get the maximum Q-value for the next state
float maxNextQ = qTable[nextState].Values.Max();
// Update the Q-value using the Q-learning formula
float newQ = currentQ + learningRate * (reward + discountFactor * maxNextQ - currentQ);
// Update the Q-table
qTable[state][action] = newQ;
}
private string GetBestAction(string state)
{
// Find the action with the highest Q-value
float maxQ = float.MinValue;
string bestAction = actions[0];
foreach (var action in actions)
{
float q = qTable[state][action];
if (q > maxQ)
{
maxQ = q;
bestAction = action;
}
}
return bestAction;
}
public void SaveQTable(string filename)
{
// In a real implementation, you would serialize the Q-table to a file
Console.WriteLine($"Saving Q-table to {filename}");
}
public void LoadQTable(string filename)
{
// In a real implementation, you would deserialize the Q-table from a file
Console.WriteLine($"Loading Q-table from {filename}");
}
}
// Example usage for a simple game
public class GridWorld
{
private int width, height;
private int playerX, playerY;
private int goalX, goalY;
private List<(int x, int y)> obstacles = new List<(int x, int y)>();
private QLearningAgent agent;
public GridWorld(int width, int height)
{
this.width = width;
this.height = height;
// Set up the player and goal positions
playerX = 0;
playerY = 0;
goalX = width - 1;
goalY = height - 1;
// Add some obstacles
obstacles.Add((1, 1));
obstacles.Add((2, 2));
obstacles.Add((3, 1));
// Create the agent
agent = new QLearningAgent(new List<string> { "up", "down", "left", "right" });
}
public void Train(int episodes)
{
for (int episode = 0; episode < episodes; episode++)
{
// Reset the player position
playerX = 0;
playerY = 0;
bool done = false;
int steps = 0;
while (!done && steps < 100)
{
// Get the current state
string state = $"{playerX},{playerY}";
// Choose an action
string action = agent.ChooseAction(state);
// Take the action
int newPlayerX = playerX;
int newPlayerY = playerY;
switch (action)
{
case "up":
newPlayerY = Math.Max(0, playerY - 1);
break;
case "down":
newPlayerY = Math.Min(height - 1, playerY + 1);
break;
case "left":
newPlayerX = Math.Max(0, playerX - 1);
break;
case "right":
newPlayerX = Math.Min(width - 1, playerX + 1);
break;
}
// Check if the new position is valid
bool isObstacle = obstacles.Contains((newPlayerX, newPlayerY));
if (!isObstacle)
{
playerX = newPlayerX;
playerY = newPlayerY;
}
// Calculate the reward
float reward = -0.1f; // Small negative reward for each step
if (playerX == goalX && playerY == goalY)
{
reward = 1.0f; // Positive reward for reaching the goal
done = true;
}
else if (isObstacle)
{
reward = -0.5f; // Negative reward for hitting an obstacle
}
// Get the new state
string newState = $"{playerX},{playerY}";
// Learn from this experience
agent.Learn(state, action, reward, newState);
steps++;
}
if (episode % 100 == 0)
{
Console.WriteLine($"Episode {episode}: {(done ? "Reached goal" : "Failed")} in {steps} steps");
}
}
// Save the learned Q-table
agent.SaveQTable("gridworld_qtable.json");
}
public void Play()
{
// Load the learned Q-table
agent.LoadQTable("gridworld_qtable.json");
// Reset the player position
playerX = 0;
playerY = 0;
bool done = false;
int steps = 0;
Console.WriteLine("Starting game...");
while (!done && steps < 100)
{
// Print the current state
PrintGrid();
// Get the current state
string state = $"{playerX},{playerY}";
// Choose the best action
string action = agent.ChooseAction(state);
Console.WriteLine($"Taking action: {action}");
// Take the action
switch (action)
{
case "up":
playerY = Math.Max(0, playerY - 1);
break;
case "down":
playerY = Math.Min(height - 1, playerY + 1);
break;
case "left":
playerX = Math.Max(0, playerX - 1);
break;
case "right":
playerX = Math.Min(width - 1, playerX + 1);
break;
}
// Check if we've reached the goal
if (playerX == goalX && playerY == goalY)
{
done = true;
Console.WriteLine("Reached the goal!");
}
steps++;
// Add a small delay to make it easier to follow
Thread.Sleep(500);
}
if (!done)
{
Console.WriteLine("Failed to reach the goal within the step limit.");
}
// Print the final state
PrintGrid();
}
private void PrintGrid()
{
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
if (x == playerX && y == playerY)
Console.Write("P ");
else if (x == goalX && y == goalY)
Console.Write("G ");
else if (obstacles.Contains((x, y)))
Console.Write("X ");
else
Console.Write(". ");
}
Console.WriteLine();
}
Console.WriteLine();
}
}
Conclusion
Game AI is a fascinating field that combines techniques from computer science, psychology, and game design to create engaging and believable behaviors for non-player characters and other game entities.
The implementations provided in this section are simplified for clarity but demonstrate the core concepts of game AI. In practice, these techniques can be combined and extended to create more complex and interesting behaviors.
When implementing AI in your games, remember to:
- Focus on Player Experience: The primary goal of game AI is to enhance the player's experience, not to create the most intelligent AI possible.
- Balance Challenge and Fairness: AI should provide an appropriate level of challenge without feeling unfair or cheating.
- Create Believable Behaviors: AI should behave in ways that make sense to the player, even if they're not optimal.
- Optimize for Performance: Game AI must run efficiently to maintain smooth gameplay.
- Test with Real Players: The ultimate test of game AI is how players perceive and interact with it.