Skip to main content

4 - Minimum Spanning Tree Algorithms

A Minimum Spanning Tree (MST) is a subset of the edges of a connected, undirected graph that connects all vertices together with the minimum possible total edge weight. MSTs have numerous applications in network design, clustering, and approximation algorithms.

Properties of Minimum Spanning Trees

  1. Connectivity: An MST connects all vertices in the graph.
  2. Acyclicity: An MST is a tree, so it contains no cycles.
  3. Minimality: The sum of edge weights is minimized.
  4. Uniqueness: If all edge weights are distinct, the MST is unique.
  5. Number of Edges: An MST of a graph with n vertices has exactly n-1 edges.

Kruskal's Algorithm

Kruskal's algorithm builds the MST by adding edges in order of increasing weight, skipping edges that would create a cycle.

How Kruskal's Algorithm Works

  1. Sort all edges in non-decreasing order of their weight.
  2. Initialize an empty MST.
  3. Iterate through the sorted edges: a. If adding the current edge doesn't create a cycle, add it to the MST. b. Otherwise, discard the edge.
  4. Continue until the MST has n-1 edges (where n is the number of vertices).

Implementation

/// <summary>
/// Represents an edge in a graph, connecting two vertices with a weight.
/// </summary>
public class Edge : IComparable<Edge>
{
/// <summary>
/// Gets or sets the source vertex of the edge.
/// </summary>
public int Source { get; set; }

/// <summary>
/// Gets or sets the destination vertex of the edge.
/// </summary>
public int Destination { get; set; }

/// <summary>
/// Gets or sets the weight of the edge.
/// </summary>
public int Weight { get; set; }

/// <summary>
/// Initializes a new instance of the Edge class with the specified source, destination, and weight.
/// </summary>
/// <param name="source">The source vertex of the edge.</param>
/// <param name="destination">The destination vertex of the edge.</param>
/// <param name="weight">The weight of the edge.</param>
public Edge(int source, int destination, int weight)
{
Source = source;
Destination = destination;
Weight = weight;
}

/// <summary>
/// Compares the current edge with another edge based on their weights.
/// </summary>
/// <param name="other">The edge to compare with this edge.</param>
/// <returns>
/// A value indicating the relative order of the edges being compared.
/// Less than zero: this edge's weight is less than the other edge's weight.
/// Zero: this edge's weight equals the other edge's weight.
/// Greater than zero: this edge's weight is greater than the other edge's weight.
/// </returns>
public int CompareTo(Edge other)
{
return Weight.CompareTo(other.Weight);
}

/// <summary>
/// Returns a string representation of the edge.
/// </summary>
/// <returns>A string in the format "Source -- Weight --> Destination".</returns>
public override string ToString()
{
return $"{Source} -- {Weight} --> {Destination}";
}
}

/// <summary>
/// Implements a disjoint-set data structure (also known as a union-find data structure)
/// for efficiently tracking a set of elements partitioned into disjoint subsets.
/// </summary>
public class DisjointSet
{
private readonly int[] parent; // Parent of each element
private readonly int[] rank; // Rank of each element (used for union by rank)

/// <summary>
/// Initializes a new instance of the DisjointSet class with n elements.
/// </summary>
/// <param name="n">The number of elements in the set.</param>
/// <exception cref="ArgumentOutOfRangeException">
/// Thrown when n is negative.
/// </exception>
public DisjointSet(int n)
{
if (n < 0)
{
throw new ArgumentOutOfRangeException(nameof(n), "Number of elements must be non-negative.");
}

parent = new int[n];
rank = new int[n];

// Initialize each element as its own parent with rank 0
for (int i = 0; i < n; i++)
{
parent[i] = i;
rank[i] = 0;
}
}

/// <summary>
/// Finds the representative (root) of the set containing element x.
/// Uses path compression to optimize future operations.
/// </summary>
/// <param name="x">The element to find the representative for.</param>
/// <returns>The representative of the set containing x.</returns>
/// <exception cref="ArgumentOutOfRangeException">
/// Thrown when x is outside the valid range.
/// </exception>
public int Find(int x)
{
if (x < 0 || x >= parent.Length)
{
throw new ArgumentOutOfRangeException(nameof(x),
$"Element must be between 0 and {parent.Length - 1}.");
}

// Path compression: make every examined node point directly to the root
if (parent[x] != x)
{
parent[x] = Find(parent[x]);
}

return parent[x];
}

/// <summary>
/// Merges the sets containing elements x and y.
/// Uses union by rank to keep the tree balanced.
/// </summary>
/// <param name="x">The first element.</param>
/// <param name="y">The second element.</param>
/// <exception cref="ArgumentOutOfRangeException">
/// Thrown when either x or y is outside the valid range.
/// </exception>
public void Union(int x, int y)
{
// Find the roots of the sets containing x and y
int rootX = Find(x);
int rootY = Find(y);

// If x and y are already in the same set, do nothing
if (rootX == rootY)
{
return;
}

// Union by rank: attach the smaller rank tree under the root of the higher rank tree
if (rank[rootX] < rank[rootY])
{
// Make rootY the parent of rootX
parent[rootX] = rootY;
}
else if (rank[rootX] > rank[rootY])
{
// Make rootX the parent of rootY
parent[rootY] = rootX;
}
else
{
// If ranks are the same, make one the parent and increment its rank
parent[rootY] = rootX;
rank[rootX]++;
}
}

/// <summary>
/// Determines whether two elements are in the same set.
/// </summary>
/// <param name="x">The first element.</param>
/// <param name="y">The second element.</param>
/// <returns>true if x and y are in the same set; otherwise, false.</returns>
public bool AreConnected(int x, int y)
{
return Find(x) == Find(y);
}
}

/// <summary>
/// Represents a graph and provides methods for finding the Minimum Spanning Tree (MST)
/// using Kruskal's algorithm.
/// </summary>
public class Graph
{
private readonly int vertices; // Number of vertices
private readonly List<Edge> edges;

/// <summary>
/// Initializes a new instance of the Graph class with the specified number of vertices.
/// </summary>
/// <param name="vertices">The number of vertices in the graph.</param>
/// <exception cref="ArgumentOutOfRangeException">
/// Thrown when the number of vertices is negative.
/// </exception>
public Graph(int vertices)
{
if (vertices < 0)
{
throw new ArgumentOutOfRangeException(nameof(vertices), "Number of vertices must be non-negative.");
}

this.vertices = vertices;
edges = new List<Edge>();
}

/// <summary>
/// Adds an edge to the graph.
/// </summary>
/// <param name="source">The source vertex of the edge.</param>
/// <param name="destination">The destination vertex of the edge.</param>
/// <param name="weight">The weight of the edge.</param>
/// <exception cref="ArgumentOutOfRangeException">
/// Thrown when either source or destination is outside the valid range.
/// </exception>
public void AddEdge(int source, int destination, int weight)
{
ValidateVertex(source);
ValidateVertex(destination);

edges.Add(new Edge(source, destination, weight));
}

/// <summary>
/// Finds the Minimum Spanning Tree (MST) of the graph using Kruskal's algorithm.
/// </summary>
/// <returns>A list of edges that form the MST.</returns>
public List<Edge> KruskalMST()
{
List<Edge> result = new List<Edge>();

// Sort edges by weight in non-decreasing order
// This is a key step in Kruskal's algorithm
edges.Sort();

// Create a disjoint set to detect cycles
DisjointSet disjointSet = new DisjointSet(vertices);

int edgeCount = 0; // Number of edges added to the MST
int index = 0; // Current edge index

// A MST has exactly (V-1) edges, where V is the number of vertices
// Continue until we have (V-1) edges or we've examined all edges
while (edgeCount < vertices - 1 && index < edges.Count)
{
// Get the next edge with the smallest weight
Edge nextEdge = edges[index++];

// Find the sets containing the source and destination vertices
int sourceRoot = disjointSet.Find(nextEdge.Source);
int destinationRoot = disjointSet.Find(nextEdge.Destination);

// If including this edge doesn't create a cycle (source and destination are in different sets)
if (sourceRoot != destinationRoot)
{
// Add the edge to the MST
result.Add(nextEdge);

// Merge the sets containing the source and destination vertices
disjointSet.Union(sourceRoot, destinationRoot);

// Increment the edge count
edgeCount++;
}
// If the edge would create a cycle, skip it
}

return result;
}

/// <summary>
/// Calculates the total weight of the Minimum Spanning Tree (MST).
/// </summary>
/// <param name="mst">The list of edges in the MST.</param>
/// <returns>The sum of the weights of all edges in the MST.</returns>
/// <exception cref="ArgumentNullException">
/// Thrown when the MST is null.
/// </exception>
public int GetMSTWeight(List<Edge> mst)
{
if (mst == null)
{
throw new ArgumentNullException(nameof(mst), "MST cannot be null.");
}

int totalWeight = 0;

foreach (Edge edge in mst)
{
totalWeight += edge.Weight;
}

return totalWeight;
}

/// <summary>
/// Gets the number of vertices in the graph.
/// </summary>
public int VertexCount => vertices;

/// <summary>
/// Gets the number of edges in the graph.
/// </summary>
public int EdgeCount => edges.Count;

/// <summary>
/// Validates that a vertex index is within the valid range.
/// </summary>
/// <param name="vertex">The vertex index to validate.</param>
/// <exception cref="ArgumentOutOfRangeException">
/// Thrown when the vertex is outside the valid range.
/// </exception>
private void ValidateVertex(int vertex)
{
if (vertex < 0 || vertex >= vertices)
{
throw new ArgumentOutOfRangeException(nameof(vertex),
$"Vertex must be between 0 and {vertices - 1}.");
}
}
}

Time and Space Complexity

  • Time Complexity: O(E log E) or O(E log V) where E is the number of edges and V is the number of vertices. The dominant operation is sorting the edges.
  • Space Complexity: O(E + V) for storing the edges and the disjoint set.

Applications

  1. Network Design: Designing minimum-cost networks.
  2. Cluster Analysis: Identifying clusters in data.
  3. Circuit Design: Minimizing wire length in circuits.
  4. Approximation Algorithms: Used as a subroutine in approximation algorithms for other problems.

Prim's Algorithm

Prim's algorithm builds the MST by growing a single tree from a starting vertex, adding the minimum-weight edge that connects a vertex in the tree to a vertex outside the tree.

How Prim's Algorithm Works

  1. Start with any vertex as a single-vertex tree.
  2. Repeatedly add the minimum-weight edge that connects the tree to a vertex not yet in the tree.
  3. Continue until all vertices are included in the tree.

Implementation

public class Graph
{
private int V; // Number of vertices
private List<(int vertex, int weight)>[] adjacencyList;

public Graph(int v)
{
V = v;
adjacencyList = new List<(int, int)>[v];

for (int i = 0; i < v; i++)
{
adjacencyList[i] = new List<(int, int)>();
}
}

public void AddEdge(int u, int v, int weight)
{
adjacencyList[u].Add((v, weight));
adjacencyList[v].Add((u, weight)); // For undirected graph
}

public List<Edge> PrimMST()
{
// Array to store constructed MST
int[] parent = new int[V];

// Key values used to pick minimum weight edge
int[] key = new int[V];

// To represent set of vertices included in MST
bool[] mstSet = new bool[V];

// Initialize all keys as INFINITE
for (int i = 0; i < V; i++)
{
key[i] = int.MaxValue;
mstSet[i] = false;
}

// Always include first vertex in MST
key[0] = 0; // Make key 0 so that this vertex is picked as first vertex
parent[0] = -1; // First node is always root of MST

// The MST will have V vertices
for (int count = 0; count < V - 1; count++)
{
// Pick the minimum key vertex from the set of vertices not yet included in MST
int u = MinKey(key, mstSet);

// Add the picked vertex to the MST Set
mstSet[u] = true;

// Update key value and parent index of the adjacent vertices of the picked vertex
foreach (var edge in adjacencyList[u])
{
int v = edge.vertex;
int weight = edge.weight;

// Update the key only if v is not in mstSet, there is an edge from u to v,
// and weight of (u,v) is smaller than current key of v
if (!mstSet[v] && weight < key[v])
{
parent[v] = u;
key[v] = weight;
}
}
}

// Construct the MST
List<Edge> result = new List<Edge>();
for (int i = 1; i < V; i++)
{
result.Add(new Edge(parent[i], i, key[i]));
}

return result;
}

private int MinKey(int[] key, bool[] mstSet)
{
int min = int.MaxValue;
int minIndex = -1;

for (int v = 0; v < V; v++)
{
if (!mstSet[v] && key[v] < min)
{
min = key[v];
minIndex = v;
}
}

return minIndex;
}

// Optimized version using priority queue
public List<Edge> PrimMSTOptimized()
{
// Array to store constructed MST
int[] parent = new int[V];

// Key values used to pick minimum weight edge
int[] key = new int[V];

// To represent set of vertices included in MST
bool[] inMST = new bool[V];

// Initialize all keys as INFINITE
for (int i = 0; i < V; i++)
{
key[i] = int.MaxValue;
inMST[i] = false;
}

// Always include first vertex in MST
key[0] = 0; // Make key 0 so that this vertex is picked as first vertex
parent[0] = -1; // First node is always root of MST

// Use priority queue to get minimum key vertex
PriorityQueue<int, int> pq = new PriorityQueue<int, int>();
pq.Enqueue(0, 0); // Insert source with key 0

while (pq.Count > 0)
{
// Extract minimum key vertex
int u = pq.Dequeue();

// If vertex is already in MST, skip it
if (inMST[u])
continue;

// Include vertex in MST
inMST[u] = true;

// Update key values of adjacent vertices
foreach (var edge in adjacencyList[u])
{
int v = edge.vertex;
int weight = edge.weight;

// If v is not in MST and weight of (u,v) is smaller than current key of v
if (!inMST[v] && weight < key[v])
{
// Update key of v
key[v] = weight;
parent[v] = u;

// Add v to priority queue
pq.Enqueue(v, key[v]);
}
}
}

// Construct the MST
List<Edge> result = new List<Edge>();
for (int i = 1; i < V; i++)
{
result.Add(new Edge(parent[i], i, key[i]));
}

return result;
}
}

Time and Space Complexity

  • Time Complexity:
    • Basic implementation: O(V²)
    • With binary heap: O(E log V)
  • Space Complexity: O(V) for the key, parent, and mstSet arrays.

Applications

  1. Network Design: Similar to Kruskal's algorithm.
  2. Traveling Salesman Problem: Used in approximation algorithms.
  3. Clustering: Identifying clusters by removing edges from the MST.
  4. Image Segmentation: Segmenting images based on pixel similarity.

Comparison of Kruskal's and Prim's Algorithms

AspectKruskal's AlgorithmPrim's Algorithm
ApproachAdds the next lightest edge that doesn't create a cycleGrows a tree from a starting vertex
Time ComplexityO(E log E)O(E log V) with binary heap
Space ComplexityO(E + V)O(V)
Best ForSparse graphs (E ≈ V)Dense graphs (E ≈ V²)
Data StructureDisjoint SetPriority Queue
Edge ProcessingProcesses edges in sorted orderProcesses edges adjacent to the current tree

Borůvka's Algorithm

Borůvka's algorithm is another approach to finding the MST, which works by merging forests of trees.

How Borůvka's Algorithm Works

  1. Start with each vertex as a separate component.
  2. For each component, find the minimum-weight edge that connects it to another component.
  3. Add all these minimum-weight edges to the MST.
  4. Merge the connected components.
  5. Repeat until there is only one component.

Implementation

public class Graph
{
private int V; // Number of vertices
private List<Edge> edges;

public Graph(int v)
{
V = v;
edges = new List<Edge>();
}

public void AddEdge(int source, int destination, int weight)
{
edges.Add(new Edge(source, destination, weight));
}

public List<Edge> BoruvkaMST()
{
List<Edge> result = new List<Edge>();

// Create disjoint set
DisjointSet ds = new DisjointSet(V);

// Initialize components (each vertex is a component)
int[] cheapest = new int[V];

// Initially, there are V different trees
int numTrees = V;

// Keep combining components until all vertices are in one MST
while (numTrees > 1)
{
// Initialize cheapest array
for (int i = 0; i < V; i++)
{
cheapest[i] = -1;
}

// For each edge, find if it's the cheapest edge connecting two components
for (int i = 0; i < edges.Count; i++)
{
int set1 = ds.Find(edges[i].Source);
int set2 = ds.Find(edges[i].Destination);

// If the two vertices belong to different components
if (set1 != set2)
{
// If this is the first edge for component set1 or it's cheaper than the current cheapest
if (cheapest[set1] == -1 ||
edges[cheapest[set1]].Weight > edges[i].Weight)
{
cheapest[set1] = i;
}

// If this is the first edge for component set2 or it's cheaper than the current cheapest
if (cheapest[set2] == -1 ||
edges[cheapest[set2]].Weight > edges[i].Weight)
{
cheapest[set2] = i;
}
}
}

// Add the cheapest edges to the MST
for (int i = 0; i < V; i++)
{
if (cheapest[i] != -1)
{
int set1 = ds.Find(edges[cheapest[i]].Source);
int set2 = ds.Find(edges[cheapest[i]].Destination);

if (set1 != set2)
{
result.Add(edges[cheapest[i]]);
ds.Union(set1, set2);
numTrees--;
}
}
}
}

return result;
}
}

Time and Space Complexity

  • Time Complexity: O(E log V) where E is the number of edges and V is the number of vertices.
  • Space Complexity: O(E + V) for storing the edges and the disjoint set.

Applications

Similar to Kruskal's and Prim's algorithms, Borůvka's algorithm is used for network design, clustering, and other MST applications. It's particularly useful in parallel computing environments.

Advanced MST Techniques

Reverse-Delete Algorithm

The Reverse-Delete algorithm is the reverse of Kruskal's algorithm. It starts with the full graph and removes edges in order of decreasing weight, as long as removing an edge doesn't disconnect the graph.

public List<Edge> ReverseDeleteMST()
{
// Start with all edges
List<Edge> result = new List<Edge>(edges);

// Sort edges in descending order of weight
result.Sort((a, b) => b.Weight.CompareTo(a.Weight));

// Create a graph with all edges
Graph g = new Graph(V);
foreach (Edge edge in result)
{
g.AddEdge(edge.Source, edge.Destination, edge.Weight);
}

for (int i = 0; i < result.Count; i++)
{
Edge edge = result[i];

// Remove the edge
g.RemoveEdge(edge.Source, edge.Destination);

// Check if the graph is still connected
if (!g.IsConnected())
{
// If not, add the edge back
g.AddEdge(edge.Source, edge.Destination, edge.Weight);
}
else
{
// If still connected, remove the edge from the result
result.RemoveAt(i);
i--; // Adjust index after removal
}
}

return result;
}

Randomized MST Algorithms

Randomized algorithms for MST construction can provide good average-case performance and are useful in certain contexts.

Dynamic MST Algorithms

Dynamic MST algorithms efficiently update the MST when the graph changes (edges are added or removed).

Applications and Examples

Network Design

public class NetworkDesigner
{
private Graph network;

public NetworkDesigner(int numNodes)
{
network = new Graph(numNodes);
}

public void AddPotentialConnection(int node1, int node2, int cost)
{
network.AddEdge(node1, node2, cost);
}

public List<Edge> DesignMinimumCostNetwork()
{
return network.KruskalMST();
}

public int CalculateTotalCost(List<Edge> design)
{
int totalCost = 0;
foreach (Edge edge in design)
{
totalCost += edge.Weight;
}
return totalCost;
}
}

Clustering

public class Clustering
{
private Graph graph;
private int V;

public Clustering(int numPoints)
{
V = numPoints;
graph = new Graph(numPoints);
}

public void AddDistance(int point1, int point2, int distance)
{
graph.AddEdge(point1, point2, distance);
}

public List<List<int>> CreateClusters(int k)
{
// Get the MST
List<Edge> mst = graph.KruskalMST();

// Sort edges by weight in descending order
mst.Sort((a, b) => b.Weight.CompareTo(a.Weight));

// Remove k-1 edges to create k clusters
for (int i = 0; i < k - 1 && i < mst.Count; i++)
{
mst.RemoveAt(0);
}

// Create disjoint set to represent clusters
DisjointSet ds = new DisjointSet(V);

// Union vertices connected by remaining edges
foreach (Edge edge in mst)
{
ds.Union(edge.Source, edge.Destination);
}

// Collect clusters
Dictionary<int, List<int>> clusters = new Dictionary<int, List<int>>();

for (int i = 0; i < V; i++)
{
int root = ds.Find(i);

if (!clusters.ContainsKey(root))
{
clusters[root] = new List<int>();
}

clusters[root].Add(i);
}

return clusters.Values.ToList();
}
}

Image Segmentation

public class ImageSegmenter
{
private int width;
private int height;
private Graph graph;

public ImageSegmenter(int width, int height)
{
this.width = width;
this.height = height;
graph = new Graph(width * height);
}

public void AddPixelDifference(int x1, int y1, int x2, int y2, int difference)
{
int vertex1 = y1 * width + x1;
int vertex2 = y2 * width + x2;

graph.AddEdge(vertex1, vertex2, difference);
}

public List<List<(int x, int y)>> SegmentImage(int numSegments)
{
// Get the MST
List<Edge> mst = graph.KruskalMST();

// Sort edges by weight in descending order
mst.Sort((a, b) => b.Weight.CompareTo(a.Weight));

// Remove numSegments-1 edges to create numSegments segments
for (int i = 0; i < numSegments - 1 && i < mst.Count; i++)
{
mst.RemoveAt(0);
}

// Create disjoint set to represent segments
DisjointSet ds = new DisjointSet(width * height);

// Union vertices connected by remaining edges
foreach (Edge edge in mst)
{
ds.Union(edge.Source, edge.Destination);
}

// Collect segments
Dictionary<int, List<(int x, int y)>> segments = new Dictionary<int, List<(int x, int y)>>();

for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
int vertex = y * width + x;
int root = ds.Find(vertex);

if (!segments.ContainsKey(root))
{
segments[root] = new List<(int x, int y)>();
}

segments[root].Add((x, y));
}
}

return segments.Values.ToList();
}
}

Best Practices and Optimization

Choosing the Right MST Algorithm

  • Use Kruskal's Algorithm when:

    • The graph is sparse (E ≈ V).
    • You can efficiently sort the edges.
    • You have an efficient disjoint-set data structure.
  • Use Prim's Algorithm when:

    • The graph is dense (E ≈ V²).
    • You have an efficient priority queue implementation.
    • You're starting from a specific vertex.
  • Use Borůvka's Algorithm when:

    • You need a parallel implementation.
    • The graph has a unique minimum-weight edge for each vertex.

Optimizing MST Algorithms

  1. Efficient Data Structures: Use optimized implementations of disjoint-set (for Kruskal's) and priority queue (for Prim's).
  2. Edge Filtering: For sparse graphs, consider filtering out obviously non-MST edges before running the algorithm.
  3. Incremental Updates: For dynamic graphs, use incremental update algorithms instead of recomputing the entire MST.
  4. Parallelization: Consider parallel implementations, especially for Borůvka's algorithm.
  5. Memory Management: For very large graphs, consider external memory algorithms or streaming approaches.

Conclusion

Minimum Spanning Tree algorithms are fundamental tools for solving a wide range of problems involving network design, clustering, and optimization. By understanding the characteristics, strengths, and limitations of each algorithm, you can choose the most appropriate one for your specific requirements.

Whether you're designing a telecommunications network, clustering data points, or segmenting images, these algorithms provide efficient solutions for finding the optimal tree structure that connects all vertices with minimum total weight. The advanced techniques and optimizations discussed can help you tackle even the most challenging MST problems in large and complex graphs.