7.3 - Concurrency and Asynchronous Programming

Modern applications often need to perform multiple operations simultaneously or handle long-running operations without blocking the main execution thread. C# provides robust tools for concurrency and asynchronous programming, enabling developers to build responsive, scalable applications that efficiently utilize system resources.

🔰 Beginner's Corner: What is Concurrency?

Think of concurrency like a restaurant kitchen with multiple chefs working at the same time:

┌───────────────────────────────────────────────────┐
│                                                   │
│  SYNCHRONOUS (Single-Threaded)                    │
│                                                   │
│  ┌─────┐     ┌─────┐     ┌─────┐     ┌─────┐     │
│  │Task1│ ──► │Task2│ ──► │Task3│ ──► │Task4│     │
│  └─────┘     └─────┘     └─────┘     └─────┘     │
│                                                   │
│  One chef cooking one dish at a time              │
│                                                   │
│  CONCURRENT (Multi-Threaded)                      │
│                                                   │
│  ┌─────┐                                          │
│  │Task1│                                          │
│  └─────┘                                          │
│     │                                             │
│     │     ┌─────┐                                 │
│     ├────►│Task2│                                 │
│     │     └─────┘                                 │
│     │                                             │
│     │     ┌─────┐                                 │
│     ├────►│Task3│                                 │
│     │     └─────┘                                 │
│     │                                             │
│     │     ┌─────┐                                 │
│     └────►│Task4│                                 │
│           └─────┘                                 │
│                                                   │
│  Multiple chefs preparing different dishes        │
│  simultaneously                                   │
│                                                   │
└───────────────────────────────────────────────────┘

💡 Concept Breakdown: Concurrency vs. Asynchronous Programming

These two related concepts are often confused:

1. Concurrency (Multithreading) - Doing multiple things at the same time

   • Like having multiple chefs in a kitchen
   • Uses multiple threads (CPU cores) for parallel execution
   • Good for CPU-intensive tasks (calculations, processing)

2. Asynchronous Programming - Starting something and not waiting for it to finish

   • Like putting bread in a toaster and doing other things while it toasts
   • Doesn't necessarily use multiple threads
   • Great for I/O-bound tasks (network requests, file operations)

🌟 Real-World Analogy: The Restaurant Kitchen

Imagine you're running a restaurant:

• Synchronous (Single-threaded): One chef does everything in sequence - prepares appetizer, then main course, then dessert. Customers wait a long time.

• Concurrent (Multithreaded): Multiple chefs work simultaneously - one prepares appetizers, another makes main courses, a third handles desserts. Much more efficient!

• Asynchronous: A chef puts a dish in the oven and, instead of standing there waiting, starts preparing the next dish. When the oven timer beeps, they go back to finish the first dish.

7.3.1 - Multithreading Basics

Multithreading allows applications to execute multiple threads concurrently, improving performance and responsiveness.

7.3.1.1 - Understanding Threads

A thread is the smallest unit of execution within a process. Each thread has its own execution path and stack but shares the process's memory space with other threads.

// Creating and starting a thread
Thread thread = new Thread(() =>
{
    Console.WriteLine("Thread is running");
    Thread.Sleep(1000);
    Console.WriteLine("Thread completed");
});

// Start the thread
thread.Start();

// Wait for the thread to complete
thread.Join();
Console.WriteLine("Main thread continues");

7.3.1.2 - Thread Properties and Methods

// Create a thread with a name
Thread thread = new Thread(() => ProcessData())
{
    Name = "DataProcessingThread",
    IsBackground = true  // Background thread will terminate when the main thread exits
};

// Get the current thread
Thread currentThread = Thread.CurrentThread;
Console.WriteLine($"Current thread ID: {currentThread.ManagedThreadId}");
Console.WriteLine($"Current thread name: {currentThread.Name ?? "Unnamed"}");

// Thread priority
thread.Priority = ThreadPriority.AboveNormal;

// Thread state
Console.WriteLine($"Thread state: {thread.ThreadState}");

7.3.1.3 - Thread Pool

The thread pool provides a set of worker threads that can be used to execute tasks, reducing the overhead of thread creation and destruction.

// Queue a work item to the thread pool
ThreadPool.QueueUserWorkItem(state =>
{
    Console.WriteLine("Work item executing on thread pool");
    Thread.Sleep(1000);
    Console.WriteLine("Work item completed");
});

// Get thread pool information
ThreadPool.GetMaxThreads(out int maxWorkerThreads, out int maxCompletionPortThreads);
Console.WriteLine($"Max worker threads: {maxWorkerThreads}");
Console.WriteLine($"Max completion port threads: {maxCompletionPortThreads}");

7.3.2 - Thread Safety and Synchronization

When multiple threads access shared resources, synchronization mechanisms are needed to prevent race conditions and ensure data integrity.

7.3.2.1 - Race Conditions

A race condition occurs when multiple threads access shared data concurrently, and at least one thread modifies the data.

// Example of a race condition
public class Counter
{
    private int _count = 0;
    
    // Not thread-safe
    public void Increment()
    {
        _count++;  // This is not an atomic operation
    }
    
    public int Count => _count;
}

// Usage that may lead to race conditions
var counter = new Counter();
var tasks = new List<Task>();

for (int i = 0; i < 1000; i++)
{
    tasks.Add(Task.Run(() => counter.Increment()));
}

Task.WaitAll(tasks.ToArray());
Console.WriteLine($"Count: {counter.Count}");  // May be less than 1000

7.3.2.2 - Lock Statement

The lock statement provides a simple way to synchronize access to a shared resource.

public class ThreadSafeCounter
{
    private int _count = 0;
    private readonly object _lock = new object();
    
    // Thread-safe
    public void Increment()
    {
        lock (_lock)
        {
            _count++;
        }
    }
    
    public int Count
    {
        get
        {
            lock (_lock)
            {
                return _count;
            }
        }
    }
}

7.3.2.3 - Monitor Class

The Monitor class provides more control over locking than the lock statement.

public void ProcessData()
{
    object lockObject = new object();
    
    try
    {
        Monitor.Enter(lockObject);
        // Critical section - only one thread can execute this at a time
        // Process data...
    }
    finally
    {
        Monitor.Exit(lockObject);
    }
}

// Monitor with timeout
public bool TryProcessData(int timeoutMilliseconds)
{
    object lockObject = new object();
    
    if (Monitor.TryEnter(lockObject, timeoutMilliseconds))
    {
        try
        {
            // Critical section
            return true;
        }
        finally
        {
            Monitor.Exit(lockObject);
        }
    }
    
    return false;  // Could not acquire the lock within the timeout
}

7.3.2.4 - Interlocked Operations

The Interlocked class provides atomic operations for simple variables.

public class AtomicCounter
{
    private int _count = 0;
    
    public void Increment()
    {
        Interlocked.Increment(ref _count);
    }
    
    public void Decrement()
    {
        Interlocked.Decrement(ref _count);
    }
    
    public int Count => Interlocked.CompareExchange(ref _count, 0, 0);
}

// Other Interlocked operations
public void InterlockedExamples()
{
    int value = 5;
    
    // Add
    Interlocked.Add(ref value, 10);  // value = 15
    
    // Exchange
    int original = Interlocked.Exchange(ref value, 20);  // original = 15, value = 20
    
    // Compare and exchange
    int comparand = 20;
    int newValue = 30;
    int result = Interlocked.CompareExchange(ref value, newValue, comparand);
    // If value == comparand, then value becomes newValue
    // result contains the original value
}

7.3.2.5 - ReaderWriterLockSlim

ReaderWriterLockSlim allows multiple readers but exclusive writers, optimizing for scenarios with frequent reads and infrequent writes.

public class ThreadSafeCache<TKey, TValue>
{
    private readonly Dictionary<TKey, TValue> _cache = new Dictionary<TKey, TValue>();
    private readonly ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();
    
    public TValue GetOrAdd(TKey key, Func<TKey, TValue> valueFactory)
    {
        // Try to get with a read lock first
        _lock.EnterReadLock();
        try
        {
            if (_cache.TryGetValue(key, out TValue value))
            {
                return value;
            }
        }
        finally
        {
            _lock.ExitReadLock();
        }
        
        // If not found, acquire a write lock
        _lock.EnterWriteLock();
        try
        {
            // Check again in case another thread added the value
            if (_cache.TryGetValue(key, out TValue value))
            {
                return value;
            }
            
            // Add the new value
            value = valueFactory(key);
            _cache[key] = value;
            return value;
        }
        finally
        {
            _lock.ExitWriteLock();
        }
    }
    
    public bool TryGetValue(TKey key, out TValue value)
    {
        _lock.EnterReadLock();
        try
        {
            return _cache.TryGetValue(key, out value);
        }
        finally
        {
            _lock.ExitReadLock();
        }
    }
    
    public void Dispose()
    {
        _lock.Dispose();
    }
}

7.3.2.6 - Mutex and Semaphore

Mutex and Semaphore provide synchronization across processes and control access to limited resources.

// Mutex example (cross-process synchronization)
public void MutexExample()
{
    // Create or open a named mutex
    using (var mutex = new Mutex(false, "MyApplicationMutex"))
    {
        // Wait to acquire the mutex
        bool acquired = mutex.WaitOne(TimeSpan.FromSeconds(5));
        
        if (acquired)
        {
            try
            {
                // Critical section
                Console.WriteLine("Mutex acquired, executing critical section");
            }
            finally
            {
                // Release the mutex
                mutex.ReleaseMutex();
            }
        }
        else
        {
            Console.WriteLine("Could not acquire mutex within timeout");
        }
    }
}

// Semaphore example (limiting concurrent access)
public void SemaphoreExample()
{
    // Allow up to 3 concurrent threads
    using (var semaphore = new SemaphoreSlim(3, 3))
    {
        var tasks = new List<Task>();
        
        for (int i = 0; i < 10; i++)
        {
            int taskId = i;
            tasks.Add(Task.Run(() =>
            {
                Console.WriteLine($"Task {taskId} waiting for semaphore");
                semaphore.Wait();
                
                try
                {
                    Console.WriteLine($"Task {taskId} acquired semaphore");
                    Thread.Sleep(1000);  // Simulate work
                }
                finally
                {
                    Console.WriteLine($"Task {taskId} releasing semaphore");
                    semaphore.Release();
                }
            }));
        }
        
        Task.WaitAll(tasks.ToArray());
    }
}

7.3.3 - Task Parallel Library (TPL)

The Task Parallel Library (TPL) provides a higher-level abstraction for parallel and asynchronous programming.

7.3.3.1 - Tasks

A Task represents an asynchronous operation that may return a value.

// Create and start a task
Task task = Task.Run(() =>
{
    Console.WriteLine("Task is running");
    Thread.Sleep(1000);
    Console.WriteLine("Task completed");
});

// Wait for the task to complete
task.Wait();

// Create a task that returns a value
Task<int> calculationTask = Task.Run(() =>
{
    Console.WriteLine("Calculating...");
    Thread.Sleep(1000);
    return 42;
});

// Get the result (blocks until the task completes)
int result = calculationTask.Result;
Console.WriteLine($"Result: {result}");

7.3.3.2 - Task Continuations

Continuations allow you to specify additional work to be performed after a task completes.

Task<int> task = Task.Run(() => 
{
    Console.WriteLine("First task running");
    return 42;
});

Task<string> continuation = task.ContinueWith(antecedent =>
{
    Console.WriteLine("Continuation running");
    return $"The answer is {antecedent.Result}";
});

string answer = continuation.Result;
Console.WriteLine(answer);  // The answer is 42

// Conditional continuations
Task task2 = Task.Run(() => { throw new Exception("Task failed"); });

task2.ContinueWith(t => Console.WriteLine("Task completed successfully"),
    TaskContinuationOptions.OnlyOnRanToCompletion);

task2.ContinueWith(t => Console.WriteLine($"Task failed: {t.Exception.InnerException.Message}"),
    TaskContinuationOptions.OnlyOnFaulted);

7.3.3.3 - Parallel.For and Parallel.ForEach

Parallel.For and Parallel.ForEach provide simple ways to parallelize loops.

// Parallel.For
Parallel.For(0, 10, i =>
{
    Console.WriteLine($"Processing item {i} on thread {Thread.CurrentThread.ManagedThreadId}");
    Thread.Sleep(100);  // Simulate work
});

// Parallel.ForEach
var items = Enumerable.Range(0, 10).ToList();
Parallel.ForEach(items, item =>
{
    Console.WriteLine($"Processing item {item} on thread {Thread.CurrentThread.ManagedThreadId}");
    Thread.Sleep(100);  // Simulate work
});

// With options
var options = new ParallelOptions
{
    MaxDegreeOfParallelism = Environment.ProcessorCount / 2
};

Parallel.ForEach(items, options, item =>
{
    Console.WriteLine($"Processing item {item} with limited parallelism");
    Thread.Sleep(100);
});

7.3.3.4 - Task.WhenAll and Task.WhenAny

Task.WhenAll and Task.WhenAny allow you to wait for multiple tasks to complete.

// Create multiple tasks
Task<int>[] tasks = new Task<int>[3];
for (int i = 0; i < tasks.Length; i++)
{
    int taskNum = i;
    tasks[i] = Task.Run(() =>
    {
        int sleepTime = (taskNum + 1) * 1000;
        Thread.Sleep(sleepTime);
        return taskNum;
    });
}

// Wait for all tasks to complete
Task.WhenAll(tasks).ContinueWith(t =>
{
    Console.WriteLine("All tasks completed");
    foreach (var result in t.Result)
    {
        Console.WriteLine($"Result: {result}");
    }
});

// Wait for any task to complete
Task<Task<int>> whenAnyTask = Task.WhenAny(tasks);
whenAnyTask.ContinueWith(t =>
{
    int firstResult = t.Result.Result;
    Console.WriteLine($"First completed task result: {firstResult}");
});

7.3.4 - Async and Await Patterns

The async and await keywords provide a simplified approach to asynchronous programming, making asynchronous code more readable and maintainable.

7.3.4.1 - Basic Async/Await

// Asynchronous method
public async Task<string> DownloadDataAsync(string url)
{
    Console.WriteLine("Starting download...");
    
    using (HttpClient client = new HttpClient())
    {
        // Asynchronously wait for the download to complete
        string data = await client.GetStringAsync(url);
        
        Console.WriteLine("Download completed");
        return data;
    }
}

// Calling an async method
public async Task ProcessDataAsync()
{
    try
    {
        string data = await DownloadDataAsync("https://example.com");
        Console.WriteLine($"Downloaded {data.Length} bytes");
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Error: {ex.Message}");
    }
}

7.3.4.2 - Async Void

Async void methods should generally be avoided except for event handlers.

// Avoid this pattern except for event handlers
public async void Button_Click(object sender, EventArgs e)
{
    try
    {
        string data = await DownloadDataAsync("https://example.com");
        UpdateUI(data);
    }
    catch (Exception ex)
    {
        DisplayError(ex.Message);
    }
}

// Prefer returning Task
public async Task ProcessButtonClickAsync()
{
    string data = await DownloadDataAsync("https://example.com");
    return data;
}

7.3.4.3 - Async Lambda Expressions

// Async lambda with Task return type
Func<string, Task<int>> countCharsAsync = async (url) =>
{
    using (HttpClient client = new HttpClient())
    {
        string data = await client.GetStringAsync(url);
        return data.Length;
    }
};

// Using the async lambda
int length = await countCharsAsync("https://example.com");
Console.WriteLine($"Content length: {length}");

// Async event handler
button.Click += async (sender, e) =>
{
    await ProcessButtonClickAsync();
};

7.3.4.4 - Task Composition

public async Task<string> GetUserDataAsync(int userId)
{
    // Run two tasks concurrently
    Task<string> profileTask = GetUserProfileAsync(userId);
    Task<string> preferencesTask = GetUserPreferencesAsync(userId);
    
    // Await both tasks
    string profile = await profileTask;
    string preferences = await preferencesTask;
    
    // Combine the results
    return $"{profile}\n{preferences}";
}

// Alternative using Task.WhenAll
public async Task<string> GetUserDataAsync2(int userId)
{
    Task<string> profileTask = GetUserProfileAsync(userId);
    Task<string> preferencesTask = GetUserPreferencesAsync(userId);
    
    // Await both tasks simultaneously
    await Task.WhenAll(profileTask, preferencesTask);
    
    // Both tasks are now complete
    return $"{profileTask.Result}\n{preferencesTask.Result}";
}

7.3.4.5 - ValueTask

ValueTask<T> reduces allocations when async methods complete synchronously.

public ValueTask<int> GetValueAsync(bool useCache)
{
    if (useCache && _cache.TryGetValue("key", out int value))
    {
        // Return a value without allocating a Task
        return new ValueTask<int>(value);
    }
    
    // Fall back to Task when actual async work is needed
    return new ValueTask<int>(GetValueSlowAsync());
}

private async Task<int> GetValueSlowAsync()
{
    await Task.Delay(1000);
    return 42;
}

7.3.5 - Parallel LINQ (PLINQ)

Parallel LINQ (PLINQ) provides a simple way to parallelize LINQ queries.

// Sequential LINQ query
var numbers = Enumerable.Range(1, 1000000);
var evenNumbers = numbers.Where(n => n % 2 == 0).ToList();

// Parallel LINQ query
var parallelEvenNumbers = numbers.AsParallel()
                                .Where(n => n % 2 == 0)
                                .ToList();

// With execution options
var orderedResults = numbers.AsParallel()
                           .WithDegreeOfParallelism(4)
                           .WithExecutionMode(ParallelExecutionMode.ForceParallelism)
                           .Where(n => n % 2 == 0)
                           .OrderBy(n => n)
                           .ToList();

// Handling exceptions
try
{
    var results = numbers.AsParallel()
                        .Where(n => 
                        {
                            if (n == 500000) throw new Exception("Sample exception");
                            return n % 2 == 0;
                        })
                        .ToList();
}
catch (AggregateException ae)
{
    foreach (var ex in ae.InnerExceptions)
    {
        Console.WriteLine($"Error: {ex.Message}");
    }
}

7.3.6 - Concurrent Collections

Thread-safe collections designed for concurrent access.

7.3.6.1 - ConcurrentDictionary

// Create a concurrent dictionary
var concurrentDict = new ConcurrentDictionary<string, int>();

// Add or update values
concurrentDict.TryAdd("one", 1);
concurrentDict.TryAdd("two", 2);

// Update with a function
concurrentDict.AddOrUpdate("one", 
    key => 10,                         // Add function if key doesn't exist
    (key, oldValue) => oldValue + 10); // Update function if key exists

// Get or add
int value = concurrentDict.GetOrAdd("three", key => 3);

// Thread-safe operations
Parallel.For(0, 100, i =>
{
    concurrentDict.AddOrUpdate("counter", 
        key => 1, 
        (key, oldValue) => oldValue + 1);
});

Console.WriteLine($"Counter: {concurrentDict["counter"]}");

7.3.6.2 - ConcurrentQueue and ConcurrentStack

// Concurrent queue
var queue = new ConcurrentQueue<int>();

// Enqueue items
queue.Enqueue(1);
queue.Enqueue(2);
queue.Enqueue(3);

// Try to dequeue
if (queue.TryDequeue(out int result))
{
    Console.WriteLine($"Dequeued: {result}");
}

// Peek at the next item
if (queue.TryPeek(out int peeked))
{
    Console.WriteLine($"Next item: {peeked}");
}

// Concurrent stack
var stack = new ConcurrentStack<int>();

// Push items
stack.Push(1);
stack.Push(2);
stack.Push(3);

// Try to pop
if (stack.TryPop(out int popped))
{
    Console.WriteLine($"Popped: {popped}");
}

// Try to peek
if (stack.TryPeek(out int peekedStack))
{
    Console.WriteLine($"Top item: {peekedStack}");
}

// Push and pop multiple items
int[] items = new int[3] { 4, 5, 6 };
stack.PushRange(items);

int[] poppedItems = new int[2];
stack.TryPopRange(poppedItems);

7.3.6.3 - ConcurrentBag

// Concurrent bag (unordered collection)
var bag = new ConcurrentBag<int>();

// Add items
bag.Add(1);
bag.Add(2);
bag.Add(3);

// Try to take an item
if (bag.TryTake(out int taken))
{
    Console.WriteLine($"Taken: {taken}");
}

// Parallel add
Parallel.For(0, 100, i =>
{
    bag.Add(i);
});

Console.WriteLine($"Bag count: {bag.Count}");

7.3.6.4 - BlockingCollection

BlockingCollection<T> provides blocking and bounding capabilities for thread-safe collections.

// Create a bounded blocking collection
using (var blockingCollection = new BlockingCollection<int>(boundedCapacity: 100))
{
    // Producer task
    Task producer = Task.Run(() =>
    {
        for (int i = 0; i < 200; i++)
        {
            blockingCollection.Add(i);
            Console.WriteLine($"Produced: {i}");
            Thread.Sleep(10);
        }
        
        // Signal that no more items will be added
        blockingCollection.CompleteAdding();
    });
    
    // Consumer tasks
    Task[] consumers = new Task[2];
    for (int i = 0; i < consumers.Length; i++)
    {
        int consumerId = i;
        consumers[i] = Task.Run(() =>
        {
            // GetConsumingEnumerable returns items until CompleteAdding is called
            foreach (var item in blockingCollection.GetConsumingEnumerable())
            {
                Console.WriteLine($"Consumer {consumerId} consumed: {item}");
                Thread.Sleep(50);
            }
        });
    }
    
    // Wait for all tasks to complete
    Task.WaitAll(new[] { producer }.Concat(consumers).ToArray());
}

7.3.7 - Asynchronous Streams with IAsyncEnumerable<T>

Asynchronous streams (introduced in C# 8.0) allow you to asynchronously iterate over a collection of items. This feature combines the benefits of asynchronous programming with the simplicity of enumeration, making it ideal for scenarios where data is produced asynchronously over time.

7.3.7.1 - Basic Usage

// Generate an asynchronous stream
public static async IAsyncEnumerable<int> GenerateNumbersAsync(int count, int delay)
{
    for (int i = 0; i < count; i++)
    {
        await Task.Delay(delay);  // Simulate async work
        yield return i;
    }
}

// Consume an asynchronous stream
public static async Task ConsumeAsyncStreamAsync()
{
    await foreach (var number in GenerateNumbersAsync(10, 100))
    {
        Console.WriteLine($"Received: {number}");
    }
}

7.3.7.2 - Processing Asynchronous Streams

You can process asynchronous streams using LINQ-like operations:

// Filtering an async stream
public static async IAsyncEnumerable<int> FilterEvenNumbersAsync(IAsyncEnumerable<int> source)
{
    await foreach (var number in source)
    {
        if (number % 2 == 0)
        {
            yield return number;
        }
    }
}

// Usage
public static async Task FilteredStreamExampleAsync()
{
    var numbers = GenerateNumbersAsync(20, 50);
    var evenNumbers = FilterEvenNumbersAsync(numbers);
    
    await foreach (var number in evenNumbers)
    {
        Console.WriteLine($"Even number: {number}");
    }
}

7.3.7.3 - Cancellation Support

Asynchronous streams support cancellation:

public static async IAsyncEnumerable<int> GenerateWithCancellationAsync(
    int count, 
    int delay,
    [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
    for (int i = 0; i < count; i++)
    {
        // Check cancellation before each delay
        cancellationToken.ThrowIfCancellationRequested();
        
        await Task.Delay(delay, cancellationToken);
        yield return i;
    }
}

public static async Task CancellableStreamExampleAsync()
{
    using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(2));
    
    try
    {
        await foreach (var number in GenerateWithCancellationAsync(100, 500)
                      .WithCancellation(cts.Token))
        {
            Console.WriteLine($"Received: {number}");
        }
    }
    catch (OperationCanceledException)
    {
        Console.WriteLine("Stream processing was cancelled");
    }
}

7.3.7.4 - Real-World Example: API Pagination

Asynchronous streams are perfect for handling paginated API results:

public class DataService
{
    private readonly HttpClient _client;
    
    public DataService(HttpClient client)
    {
        _client = client;
    }
    
    // Return all results as an asynchronous stream
    public async IAsyncEnumerable<Product> GetAllProductsAsync(
        int pageSize = 25,
        [EnumeratorCancellation] CancellationToken cancellationToken = default)
    {
        int pageNumber = 1;
        bool hasMoreData = true;
        
        while (hasMoreData)
        {
            // Get a page of results
            var page = await GetProductPageAsync(pageNumber, pageSize, cancellationToken);
            
            // Yield each product in the current page
            foreach (var product in page.Items)
            {
                yield return product;
            }
            
            // Check if we've reached the last page
            hasMoreData = page.HasNextPage;
            pageNumber++;
        }
    }
    
    private async Task<PagedResult<Product>> GetProductPageAsync(
        int pageNumber, int pageSize, CancellationToken cancellationToken)
    {
        var response = await _client.GetAsync(
            $"api/products?page={pageNumber}&pageSize={pageSize}", 
            cancellationToken);
        
        response.EnsureSuccessStatusCode();
        
        return await response.Content.ReadFromJsonAsync<PagedResult<Product>>(
            cancellationToken: cancellationToken);
    }
}

// Usage
public static async Task ProcessAllProductsAsync()
{
    var service = new DataService(new HttpClient());
    
    // Process all products without loading everything into memory
    await foreach (var product in service.GetAllProductsAsync())
    {
        Console.WriteLine($"Processing product: {product.Name}");
        // Process each product as it arrives
    }
}

7.3.7.5 - Comparison with Traditional Approaches

| Approach              | Pros                                                                 | Cons                                               |
|-----------------------|----------------------------------------------------------------------|----------------------------------------------------|
| **IAsyncEnumerable<T>** | - Process items as they arrive<br>- Memory efficient<br>- Natural syntax with await foreach | - Requires C# 8.0+<br>- Not supported in all libraries |
| **Task<IEnumerable<T>>** | - Simple return type<br>- Widely supported                        | - Must wait for all items<br>- Loads everything into memory |
| **IObservable<T>**       | - Push-based model<br>- Supports complex event streams            | - More complex API<br>- Steeper learning curve     |

7.3.8 - Cancellation and Progress Reporting

Cancellation tokens and progress reporting enable responsive asynchronous operations.

7.3.8.1 - Cancellation Tokens

public static async Task CancellationExampleAsync()
{
    // Create a cancellation token source
    using (var cts = new CancellationTokenSource())
    {
        // Get the token
        CancellationToken token = cts.Token;
        
        // Start a task that can be canceled
        Task task = LongRunningOperationAsync(token);
        
        // Cancel after 3 seconds
        await Task.Delay(3000);
        Console.WriteLine("Cancelling operation...");
        cts.Cancel();
        
        try
        {
            await task;
            Console.WriteLine("Task completed successfully");
        }
        catch (OperationCanceledException)
        {
            Console.WriteLine("Task was canceled");
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Task failed: {ex.Message}");
        }
    }
}

public static async Task LongRunningOperationAsync(CancellationToken cancellationToken)
{
    for (int i = 0; i < 10; i++)
    {
        // Check for cancellation
        cancellationToken.ThrowIfCancellationRequested();
        
        Console.WriteLine($"Operation step {i}");
        await Task.Delay(1000, cancellationToken);
    }
}

7.3.8.2 - Cancellation with Timeout

public static async Task TimeoutExampleAsync()
{
    using (var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5)))
    {
        try
        {
            await LongRunningOperationAsync(cts.Token);
        }
        catch (OperationCanceledException)
        {
            Console.WriteLine("Operation timed out after 5 seconds");
        }
    }
}

// Combining cancellation tokens
public static async Task CombinedCancellationExampleAsync()
{
    using (var userCts = new CancellationTokenSource())
    using (var timeoutCts = new CancellationTokenSource(TimeSpan.FromSeconds(10)))
    using (var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(
        userCts.Token, timeoutCts.Token))
    {
        try
        {
            // This will be canceled if either the user cancels or the timeout occurs
            await LongRunningOperationAsync(linkedCts.Token);
        }
        catch (OperationCanceledException)
        {
            if (timeoutCts.Token.IsCancellationRequested)
            {
                Console.WriteLine("Operation timed out");
            }
            else
            {
                Console.WriteLine("Operation was canceled by the user");
            }
        }
    }
}

7.3.8.3 - Progress Reporting

public static async Task ProgressExampleAsync()
{
    // Create a progress reporter
    var progress = new Progress<int>(percent =>
    {
        Console.WriteLine($"Progress: {percent}%");
    });
    
    // Start the operation with progress reporting
    await LongRunningOperationWithProgressAsync(progress);
}

public static async Task LongRunningOperationWithProgressAsync(IProgress<int> progress)
{
    for (int i = 0; i <= 100; i += 10)
    {
        // Report progress
        progress?.Report(i);
        
        await Task.Delay(500);
    }
}

// Progress with custom data
public class OperationProgress
{
    public int PercentComplete { get; set; }
    public string CurrentOperation { get; set; }
    public int ItemsProcessed { get; set; }
}

public static async Task DetailedProgressExampleAsync()
{
    var progress = new Progress<OperationProgress>(p =>
    {
        Console.WriteLine($"Operation: {p.CurrentOperation}, " +
                         $"{p.PercentComplete}% complete, " +
                         $"{p.ItemsProcessed} items processed");
    });
    
    await ProcessFilesWithProgressAsync(progress);
}

public static async Task ProcessFilesWithProgressAsync(IProgress<OperationProgress> progress)
{
    string[] files = Directory.GetFiles(".", "*.txt");
    int totalFiles = files.Length;
    
    for (int i = 0; i < totalFiles; i++)
    {
        string file = files[i];
        
        // Report progress
        progress?.Report(new OperationProgress
        {
            CurrentOperation = $"Processing {Path.GetFileName(file)}",
            PercentComplete = (i * 100) / totalFiles,
            ItemsProcessed = i
        });
        
        // Process the file
        await Task.Delay(500);  // Simulate processing
    }
    
    // Final progress report
    progress?.Report(new OperationProgress
    {
        CurrentOperation = "Complete",
        PercentComplete = 100,
        ItemsProcessed = totalFiles
    });
}

7.3.9 - Synchronization Contexts

Synchronization contexts control where asynchronous continuations execute, which is particularly important for UI applications and other scenarios where thread affinity matters.

7.3.9.1 - Understanding Synchronization Contexts

A SynchronizationContext captures the execution environment and allows code to run in that environment later. Different application types have different synchronization contexts:

Application Type	Synchronization Context	Behavior
Console/Service	null or ThreadPoolSynchronizationContext	Continuations run on thread pool threads
Windows Forms	WindowsFormsSynchronizationContext	Continuations run on the UI thread
WPF	DispatcherSynchronizationContext	Continuations run on the UI thread
ASP.NET	AspNetSynchronizationContext	Continuations run on the request thread
ASP.NET Core	null	No synchronization context by default

7.3.9.2 - How Await Uses Synchronization Context

public static async Task SynchronizationContextExampleAsync()
{
    // Capture the current synchronization context
    SynchronizationContext originalContext = SynchronizationContext.Current;
    Console.WriteLine($"Original thread: {Thread.CurrentThread.ManagedThreadId}");
    Console.WriteLine($"Has sync context: {originalContext != null}");
    
    await Task.Run(async () =>
    {
        // This code runs on a thread pool thread
        Console.WriteLine($"Background thread: {Thread.CurrentThread.ManagedThreadId}");
        
        // No synchronization context in the thread pool
        Console.WriteLine($"SynchronizationContext: {SynchronizationContext.Current}");
        
        await Task.Delay(1000);
        
        // Still on a thread pool thread after the await
        Console.WriteLine($"After await in Task.Run: {Thread.CurrentThread.ManagedThreadId}");
    });
    
    // Back to the original context after the await
    Console.WriteLine($"Back to thread: {Thread.CurrentThread.ManagedThreadId}");
    Console.WriteLine($"Original context: {SynchronizationContext.Current == originalContext}");
}

7.3.9.3 - ConfigureAwait

The ConfigureAwait method controls whether continuations after an await should run in the captured context:

public static async Task ConfigureAwaitExampleAsync()
{
    Console.WriteLine($"Starting on thread: {Thread.CurrentThread.ManagedThreadId}");
    
    // With ConfigureAwait(true) - default behavior
    await Task.Delay(1000);
    Console.WriteLine($"After regular await: {Thread.CurrentThread.ManagedThreadId}");
    
    // With ConfigureAwait(false) - don't return to the original context
    await Task.Delay(1000).ConfigureAwait(false);
    Console.WriteLine($"After ConfigureAwait(false): {Thread.CurrentThread.ManagedThreadId}");
    
    // The rest of the method might run on a different thread
}

7.3.9.4 - When to Use ConfigureAwait(false)

Using ConfigureAwait(false) can improve performance by avoiding unnecessary context switches:

public class DataService
{
    // Library code should use ConfigureAwait(false)
    public async Task<string> GetDataAsync()
    {
        // No need to return to the caller's context for internal operations
        var response = await httpClient.GetAsync("api/data").ConfigureAwait(false);
        var content = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
        var processed = await ProcessDataAsync(content).ConfigureAwait(false);
        
        return processed;
    }
    
    private async Task<string> ProcessDataAsync(string data)
    {
        // Internal method also uses ConfigureAwait(false)
        await Task.Delay(100).ConfigureAwait(false);
        return data.ToUpperInvariant();
    }
}

7.3.9.5 - UI Applications and Synchronization Context

In UI applications, the synchronization context ensures UI updates happen on the UI thread:

// WPF/Windows Forms application example
private async void Button_Click(object sender, EventArgs e)
{
    // Disable the button while operation is in progress
    button.Enabled = false;
    progressBar.Visible = true;
    
    try
    {
        // This runs on a background thread
        var result = await Task.Run(() => 
        {
            // Long-running operation
            return PerformCalculation();
        });
        
        // This automatically runs on the UI thread due to SynchronizationContext
        resultTextBox.Text = result.ToString();
    }
    catch (Exception ex)
    {
        // Error handling on UI thread
        MessageBox.Show(ex.Message);
    }
    finally
    {
        // UI cleanup on UI thread
        button.Enabled = true;
        progressBar.Visible = false;
    }
}

7.3.9.6 - Custom Synchronization Contexts

You can create custom synchronization contexts for specialized scenarios:

public class CustomSynchronizationContext : SynchronizationContext
{
    private readonly ConcurrentQueue<(SendOrPostCallback, object)> _workItems = 
        new ConcurrentQueue<(SendOrPostCallback, object)>();
    private readonly AutoResetEvent _workItemsAvailable = new AutoResetEvent(false);
    
    public override void Post(SendOrPostCallback callback, object state)
    {
        _workItems.Enqueue((callback, state));
        _workItemsAvailable.Set();
    }
    
    public void RunMessageLoop()
    {
        while (true)
        {
            _workItemsAvailable.WaitOne();
            
            while (_workItems.TryDequeue(out var workItem))
            {
                workItem.Item1(workItem.Item2);
            }
        }
    }
}

7.3.10 - Thread Synchronization with Lock Type (C# 13+)

C# 13 introduces a new System.Threading.Lock type that provides improved thread synchronization capabilities compared to the traditional lock statement with Monitor.

7.3.10.1 - Understanding the Lock Type

The Lock type is a value type designed specifically for thread synchronization, offering better performance and more features than the traditional Monitor-based locking.

using System;
using System.Threading;
using System.Threading.Tasks;

class Program
{
    // Create a Lock instance
    private static readonly Lock _lock = new Lock();
    private static int _counter = 0;
    
    static async Task Main()
    {
        // Create multiple tasks that increment the counter
        var tasks = new Task[10];
        for (int i = 0; i < tasks.Length; i++)
        {
            tasks[i] = IncrementCounterAsync();
        }
        
        await Task.WhenAll(tasks);
        Console.WriteLine($"Final counter value: {_counter}");
    }
    
    static async Task IncrementCounterAsync()
    {
        for (int i = 0; i < 1000; i++)
        {
            // Use the lock statement with the Lock type
            lock (_lock)
            {
                _counter++;
            }
            
            await Task.Delay(1); // Simulate some async work
        }
    }
}

7.3.10.2 - Lock vs Monitor

The Lock type offers several advantages over the traditional Monitor-based locking:

// Traditional lock statement (uses Monitor internally)
private static readonly object _traditionalLock = new object();

public void TraditionalLockExample()
{
    lock (_traditionalLock)
    {
        // Critical section
        Console.WriteLine("Inside critical section with traditional lock");
    }
}

// New Lock type
private static readonly Lock _newLock = new Lock();

public void NewLockExample()
{
    lock (_newLock)
    {
        // Critical section
        Console.WriteLine("Inside critical section with new Lock type");
    }
}

The C# compiler recognizes when the target of a lock statement is a Lock object and generates more efficient code using the Lock.EnterScope() method instead of Monitor.Enter and Monitor.Exit.

7.3.10.3 - Advanced Lock Features

The Lock type provides additional features beyond what's available with Monitor:

using System;
using System.Threading;
using System.Threading.Tasks;

class AdvancedLockExample
{
    private readonly Lock _lock = new Lock();
    
    // Using Lock.EnterScope directly for more control
    public void ExplicitScopeExample()
    {
        using (var scope = _lock.EnterScope())
        {
            // Critical section
            Console.WriteLine("Inside critical section with explicit scope");
            
            // You can check if you have the lock
            if (scope.HasLock)
            {
                Console.WriteLine("We have the lock");
            }
        }
        // Lock is automatically released when scope is disposed
    }
    
    // Trying to acquire a lock with timeout
    public bool TryLockExample(TimeSpan timeout)
    {
        if (_lock.TryEnterScope(timeout, out var scope))
        {
            using (scope)
            {
                Console.WriteLine("Acquired lock with timeout");
                return true;
            }
        }
        
        Console.WriteLine("Failed to acquire lock within timeout");
        return false;
    }
    
    // Asynchronous locking
    public async Task AsyncLockExample()
    {
        await using (await _lock.EnterScopeAsync())
        {
            Console.WriteLine("Inside critical section with async lock");
            await Task.Delay(100); // Can safely await inside the lock
        }
        // Lock is automatically released when scope is disposed
    }
}

7.3.10.4 - Lock Type Performance Benefits

The Lock type is designed to be more efficient than traditional locking mechanisms:

1. Value Type: As a value type, it avoids heap allocations.
2. Specialized API: The API is designed specifically for locking, unlike object which is used for many purposes.
3. Compiler Integration: The C# compiler generates optimized code when using Lock with the lock statement.
4. Async Support: Built-in support for asynchronous operations with EnterScopeAsync.

// Performance comparison example
public void PerformanceComparison()
{
    const int iterations = 1000000;
    
    // Traditional lock
    object traditionalLock = new object();
    var sw1 = Stopwatch.StartNew();
    for (int i = 0; i < iterations; i++)
    {
        lock (traditionalLock)
        {
            // Empty critical section
        }
    }
    sw1.Stop();
    
    // New Lock type
    Lock newLock = new Lock();
    var sw2 = Stopwatch.StartNew();
    for (int i = 0; i < iterations; i++)
    {
        lock (newLock)
        {
            // Empty critical section
        }
    }
    sw2.Stop();
    
    Console.WriteLine($"Traditional lock: {sw1.ElapsedMilliseconds}ms");
    Console.WriteLine($"New Lock type: {sw2.ElapsedMilliseconds}ms");
}

7.3.10.5 - Best Practices for Using Lock Type

When using the new Lock type, follow these best practices:

1. Prefer Lock over object: Use the Lock type instead of object for new code that requires synchronization.
2. Keep Critical Sections Short: Minimize the amount of code inside lock blocks to reduce contention.
3. Avoid Nested Locks: Nested locks can lead to deadlocks; design your code to avoid them.
4. Use Async Locking for Async Code: Use EnterScopeAsync when working with asynchronous code.
5. Consider Timeouts: Use TryEnterScope with a timeout to avoid indefinite waiting.

// Example of good Lock usage
public class ThreadSafeCounter
{
    private readonly Lock _lock = new Lock();
    private int _count = 0;
    
    public int Increment()
    {
        lock (_lock)
        {
            return ++_count;
        }
    }
    
    public async Task<int> IncrementAsync()
    {
        await using (await _lock.EnterScopeAsync())
        {
            return ++_count;
        }
    }
    
    public bool TryIncrement(TimeSpan timeout, out int newValue)
    {
        newValue = 0;
        if (_lock.TryEnterScope(timeout, out var scope))
        {
            using (scope)
            {
                newValue = ++_count;
                return true;
            }
        }
        return false;
    }
}

7.3.11 - Asynchronous Programming Best Practices

7.3.11.1 - General Guidelines

1. Use async/await for I/O-bound operations

   // Good - asynchronous I/O
   public async Task<string> ReadFileAsync(string path)
   {
       using (var reader = new StreamReader(path))
       {
           return await reader.ReadToEndAsync();
       }
   }

2. Use Task.Run for CPU-bound operations

   // Good - offload CPU-intensive work
   public async Task<int> CalculateComplexValueAsync(int input)
   {
       return await Task.Run(() =>
       {
           // CPU-intensive calculation
           int result = 0;
           for (int i = 0; i < input; i++)
           {
               result += PerformComplexCalculation(i);
           }
           return result;
       });
   }

3. Avoid async void except for event handlers

   // Good - event handler can be async void
   private async void Button_Click(object sender, EventArgs e)
   {
       try
       {
           await ProcessDataAsync();
       }
       catch (Exception ex)
       {
           LogError(ex);
       }
   }
   
   // Good - return Task for all other async methods
   public async Task ProcessDataAsync()
   {
       // Implementation
   }

4. Use ConfigureAwait(false) in library code

   // Good - library method that doesn't need context
   public async Task<int> LibraryMethodAsync()
   {
       // Use ConfigureAwait(false) to avoid context switches
       var data = await FetchDataAsync().ConfigureAwait(false);
       var processed = await ProcessDataAsync(data).ConfigureAwait(false);
       return processed.Length;
   }

5. Handle exceptions in async code

   // Good - proper exception handling
   public async Task<string> GetDataWithRetryAsync(string url, int maxRetries = 3)
   {
       for (int i = 0; i < maxRetries; i++)
       {
           try
           {
               using (var client = new HttpClient())
               {
                   return await client.GetStringAsync(url).ConfigureAwait(false);
               }
           }
           catch (HttpRequestException ex) when (i < maxRetries - 1)
           {
               // Log and retry
               Console.WriteLine($"Attempt {i + 1} failed: {ex.Message}. Retrying...");
               await Task.Delay(1000 * (i + 1)).ConfigureAwait(false);
           }
       }
       
       throw new TimeoutException($"Failed to get data after {maxRetries} attempts");
   }

6. Avoid mixing synchronous and asynchronous code

   // Bad - blocking on async code
   public string GetDataSync(string url)
   {
       // This can lead to deadlocks
       return GetDataAsync(url).Result;
   }
   
   // Good - keep the async flow
   public async Task<string> GetDataAsync(string url)
   {
       using (var client = new HttpClient())
       {
           return await client.GetStringAsync(url);
       }
   }

7. Use cancellation tokens for long-running operations

   // Good - supports cancellation
   public async Task ProcessLargeDataAsync(Stream data, CancellationToken cancellationToken = default)
   {
       byte[] buffer = new byte[4096];
       int bytesRead;
       
       while ((bytesRead = await data.ReadAsync(buffer, 0, buffer.Length, cancellationToken)) > 0)
       {
           cancellationToken.ThrowIfCancellationRequested();
           await ProcessChunkAsync(buffer, bytesRead, cancellationToken);
       }
   }

8. Compose asynchronous operations efficiently

   // Good - run independent operations concurrently
   public async Task<UserProfile> GetUserProfileAsync(int userId)
   {
       Task<UserData> userDataTask = GetUserDataAsync(userId);
       Task<UserPreferences> preferencesTask = GetUserPreferencesAsync(userId);
       Task<List<Order>> ordersTask = GetUserOrdersAsync(userId);
       
       // Wait for all tasks to complete
       await Task.WhenAll(userDataTask, preferencesTask, ordersTask);
       
       // Combine the results
       return new UserProfile
       {
           UserData = userDataTask.Result,
           Preferences = preferencesTask.Result,
           RecentOrders = ordersTask.Result
       };
   }

7.3.10.2 - Performance Considerations

1. Minimize allocations with ValueTask

   // Good - use ValueTask for frequently cached results
   public ValueTask<int> GetCachedValueAsync(string key)
   {
       if (_cache.TryGetValue(key, out int value))
       {
           return new ValueTask<int>(value);
       }
       
       return new ValueTask<int>(GetValueFromDatabaseAsync(key));
   }

2. Use pooled objects for frequent operations

   // Good - use ArrayPool for buffer management
   public async Task CopyStreamAsync(Stream source, Stream destination)
   {
       byte[] buffer = ArrayPool<byte>.Shared.Rent(4096);
       try
       {
           int bytesRead;
           while ((bytesRead = await source.ReadAsync(buffer, 0, buffer.Length)) > 0)
           {
               await destination.WriteAsync(buffer, 0, bytesRead);
           }
       }
       finally
       {
           ArrayPool<byte>.Shared.Return(buffer);
       }
   }

3. Avoid excessive Task creation

   // Bad - creates a Task for a completed operation
   public Task<int> GetValueBad(bool useCache)
   {
       if (useCache && _cache.TryGetValue("key", out int value))
       {
           return Task.FromResult(value);
       }
       
       return GetValueFromDatabaseAsync("key");
   }
   
   // Good - uses ValueTask to avoid allocation
   public ValueTask<int> GetValueGood(bool useCache)
   {
       if (useCache && _cache.TryGetValue("key", out int value))
       {
           return new ValueTask<int>(value);
       }
       
       return new ValueTask<int>(GetValueFromDatabaseAsync("key"));
   }

Summary

Concurrency and asynchronous programming are essential for building responsive, scalable applications in C#. The language provides a rich set of tools, from low-level thread management to high-level abstractions like async/await and PLINQ. By understanding these concepts and following best practices, you can write efficient, maintainable code that effectively utilizes system resources and provides a responsive user experience.

Additional Resources

• Microsoft Docs: Asynchronous programming
• Microsoft Docs: Task Parallel Library
• Microsoft Docs: Thread Synchronization
• Microsoft Docs: Concurrent Collections