Welcome! In our previous lessons, we explored foundational concepts of concurrency, focusing on how multiple tasks can run simultaneously to improve performance and resource utilization. Today, we will build on that knowledge and dive into a practical application by creating a Multi-Threaded Download Manager with Resource Limiting. This lesson will show you how to manage concurrent downloads while controlling resource usage, a critical skill for building efficient applications.

In this lesson, you will learn how to:

• Configure custom thread pools using ThreadPoolExecutor.
• Manage concurrent tasks with Semaphore for resource control.
• Synchronize task execution and handle termination.

By the end, you'll have built a download manager that handles multiple downloads efficiently while limiting resource usage.

In this section, we’ll create a Multi-Threaded Download Manager that allows multiple downloads to occur concurrently while staying within system resource limits. To achieve this, we’ll combine a custom thread pool with a semaphore for fine-grained control over concurrency.

ThreadPoolExecutor manages a pool of threads, determining how many can operate at once based on system resources and the pool's configuration. However, on its own, ThreadPoolExecutor does not directly control the number of simultaneous downloads. This is where Semaphore becomes essential: it limits access to the shared download resource by controlling the number of active permits.

Together, these tools enable precise control over concurrent downloads, balancing the workload across available system resources for efficient management.

We'll start by defining the DownloadManager class. This class is responsible for managing the downloads by limiting how many can run at once, using a thread pool and a semaphore.

Java
Copy to clipboard
1import java.util.List;
2import java.util.concurrent.*;
3
4public class DownloadManager {
5
6    private final Semaphore connectionSemaphore;
7    private final ExecutorService executor;
8
9    public DownloadManager(Semaphore connectionSemaphore, ExecutorService executor) {
10        this.connectionSemaphore = connectionSemaphore;
11        this.executor = executor;
12    }
13}

The DownloadManager initializes with:

• Semaphore to limit the number of concurrent downloads by blocking or allowing threads based on permits.
• ExecutorService to manage the execution of tasks in a thread pool, improving resource management for handling multiple downloads concurrently.

Next, let’s implement the method that handles downloading files. This method submits tasks to the thread pool for execution and ensures that only a limited number of downloads can run simultaneously.

Java
Copy to clipboard
1public void downloadFiles(List<String> fileUrls) {
2    for (String url : fileUrls) {
3        executor.submit(() -> {
4            try {
5                connectionSemaphore.acquire();  // Limit concurrent downloads
6                downloadFile(url);
7            } catch (InterruptedException e) {
8                Thread.currentThread().interrupt();  // Handle thread interruption
9            } finally {
10                connectionSemaphore.release();  // Release the semaphore when done
11            }
12        });
13    }
14    executor.shutdown();  // Prevent further submissions
15    awaitTermination();  // Wait for downloads to complete
16}

Here are the key methods to pay attention to in the above snippet:

• connectionSemaphore.acquire(): This blocks the thread until a permit is available, limiting the number of concurrent downloads.
• connectionSemaphore.release(): After a download completes, a permit is released, allowing another download to start.
• executor.shutdown(): Once all tasks are submitted, the executor is shut down to prevent further task submissions.

The method loops over the URLs, ensuring that only a specified number of downloads run concurrently while managing thread synchronization and resource limits.

Let’s now define the downloadFile() method, which simulates downloading a file.

Java
Copy to clipboard
1private void downloadFile(String url) {
2    System.out.println("Downloading " + url + " on " + Thread.currentThread().getName());
3    try {
4        Thread.sleep((long) (Math.random() * 3000 + 1000));  // Simulate download time
5    } catch (InterruptedException e) {
6        Thread.currentThread().interrupt();  // Handle interruption
7    }
8    System.out.println("Completed " + url + " on " + Thread.currentThread().getName());
9}

This method simulates the process of downloading a file, pausing the thread to mimic the download time. It logs the start and end of each download for visibility.

To ensure that all downloads complete before shutting down, we implement the awaitTermination() method.

Java
Copy to clipboard
1private void awaitTermination() {
2    try {
3        if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {
4            executor.shutdownNow();  // Force shutdown if not completed
5        }
6    } catch (InterruptedException e) {
7        executor.shutdownNow();
8        Thread.currentThread().interrupt();
9    }
10}

This method waits for all tasks to finish, enforcing a timeout of 60 seconds. If tasks are still running after this period, the executor is forcibly shut down to ensure proper termination.

Now, let’s see how the Main class ties everything together and starts the download manager.

Java
Copy to clipboard
1import java.util.*;
2import java.util.concurrent.*;
3
4public class Main {
5    public static void main(String[] args) {
6        new Main().startDownloads();
7    }
8
9    public void startDownloads() {
10        List<String> fileUrls = Arrays.asList("file1.dat", "file2.dat", "file3.dat", "file4.dat", "file5.dat");
11
12        Semaphore connectionSemaphore = new Semaphore(2);  // Limit to 2 concurrent downloads
13        ExecutorService executor = new ThreadPoolExecutor(
14            2,            // Minimum active threads
15            4,            // Maximum threads allowed
16            10,           // Idle time before excess threads terminate
17            TimeUnit.SECONDS, // Unit for idle time
18            new ArrayBlockingQueue<>(2) // Queue for pending tasks
19        );
20
21        DownloadManager manager = new DownloadManager(connectionSemaphore, executor);
22        manager.downloadFiles(fileUrls);
23    }
24}

In the Main class:

• Semaphore connectionSemaphore = new Semaphore(2) limits the number of active downloads to two, ensuring that no more than two downloads are running at any given time.
• ThreadPoolExecutor manages the pool of threads responsible for downloading files, ensuring that the system resources are used efficiently for managing multiple downloads.

This setup allows us to manage downloads efficiently with controlled concurrency using the semaphore, while the ThreadPoolExecutor parameters enable fine-tuning of the pool behavior for optimal resource management.

Managing concurrent tasks efficiently is crucial for scalable applications. By using ThreadPoolExecutor and Semaphore, we ensure that resources are not overwhelmed, limiting the number of active downloads to what the system can handle. This approach is common in scenarios like web servers, file processors, or batch systems that need to manage multiple requests or tasks simultaneously without sacrificing performance.

Now that you’ve learned how to create a multi-threaded download manager, it’s time to apply these concepts in the practice section. Let’s see how this implementation handles concurrent downloads while efficiently managing system resources!