Updated to v3.2.0

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,37 @@
+---
+name: Bug report
+about: Found a bug? Report it here.
+title: "[BUG]"
+labels: bug
+assignees: bshoshany
+
+---
+
+**Describe the bug**
+
+A clear and concise description of what the bug is.
+
+**Minimal working example**
+
+A short but complete program that can be compiled to reproduce the error. Paste the program between the two code fences. If it's too long or requires multiple files, attach the file(s) instead.
+
+```cpp
+```
+
+**Behavior**
+
+What behavior did you expect to get? What actually happened? If the code failed to compile, please include the full output of the compiler.
+
+**System information**
+
+* CPU model, architecture, # of cores and threads:
+* Operating system:
+* Name and version of C++ compiler:
+* Full command used for compiling, including all compiler flags:
+* Thread pool library version:
+
+(Please note that only the latest version of the thread pool library is supported.)
+
+**Additional information**
+
+Include any additional information here.

.github/ISSUE_TEMPLATE/failed-tests.md

@@ -0,0 +1,26 @@
+---
+name: Failed tests
+about: The provided automated tests failed on your system? Report it here.
+title: "[TEST]"
+labels: bug
+assignees: bshoshany
+
+---
+
+**System information**
+
+* CPU model, architecture, # of cores and threads:
+* Operating system:
+* Name and version of C++ compiler:
+* Full command used for compiling, including all compiler flags:
+* Thread pool library version:
+
+(Please note that only the latest version of the thread pool library is supported.)
+
+**Log file**
+
+Please attach the log file generated by the automated test program to this issue.
+
+**Additional information**
+
+Include any additional information here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Want a new feature? Suggest it here.
+title: "[REQ]"
+labels: enhancement
+assignees: bshoshany
+
+---
+
+**Describe the new feature**
+
+A clear and concise description of the feature you want.
+
+**Code example**
+
+An example of code that utilizes the suggested feature. Paste or write it between the two code fences.
+
+```cpp
+```
+
+**Additional information**
+
+Include any additional information here.

.github/pull_request_template.md

@@ -0,0 +1,20 @@
+**Pull request policy (please read)**
+
+> Contributions are always welcome. However, I release my projects in cumulative updates after editing and testing them locally on my system, so my policy is not to accept any pull requests. If you open a pull request, and I decide to incorporate your suggestion into the project, I will first modify your code to comply with the project's coding conventions (formatting, syntax, naming, comments, programming practices, etc.), and perform some tests to ensure that the change doesn't break anything. I will then merge it into the next release of the project, possibly together with some other changes. The new release will also include a note in `CHANGELOG.md` with a link to your pull request, and modifications to the documentation in `README.md` as needed.
+
+**Describe the changes**
+
+What does your pull request fix or add to the library?
+
+**Testing**
+
+Have you tested the new code using the provided automated test program and/or performed any other tests to ensure that it works correctly? If so, please provide information about the test system(s):
+
+* CPU model, architecture, # of cores and threads:
+* Operating system:
+* Name and version of C++ compiler:
+* Full command used for compiling, including all compiler flags:
+
+**Additional information**
+
+Include any additional information here.

BS_thread_pool.hpp

@@ -0,0 +1,705 @@
+#pragma once
+
+/**
+ * @file BS_thread_pool.hpp
+ * @author Barak Shoshany (baraksh@gmail.com) (http://baraksh.com)
+ * @version 3.2.0
+ * @date 2022-07-28
+ * @copyright Copyright (c) 2022 Barak Shoshany. Licensed under the MIT license. If you found this project useful, please consider starring it on GitHub! If you use this library in software of any kind, please provide a link to the GitHub repository https://github.com/bshoshany/thread-pool in the source code and documentation. If you use this library in published research, please cite it as follows: Barak Shoshany, "A C++17 Thread Pool for High-Performance Scientific Computing", doi:10.5281/zenodo.4742687, arXiv:2105.00613 (May 2021)
+ *
+ * @brief BS::thread_pool: a fast, lightweight, and easy-to-use C++17 thread pool library. This header file contains the entire library, including the main BS::thread_pool class and the helper classes BS::multi_future, BS::blocks, BS:synced_stream, and BS::timer.
+ */
+
+#define BS_THREAD_POOL_VERSION "v3.2.0 (2022-07-28)"
+
+#include <atomic>             // std::atomic
+#include <chrono>             // std::chrono
+#include <condition_variable> // std::condition_variable
+#include <exception>          // std::current_exception
+#include <functional>         // std::bind, std::function, std::invoke
+#include <future>             // std::future, std::promise
+#include <iostream>           // std::cout, std::endl, std::flush, std::ostream
+#include <memory>             // std::make_shared, std::make_unique, std::shared_ptr, std::unique_ptr
+#include <mutex>              // std::mutex, std::scoped_lock, std::unique_lock
+#include <queue>              // std::queue
+#include <thread>             // std::thread
+#include <type_traits>        // std::common_type_t, std::conditional_t, std::decay_t, std::invoke_result_t, std::is_void_v
+#include <utility>            // std::forward, std::move, std::swap
+#include <vector>             // std::vector
+
+namespace BS
+{
+/**
+ * @brief A convenient shorthand for the type of std::thread::hardware_concurrency(). Should evaluate to unsigned int.
+ */
+using concurrency_t = std::invoke_result_t<decltype(std::thread::hardware_concurrency)>;
+
+/**
+ * @brief Explicit casts of the flushing stream manipulators, to enable using them with synced_stream, e.g. sync_out.print(BS::flush).
+ */
+std::ostream& (&endl)(std::ostream&) = static_cast<std::ostream& (&)(std::ostream&)>(std::endl);
+std::ostream& (&flush)(std::ostream&) = static_cast<std::ostream& (&)(std::ostream&)>(std::flush);
+
+// ============================================================================================= //
+//                                    Begin class multi_future                                   //
+
+/**
+ * @brief A helper class to facilitate waiting for and/or getting the results of multiple futures at once.
+ *
+ * @tparam T The return type of the futures.
+ */
+template <typename T>
+class [[nodiscard]] multi_future
+{
+public:
+    /**
+     * @brief Construct a multi_future object with the given number of futures.
+     *
+     * @param num_futures_ The desired number of futures to store.
+     */
+    multi_future(const size_t num_futures_ = 0) : f(num_futures_) {}
+
+    /**
+     * @brief Get the results from all the futures stored in this multi_future object, rethrowing any stored exceptions.
+     *
+     * @return If the futures return void, this function returns void as well. Otherwise, it returns a vector containing the results.
+     */
+    [[nodiscard]] std::conditional_t<std::is_void_v<T>, void, std::vector<T>> get()
+    {
+        if constexpr (std::is_void_v<T>)
+        {
+            for (size_t i = 0; i < f.size(); ++i)
+                f[i].get();
+            return;
+        }
+        else
+        {
+            std::vector<T> results(f.size());
+            for (size_t i = 0; i < f.size(); ++i)
+                results[i] = f[i].get();
+            return results;
+        }
+    }
+
+    /**
+     * @brief Wait for all the futures stored in this multi_future object.
+     */
+    void wait() const
+    {
+        for (size_t i = 0; i < f.size(); ++i)
+            f[i].wait();
+    }
+
+    /**
+     * @brief A vector to store the futures.
+     */
+    std::vector<std::future<T>> f;
+};
+
+//                                     End class multi_future                                    //
+// ============================================================================================= //
+
+// ============================================================================================= //
+//                                       Begin class blocks                                      //
+
+/**
+ * @brief A helper class to divide a range into blocks. Used by parallelize_loop() and push_loop().
+ *
+ * @tparam T1 The type of the first index in the range. Should be a signed or unsigned integer.
+ * @tparam T2 The type of the index after the last index in the range. Should be a signed or unsigned integer. If T1 is not the same as T2, a common type will be automatically inferred.
+ * @tparam T The common type of T1 and T2.
+ */
+template <typename T1, typename T2, typename T = std::common_type_t<T1, T2>>
+class [[nodiscard]] blocks
+{
+public:
+    /**
+     * @brief Construct a blocks object with the given specifications.
+     *
+     * @param first_index_ The first index in the range.
+     * @param index_after_last_ The index after the last index in the range.
+     * @param num_blocks_ The desired number of blocks to divide the range into.
+     */
+    blocks(const T1 first_index_, const T2 index_after_last_, const size_t num_blocks_) : first_index(static_cast<T>(first_index_)), index_after_last(static_cast<T>(index_after_last_)), num_blocks(num_blocks_)
+    {
+        if (index_after_last < first_index)
+            std::swap(index_after_last, first_index);
+        total_size = static_cast<size_t>(index_after_last - first_index);
+        block_size = static_cast<size_t>(total_size / num_blocks);
+        if (block_size == 0)
+        {
+            block_size = 1;
+            num_blocks = (total_size > 1) ? total_size : 1;
+        }
+    }
+
+    /**
+     * @brief Get the first index of a block.
+     *
+     * @param i The block number.
+     * @return The first index.
+     */
+    [[nodiscard]] T start(const size_t i) const
+    {
+        return static_cast<T>(i * block_size) + first_index;
+    }
+
+    /**
+     * @brief Get the index after the last index of a block.
+     *
+     * @param i The block number.
+     * @return The index after the last index.
+     */
+    [[nodiscard]] T end(const size_t i) const
+    {
+        return (i == num_blocks - 1) ? index_after_last : (static_cast<T>((i + 1) * block_size) + first_index);
+    }
+
+    /**
+     * @brief Get the number of blocks. Note that this may be different than the desired number of blocks that was passed to the constructor.
+     *
+     * @return The number of blocks.
+     */
+    [[nodiscard]] size_t get_num_blocks() const
+    {
+        return num_blocks;
+    }
+
+    /**
+     * @brief Get the total number of indices in the range.
+     *
+     * @return The total number of indices.
+     */
+    [[nodiscard]] size_t get_total_size() const
+    {
+        return total_size;
+    }
+
+private:
+    /**
+     * @brief The size of each block (except possibly the last block).
+     */
+    size_t block_size = 0;
+
+    /**
+     * @brief The first index in the range.
+     */
+    T first_index = 0;
+
+    /**
+     * @brief The index after the last index in the range.
+     */
+    T index_after_last = 0;
+
+    /**
+     * @brief The number of blocks.
+     */
+    size_t num_blocks = 0;
+
+    /**
+     * @brief The total number of indices in the range.
+     */
+    size_t total_size = 0;
+};
+
+//                                        End class blocks                                       //
+// ============================================================================================= //
+
+// ============================================================================================= //
+//                                    Begin class thread_pool                                    //
+
+/**
+ * @brief A fast, lightweight, and easy-to-use C++17 thread pool class.
+ */
+class [[nodiscard]] thread_pool
+{
+public:
+    // ============================
+    // Constructors and destructors
+    // ============================
+
+    /**
+     * @brief Construct a new thread pool.
+     *
+     * @param thread_count_ The number of threads to use. The default value is the total number of hardware threads available, as reported by the implementation. This is usually determined by the number of cores in the CPU. If a core is hyperthreaded, it will count as two threads.
+     */
+    thread_pool(const concurrency_t thread_count_ = 0) : thread_count(determine_thread_count(thread_count_)), threads(std::make_unique<std::thread[]>(determine_thread_count(thread_count_)))
+    {
+        create_threads();
+    }
+
+    /**
+     * @brief Destruct the thread pool. Waits for all tasks to complete, then destroys all threads. Note that if the variable paused is set to true, then any tasks still in the queue will never be executed.
+     */
+    ~thread_pool()
+    {
+        wait_for_tasks();
+        destroy_threads();
+    }
+
+    // =======================
+    // Public member functions
+    // =======================
+
+    /**
+     * @brief Get the number of tasks currently waiting in the queue to be executed by the threads.
+     *
+     * @return The number of queued tasks.
+     */
+    [[nodiscard]] size_t get_tasks_queued() const
+    {
+        const std::scoped_lock tasks_lock(tasks_mutex);
+        return tasks.size();
+    }
+
+    /**
+     * @brief Get the number of tasks currently being executed by the threads.
+     *
+     * @return The number of running tasks.
+     */
+    [[nodiscard]] size_t get_tasks_running() const
+    {
+        const std::scoped_lock tasks_lock(tasks_mutex);
+        return tasks_total - tasks.size();
+    }
+
+    /**
+     * @brief Get the total number of unfinished tasks: either still in the queue, or running in a thread. Note that get_tasks_total() == get_tasks_queued() + get_tasks_running().
+     *
+     * @return The total number of tasks.
+     */
+    [[nodiscard]] size_t get_tasks_total() const
+    {
+        return tasks_total;
+    }
+
+    /**
+     * @brief Get the number of threads in the pool.
+     *
+     * @return The number of threads.
+     */
+    [[nodiscard]] concurrency_t get_thread_count() const
+    {
+        return thread_count;
+    }
+
+    /**
+     * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue. Returns a multi_future object that contains the futures for all of the blocks.
+     *
+     * @tparam F The type of the function to loop through.
+     * @tparam T1 The type of the first index in the loop. Should be a signed or unsigned integer.
+     * @tparam T2 The type of the index after the last index in the loop. Should be a signed or unsigned integer. If T1 is not the same as T2, a common type will be automatically inferred.
+     * @tparam T The common type of T1 and T2.
+     * @tparam R The return value of the loop function F (can be void).
+     * @param first_index The first index in the loop.
+     * @param index_after_last The index after the last index in the loop. The loop will iterate from first_index to (index_after_last - 1) inclusive. In other words, it will be equivalent to "for (T i = first_index; i < index_after_last; ++i)". Note that if index_after_last == first_index, no blocks will be submitted.
+     * @param loop The function to loop through. Will be called once per block. Should take exactly two arguments: the first index in the block and the index after the last index in the block. loop(start, end) should typically involve a loop of the form "for (T i = start; i < end; ++i)".
+     * @param num_blocks The maximum number of blocks to split the loop into. The default is to use the number of threads in the pool.
+     * @return A multi_future object that can be used to wait for all the blocks to finish. If the loop function returns a value, the multi_future object can also be used to obtain the values returned by each block.
+     */
+    template <typename F, typename T1, typename T2, typename T = std::common_type_t<T1, T2>, typename R = std::invoke_result_t<std::decay_t<F>, T, T>>
+    [[nodiscard]] multi_future<R> parallelize_loop(const T1 first_index, const T2 index_after_last, F&& loop, const size_t num_blocks = 0)
+    {
+        blocks blks(first_index, index_after_last, num_blocks ? num_blocks : thread_count);
+        if (blks.get_total_size() > 0)
+        {
+            multi_future<R> mf(blks.get_num_blocks());
+            for (size_t i = 0; i < blks.get_num_blocks(); ++i)
+                mf.f[i] = submit(std::forward<F>(loop), blks.start(i), blks.end(i));
+            return mf;
+        }
+        else
+        {
+            return multi_future<R>();
+        }
+    }
+
+    /**
+     * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue. Returns a multi_future object that contains the futures for all of the blocks. This overload is used for the special case where the first index is 0.
+     *
+     * @tparam F The type of the function to loop through.
+     * @tparam T The type of the loop indices. Should be a signed or unsigned integer.
+     * @tparam R The return value of the loop function F (can be void).
+     * @param index_after_last The index after the last index in the loop. The loop will iterate from 0 to (index_after_last - 1) inclusive. In other words, it will be equivalent to "for (T i = 0; i < index_after_last; ++i)". Note that if index_after_last == 0, no blocks will be submitted.
+     * @param loop The function to loop through. Will be called once per block. Should take exactly two arguments: the first index in the block and the index after the last index in the block. loop(start, end) should typically involve a loop of the form "for (T i = start; i < end; ++i)".
+     * @param num_blocks The maximum number of blocks to split the loop into. The default is to use the number of threads in the pool.
+     * @return A multi_future object that can be used to wait for all the blocks to finish. If the loop function returns a value, the multi_future object can also be used to obtain the values returned by each block.
+     */
+    template <typename F, typename T, typename R = std::invoke_result_t<std::decay_t<F>, T, T>>
+    [[nodiscard]] multi_future<R> parallelize_loop(const T index_after_last, F&& loop, const size_t num_blocks = 0)
+    {
+        return parallelize_loop(0, index_after_last, std::forward<F>(loop), num_blocks);
+    }
+
+    /**
+     * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue. Does not return a multi_future, so the user must use wait_for_tasks() or some other method to ensure that the loop finishes executing, otherwise bad things will happen.
+     *
+     * @tparam F The type of the function to loop through.
+     * @tparam T1 The type of the first index in the loop. Should be a signed or unsigned integer.
+     * @tparam T2 The type of the index after the last index in the loop. Should be a signed or unsigned integer. If T1 is not the same as T2, a common type will be automatically inferred.
+     * @tparam T The common type of T1 and T2.
+     * @param first_index The first index in the loop.
+     * @param index_after_last The index after the last index in the loop. The loop will iterate from first_index to (index_after_last - 1) inclusive. In other words, it will be equivalent to "for (T i = first_index; i < index_after_last; ++i)". Note that if index_after_last == first_index, no blocks will be submitted.
+     * @param loop The function to loop through. Will be called once per block. Should take exactly two arguments: the first index in the block and the index after the last index in the block. loop(start, end) should typically involve a loop of the form "for (T i = start; i < end; ++i)".
+     * @param num_blocks The maximum number of blocks to split the loop into. The default is to use the number of threads in the pool.
+     */
+    template <typename F, typename T1, typename T2, typename T = std::common_type_t<T1, T2>>
+    void push_loop(const T1 first_index, const T2 index_after_last, F&& loop, const size_t num_blocks = 0)
+    {
+        blocks blks(first_index, index_after_last, num_blocks ? num_blocks : thread_count);
+        if (blks.get_total_size() > 0)
+        {
+            for (size_t i = 0; i < blks.get_num_blocks(); ++i)
+                push_task(std::forward<F>(loop), blks.start(i), blks.end(i));
+        }
+    }
+
+    /**
+     * @brief Parallelize a loop by automatically splitting it into blocks and submitting each block separately to the queue. Does not return a multi_future, so the user must use wait_for_tasks() or some other method to ensure that the loop finishes executing, otherwise bad things will happen. This overload is used for the special case where the first index is 0.
+     *
+     * @tparam F The type of the function to loop through.
+     * @tparam T The type of the loop indices. Should be a signed or unsigned integer.
+     * @param index_after_last The index after the last index in the loop. The loop will iterate from 0 to (index_after_last - 1) inclusive. In other words, it will be equivalent to "for (T i = 0; i < index_after_last; ++i)". Note that if index_after_last == 0, no blocks will be submitted.
+     * @param loop The function to loop through. Will be called once per block. Should take exactly two arguments: the first index in the block and the index after the last index in the block. loop(start, end) should typically involve a loop of the form "for (T i = start; i < end; ++i)".
+     * @param num_blocks The maximum number of blocks to split the loop into. The default is to use the number of threads in the pool.
+     */
+    template <typename F, typename T>
+    void push_loop(const T index_after_last, F&& loop, const size_t num_blocks = 0)
+    {
+        push_loop(0, index_after_last, std::forward<F>(loop), num_blocks);
+    }
+
+    /**
+     * @brief Push a function with zero or more arguments, but no return value, into the task queue. Does not return a future, so the user must use wait_for_tasks() or some other method to ensure that the task finishes executing, otherwise bad things will happen.
+     *
+     * @tparam F The type of the function.
+     * @tparam A The types of the arguments.
+     * @param task The function to push.
+     * @param args The zero or more arguments to pass to the function. Note that if the task is a class member function, the first argument must be a pointer to the object, i.e. &object (or this), followed by the actual arguments.
+     */
+    template <typename F, typename... A>
+    void push_task(F&& task, A&&... args)
+    {
+        std::function<void()> task_function = std::bind(std::forward<F>(task), std::forward<A>(args)...);
+        {
+            const std::scoped_lock tasks_lock(tasks_mutex);
+            tasks.push(task_function);
+        }
+        ++tasks_total;
+        task_available_cv.notify_one();
+    }
+
+    /**
+     * @brief Reset the number of threads in the pool. Waits for all currently running tasks to be completed, then destroys all threads in the pool and creates a new thread pool with the new number of threads. Any tasks that were waiting in the queue before the pool was reset will then be executed by the new threads. If the pool was paused before resetting it, the new pool will be paused as well.
+     *
+     * @param thread_count_ The number of threads to use. The default value is the total number of hardware threads available, as reported by the implementation. This is usually determined by the number of cores in the CPU. If a core is hyperthreaded, it will count as two threads.
+     */
+    void reset(const concurrency_t thread_count_ = 0)
+    {
+        const bool was_paused = paused;
+        paused = true;
+        wait_for_tasks();
+        destroy_threads();
+        thread_count = determine_thread_count(thread_count_);
+        threads = std::make_unique<std::thread[]>(thread_count);
+        paused = was_paused;
+        create_threads();
+    }
+
+    /**
+     * @brief Submit a function with zero or more arguments into the task queue. If the function has a return value, get a future for the eventual returned value. If the function has no return value, get an std::future<void> which can be used to wait until the task finishes.
+     *
+     * @tparam F The type of the function.
+     * @tparam A The types of the zero or more arguments to pass to the function.
+     * @tparam R The return type of the function (can be void).
+     * @param task The function to submit.
+     * @param args The zero or more arguments to pass to the function. Note that if the task is a class member function, the first argument must be a pointer to the object, i.e. &object (or this), followed by the actual arguments.
+     * @return A future to be used later to wait for the function to finish executing and/or obtain its returned value if it has one.
+     */
+    template <typename F, typename... A, typename R = std::invoke_result_t<std::decay_t<F>, std::decay_t<A>...>>
+    [[nodiscard]] std::future<R> submit(F&& task, A&&... args)
+    {
+        std::function<R()> task_function = std::bind(std::forward<F>(task), std::forward<A>(args)...);
+        std::shared_ptr<std::promise<R>> task_promise = std::make_shared<std::promise<R>>();
+        push_task(
+            [task_function, task_promise]
+            {
+                try
+                {
+                    if constexpr (std::is_void_v<R>)
+                    {
+                        std::invoke(task_function);
+                        task_promise->set_value();
+                    }
+                    else
+                    {
+                        task_promise->set_value(std::invoke(task_function));
+                    }
+                }
+                catch (...)
+                {
+                    try
+                    {
+                        task_promise->set_exception(std::current_exception());
+                    }
+                    catch (...)
+                    {
+                    }
+                }
+            });
+        return task_promise->get_future();
+    }
+
+    /**
+     * @brief Wait for tasks to be completed. Normally, this function waits for all tasks, both those that are currently running in the threads and those that are still waiting in the queue. However, if the pool is paused, this function only waits for the currently running tasks (otherwise it would wait forever). Note: To wait for just one specific task, use submit() instead, and call the wait() member function of the generated future.
+     */
+    void wait_for_tasks()
+    {
+        waiting = true;
+        std::unique_lock<std::mutex> tasks_lock(tasks_mutex);
+        task_done_cv.wait(tasks_lock, [this] { return (tasks_total == (paused ? tasks.size() : 0)); });
+        waiting = false;
+    }
+
+    // ===========
+    // Public data
+    // ===========
+
+    /**
+     * @brief An atomic variable indicating whether the workers should pause. When set to true, the workers temporarily stop retrieving new tasks out of the queue, although any tasks already executed will keep running until they are finished. Set to false again to resume retrieving tasks.
+     */
+    std::atomic<bool> paused = false;
+
+private:
+    // ========================
+    // Private member functions
+    // ========================
+
+    /**
+     * @brief Create the threads in the pool and assign a worker to each thread.
+     */
+    void create_threads()
+    {
+        running = true;
+        for (concurrency_t i = 0; i < thread_count; ++i)
+        {
+            threads[i] = std::thread(&thread_pool::worker, this);
+        }
+    }
+
+    /**
+     * @brief Destroy the threads in the pool.
+     */
+    void destroy_threads()
+    {
+        running = false;
+        task_available_cv.notify_all();
+        for (concurrency_t i = 0; i < thread_count; ++i)
+        {
+            threads[i].join();
+        }
+    }
+
+    /**
+     * @brief Determine how many threads the pool should have, based on the parameter passed to the constructor or reset().
+     *
+     * @param thread_count_ The parameter passed to the constructor or reset(). If the parameter is a positive number, then the pool will be created with this number of threads. If the parameter is non-positive, or a parameter was not supplied (in which case it will have the default value of 0), then the pool will be created with the total number of hardware threads available, as obtained from std::thread::hardware_concurrency(). If the latter returns a non-positive number for some reason, then the pool will be created with just one thread.
+     * @return The number of threads to use for constructing the pool.
+     */
+    [[nodiscard]] concurrency_t determine_thread_count(const concurrency_t thread_count_)
+    {
+        if (thread_count_ > 0)
+            return thread_count_;
+        else
+        {
+            if (std::thread::hardware_concurrency() > 0)
+                return std::thread::hardware_concurrency();
+            else
+                return 1;
+        }
+    }
+
+    /**
+     * @brief A worker function to be assigned to each thread in the pool. Waits until it is notified by push_task() that a task is available, and then retrieves the task from the queue and executes it. Once the task finishes, the worker notifies wait_for_tasks() in case it is waiting.
+     */
+    void worker()
+    {
+        while (running)
+        {
+            std::function<void()> task;
+            std::unique_lock<std::mutex> tasks_lock(tasks_mutex);
+            task_available_cv.wait(tasks_lock, [this] { return !tasks.empty() || !running; });
+            if (running && !paused)
+            {
+                task = std::move(tasks.front());
+                tasks.pop();
+                tasks_lock.unlock();
+                task();
+                tasks_lock.lock();
+                --tasks_total;
+                if (waiting)
+                    task_done_cv.notify_one();
+            }
+        }
+    }
+
+    // ============
+    // Private data
+    // ============
+
+    /**
+     * @brief An atomic variable indicating to the workers to keep running. When set to false, the workers permanently stop working.
+     */
+    std::atomic<bool> running = false;
+
+    /**
+     * @brief A condition variable used to notify worker() that a new task has become available.
+     */
+    std::condition_variable task_available_cv = {};
+
+    /**
+     * @brief A condition variable used to notify wait_for_tasks() that a tasks is done.
+     */
+    std::condition_variable task_done_cv = {};
+
+    /**
+     * @brief A queue of tasks to be executed by the threads.
+     */
+    std::queue<std::function<void()>> tasks = {};
+
+    /**
+     * @brief An atomic variable to keep track of the total number of unfinished tasks - either still in the queue, or running in a thread.
+     */
+    std::atomic<size_t> tasks_total = 0;
+
+    /**
+     * @brief A mutex to synchronize access to the task queue by different threads.
+     */
+    mutable std::mutex tasks_mutex = {};
+
+    /**
+     * @brief The number of threads in the pool.
+     */
+    concurrency_t thread_count = 0;
+
+    /**
+     * @brief A smart pointer to manage the memory allocated for the threads.
+     */
+    std::unique_ptr<std::thread[]> threads = nullptr;
+
+    /**
+     * @brief An atomic variable indicating that wait_for_tasks() is active and expects to be notified whenever a task is done.
+     */
+    std::atomic<bool> waiting = false;
+};
+
+//                                     End class thread_pool                                     //
+// ============================================================================================= //
+
+// ============================================================================================= //
+//                                   Begin class synced_stream                                   //
+
+/**
+ * @brief A helper class to synchronize printing to an output stream by different threads.
+ */
+class [[nodiscard]] synced_stream
+{
+public:
+    /**
+     * @brief Construct a new synced stream.
+     *
+     * @param out_stream_ The output stream to print to. The default value is std::cout.
+     */
+    synced_stream(std::ostream& out_stream_ = std::cout) : out_stream(out_stream_) {}
+
+    /**
+     * @brief Print any number of items into the output stream. Ensures that no other threads print to this stream simultaneously, as long as they all exclusively use the same synced_stream object to print.
+     *
+     * @tparam T The types of the items
+     * @param items The items to print.
+     */
+    template <typename... T>
+    void print(T&&... items)
+    {
+        const std::scoped_lock lock(stream_mutex);
+        (out_stream << ... << std::forward<T>(items));
+    }
+
+    /**
+     * @brief Print any number of items into the output stream, followed by a newline character. Ensures that no other threads print to this stream simultaneously, as long as they all exclusively use the same synced_stream object to print.
+     *
+     * @tparam T The types of the items
+     * @param items The items to print.
+     */
+    template <typename... T>
+    void println(T&&... items)
+    {
+        print(std::forward<T>(items)..., '\n');
+    }
+
+private:
+    /**
+     * @brief The output stream to print to.
+     */
+    std::ostream& out_stream;
+
+    /**
+     * @brief A mutex to synchronize printing.
+     */
+    mutable std::mutex stream_mutex = {};
+};
+
+//                                    End class synced_stream                                    //
+// ============================================================================================= //
+
+// ============================================================================================= //
+//                                       Begin class timer                                       //
+
+/**
+ * @brief A helper class to measure execution time for benchmarking purposes.
+ */
+class [[nodiscard]] timer
+{
+public:
+    /**
+     * @brief Start (or restart) measuring time.
+     */
+    void start()
+    {
+        start_time = std::chrono::steady_clock::now();
+    }
+
+    /**
+     * @brief Stop measuring time and store the elapsed time since start().
+     */
+    void stop()
+    {
+        elapsed_time = std::chrono::steady_clock::now() - start_time;
+    }
+
+    /**
+     * @brief Get the number of milliseconds that have elapsed between start() and stop().
+     *
+     * @return The number of milliseconds.
+     */
+    [[nodiscard]] std::chrono::milliseconds::rep ms() const
+    {
+        return (std::chrono::duration_cast<std::chrono::milliseconds>(elapsed_time)).count();
+    }
+
+private:
+    /**
+     * @brief The time point when measuring started.
+     */
+    std::chrono::time_point<std::chrono::steady_clock> start_time = std::chrono::steady_clock::now();
+
+    /**
+     * @brief The duration that has elapsed between start() and stop().
+     */
+    std::chrono::duration<double> elapsed_time = std::chrono::duration<double>::zero();
+};
+
+//                                        End class timer                                        //
+// ============================================================================================= //
+
+} // namespace BS

CHANGELOG.md

@@ -0,0 +1,198 @@
+[![DOI:10.5281/zenodo.4742687](https://zenodo.org/badge/DOI/10.5281/zenodo.4742687.svg)](https://doi.org/10.5281/zenodo.4742687)
+[![arXiv:2105.00613](https://img.shields.io/badge/arXiv-2105.00613-b31b1b.svg)](https://arxiv.org/abs/2105.00613)
+[![License: MIT](https://img.shields.io/github/license/bshoshany/thread-pool)](https://github.com/bshoshany/thread-pool/blob/master/LICENSE.txt)
+![Language: C++17](https://img.shields.io/badge/Language-C%2B%2B17-yellow)
+![File size in bytes](https://img.shields.io/github/size/bshoshany/thread-pool/BS_thread_pool.hpp)
+![GitHub last commit](https://img.shields.io/github/last-commit/bshoshany/thread-pool)
+[![GitHub repo stars](https://img.shields.io/github/stars/bshoshany/thread-pool?style=social)](https://github.com/bshoshany/thread-pool)
+[![Twitter @BarakShoshany](https://img.shields.io/twitter/follow/BarakShoshany?style=social)](https://twitter.com/BarakShoshany)
+[![Open in Visual Studio Code](https://img.shields.io/badge/-Open%20in%20Visual%20Studio%20Code-007acc)](https://vscode.dev/github/bshoshany/thread-pool)
+
+# `BS::thread_pool`: a fast, lightweight, and easy-to-use C++17 thread pool library
+
+By Barak Shoshany<br />
+Email: [baraksh@gmail.com](mailto:baraksh@gmail.com)<br />
+Website: [https://baraksh.com/](https://baraksh.com/)<br />
+GitHub: [https://github.com/bshoshany](https://github.com/bshoshany)<br />
+
+* [Version history](#version-history)
+    * [v3.2.0 (2022-07-28)](#v320-2022-07-28)
+    * [v3.1.0 (2022-07-13)](#v310-2022-07-13)
+    * [v3.0.0 (2022-05-30)](#v300-2022-05-30)
+    * [v2.0.0 (2021-08-14)](#v200-2021-08-14)
+    * [v1.9 (2021-07-29)](#v19-2021-07-29)
+    * [v1.8 (2021-07-28)](#v18-2021-07-28)
+    * [v1.7 (2021-06-02)](#v17-2021-06-02)
+    * [v1.6 (2021-05-26)](#v16-2021-05-26)
+    * [v1.5 (2021-05-07)](#v15-2021-05-07)
+    * [v1.4 (2021-05-05)](#v14-2021-05-05)
+    * [v1.3 (2021-05-03)](#v13-2021-05-03)
+    * [v1.2 (2021-04-29)](#v12-2021-04-29)
+    * [v1.1 (2021-04-24)](#v11-2021-04-24)
+    * [v1.0 (2021-01-15)](#v10-2021-01-15)
+
+## Version history
+
+### v3.2.0 (2022-07-28)
+
+* `BS_thread_pool.hpp`:
+    * Main `BS::thread_pool` class:
+        * Added a new member function, `push_loop()`, which does the same thing as `parallelize_loop()`, except that it does not return a `BS::multi_future` with the futures for each block. Just like `push_task()` vs. `submit()`, this avoids the overhead of creating the futures, but the user must use `wait_for_tasks()` or some other method to ensure that the loop finishes executing, otherwise bad things will happen.
+        * `push_task()` and `submit()` now utilize perfect forwarding in order to support more types of tasks - in particular member functions, which in previous versions could not be submitted unless wrapped in a lambda. To submit a member function, use the syntax `submit(&class::function, &object, args)`. More information can be found in `README.md`. See [#9](https://github.com/bshoshany/thread-pool/issues/9).
+        * `push_loop()` and `parallelize_loop()` now have overloads where the first argument (the first index in the loop) is omitted, in which case it is assumed to be 0. This is for convenience, as the case where the first index is 0 is very common.
+    * Helper classes:
+        * `BS::synced_stream` now utilizes perfect forwarding in the member functions `print()` and `println()`.
+        * Previously, it was impossible to pass the flushing manipulators `std::endl` and `std::flush` to `print()` and `println()`, since the compiler could not figure out which template specializations to use. The new objects `BS::endl` and `BS::flush` are explicit casts of these manipulators, whose sole purpose is to enable passing them to `print()` and `println()`.
+        * `BS::multi_future::get()` now rethrows exceptions generated by the futures, even if the futures return `void`. See [#62](https://github.com/bshoshany/thread-pool/pull/62).
+        * Added a new helper class, `BS::blocks`, which is used by `parallelize_loop()` and `push_loop()` to divide a range into blocks. This class is not documented in `README.md`, as it most likely will not be of interest to most users, but it is still publicly available, in case you want to parallelize something manually but still benefit from the built-in algorithm for splitting a range into blocks.
+* `BS_thread_pool_test.cpp`:
+    * Added plenty of new tests for the new features described above.
+    * Fixed a bug in `count_unique_threads()` that caused it to get stuck on certain systems.
+    * `dual_println()` now also flushes the stream using `BS::endl`, so that if the test gets stuck, the log file will still contain everything up to that point. (Note: It is a common misconception that `std::endl` and `'\n'` are interchangeable. `std::endl` not only prints a newline character, it also flushes the stream, which is not always desirable, as it may reduce performance.)
+    * The performance test has been modified as follows:
+        * Instead of generating random vectors using `std::mersenne_twister_engine`, which proved to be inconsistent across different compilers and systems, the test now generates each element via an arbitrarily-chosen numerical operation. In my testing, this provided much more consistent results.
+        * Instead of using a hard-coded vector size, a suitable vector size is now determined dynamically at runtime.
+        * Instead of using `parallelize_loop()`, the test now uses the new `push_loop()` function to squeeze out a bit more performance.
+        * Instead of setting the test parameters to achieve a fixed single-threaded mean execution time of 300 ms, the test now aims to achieve a fixed multi-threaded mean execution time of 50 ms when the number of blocks is equal to the number of threads. This allows for more reliable results on very fast CPUs with a very large number of threads, where the mean execution time when using all the threads could previously be below a statistically significant value.
+        * The number of vectors is now restricted to be a multiple of the number of threads, so that the blocks are always all of the same size.
+* `README.md`:
+    * Added instructions and examples for the new features described above.
+    * Rewrote the documentation for `parallelize_loop()` to make it clearer.
+
+### v3.1.0 (2022-07-13)
+
+* `BS_thread_pool.hpp`:
+    * Fixed an issue where `wait_for_tasks()` would sometimes get stuck if `push_task()` was executed immediately before `wait_for_tasks()`.
+    * Both the thread pool constructor and the `reset()` member function now determine the number of threads to use in the pool as follows. If the parameter is a positive number, then the pool will be created with this number of threads. If the parameter is non-positive, or a parameter was not supplied, then the pool will be created with the total number of hardware threads available, as obtained from `std::thread::hardware_concurrency()`. If the latter returns a non-positive number for some reason, then the pool will be created with just one thread. See [#51](https://github.com/bshoshany/thread-pool/issues/51) and [#52](https://github.com/bshoshany/thread-pool/issues/52).
+    * Added the `[[nodiscard]]` attribute to classes and class members, in order to warn the user when accidentally discarding an important return value, such as a future or the return value of a function with no useful side-effects. For example, if you use `submit()` and don't save the future it returns, the compiler will now generate a warning. (If a future is not needed, then you should use `push_task()` instead.)
+    * Removed the `explicit` specifier from all constructors, as it prevented the default constructor from being used with static class members. See [#48](https://github.com/bshoshany/thread-pool/issues/48>).
+* `BS_thread_pool_test.cpp`:
+    * Improved `count_unique_threads()` using condition variables, to ensure that each thread in the pool runs at least one task regardless of how fast it takes to run the tasks.
+    * When appropriate, `check()` now explicitly reports what the obtained result was and what it was expected to be.
+    * `check_task_monitoring()` and `check_pausing()` now explicitly report the results of the monitoring at each step.
+    * Changed all instances of `std::vector<std::atomic<bool>>` to `std::unique_ptr<std::atomic<bool>[]>`. See [#44](https://github.com/bshoshany/thread-pool/issues/44).
+    * Converted a few more C-style casts to C++ cast expressions.
+* `README.md`:
+    * Added instructions for using this package with the [Conan](https://conan.io/) C/C++ package manager. Please refer to [this package's page on ConanCenter](https://conan.io/center/bshoshany-thread-pool) to learn how to use Conan to include this package in your project with various build systems.
+* If you found this project useful, please consider [starring it on GitHub](https://github.com/bshoshany/thread-pool/stargazers)! This allows me to see how many people are using my code, and motivates me to keep working to improve it.
+
+### v3.0.0 (2022-05-30)
+
+* This is a major new release with many changes and improvements! Please note that code written using previous releases will need to be slightly modified to work with the new release. The changes needed to migrate to the new API are explicitly indicated below for your convenience.
+* Breaking changes to the library header file:
+    * The header file has been renamed to `BS_thread_pool.hpp` to avoid potential conflict with other thread pool libraries.
+        * **API migration:** The library must now be included by invoking `#include "BS_thread_pool.hpp"`.
+    * All the definitions in the library, including the `thread_pool` class and the helper classes, are now located in the namespace `BS`. This namespace will also be used for my other C++ projects, and is intended to ensure consistency between my projects while avoiding potential name conflicts with other libraries.
+        * **API migration:** The thread pool class should now be invoked as `BS::thread_pool`. Alternatively, it is possible to employ `using BS::thread_pool` or even `using namespace BS` and then invoke `thread_pool` directly. Same for the `BS::synced_stream` and `BS::timer` helper classes.
+    * The macro `THREAD_POOL_VERSION`, which contains the version number and release date of the library, has been renamed to `BS_THREAD_POOL_VERSION` to avoid potential conflicts.
+        * **API migration:** The version must now be read from the macro `BS_THREAD_POOL_VERSION`.
+    * The public member `sleep_duration` has been removed. The thread pool now uses condition variables instead of sleep to facilitate waiting. This significantly improves performance (by 10%-50% in my testing), drastically decreases idle CPU utilization, and eliminates the need to set an optimal sleep time. This was a highly-requested change; see [issue #1](https://github.com/bshoshany/thread-pool/issues/1), [issue #12](https://github.com/bshoshany/thread-pool/issues/12), and [pull request #23](https://github.com/bshoshany/thread-pool/pull/23).
+        * **API migration:** Remove any code that relates to the public member `sleep_duration`.
+    * The template specializations for `submit()` have been merged. Now instead of two versions, one for functions with a return value and one for functions without a return value, there is just one version, which can accept any function. This makes the code more compact (and elegant). If a function with no return value is submitted, an `std::future<void>` is returned (the previous version returned an `std::future<bool>`)
+        * **API migration:** To wait for a task with no return value, simply call `wait()` or `get()` on the corresponding `std::future<void>`.
+    * `parallelize_loop()` now returns a future in the form of a new `BS::multi_future` helper class template. The member function `wait()` of this future allows waiting until all of the loop's blocks finish executing. In previous versions, calling `parallelize_loop()` both parallelized the loop and waited for the blocks to finish; now it is possible to do other stuff while the loop executes.
+        * **API migration:** Since `parallelize_loop()` no longer automatically blocks, you should either store the result in a `BS::multi_future` object and call its `wait()` member function, or simply call `parallelize_loop().wait()` to reproduce the old behavior.
+* Non-breaking changes to the library header file:
+    * It is now possible to use `parallelize_loop()` with functions that have return values and get these values from all blocks at once through the `get()` member function of the `BS::multi_future`.
+    * The template specializations for `push_task()` have been merged. Now instead of two versions, one for functions with arguments and one for functions without arguments, there is just one version, which can accept any function.
+    * Constructors have been made `explicit`. See [issue #28](https://github.com/bshoshany/thread-pool/issues/28).
+    * `submit()` now uses `std::make_shared` instead of `new` to create the shared pointer. This means only one memory allocation is performed instead of two, which should improve performance. In addition, all unique pointers are now created using `std::make_unique`.
+    * A new helper class template, `BS::multi_future`, has been added. It's basically just a wrapper around `std::vector<std::future<T>>`. This class is used by the new implementation of `parallelize_loop()` to allow waiting for the entire loop, consisting of multiple tasks with their corresponding futures, to finish executing.
+    * `BS::multi_future` can also be used independently to handle multiple futures at once. For example, you can now keep track of several groups of tasks by storing their futures inside separate `BS::multi_future` objects and use either `wait()` to wait for all tasks in a specific group to finish or `get()` to get an `std::vector` with the return values of every task in the group.
+    * Integer types are now chosen in a smarter way to improve portability, allow for better compatibility with 32-bit systems, and prevent potential conversion errors.
+    * Added a new type, `BS::concurrency_t`, equal to the return type of `std::thread::hardware_concurrency()`. This is probably pointless, since the C++ standard requires this to be `unsigned int`, but it seems to me to make the code slightly more portable, in case some non-conforming compiler chooses to use a different integer type.
+    * C-style casts have been converted to C++ cast expressions for added clarity.
+    * Miscellaneous minor optimizations and style improvements.
+* Changes to the test program:
+    * The program has been renamed to `BS_thread_pool_test.cpp` to avoid potential conflict with other thread pool libraries.
+    * The program now returns `EXIT_FAILURE` if any of the tests failed, for automation purposes. See [pull request #42](https://github.com/bshoshany/thread-pool/pull/42).
+    * Fixed incorrect check order in `check_task_monitoring()`. See [pull request #43](https://github.com/bshoshany/thread-pool/pull/43).
+    * Added a new test for `parallelize_loop()` with a return value.
+    * Improved some of the tests to make them more reliable. For example, `count_unique_threads()` now uses futures (stored in a `BS::multi_future<void>` object).
+    * The program now uses `std::vector` instead of matrices, for both consistency checks and benchmarks, in order to simplify the code and considerably reduce its length.
+    * The benchmarks have been simplified. There's now only one test: filling a specific number of vectors of fixed size with random values. This may be replaced with something more practical in a future released, but at least on the systems I've tested on, it does demonstrate a very significant multi-threading speedup.
+    * In addition to multi-threaded tests with different numbers of tasks, the benchmark now also includes a single-threaded test. This allows for more accurate benchmarks compared to previous versions, as the (slight) parallelization overhead is now taken into account when calculating the maximum speedup.
+    * The program decides how many vectors to use for benchmarking by testing how many are needed to reach a target duration in the single-threaded test. This ensures that the test takes approximately the same amount of time on different systems, and is thus more consistent and portable.
+    * Miscellaneous minor optimizations and style improvements.
+* Changes to `README.md`:
+    * Many sections have been rewritten and/or polished.
+    * Explanations and examples of all the new features have been added.
+    * Added an acknowledgements section.
+* Miscellaneous changes:
+    * Added a `CITATION.bib` file (in BibTeX format) to the GitHub repository. You can use it to easily cite this package if you use it in any research papers.
+    * Added a `CITATION.cff` file (in YAML format) to the GitHub repository. This should add [an option to get a citation in different formats](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-on-github/about-citation-files) directly from GitHub repository by clicking on "cite this repository" on the sidebar to the right.
+    * Added templates for GitHub issues and pull requests.
+
+### v2.0.0 (2021-08-14)
+
+* From now on, version numbers will adhere to the [Semantic Versioning](https://semver.org/) specification in the format **major.minor.patch**.
+* A file named `thread_pool_test.cpp` has been added to the package. It will perform automated tests of all aspects of the package, and benchmark some multithreaded matrix operations. Please run it on your system and [submit a bug report](https://github.com/bshoshany/thread-pool/issues) if any of the tests fail. In addition, the code is thoroughly documented, and is meant to serve as an extensive example of how to properly use the package.
+* The package is now available through [vcpkg](https://github.com/microsoft/vcpkg). Instructions for how to install it have been added to `README.md`. See [this pull request](https://github.com/bshoshany/thread-pool/pull/18).
+* The package now defines a macro `THREAD_POOL_VERSION`, which returns the version number and release date of the thread pool library as a string.
+* `parallelize_loop()` has undergone some major changes (and is now incompatible with v1.x):
+    * The second argument is now the index **after** the last index, instead of the last index itself. This is more consistent with C++ conventions (e.g. standard library algorithms) where the range is always `[first, last)`. For example, for an array with `n` indices, instead of `parallelize_loop(0, n - 1, ...)` you should now write `parallelize_loop(0, n, ...)`.
+    * The `loop` function is now only called once per block, instead of once per index, as was the case before. This should provide a performance boost due to significantly reducing the number of function calls, and it also allows you to conserve resources by using them only once per block instead of once per index (an example can be found in the `random_matrix_generator` class in `thread_pool_test.cpp`). It also means that `loop` now takes two arguments: the first index in the block and the index after the last index in the block. Thus, `loop(start, end)` should typically involve a loop of the form `for (T i = start; i < end; i++)`.
+    * The first and last indices can now be of two different integer types. Previously, `parallelize_loop(0, i, ...)` did not work if `i` was not an `int`, because `0` was interpreted as `int`, and the two arguments had to be of the same type. Therefore, one had to use casting, e.g. `parallelize_loop((size_t)0, i)`, to make it work. Now this is no longer necessary; the common type is inferred automatically using `std::common_type_t`.
+
+### v1.9 (2021-07-29)
+
+* Fixed a bug in `reset()` which caused it to create the wrong number of threads.
+
+### v1.8 (2021-07-28)
+
+* The version history has become too long to be included in `README.md`, so I moved it to a separate file, `CHANGELOG.md`.
+* A button to open this repository directly in Visual Studio Code has been added to the badges in `README.md`.
+* An internal variable named `promise` has been renamed to `task_promise` to avoid any potential errors in case the user invokes `using namespace std`.
+* `submit()` now catches exceptions thrown by the submitted task and forwards them to the future. See [this issue](https://github.com/bshoshany/thread-pool/issues/14).
+* Eliminated compiler warnings that appeared when using the `-Weffc++` flag in GCC. See [this pull request](https://github.com/bshoshany/thread-pool/pull/17).
+
+### v1.7 (2021-06-02)
+
+* Fixed a bug in `parallelize_loop()` which prevented it from actually running loops in parallel, see [this issue](https://github.com/bshoshany/thread-pool/issues/11).
+
+### v1.6 (2021-05-26)
+
+* Since MSVC does not interpret `and` as `&&` by default, the previous release did not compile with MSVC unless the `/permissive-` or `/Za` compiler flags were used. This has been fixed in this version, and the code now successfully compiles with GCC, Clang, and MSVC. See [this pull request](https://github.com/bshoshany/thread-pool/pull/10).
+
+### v1.5 (2021-05-07)
+
+* This library now has a DOI for citation purposes. Information on how to cite it in publications has been added to the source code and to `README.md`.
+* Added GitHub badges to `README.md`.
+
+### v1.4 (2021-05-05)
+
+* Added three new public member functions to monitor the tasks submitted to the pool:
+    * `get_tasks_queued()` gets the number of tasks currently waiting in the queue to be executed by the threads.
+    * `get_tasks_running()` gets the number of tasks currently being executed by the threads.
+    * `get_tasks_total()` gets the total number of unfinished tasks - either still in the queue, or running in a thread.
+    * Note that `get_tasks_running() == get_tasks_total() - get_tasks_queued()`.
+    * Renamed the private member variable `tasks_waiting` to `tasks_total` to make its purpose clearer.
+* Added an option to temporarily pause the workers:
+    * When public member variable `paused` is set to `true`, the workers temporarily stop popping new tasks out of the queue, although any tasks already executed will keep running until they are done. Set to `false` again to resume popping tasks.
+    * While the workers are paused, `wait_for_tasks()` will wait for the running tasks instead of all tasks (otherwise it would wait forever).
+    * By utilizing the new pausing mechanism, `reset()` can now change the number of threads on-the-fly while there are still tasks waiting in the queue. The new thread pool will resume executing tasks from the queue once it is created.
+* `parallelize_loop()` and `wait_for_tasks()` now have the same behavior as the worker function with regards to waiting for tasks to complete. If the relevant tasks are not yet complete, then before checking again, they will sleep for `sleep_duration` microseconds, unless that variable is set to zero, in which case they will call `std::this_thread::yield()`. This should improve performance and reduce CPU usage.
+* Merged [this commit](https://github.com/bshoshany/thread-pool/pull/8): Fixed weird error when using MSVC and including `windows.h`.
+* The `README.md` file has been reorganized and expanded.
+
+### v1.3 (2021-05-03)
+
+* Fixed [this issue](https://github.com/bshoshany/thread-pool/issues/3): Removed `std::move` from the `return` statement in `push_task()`. This previously generated a `-Wpessimizing-move` warning in Clang. The assembly code generated by the compiler seems to be the same before and after this change, presumably because the compiler eliminates the `std::move` automatically, but this change gets rid of the Clang warning.
+* Fixed [this issue](https://github.com/bshoshany/thread-pool/issues/5): Removed a debugging message printed to `std::cout`, which was left in the code by mistake.
+* Fixed [this issue](https://github.com/bshoshany/thread-pool/issues/6): `parallelize_loop()` no longer sends references for the variables `start` and `stop` when calling `push_task()`, which may lead to undefined behavior.
+* A companion paper is now published at <a href="https://arxiv.org/abs/2105.00613">arXiv:2105.00613</a>, including additional information such as performance tests on systems with up to 80 hardware threads. The `README.md` has been updated, and it is now roughly identical in content to the paper.
+
+### v1.2 (2021-04-29)
+
+* The worker function, which controls the execution of tasks by each thread, now sleeps by default instead of yielding. Previously, when the worker could not find any tasks in the queue, it called `std::this_thread::yield()` and then tried again. However, this caused the workers to have high CPU usage when idle, [as reported by some users](https://github.com/bshoshany/thread-pool/issues/1). Now, when the worker function cannot find a task to run, it instead sleeps for a duration given by the public member variable `sleep_duration` (in microseconds) before checking the queue again. The default value is `1000` microseconds, which I found to be optimal in terms of both CPU usage and performance, but your own optimal value may be different.
+* If the constructor is called with an argument of zero for the number of threads, then the default value, `std::thread::hardware_concurrency()`, is used instead.
+* Added a simple helper class, `timer`, which can be used to measure execution time for benchmarking purposes.
+* Improved and expanded the documentation.
+
+### v1.1 (2021-04-24)
+
+* Cosmetic changes only. Fixed a typo in the Doxygen comments and added a link to the GitHub repository.
+
+### v1.0 (2021-01-15)
+
+* Initial release.

CITATION.bib

@@ -0,0 +1,13 @@
+@article{Shoshany2021_ThreadPool,
+    archiveprefix = {arXiv},
+    author        = {Barak Shoshany},
+    doi           = {10.5281/zenodo.4742687},
+    eid           = {arXiv:2105.00613},
+    eprint        = {2105.00613},
+    journal       = {arXiv e-prints},
+    keywords      = {Computer Science - Distributed, Parallel, and Cluster Computing, D.1.3, D.1.5},
+    month         = {May},
+    primaryclass  = {cs.DC},
+    title         = {{A C++17 Thread Pool for High-Performance Scientific Computing}},
+    year          = {2021}
+}

CITATION.cff

@@ -0,0 +1,24 @@
+---
+authors:
+    - family-names: "Shoshany"
+      given-names: "Barak"
+      orcid: "https://orcid.org/0000-0003-2222-127X"
+cff-version: "1.2.0"
+date-released: "2021-05-03"
+doi: "10.5281/zenodo.4742687"
+license: "MIT"
+message: "If you use this package in published research, please cite it as follows."
+repository-code: "https://github.com/bshoshany/thread-pool"
+title: "A C++17 Thread Pool for High-Performance Scientific Computing"
+preferred-citation:
+    type: "article"
+    authors:
+        - family-names: "Shoshany"
+          given-names: "Barak"
+          orcid: "https://orcid.org/0000-0003-2222-127X"
+    doi: "10.5281/zenodo.4742687"
+    journal: "arXiv"
+    month: 5
+    title: "A C++17 Thread Pool for High-Performance Scientific Computing"
+    url: "https://arxiv.org/abs/2105.00613"
+    year: 2021

LICENSE.txt

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2021 Barak Shoshany
+Copyright (c) 2022 Barak Shoshany
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

thread_pool.hpp

@@ -1,503 +0,0 @@
-#pragma once
-
-/**
- * @file thread_pool.hpp
- * @author Barak Shoshany (baraksh@gmail.com) (http://baraksh.com)
- * @version 1.6
- * @date 2021-05-26
- * @copyright Copyright (c) 2021 Barak Shoshany. Licensed under the MIT license. If you use this library in published research, please cite it as follows:
- *  - Barak Shoshany, "A C++17 Thread Pool for High-Performance Scientific Computing", doi:10.5281/zenodo.4742687, arXiv:2105.00613 (May 2021)
- *
- * @brief A C++17 thread pool for high-performance scientific computing.
- * @details A modern C++17-compatible thread pool implementation, built from scratch with high-performance scientific computing in mind. The thread pool is implemented as a single lightweight and self-contained class, and does not have any dependencies other than the C++17 standard library, thus allowing a great degree of portability. In particular, this implementation does not utilize OpenMP or any other high-level multithreading APIs, and thus gives the programmer precise low-level control over the details of the parallelization, which permits more robust optimizations. The thread pool was extensively tested on both AMD and Intel CPUs with up to 40 cores and 80 threads. Other features include automatic generation of futures and easy parallelization of loops. Two helper classes enable synchronizing printing to an output stream by different threads and measuring execution time for benchmarking purposes. Please visit the GitHub repository for documentation and updates, or to submit feature requests and bug reports.
- */
-
-#include <atomic>      // std::atomic
-#include <chrono>      // std::chrono
-#include <cstdint>     // std::int_fast64_t, std::uint_fast32_t
-#include <functional>  // std::function
-#include <future>      // std::future, std::promise
-#include <iostream>    // std::cout, std::ostream
-#include <memory>      // std::shared_ptr, std::unique_ptr
-#include <mutex>       // std::mutex, std::scoped_lock
-#include <queue>       // std::queue
-#include <thread>      // std::this_thread, std::thread
-#include <type_traits> // std::decay_t, std::enable_if_t, std::is_void_v, std::invoke_result_t
-#include <utility>     // std::move, std::swap
-
-// ============================================================================================= //
-//                                    Begin class thread_pool                                    //
-
-/**
- * @brief A C++17 thread pool class. The user submits tasks to be executed into a queue. Whenever a thread becomes available, it pops a task from the queue and executes it. Each task is automatically assigned a future, which can be used to wait for the task to finish executing and/or obtain its eventual return value.
- */
-class thread_pool
-{
-    typedef std::uint_fast32_t ui32;
-
-public:
-    // ============================
-    // Constructors and destructors
-    // ============================
-
-    /**
-     * @brief Construct a new thread pool.
-     *
-     * @param _thread_count The number of threads to use. The default value is the total number of hardware threads available, as reported by the implementation. With a hyperthreaded CPU, this will be twice the number of CPU cores. If the argument is zero, the default value will be used instead.
-     */
-    thread_pool(const ui32 &_thread_count = std::thread::hardware_concurrency())
-        : thread_count(_thread_count ? _thread_count : std::thread::hardware_concurrency()), threads(new std::thread[_thread_count ? _thread_count : std::thread::hardware_concurrency()])
-    {
-        create_threads();
-    }
-
-    /**
-     * @brief Destruct the thread pool. Waits for all tasks to complete, then destroys all threads. Note that if the variable paused is set to true, then any tasks still in the queue will never be executed.
-     */
-    ~thread_pool()
-    {
-        wait_for_tasks();
-        running = false;
-        destroy_threads();
-    }
-
-    // =======================
-    // Public member functions
-    // =======================
-
-    /**
-     * @brief Get the number of tasks currently waiting in the queue to be executed by the threads.
-     *
-     * @return The number of queued tasks.
-     */
-    size_t get_tasks_queued() const
-    {
-        const std::scoped_lock lock(queue_mutex);
-        return tasks.size();
-    }
-
-    /**
-     * @brief Get the number of tasks currently being executed by the threads.
-     *
-     * @return The number of running tasks.
-     */
-    ui32 get_tasks_running() const
-    {
-        return tasks_total - (ui32)get_tasks_queued();
-    }
-
-    /**
-     * @brief Get the total number of unfinished tasks - either still in the queue, or running in a thread.
-     *
-     * @return The total number of tasks.
-     */
-    ui32 get_tasks_total() const
-    {
-        return tasks_total;
-    }
-
-    /**
-     * @brief Get the number of threads in the pool.
-     *
-     * @return The number of threads.
-     */
-    ui32 get_thread_count() const
-    {
-        return thread_count;
-    }
-
-    /**
-     * @brief Parallelize a loop by splitting it into blocks, submitting each block separately to the thread pool, and waiting for all blocks to finish executing. The loop will be equivalent to: for (T i = first_index; i <= last_index; i++) loop(i);
-     *
-     * @tparam T The type of the loop index. Should be a signed or unsigned integer.
-     * @tparam F The type of the function to loop through.
-     * @param first_index The first index in the loop (inclusive).
-     * @param last_index The last index in the loop (inclusive).
-     * @param loop The function to loop through. Should take exactly one argument, the loop index.
-     * @param num_tasks The maximum number of tasks to split the loop into. The default is to use the number of threads in the pool.
-     */
-    template <typename T, typename F>
-    void parallelize_loop(T first_index, T last_index, const F &loop, ui32 num_tasks = 0)
-    {
-        if (num_tasks == 0)
-            num_tasks = thread_count;
-        if (last_index < first_index)
-            std::swap(last_index, first_index);
-        size_t total_size = last_index - first_index + 1;
-        size_t block_size = total_size / num_tasks;
-        if (block_size == 0)
-        {
-            block_size = 1;
-            num_tasks = (ui32)total_size > 1 ? (ui32)total_size : 1;
-        }
-        std::atomic<ui32> blocks_running = 0;
-        for (ui32 t = 0; t < num_tasks; t++)
-        {
-            T start = (T)(t * block_size + first_index);
-            T end = (t == num_tasks - 1) ? last_index : (T)((t + 1) * block_size + first_index - 1);
-            blocks_running++;
-            push_task([start, end, &loop, &blocks_running]
-                      {
-                          for (T i = start; i <= end; i++)
-                              loop(i);
-                          blocks_running--;
-                      });
-            while (blocks_running != 0)
-            {
-                sleep_or_yield();
-            }
-        }
-    }
-
-    /**
-     * @brief Push a function with no arguments or return value into the task queue.
-     *
-     * @tparam F The type of the function.
-     * @param task The function to push.
-     */
-    template <typename F>
-    void push_task(const F &task)
-    {
-        tasks_total++;
-        {
-            const std::scoped_lock lock(queue_mutex);
-            tasks.push(std::function<void()>(task));
-        }
-    }
-
-    /**
-     * @brief Push a function with arguments, but no return value, into the task queue.
-     * @details The function is wrapped inside a lambda in order to hide the arguments, as the tasks in the queue must be of type std::function<void()>, so they cannot have any arguments or return value. If no arguments are provided, the other overload will be used, in order to avoid the (slight) overhead of using a lambda.
-     *
-     * @tparam F The type of the function.
-     * @tparam A The types of the arguments.
-     * @param task The function to push.
-     * @param args The arguments to pass to the function.
-     */
-    template <typename F, typename... A>
-    void push_task(const F &task, const A &...args)
-    {
-        push_task([task, args...]
-                  { task(args...); });
-    }
-
-    /**
-     * @brief Reset the number of threads in the pool. Waits for all currently running tasks to be completed, then destroys all threads in the pool and creates a new thread pool with the new number of threads. Any tasks that were waiting in the queue before the pool was reset will then be executed by the new threads. If the pool was paused before resetting it, the new pool will be paused as well.
-     *
-     * @param _thread_count The number of threads to use. The default value is the total number of hardware threads available, as reported by the implementation. With a hyperthreaded CPU, this will be twice the number of CPU cores. If the argument is zero, the default value will be used instead.
-     */
-    void reset(const ui32 &_thread_count = std::thread::hardware_concurrency())
-    {
-        bool was_paused = paused;
-        paused = true;
-        wait_for_tasks();
-        running = false;
-        destroy_threads();
-        thread_count = _thread_count ? _thread_count : std::thread::hardware_concurrency();
-        threads.reset(new std::thread[thread_count]);
-        paused = was_paused;
-        create_threads();
-        running = true;
-    }
-
-    /**
-     * @brief Submit a function with zero or more arguments and no return value into the task queue, and get an std::future<bool> that will be set to true upon completion of the task.
-     *
-     * @tparam F The type of the function.
-     * @tparam A The types of the zero or more arguments to pass to the function.
-     * @param task The function to submit.
-     * @param args The zero or more arguments to pass to the function.
-     * @return A future to be used later to check if the function has finished its execution.
-     */
-    template <typename F, typename... A, typename = std::enable_if_t<std::is_void_v<std::invoke_result_t<std::decay_t<F>, std::decay_t<A>...>>>>
-    std::future<bool> submit(const F &task, const A &...args)
-    {
-        std::shared_ptr<std::promise<bool>> promise(new std::promise<bool>);
-        std::future<bool> future = promise->get_future();
-        push_task([task, args..., promise]
-                  {
-                      task(args...);
-                      promise->set_value(true);
-                  });
-        return future;
-    }
-
-    /**
-     * @brief Submit a function with zero or more arguments and a return value into the task queue, and get a future for its eventual returned value.
-     *
-     * @tparam F The type of the function.
-     * @tparam A The types of the zero or more arguments to pass to the function.
-     * @tparam R The return type of the function.
-     * @param task The function to submit.
-     * @param args The zero or more arguments to pass to the function.
-     * @return A future to be used later to obtain the function's returned value, waiting for it to finish its execution if needed.
-     */
-    template <typename F, typename... A, typename R = std::invoke_result_t<std::decay_t<F>, std::decay_t<A>...>, typename = std::enable_if_t<!std::is_void_v<R>>>
-    std::future<R> submit(const F &task, const A &...args)
-    {
-        std::shared_ptr<std::promise<R>> promise(new std::promise<R>);
-        std::future<R> future = promise->get_future();
-        push_task([task, args..., promise]
-                  { promise->set_value(task(args...)); });
-        return future;
-    }
-
-    /**
-     * @brief Wait for tasks to be completed. Normally, this function waits for all tasks, both those that are currently running in the threads and those that are still waiting in the queue. However, if the variable paused is set to true, this function only waits for the currently running tasks (otherwise it would wait forever). To wait for a specific task, use submit() instead, and call the wait() member function of the generated future.
-     */
-    void wait_for_tasks()
-    {
-        while (true)
-        {
-            if (!paused)
-            {
-                if (tasks_total == 0)
-                    break;
-            }
-            else
-            {
-                if (get_tasks_running() == 0)
-                    break;
-            }
-            sleep_or_yield();
-        }
-    }
-
-    // ===========
-    // Public data
-    // ===========
-
-    /**
-     * @brief An atomic variable indicating to the workers to pause. When set to true, the workers temporarily stop popping new tasks out of the queue, although any tasks already executed will keep running until they are done. Set to false again to resume popping tasks.
-     */
-    std::atomic<bool> paused = false;
-
-    /**
-     * @brief The duration, in microseconds, that the worker function should sleep for when it cannot find any tasks in the queue. If set to 0, then instead of sleeping, the worker function will execute std::this_thread::yield() if there are no tasks in the queue. The default value is 1000.
-     */
-    ui32 sleep_duration = 1000;
-
-private:
-    // ========================
-    // Private member functions
-    // ========================
-
-    /**
-     * @brief Create the threads in the pool and assign a worker to each thread.
-     */
-    void create_threads()
-    {
-        for (ui32 i = 0; i < thread_count; i++)
-        {
-            threads[i] = std::thread(&thread_pool::worker, this);
-        }
-    }
-
-    /**
-     * @brief Destroy the threads in the pool by joining them.
-     */
-    void destroy_threads()
-    {
-        for (ui32 i = 0; i < thread_count; i++)
-        {
-            threads[i].join();
-        }
-    }
-
-    /**
-     * @brief Try to pop a new task out of the queue.
-     *
-     * @param task A reference to the task. Will be populated with a function if the queue is not empty.
-     * @return true if a task was found, false if the queue is empty.
-     */
-    bool pop_task(std::function<void()> &task)
-    {
-        const std::scoped_lock lock(queue_mutex);
-        if (tasks.empty())
-            return false;
-        else
-        {
-            task = std::move(tasks.front());
-            tasks.pop();
-            return true;
-        }
-    }
-
-    /**
-     * @brief Sleep for sleep_duration microseconds. If that variable is set to zero, yield instead.
-     *
-     */
-    void sleep_or_yield()
-    {
-        if (sleep_duration)
-            std::this_thread::sleep_for(std::chrono::microseconds(sleep_duration));
-        else
-            std::this_thread::yield();
-    }
-
-    /**
-     * @brief A worker function to be assigned to each thread in the pool. Continuously pops tasks out of the queue and executes them, as long as the atomic variable running is set to true.
-     */
-    void worker()
-    {
-        while (running)
-        {
-            std::function<void()> task;
-            if (!paused && pop_task(task))
-            {
-                task();
-                tasks_total--;
-            }
-            else
-            {
-                sleep_or_yield();
-            }
-        }
-    }
-
-    // ============
-    // Private data
-    // ============
-
-    /**
-     * @brief A mutex to synchronize access to the task queue by different threads.
-     */
-    mutable std::mutex queue_mutex;
-
-    /**
-     * @brief An atomic variable indicating to the workers to keep running. When set to false, the workers permanently stop working.
-     */
-    std::atomic<bool> running = true;
-
-    /**
-     * @brief A queue of tasks to be executed by the threads.
-     */
-    std::queue<std::function<void()>> tasks;
-
-    /**
-     * @brief The number of threads in the pool.
-     */
-    ui32 thread_count;
-
-    /**
-     * @brief A smart pointer to manage the memory allocated for the threads.
-     */
-    std::unique_ptr<std::thread[]> threads;
-
-    /**
-     * @brief An atomic variable to keep track of the total number of unfinished tasks - either still in the queue, or running in a thread.
-     */
-    std::atomic<ui32> tasks_total = 0;
-};
-
-//                                     End class thread_pool                                     //
-// ============================================================================================= //
-
-// ============================================================================================= //
-//                                   Begin class synced_stream                                   //
-
-/**
- * @brief A helper class to synchronize printing to an output stream by different threads.
- */
-class synced_stream
-{
-public:
-    /**
-     * @brief Construct a new synced stream.
-     *
-     * @param _out_stream The output stream to print to. The default value is std::cout.
-     */
-    synced_stream(std::ostream &_out_stream = std::cout)
-        : out_stream(_out_stream){};
-
-    /**
-     * @brief Print any number of items into the output stream. Ensures that no other threads print to this stream simultaneously, as long as they all exclusively use this synced_stream object to print.
-     *
-     * @tparam T The types of the items
-     * @param items The items to print.
-     */
-    template <typename... T>
-    void print(const T &...items)
-    {
-        const std::scoped_lock lock(stream_mutex);
-        (out_stream << ... << items);
-    }
-
-    /**
-     * @brief Print any number of items into the output stream, followed by a newline character. Ensures that no other threads print to this stream simultaneously, as long as they all exclusively use this synced_stream object to print.
-     *
-     * @tparam T The types of the items
-     * @param items The items to print.
-     */
-    template <typename... T>
-    void println(const T &...items)
-    {
-        print(items..., '\n');
-    }
-
-private:
-    /**
-     * @brief A mutex to synchronize printing.
-     */
-    mutable std::mutex stream_mutex;
-
-    /**
-     * @brief The output stream to print to.
-     */
-    std::ostream &out_stream;
-};
-
-//                                    End class synced_stream                                    //
-// ============================================================================================= //
-
-// ============================================================================================= //
-//                                       Begin class timer                                       //
-
-/**
- * @brief A helper class to measure execution time for benchmarking purposes.
- */
-class timer
-{
-    typedef std::int_fast64_t i64;
-
-public:
-    /**
-     * @brief Start (or restart) measuring time.
-     */
-    void start()
-    {
-        start_time = std::chrono::steady_clock::now();
-    }
-
-    /**
-     * @brief Stop measuring time and store the elapsed time since start().
-     */
-    void stop()
-    {
-        elapsed_time = std::chrono::steady_clock::now() - start_time;
-    }
-
-    /**
-     * @brief Get the number of milliseconds that have elapsed between start() and stop().
-     *
-     * @return The number of milliseconds.
-     */
-    i64 ms() const
-    {
-        return (std::chrono::duration_cast<std::chrono::milliseconds>(elapsed_time)).count();
-    }
-
-private:
-    /**
-     * @brief The time point when measuring started.
-     */
-    std::chrono::time_point<std::chrono::steady_clock> start_time = std::chrono::steady_clock::now();
-
-    /**
-     * @brief The duration that has elapsed between start() and stop().
-     */
-    std::chrono::duration<double> elapsed_time = std::chrono::duration<double>::zero();
-};
-
-//                                        End class timer                                        //
-// ============================================================================================= //