thread-pool/CHANGELOG.md

BS::thread_pool: a fast, lightweight, and easy-to-use C++17 thread pool library

By Barak Shoshany (baraksh@gmail.com) (https://baraksh.com/)

• Version history
  • v3.0.0 (2022-05-30)
  • v2.0.0 (2021-08-14)
  • v1.9 (2021-07-29)
  • v1.8 (2021-07-28)
  • v1.7 (2021-06-02)
  • v1.6 (2021-05-26)
  • v1.5 (2021-05-07)
  • v1.4 (2021-05-05)
  • v1.3 (2021-05-03)
  • v1.2 (2021-04-29)
  • v1.1 (2021-04-24)
  • v1.0 (2021-01-15)

Version history

v3.0.0 (2022-05-30)

• This is a major new release with many changes and improvements! Please note that code written using previous releases will need to be slightly modified to work with the new release. The changes needed to migrate to the new API are explicitly indicated below for your convenience.
• Breaking changes to the library header file:
  • The header file has been renamed to BS_thread_pool.hpp to avoid potential conflict with other thread pool libraries.
    • API migration: The library must now be included by invoking #include "BS_thread_pool.hpp".
  • All the definitions in the library, including the thread_pool class and the helper classes, are now located in the namespace BS. This namespace will also be used for my other C++ projects, and is intended to ensure consistency between my projects while avoiding potential name conflicts with other libraries.
    • API migration: The thread pool class should now be invoked as BS::thread_pool. Alternatively, it is possible to employ using BS::thread_pool or even using namespace BS and then invoke thread_pool directly. Same for the BS::synced_stream and BS::timer helper classes.
  • The macro THREAD_POOL_VERSION, which contains the version number and release date of the library, has been renamed to BS_THREAD_POOL_VERSION to avoid potential conflicts.
    • API migration: The version must now be read from the macro BS_THREAD_POOL_VERSION.
  • The public member sleep_duration has been removed. The thread pool now uses condition variables instead of sleep to facilitate waiting. This significantly improves performance (by 10%-50% in my testing), drastically decreases idle CPU utilization, and eliminates the need to set an optimal sleep time. This was a highly-requested change; see issue #1, issue #12, and pull request #23.
    • API migration: Remove any code that relates to the public member sleep_duration.
  • The template specializations for submit() have been merged. Now instead of two versions, one for functions with a return value and one for functions without a return value, there is just one version, which can accept any function. This makes the code more compact (and elegant). If a function with no return value is submitted, an std::future<void> is returned (the previous version returned an std::future<bool>)
    • API migration: To wait for a task with no return value, simply call wait() or get() on the corresponding std::future<void>.
  • parallelize_loop() now returns a future in the form of a new BS::multi_future helper class template. The member function wait() of this future allows waiting until all of the loop's blocks finish executing. In previous versions, calling parallelize_loop() both parallelized the loop and waited for the blocks to finish; now it is possible to do other stuff while the loop executes.
    • API migration: Since parallelize_loop() no longer automatically blocks, you should either store the result in a BS::multi_future object and call its wait() member function, or simply call parallelize_loop().wait() to reproduce the old behavior.
• Non-breaking changes to the library header file:
  • It is now possible to use parallelize_loop() with functions that have return values and get these values from all blocks at once through the get() member function of the BS::multi_future.
  • The template specializations for push_task() have been merged. Now instead of two versions, one for functions with arguments and one for functions without arguments, there is just one version, which can accept any function.
  • Constructors have been made explicit. See issue #28.
  • submit() now uses std::make_shared instead of new to create the shared pointer. This means only one memory allocation is performed instead of two, which should improve performance. In addition, all unique pointers are now created using std::make_unique.
  • A new helper class template, BS::multi_future, has been added. It's basically just a wrapper around std::vector<std::future<T>>. This class is used by the new implementation of parallelize_loop() to allow waiting for the entire loop, consisting of multiple tasks with their corresponding futures, to finish executing.
  • BS::multi_future can also be used independently to handle multiple futures at once. For example, you can now keep track of several groups of tasks by storing their futures inside separate BS::multi_future objects and use either wait() to wait for all tasks in a specific group to finish or get() to get an std::vector with the return values of every task in the group.
  • Integer types are now chosen in a smarter way to improve portability, allow for better compatibility with 32-bit systems, and prevent potential conversion errors.
  • Added a new type, BS::concurrency_t, equal to the return type of std::thread::hardware_concurrency(). This is probably pointless, since the C++ standard requires this to be unsigned int, but it seems to me to make the code slightly more portable, in case some non-conforming compiler chooses to use a different integer type.
  • C-style casts have been converted to C++ cast expressions for added clarity.
  • Miscellaneous minor optimizations and style improvements.
• Changes to the test program:
  • The program has been renamed to BS_thread_pool_test.cpp to avoid potential conflict with other thread pool libraries.
  • The program now returns EXIT_FAILURE if any of the tests failed, for automation purposes. See pull request #42.
  • Fixed incorrect check order in check_task_monitoring(). See pull request #43.
  • Added a new test for parallelize_loop() with a return value.
  • Improved some of the tests to make them more reliable. For example, count_unique_threads() now uses futures (stored in a BS::multi_future<void> object).
  • The program now uses std::vector instead of matrices, for both consistency checks and benchmarks, in order to simplify the code and considerably reduce its length.
  • The benchmarks have been simplified. There's now only one test: filling a specific number of vectors of fixed size with random values. This may be replaced with something more practical in a future released, but at least on the systems I've tested on, it does demonstrate a very significant multi-threading speedup.
  • In addition to multi-threaded tests with different numbers of tasks, the benchmark now also includes a single-threaded test. This allows for more accurate benchmarks compared to previous versions, as the (slight) parallelization overhead is now taken into account when calculating the maximum speedup.
  • The program decides how many vectors to use for benchmarking by testing how many are needed to reach a target duration in the single-threaded test. This ensures that the test takes approximately the same amount of time on different systems, and is thus more consistent and portable.
  • Miscellaneous minor optimizations and style improvements.
• Changes to README.md:
  • Many sections have been rewritten and/or polished.
  • Explanations and examples of all the new features have been added.
  • Added an acknowledgements section.
• Miscellaneous changes:
  • Added a CITATION.bib file (in BibTeX format) to the GitHub repository. You can use it to easily cite this package if you use it in any research papers.
  • Added a CITATION.cff file (in YAML format) to the GitHub repository. This should add an option to get a citation in different formats directly from GitHub repository by clicking on "cite this repository" on the sidebar to the right.
  • Added templates for GitHub issues and pull requests.

v2.0.0 (2021-08-14)

• From now on, version numbers will adhere to the Semantic Versioning specification in the format major.minor.patch.
• A file named thread_pool_test.cpp has been added to the package. It will perform automated tests of all aspects of the package, and benchmark some multithreaded matrix operations. Please run it on your system and submit a bug report if any of the tests fail. In addition, the code is thoroughly documented, and is meant to serve as an extensive example of how to properly use the package.
• The package is now available through vcpkg. Instructions for how to install it have been added to README.md. See this pull request.
• The package now defines a macro THREAD_POOL_VERSION, which returns the version number and release date of the thread pool library as a string.
• parallelize_loop() has undergone some major changes (and is now incompatible with v1.x):
  • The second argument is now the index after the last index, instead of the last index itself. This is more consistent with C++ conventions (e.g. standard library algorithms) where the range is always [first, last). For example, for an array with n indices, instead of parallelize_loop(0, n - 1, ...) you should now write parallelize_loop(0, n, ...).
  • The loop function is now only called once per block, instead of once per index, as was the case before. This should provide a performance boost due to significantly reducing the number of function calls, and it also allows you to conserve resources by using them only once per block instead of once per index (an example can be found in the random_matrix_generator class in thread_pool_test.cpp). It also means that loop now takes two arguments: the first index in the block and the index after the last index in the block. Thus, loop(start, end) should typically involve a loop of the form for (T i = start; i < end; i++).
  • The first and last indices can now be of two different integer types. Previously, parallelize_loop(0, i, ...) did not work if i was not an int, because 0 was interpreted as int, and the two arguments had to be of the same type. Therefore, one had to use casting, e.g. parallelize_loop((size_t)0, i), to make it work. Now this is no longer necessary; the common type is inferred automatically using std::common_type_t.

v1.9 (2021-07-29)

• Fixed a bug in reset() which caused it to create the wrong number of threads.

v1.8 (2021-07-28)

• The version history has become too long to be included in README.md, so I moved it to a separate file, CHANGELOG.md.
• A button to open this repository directly in Visual Studio Code has been added to the badges in README.md.
• An internal variable named promise has been renamed to task_promise to avoid any potential errors in case the user invokes using namespace std.
• submit() now catches exceptions thrown by the submitted task and forwards them to the future. See this issue.
• Eliminated compiler warnings that appeared when using the -Weffc++ flag in GCC. See this pull request.

v1.7 (2021-06-02)

• Fixed a bug in parallelize_loop() which prevented it from actually running loops in parallel, see this issue.

v1.6 (2021-05-26)

• Since MSVC does not interpret and as && by default, the previous release did not compile with MSVC unless the /permissive- or /Za compiler flags were used. This has been fixed in this version, and the code now successfully compiles with GCC, Clang, and MSVC. See this pull request.

v1.5 (2021-05-07)

• This library now has a DOI for citation purposes. Information on how to cite it in publications has been added to the source code and to README.md.
• Added GitHub badges to README.md.

v1.4 (2021-05-05)

• Added three new public member functions to monitor the tasks submitted to the pool:
  • get_tasks_queued() gets the number of tasks currently waiting in the queue to be executed by the threads.
  • get_tasks_running() gets the number of tasks currently being executed by the threads.
  • get_tasks_total() gets the total number of unfinished tasks - either still in the queue, or running in a thread.
  • Note that get_tasks_running() == get_tasks_total() - get_tasks_queued().
  • Renamed the private member variable tasks_waiting to tasks_total to make its purpose clearer.
• Added an option to temporarily pause the workers:
  • When public member variable paused is set to true, the workers temporarily stop popping new tasks out of the queue, although any tasks already executed will keep running until they are done. Set to false again to resume popping tasks.
  • While the workers are paused, wait_for_tasks() will wait for the running tasks instead of all tasks (otherwise it would wait forever).
  • By utilizing the new pausing mechanism, reset() can now change the number of threads on-the-fly while there are still tasks waiting in the queue. The new thread pool will resume executing tasks from the queue once it is created.
• parallelize_loop() and wait_for_tasks() now have the same behavior as the worker function with regards to waiting for tasks to complete. If the relevant tasks are not yet complete, then before checking again, they will sleep for sleep_duration microseconds, unless that variable is set to zero, in which case they will call std::this_thread::yield(). This should improve performance and reduce CPU usage.
• Merged this commit: Fixed weird error when using MSVC and including windows.h.
• The README.md file has been reorganized and expanded.

v1.3 (2021-05-03)

• Fixed this issue: Removed std::move from the return statement in push_task(). This previously generated a -Wpessimizing-move warning in Clang. The assembly code generated by the compiler seems to be the same before and after this change, presumably because the compiler eliminates the std::move automatically, but this change gets rid of the Clang warning.
• Fixed this issue: Removed a debugging message printed to std::cout, which was left in the code by mistake.
• Fixed this issue: parallelize_loop() no longer sends references for the variables start and stop when calling push_task(), which may lead to undefined behavior.
• A companion paper is now published at arXiv:2105.00613, including additional information such as performance tests on systems with up to 80 hardware threads. The README.md has been updated, and it is now roughly identical in content to the paper.

v1.2 (2021-04-29)

• The worker function, which controls the execution of tasks by each thread, now sleeps by default instead of yielding. Previously, when the worker could not find any tasks in the queue, it called std::this_thread::yield() and then tried again. However, this caused the workers to have high CPU usage when idle, as reported by some users. Now, when the worker function cannot find a task to run, it instead sleeps for a duration given by the public member variable sleep_duration (in microseconds) before checking the queue again. The default value is 1000 microseconds, which I found to be optimal in terms of both CPU usage and performance, but your own optimal value may be different.
• If the constructor is called with an argument of zero for the number of threads, then the default value, std::thread::hardware_concurrency(), is used instead.
• Added a simple helper class, timer, which can be used to measure execution time for benchmarking purposes.
• Improved and expanded the documentation.

v1.1 (2021-04-24)

• Cosmetic changes only. Fixed a typo in the Doxygen comments and added a link to the GitHub repository.

v1.0 (2021-01-15)

• Initial release.