diff --git a/sources/modules/concurrency-parallelism/async-programming.md b/sources/modules/concurrency-parallelism/async-programming.md new file mode 100644 index 0000000..97371f7 --- /dev/null +++ b/sources/modules/concurrency-parallelism/async-programming.md @@ -0,0 +1,124 @@ +## Module name: Asynchronous programming + +_Skeleton descriptions are typeset in italic text,_ +_so please don't remove these descriptions when editing the topic._ + +### Overview + +_Provides a short natural language abstract of the module’s contents._ +_Specifies the different levels of teaching._ + +------------------------------------------------------------------------ +Level Objective +----------------- ------------------------------------------------------ +Foundational --- Knowledge about build systems + +Main --- Usage of build system to compile a executable + +Advanced --- Add external libraries as a dependencies + +------------------------------------------------------------------------ + +### Motivation + +_Why is this important?_ +_Why do we want to learn/teach this topic?_ + +* Asynchronous programming allows for non-blocking functions launches and enables that functions are executed asynchronously. +* Asynchronous programming is one form of parallelism and concurrency included in the C++ standard + +### Topic introduction + +_Very brief introduction to the topic._ + +Build systems are used to configure, build, and install complex C++ projects. + + +### Foundational: Knowledge about build systems + +#### Background/Required Knowledge + +A student: +* Should know [lambdas](../functions/lambdas.md) + +#### Student outcomes + +_A list of things "a student should be able to" after the curriculum._ +_The next word should be an action word and testable in an exam._ +_Max 5 items._ + +A student should be able to: + +1. To explain what asynchronous programming is +2. To explain what a futures and promise are + +#### Caveats + +_This section mentions subtle points to understand, like anything resulting in +implementation-defined, unspecified, or undefined behavior._ + +None + +#### Points to cover + +_This section lists important details for each point._ + +* Mention how to launch a function or lambda asynchronously using `std::async` +* Mention how to use a `std::future` as a place holder for the result + +### Main: Launch functions asynchronous and obtain the results + +#### Background/Required Knowledge + +* All of the above. + +#### Student outcomes + +A student should be able to: + +1. Do define a function or lambda for the computational task +2. Launch the function or lambda asynchronous and obtain the results + +#### Caveats + +The concept of asynchronous programming is no easy digestible for most students. + + +#### Points to cover + +* The header `` needs to be includes +* The return type of the function or lambda will the the template type of the future +* The first argument of `std::async` is the function or lambda and after that all arguments are provided + +Example using a function +``` +void print_square(double a) +{ + std::cout << "Result=" << a * a << std::endl; +} + +std::future f = std::async(print,5.0); +// We could do other work here +f.get() +``` + +Example using lambdas +``` +// Compute the sum +std::future f1 = std::async([](double a, double b){ return a + b;}); +// Compute the square +std::future f2 = std::async([](double a){ return a * a;}); + +// Gather the results and add them up +double res = f1.get() + f2.get(); +``` + +### Advanced + +_These are important topics that are not expected to be covered but provide +guidance where one can continue to investigate this topic in more depth._ + +* How to build libraries +* How to have external libraries be downloaded during the build process +* Mention that build systems provide support for unit testing + diff --git a/sources/modules/concurrency-parallelism/parallel-algorithms.md b/sources/modules/concurrency-parallelism/parallel-algorithms.md new file mode 100644 index 0000000..e0d5bda --- /dev/null +++ b/sources/modules/concurrency-parallelism/parallel-algorithms.md @@ -0,0 +1,140 @@ +## Module name: Parallel algorithms + +_Skeleton descriptions are typeset in italic text,_ +_so please don't remove these descriptions when editing the topic._ + +### Overview + +_Provides a short natural language abstract of the module’s contents._ +_Specifies the different levels of teaching._ + +------------------------------------------------------------------------ +Level Objective +----------------- ------------------------------------------------------ +Foundational --- Knowledge about build systems + +Main --- Usage of build system to compile a executable + +Advanced --- Add external libraries as a dependencies + +------------------------------------------------------------------------ + +### Motivation + +_Why is this important?_ +_Why do we want to learn/teach this topic?_ + +* Allows to execute *most* of the algorithms in the C++ standard in parallel +* Parallel algorithms are one form of parallelism included in the C++ standard + +### Topic introduction + +_Very brief introduction to the topic._ + +Algorithms that range over a large data set, can be accelerated by parallel execution. +Exectution policies allow to executed the algorithms in the C++ standard on multiple cores or a single core. + +### Foundational: Knowledge about build systems + +#### Background/Required Knowledge + +A student: +* Should know [lambdas](../functions/lambdas.md) +* Should know [algorithms](../program-design/algorithms.md) +* Should know [iterators](../program-design/iterators.md) +* Should know [containers](../program-design/containers.md) + +#### Student outcomes + +_A list of things "a student should be able to" after the curriculum._ +_The next word should be an action word and testable in an exam._ +_Max 5 items._ + +A student should be able to: + +1. To explain parallel and sequential execution +2. Specify the appropriate execution policy for sequntial or parallel execution + +#### Caveats + +_This section mentions subtle points to understand, like anything resulting in +implementation-defined, unspecified, or undefined behavior._ + +1. The programmer has to make sure that the algorithm is theoretical parallel. For example, specifying parallel execution using state-full lambdas might give wrong results. +2. The programmer has to make sure that the parallel execution does not lead to race conditions. + +#### Points to cover + +_This section lists important details for each point._ + +* Add execution policies as an additonal argument to the algorithm +* The C++ 17 standard is required +* Mention `std::atomic` or `std::mutex` to avoid race conditions + +### Main: + +#### Background/Required Knowledge + +* All of the above. + +#### Student outcomes + +A student should be able to: + +1. Do define a function or lambda for the compute kernel +2. Split the work in independent tasks to avoid race conditions +3. Explain the meaning of the four policies (`std::execution::seq`, `std::execution::par`, `std::execution::par_unseq`, and `std::execution::unseq`) + +#### Caveats + +The concept of parallel programming introduces bugs introdcues via race conditions + + +#### Points to cover + +* The header `` needs to be included +* The first argument of the algorithm is the execution policy + +Example using a function +``` +std::vector values = {1,2,3,4,5,6}; +void square(double& a) +{ + a = a * a; +} + +// Parallel execution +std::for_each(std::execution::par_unseq,std::begin(values),std::end(values),square); + +// Serial execution +std::for_each(std::execution::seq,std::begin(values),std::end(values),square); +``` + +Example using a predefined algorithm +``` +std::vector values(10000); + +// Seed the random number generator +std::random_device rd; +std::mt19937 gen(rd()); + +// Define the range for the random numbers +std::uniform_int_distribution<> distrib(1, 100); // Generates numbers between 1 and 100 + +// Fill the vector with random numbers +std::generate(random_vector.begin(), random_vector.end(), [&]() { return distrib(gen); }); + +// Sort the vector in parallel +std::sort(std::exeuction::par,values.begin(),values.end()) +``` + +### Advanced + +_These are important topics that are not expected to be covered but provide +guidance where one can continue to investigate this topic in more depth._ + +* If the implementation cannot parallelize or vectorize (e.g. due to lack of resources), all standard execution policies can fall back to sequential execution. +* None of the execution policies allow for reproducibilty. This is obvious for the parallel execution policies. But even `std::ececution::seq` can execute the iterations in any order. +* Nvidia supports to run `std::execution::par` on Nvidia GPUs. However, that is not yet in the C++ standard and will only work with Nvidia's HPC compiler. +* Currently, parallel algorithms are implemented using Intel's TBB library in GCC. You can set the number of used cores using `tbb::global_control(tbb::global_control::max_allowed_parallelism, nthreads);` provided by the header `#include "tbb/tbb.h"`. +