Parallel Programming in C#, TPL API Introduction

1. Introduction
Multicore processors have been around for many years. However,
many developers are doing what they’ve always done: creating
single-threaded programs. They’re not taking advantage of all the
extra processing power.
Example: Imagine you have many tasks to perform and many
people to perform them, but you are using only one person
because you don’t know how to ask for more. It’s inefficient. Users
are paying for extra power, but their software is not allowing them
to use it.

2. Free lunch is over
• Don’t expect your sequential program to run faster on new
processors.
• Today’s desktops typically have 4 cores. Latest Intel multi-core chip
has 48 cores. Expect 100s of cores in the near future.
• To take advantage of the hardware of today and tomorrow, we have
to parallelize our code to distribute work across multiple processor
cores.
• The .NET Framework provides several ways to use multiple threads of
execution to keep application responsive to our user while
maximizing the performance.

3. Parallel Programming in the .NET Framework
• In the past, parallelization required low-level manipulation of threads,
thread pools and locks.
• Visual Studio 2010 and the .NET Framework 4.0:
• new runtime
• new class library types
• new diagnostic tools
• Features simplify parallel development to write efficient, fine-grained, and
scalable parallel code.

4. Parallel programming architecture.NET 4:
High-level overview

5. Task Parallel Library (TPL)
• Is a set of public types and APIs in
the System.Threading and System.Threading.Tasks namespaces.
• The TPL scales the degree of concurrency dynamically to most efficiently
use all the processors that are available.
• TPL handles the partitioning of the work, the scheduling of threads on the
threadpool, cancellation support, state management, and other low-level
details.
• To remember:
• Not all code is suitable for parallelization; e.g. for small/cheap loops, overhead of
parallelization can cause the code to run more slowly.
• Parallelization adds complexity to program execution.
• Need understanding of threading concepts: locks, deadlocks, and race conditions, to
use TPL effectively.

6. Data Parallelism (TPL)
• Data parallelism refers to scenarios in which the same operation is
performed concurrently (that is, in parallel) on elements in a source
collection or array.
• Through System.Threading.Tasks.Parallel class.
• Imperative way:
• Describes how to create parallel for and foreach loops.
• When to use: Strategy can work well if you have either of these
• Lots of items
• Lots of work for each item
• DEMO

7. Data Parallelism...
• Declarative way: PLINQ (Parallel LINQ)
• PLINQ)is a parallel implementation of the LINQ pattern
• Partition the data source into segments, and then executing the
query on each segment on separate worker threads in parallel
on multiple cores.
• PLINQ can achieve significant performance improvements over
legacy code for many cases.
• However, parallelization complexity can actually slows down
certain queries, if not used wisely.

8. Task Parallelism (TPL)
• Based on the concept of a task, which represents an asynchronous
operation.
• A task resembles a thread or ThreadPool work item, but at a higher
level of abstraction.
• The term task parallelism refers to one or more independent tasks
running concurrently.

9. Task benefits…
• More programmatic control than is possible with a thread: waiting,
cancellation, continuations, robust exception handling, detailed
status, custom scheduling, and more.
• More efficient and scalable use of system resources: tasks are
queued to the ThreadPool, enhanced with algorithms that determine
and adjust to the number of threads and that provide load balancing
to maximize throughput. This makes tasks relatively lightweight.
• For above reasons: TPL is the preferred API for writing multi-
threaded, asynchronous, and parallel code.

10. Creating and Running Tasks
• Implicit:
• Parallel.Invoke method to run any number tasks concurrently.
• Action delegate is passed for each item of work (task).
• Syntax: Parallel.Invoke(() => DoSomeWork(), () =>
DoSomeOtherWork());
• Demo ( parallelizes the operations, not the data)
• Explicit:
• Create Task or Task<TResult> with user delegate (named delegate,
anonymous method, or lambda expression) that encapsulates the code task
will execute.
• Execute with task1.Start() with other functions (task1.Wait() etc)
• Task.Run() methods to create and start a task in one operation
• Preferred way when more control over the creation and scheduling of the task is not
needed.

11. Tasks...
• The tasks run asynchronously and may complete in any order. If
the Result property is accessed before the computation finishes,
the property blocks the calling thread until the value is available.

12. Options
• C# provides several mechanisms for parallel programming:
• Explicit threads:
• with synchronization via locks, critical regions etc.
• The user gets full control over the parallel code.
• BUT orchestrating the parallel threads is tricky and error prone (race conditions,
deadlocks etc)
• This technique requires a shared-memory model
• Explicit threads with a message-passing library:
• Threads communicate by explicitly sending messages, with data required/produced,
between workstations. Parallel code can run on a distributed-memory architecture,
eg. a network of workstations. The programmer has to write code for (un-)serializing
the data that is sent between machines. BUT threads are still explicit, and the
difficulties in orchestrating the threads are the same. A common configuration is
C+MPI.

13. Wait a minute... Quantum Computation on a
way!
• Calls modern day computing classical one.
• Uses quantum superposition principle on superconductors.
• Can do:
• Integer prime factorization (security of public key cryptographic systems)
• Quadratic speedup over NP-complete problems (Viz. TSP, Subset sum problem
etc.)
• Much more...
• Example: D-wave 2x system

14. Resources
• https://github.com/bsonnino/ParallelProgramming
• QC: Dwave, ScienceDaily, Wikipedia