User:Alexkachanov/Parallel Programming

Hardware

 * processors and cores


 * symmetric mutliprocessing (SMP) отмирает
 * non-uniform memory access (NUMA) -> NUMA nodes
 * Asymmetric Multiprocessing (ASMP)
 * Clustered Multiprocessing


 * hardware threads - software threads


 * shared memory multicore
 * distributed memory systems


 * chip multithreading (CMT) — one chip, many threads. Most multicore processors use a combination of multiple cores and multiple threads per core. The simplest example of this would be a processor with two cores with each core being capable of supporting two threads, making a total of four threads for the entire processor.


 * Flynn's taxonomy (1966)

Intel

 * CMT - hyperthreading

UltraSPARC

 * The UltraSPARC T2 is a good example of a CMT processor. It has eight replicated cores, and each core is capable of running eight threads, making the processor capable of running 64 software threads simultaneously

Programming

 * sequential programming
 * parallel programming
 * multi-core programming - частный случай parallel programming


 * heavyweight concurrency - threads
 * lightweight concurrency - fibers, processes, actors - Haskell, Erlang, Scala(?)


 * Amdahl's Law -> Gustafson's Law


 * interleaved concurrency и просто concurrency

Properties of concurrent solutions:
 * deadlock-freedom property
 * starvation-freedom (sometimes called lockout-freedom) property
 * waiting property; fault-tolerance

Two kinds of communication occur naturally in concurrent systems:
 * Transient communication
 * Persistent communication


 * Mutual exclusion problem: Mutual exclusion requires persistent communication
 * Producer–consumer problem
 * Reader-writer problem

Java
Thread:
 * interrupt; wait; notify; notifyAll;
 * join
 * sleep

C#

 * .NET 4 Parallel Extensions: Task Parallel Library (TPL), Parallel LINQ

Книги

 * (2010) Professional Parallel Programming with C# - Master Parallel Extensions with .NET 4 (ISBN 0470495995)
 * (2010) Pro .NET 4 Parallel Programming in Csharp (ISBN 1430229675)