The Application Programming Interface (API) supports multi-platform Shared Memory Multiprocessing programming in C / C++ and Fortran on many architectures, including Unix and Microsoft Windows platforms. It consists of a set of Compiler Directive s, library routines, and Environment Variable s that influence run-time behavior.
Jointly defined by a group of major computer hardware and software vendors, OpenMP is a portable, scalable model that gives Programmer s a simple and flexible interface for developing parallel applications for platforms ranging from the desktop to the Supercomputer .
Often a so-called hybrid-model for Parallel Programming , using both OpenMP and MPI ( Message Passing Interface ), is used for programming Computer Cluster s.
The OpenMP Architecture Review Board (ARB) published its first standard, OpenMP for FORTRAN 1.0, in October of 1997. October the following year they released the C/C++ standard. 2000 saw version 2.0 of the FORTRAN standard with version 2.0 of the C/C++ standard being released in 2002. The current version is 2.5. It is a combined C/C++/FORTRAN standard, which was released in 2005.
The core elements of OpenMP are the constructs for thread creation, work load distribution (work sharing), data environment management, thread synchronization, user level runtime routines and environment variables.
- Thread creation: ''omp parallel''. It is used to fork additional threads to carry out the work enclosed in the construct in parallel. The original process will denoted as with thread ID 0.
Example: Display "Hello, world" using multiple threads.
#include
- using conditional compilation to let sequential compilers ignore the omp.h header---/
#include
#endif
#pragma omp parallel
printf("Hello, world.
");
return 0;
}
- work-sharing constructs: used to specify how to assign independent work to one or all of the threads.
- --- ''omp for'' or ''omp do'': used to splits up loop iterations among the threads
- --- ''sections'': assigning consecutive but independent code blocks to different threads
- ---''single'': specifying a code block that is executed by only one thread, a barrier is implied in the end
- --- ''master'': similar to single, but the code block will be executed by the master thread only and no barrier implied in the end.
Example: initialize the value of a large array in parallel, using each thread to do a portion of the work
#include
- using conditional compilation to let sequential compilers ignore the omp.h header---/
#include
#endif
#define N 100000
int i, a {Link without Title} ;
#pragma omp parallel for
}
- Data environment management: Since OpenMP is a shared memory programming model, most variables in OpenMP code are visible to all threads by default. But sometimes private variables are necessary and there is a need to pass values between the sequential part and the parallel region(the code block executed in parallel), so data environment management is introduced as ''data clauses''.
- --- ''shared'': the data is shared, which means visible and accessible by all threads simultaneously
- --- ''private'': the data is private to each thread, which means each thread will have a loal copy and use it as a temporary variable. A private variable is not initialized and the value is not maintained for use outside the parallel region.
- --- ''firstprivate'': the data is private to each thread, but initialized using the value of the variable using the same name from the master thread.
- --- ''lastprivate'': the data is private to each thread. The value of this private data will be copied to a global variable using the same name outside the parallel region if current iteration is the last iteration in the parallelized loop. A data can be both ''firstprivate'' and ''lastprivate''.
- --- ''threadprivate'': The data is a global data, but it is private in each parallel region during the runtime. The difference between ''threadprivate'' and ''private'' is the global scope associated with threadprivate and the preserved value across prallel regions.
- --- ''copyin'': similar to ''firstprivate''for ''private'' variables, ''threadprivate'' variables are not initialized, unless using ''copyin'' to pass the value from the corresponding global variables. No ''copyout'' is needed because the value of a threadprivate variable is maintained throughout the execution of the whole program.
- --- ''reduction'': the variable has a local copy in each thread, but the values of the local copies will be summarized(reduced) into a global shared variable.
- synchronization constructs:
- --- ''critical section'': the enclosed code block will be executed by all threads but only one thread at a time, not simultaneously execution. It is often used to protect shared data from Race Condition .
- --- ''atomic'': similar to ''critical section'', but advise the compiler to use special hardware instructions for better performance. Compilers may choose to ignore this suggestion from users and use ''critical section'' instead.
- --- ''barrier'':
- --- ''ordered''
- --- ''flush''
- --- ''locks''
- User-level runtime routines: used to modify/check the number of threads, detect if the execution context is in a parallel region, how many processors in current system, etc
- Environment variables: a method to alter the execution features of OpenMP applications. Used to control loop iterations scheduling, default number of threads, etc.
#include
#include
#pragma omp parallel private(id)
{
id = omp_get_thread_num();
printf("Hello World from thread %d
", id);
#pragma omp barrier
if ( id == 0 ) {
nthreads = omp_get_num_threads();
printf("There are %d threads
",nthreads);
}
}
return 0;
}
PROGRAM HELLO
INTEGER ID, NTHRDS
INTEGER OMP_GET_THREAD_NUM, OMP_GET_NUM_THREADS
C PARALLEL PRIVATE(ID)
ID = OMP_GET_THREAD_NUM()
- , 'HELLO WORLD FROM THREAD', ID
C BARRIER
IF ( ID .EQ. 0 ) THEN
NTHRDS = OMP_GET_NUM_THREADS()
- , 'THERE ARE', NTHRDS, 'THREADS'
END IF
C END PARALLEL
END
program hello90
use omp_lib
integer :: id, nthreads
! parallel private(id)
id = omp_get_thread_num()
- ,---) 'Hello World from thread', id
! barrier
if ( id .eq. 0 ) then
nthreads = omp_get_num_threads()
- ,---) 'There are', nthreads, 'threads'
end if
! end parallel
end program
Pros
- Simple: need not deal with message passing as MPI does
- Data layout and decomposition is handled automatically by directives.
- Incremental parallelism: can work on one portion of the program at one time, no dramatic change to code is needed.
- Unified code for both serial and parallel applications: OpenMP constructs are treated as comments when sequential compilers are used.
- Original (serial) code statements need not, in general, be modified when parallelized with OpenMP. This reduces the chance of inadvertently introducing bugs.
Cons
- Currently only run efficiently in shared-memory multiprocessor platforms
- Requires a compiler that supports OpenMP. Visual C++ 2005 supports it, and so do the Intel compilers for their x86 and IPF product series. GCC 4.2 will support OpenMP, though it is likely that some distributors will add OpenMP support already to their GCC 4.1 based system compilers.
- Low parallel efficiency: rely more on parallelizable loops, leaving out a relatively high percentage of a non-loop code in sequential part.
One may expect to get N times less wall clock execution time (or N times Speedup ) when running a program parallelized using OpenMP on a N processor platform. However, this is seldom the case due to the following reasons:
- A large portion of the program may not be parallelized by OpenMP, which sets theoretical upper limit of speedup according to Amdahl's Law .
- N processors in a SMP may bring N times computation power, but the Memory Bandwidth usually does not scale up N times. Quite often, the original memory path is shared by multiple processors and performance degradation may be observed when they compete for the shared memory bandwidth.
- Many other common problems affecting the final speedup in parallel computing also apply to OpenMP, like Load Balancing and synchronization overhead.
There are some public domain OpenMP benchmarks for users to try.
|
