8.5. Fundamentals
The mapping of processes to nodes can be defined not just with general
policies but also, if necessary, using arbitrary mappings that cannot
be described by a simple policy. Supported directives, given on the
command line via the --map-by option, include:
SEQ: (often accompanied by thefile=<path>qualifier) assigns one process to each node specified in the file. The sequential file is to contain an entry for each desired process, one per line of the file.RANKFILE: (often accompanied by thefile=<path>qualifier) assigns one process to the node/resource specified in each entry of the file, one per line of the file.
For example, using the hostfile below:
$ cat myhostfile
aa slots=4
bb slots=4
cc slots=4
The command below will launch three processes, one on each of nodes
aa, bb, and cc, respectively. The slot counts don’t
matter; one process is launched per line on whatever node is listed on
the line.
$ prun --hostfile myhostfile --map-by seq ./a.out
Impact of the ranking option is best illustrated by considering the
following hostfile and test cases where each node contains two
packages (each package with two cores). Using the --map-by
ppr:2:package option, we map two processes onto each package and
utilize the --rank-by option as show below:
$ cat myhostfile
aa
bb
Command |
Ranks on |
Ranks on |
|---|---|---|
|
0 1 ! 2 3 |
4 5 ! 6 7 |
|
0 2 ! 1 3 |
4 6 ! 5 7 |
|
0 4 ! 1 5 |
2 6 ! 3 7 |
Ranking by slot provides the identical result as ranking by core in
this case — a simple progression of ranks across each
node. Ranking by package does a round-robin ranking across packages
within each node until all processes have been assigned a rank, and
then progresses to the next node. Adding the :SPAN qualifier to
the ranking directive causes the ranking algorithm to treat the entire
allocation as a single entity — thus, the process ranks are
assigned across all packages before circling back around to the
beginning.
The binding operation restricts the process to a subset of the CPU resources on the node.
The processors to be used for binding can be identified in terms of
topological groupings — e.g., binding to an l3cache will bind
each process to all processors within the scope of a single L3 cache
within their assigned location. Thus, if a process is assigned by the
mapper to a certain package, then a --bind-to l3cache directive
will cause the process to be bound to the processors that share a
single L3 cache within that package.
To help balance loads, the binding directive uses a round-robin method, binding a process to the first available specified object type within the object where the process was mapped. For example, consider the case where a job is mapped to the package level, and then bound to core. Each package will have multiple cores, so if multiple processes are mapped to a given package, the binding algorithm will assign each process located to a package to a unique core in a round-robin manner.
Binding can only be done to the mapped object or to a resource located within that object.
An object is considered completely consumed when the number of processes bound to it equals the number of CPUs within it. Unbound processes are not considered in this computation. Additional processes cannot be mapped to consumed objects unless the OVERLOAD qualifier is provided via the “–bind-to” command line option.
Default process mapping/ranking/binding policies can also be set with MCA
parameters, overridden by the command line options when provided. MCA
parameters can be set on the prte command line when starting the
DVM (or in the prterun command line for a single-execution job), but
also in a system or user mca-params.conf file or as environment
variables, as described in the MCA section below. Some examples include:
|
MCA parameter key |
Value |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|