Accessing the worker ID¶
Each worker in MPIRE is given an integer ID to distinguish them. Worker #1 will have ID
0, #2 will have ID
etc. Sometimes it can be useful to have access to this ID.
By default, the worker ID is not passed on. You can enable/disable this by setting the
def task(worker_id, x): pass with WorkerPool(n_jobs=4, pass_worker_id=True) as pool: pool.map(task, range(10))
The worker ID will always be the first argument passed on to the provided function.
with WorkerPool(n_jobs=4) as pool: pool.pass_on_worker_id() pool.map(task, range(10))
Here’s a more elaborate example of using the worker ID together with a shared array, where each worker can only access the element corresponding to its worker ID, making the use of locking unnecessary:
def square_sum(worker_id, shared_objects, x): # Even though the shared objects is a single container, we 'unpack' it anyway results_container = shared_objects # Square and sum results_container[worker_id] += x * x # Use a shared array of size equal to the number of jobs to store the results results_container = Array('f', 4, lock=False) with WorkerPool(n_jobs=4, shared_objects=results_container, pass_worker_id=True) as pool: # Square the results and store them in the results container pool.map_unordered(square_sum, range(100))