0

This is my parallel processing code where I am comparing the runtimes of parallel codes and linear codes of the same task in Python. I am getting the output time for parallel code to be more in every run. This is completely unexpected. Why is it happening? What's wrong?

from multiprocessing import Pool
import matplotlib.pyplot as plt
import numpy as np
import time

def scatter_join(args):
    plt.scatter(s=1,*args)
    plt.savefig('test_p.png')


if __name__ == '__main__':
    p = Pool()
    snap = np.random.randint(0,20,(100000,2))
    task = [(snap[:,0],snap[:,1])]
    t = time.time()
    p.map(scatter_join,task)
    p.close()
    p.join()
    print('Time Elapsed(Parallel): ', abs((time.time()-t)))

plt.clf()
t = time.time()
plt.scatter(snap[:,0],snap[:,1],s=1)
plt.savefig('test_l.png')
print('Time Elapsed(Linear): ',abs(time.time()-t))
  • what times you get ? multiprocessing needs time to start it. – furas Dec 31 '16 at 20:45
  • Multiprocessing is technically slower because there is an extra overhead to starting. The gains of being able to process multiple things at a single time is what gives it the advantage. If the problem can be done quickly enough on a single core it will beat out multiprocessing because of the extra start time needed. – SudoKid Dec 31 '16 at 20:59
  • 1
    Your code doesn't seem to parallelize anything. Your `task` list contains a single set of arguments, so `p.map` only uses a single worker. You essentially are incurring all the overhead of `multiprocessing` with none of the benefits. – juanpa.arrivillaga Dec 31 '16 at 21:01

1 Answers1

1

Parallel code isn't guaranteed to be faster than a single thread. It can either accelerate or decelerate a workload.

If the serial(i.e non-parallelisable) parts of a code are too great a percentage, then you won't see much benefit in parallelising the algorithm.

From what it looks like here, your workload (a single matplotlib plot?) isn't going to benefit from parallel execution.

Also might want to look at this:

Python Multiprocessing with Matplotlib

Output was

Time Elapsed(Parallel): 1.8148565292358398

Time Elapsed(Linear): 0.17187786102294922

Community
  • 1
  • 1
JKent
  • 23
  • 1
  • 5