parallel processing - Python class methods parallelization -
i need boost python application. solution supposed trivial:
import time multiprocessing import pool class a: def method1(self): time.sleep(1) print('method1') return 'method1' def method2(self): time.sleep(1) print('method2') return 'method2' def method3(self): pool = pool() time1 = time.time() res1 = pool.apply_async(self.method1, []) res2 = pool.apply_async(self.method2, []) res1 = res1.get() res2 = res2.get() time2 = time.time() print('res1 = {0}'.format(res1)) print('res2 = {0}'.format(res2)) print('time = {0}'.format(time2 - time1)) = a() a.method3() but every time launch simple program exception:
exception in thread thread-2: traceback (most recent call last): file "/usr/lib/python3.2/threading.py", line 740, in _bootstrap_inner self.run() file "/usr/lib/python3.2/threading.py", line 693, in run self._target(*self._args, **self._kwargs) file "/usr/lib/python3.2/multiprocessing/pool.py", line 346, in _handle_tasks put(task) _pickle.picklingerror: can't pickle <class 'method'>: attribute lookup builtins.method failed it seems python can't parallelize class methods. have been desperately googling whole day still don't understand how work around (possibly googling not enough good). python multiprocessing documentation seems poor.
i don't want destroy class separating on global methods. following code seems workable:
class b: def method1(self): time.sleep(1) print "b.method1" return "b.method1" def method2(self): time.sleep(1) print "b.method2" return "b.method2" def method1(b): time.sleep(1) print('method1') return b.method1() def method2(b): time.sleep(1) print('method2') return b.method2() def method3(): pool = pool() time1 = time.time() b = b() res1 = pool.apply_async(method1, [b]) res2 = pool.apply_async(method2, [b]) res1 = res1.get() res2 = res2.get() time2 = time.time() print('res1 = {0}'.format(res1)) print('res2 = {0}'.format(res2)) print('time = {0}'.format(time2 - time1)) method3() but still don't understand how make python parallelization work inside class method. me work around obstacle? possibly there other ways of parallelization don't know can applied in case? need workable code example. appreciated.
Comments
Post a Comment