Python 2.7 multiprocessing get process result whithout using pool -
how can result process without using pool ?
(i'm willing conserve eye on progression:
(print "\r",float(done)/total,"%",)
which can't done using pool far know)
def multiprocess(function, argslist, ncpu): total = len(argslist) done = 0 jobs = [] while argslist != []: if len(mp.active_children()) < ncpu: p = mp.process(target=function,args=(argslist.pop(),)) jobs.append(p) p.start() done+=1 print "\r",float(done)/total,"%", #get results here job in jobs: job.get_my_result()???
the processes short (<0.5 seconds) have around 1 million of them.
i saw thread can return value multiprocessing.process? tried reproduce couldn't make work properly.
at entire disposal further information.
this question may considered duplicate anyway here solution problem:
def multiprocess(function, argslist, ncpu): total = len(argslist) done = 0 result_queue = mp.queue() jobs = [] while argslist != [] , done<10 : if len(mp.active_children()) < ncpu: p = mp.process(target=function,args=(result_queue, argslist.pop(),)) jobs.append(p) p.start() done+=1 print "\r",float(done)/total,"%", #get results here res = [result_queue.get() p in jobs] print res
and had change
return function_result
into
result_queue.put(function_result)
Comments
Post a Comment