20 August 2015

gevent not so parallel

2 issue here.

1) gevent.pool is not so perfect. It would not start new thread untill all running thread exit.
Perfect implementation should start a thread immediately when there is a vacant in the pool.

2) the thread seem not execute parallel. In below example, function do_work_function clearly not running in parallel.

Really feel been bluffing all this while..... thanx 0xpcode :)



from gevent.pool import Pool
import time
import logging

rows = range(10)
CONCURRENCY = 4
pool = Pool(CONCURRENCY)

def do_work_function(param1):
print "start" , str(param1 )
    time.sleep(4)
    print "--finish" , str(param1 )
 
for row in rows:
  #logging.info(count)
  pool.spawn(do_work_function,row) # blocks here when pool size == CONCURRENCY
  print "spawn thread %s" % str(row)

print "start join"
pool.join(timeout=1) #blocks here until the last 10 are complete
print "end join" 

No comments:

Post a Comment

Terima kasih