multithreading - Elasticsearch transport client spawning a large number of threads -


from apache storm bolt, using elasticsearch's transport client remotely connect es cluster. when take jstack output of storm process, notice there 1000 threads es stack trace like:

elasticsearch[flying tiger][transport_client_worker][t#22]{new i/o worker #269}" daemon prio=10 tid=0x00007f80ac3cb000 nid=0x356b runnable [0x00007f7e96b2a000]    java.lang.thread.state: runnable     @ sun.nio.ch.epollarraywrapper.epollwait(native method)     @ sun.nio.ch.epollarraywrapper.poll(epollarraywrapper.java:269)     @ sun.nio.ch.epollselectorimpl.doselect(epollselectorimpl.java:79)     @ sun.nio.ch.selectorimpl.lockanddoselect(selectorimpl.java:87)     - locked <0x00000000d148d138> (a sun.nio.ch.util$2)     - locked <0x00000000d148d128> (a java.util.collections$unmodifiableset)     - locked <0x00000000d148c9b8> (a sun.nio.ch.epollselectorimpl)     @ sun.nio.ch.selectorimpl.select(selectorimpl.java:98)     @ org.elasticsearch.common.netty.channel.socket.nio.selectorutil.select(selectorutil.java:68)     @ org.elasticsearch.common.netty.channel.socket.nio.abstractnioselector.select(abstractnioselector.java:415)     @ org.elasticsearch.common.netty.channel.socket.nio.abstractnioselector.run(abstractnioselector.java:212)     @ org.elasticsearch.common.netty.channel.socket.nio.abstractnioworker.run(abstractnioworker.java:89)     @ org.elasticsearch.common.netty.channel.socket.nio.nioworker.run(nioworker.java:178)     @ org.elasticsearch.common.netty.util.threadrenamingrunnable.run(threadrenamingrunnable.java:108)     @ org.elasticsearch.common.netty.util.internal.deadlockproofworker$1.run(deadlockproofworker.java:42)     @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145)     @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615)     @ java.lang.thread.run(thread.java:745) 

i using single instance of es transport client across storm topology, has 18 output streams invoking es client write data es cluster.

why es transport client spawn many threads ? there way can tune ? tried looking es documentation not provide internal details on threading mechanism of transport client nor give option tune number of threads of client.

i had similar experience before. you've mentioned, 1 transport client creates tens of threads including timers , on.

what have check whether there 1 transport client per worker process. in earlier days, when used 32 transport clients, there more 1 thousand threads , after i've correctly modified singleton instance, number of threads decreased less 2 hundreds(including other threads created in topology)


Comments

Popular posts from this blog

android - MPAndroidChart - How to add Annotations or images to the chart -

javascript - Add class to another page attribute using URL id - Jquery -

firefox - Where is 'webgl.osmesalib' parameter? -