Understanding JVM's "Attach Listener" thread - multithreading

When I use
jstack -l pid
to see the thread's dump info, I get result as follow:
"Attach Listener" daemon prio=10 tid=0x01e4a800 nid=0x109c waiting on condition
[0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
My questions are as follow:
What's the function of "Attach Listener" thread?
When does this thread be created? Where? I can't find it.
Thank you very much.

See here http://openjdk.java.net/groups/hotspot/docs/Serviceability.html
Find "Dynamic Attach”
Dynamic attach has an attach listener thread in the target JVM. This
is a thread that is started when the first attach request occurs.

Related

16 threads waiting the same lock

I'm having a problem in a glassfish 3 server. The log is throwing messages
java.util.concurrent.RejectedExecutionException: The thread pool's task queue is full, limit: 256
I did a couple of thread dumps and found that there are 16 threads waiting for the same lock, for example:
"__ejb-thread-pool1" daemon prio=6 tid=0x39657c00 nid=0x1c08 waiting on condition [0x3297f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x117b2cb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Locked ownable synchronizers:
- None
"__ejb-thread-pool2" daemon prio=6 tid=0x38408c00 nid=0x1a3c waiting on condition [0x3ad3f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x117b2cb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Locked ownable synchronizers:
- None
...
is this normal? I thought I was going to find a thread owning the lock in the thread dump, but there isn't. Please help.. I'm new to thread analysis.
If some executor service has 16 theads doing nothing, then yes it's perfectly normal that they are all waiting for something to show in the task queue.
The rejected executions likely occured at another moment than when you snapshot threads. It only means there was a spike in submissions and the queue overflowed earlier.
Careful though, these rejected executions may not even be on the executor service you are looking at. It depends how many other executor services there are elsewhere.

Is it possible to find the creation time of a thread using thread dump?

I use jmap -dump:format=b,file=<file.dump> <PID> command to generate thread dump of a JVM instance. I just want to know if it's possible to find the time of creation of a thread running in the JVM instance, using the thread dump generated.
This is a heap dump, not a thread dump. There is no info about thread creation time in a heap dump. However, you can find it from a thread dump as described below.
Dump threads using jstack <PID>
Find nid of the thread you are interested in:
"Thread-7" #30 daemon prio=5 os_prio=0 tid=0x00002aaac9688800 nid=0x6945 runnable [0x00000000429c5000]
^^^^^^
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPoll.epollWait(Native Method)
at sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:194)
at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:268)
at java.lang.Thread.run(Thread.java:745)
Convert it to decimal: TID = 0x6945 = 26949
Get the start time using ps -Lo tid,lstart <PID> | grep <TID>
26949 Tue May 30 19:16:29 2017

jvm stack threads increase

There are many Timer threads in my jvm. From thread dump, i get this:
"Timer-19" - Thread t#159
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Native Method)
- waiting on <134c08ad> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
at java.util.TimerThread.run(Timer.java:462)
Locked ownable synchronizers:
- None
In my program i don't use Timer, it confused me a lot.
So please help!
attach a debugger
set a breakpoint on the Timer constructor
wait until it triggers
inspect the call stack to find the caller

message-driven-channel-adapter stopped processing messages

I have noticed that after few days running this stopped consuming messages from the queue with no known reason or trace, i have to restart it to continue processing.
Are there any known reasons behind this, also how to find out what happened in such cases and ways to fix it.
I am using WMQ and need to maintain transaction while processing a message. Thanks You.
<jms:message-driven-channel-adapter
acknowledge="transacted"
destination="inboundQueue"
channel="inboundChannel"
auto-startup="false"
max-concurrent-consumers="5"
transaction-manager="transactionManager"
recovery-interval="60000"/>
Also i will like to turn on debug on this consumer thread(s) to print that its trying to get a message, is there a way to do that.
EDITED*
I see that this had deadlocked, please let me know the cause and possible resolution, thanks.
"receivingTaskExecutor-12" - Thread t#961
java.lang.Thread.State: BLOCKED
at org.springframework.jms.listener.AbstractJmsListeningContainer.isRunning(AbstractJmsListeningContainer.java:348)
- waiting to lock <66a37d72> (a java.lang.Object) owned by "mqReceivingTaskExecutor-10" t#878
at org.springframework.jms.listener.DefaultMessageListenerContainer.scheduleNewInvokerIfAppropriate(DefaultMessageListenerContainer.java:715)
at org.springframework.jms.listener.DefaultMessageListenerContainer.messageReceived(DefaultMessageListenerContainer.java:692)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:243)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1102)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:996)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Locked ownable synchronizers:
- locked <5a9e40d2> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"receivingTaskExecutor-10" - Thread t#878
java.lang.Thread.State: TIMED_WAITING
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <35212499> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.LinkedBlockingQueue.offer(LinkedBlockingQueue.java:328)
at org.springframework.integration.util.CallerBlocksPolicy.rejectedExecution(CallerBlocksPolicy.java:58)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:822)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1373)
at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.execute(ThreadPoolTaskExecutor.java:254)
at org.springframework.jms.listener.DefaultMessageListenerContainer.doRescheduleTask(DefaultMessageListenerContainer.java:682)
at org.springframework.jms.listener.AbstractJmsListeningContainer.rescheduleTaskIfNecessary(AbstractJmsListeningContainer.java:519)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1036)
- locked <66a37d72> (a java.lang.Object)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Locked ownable synchronizers:
- locked <78ff22ed> (a java.util.concurrent.ThreadPoolExecutor$Worker)
Most likely is some problem up-stack (hung in user code somewhere). Use jstack <pid> to see what the container thread is doing when you hit this condition.

If non-daemon thread is waiting for daemon one?

If non-daemon thread is waiting for daemon thread (with join() method), will this prevent JVM from exit?
will this prevent JVM from exit?
Yes:-)
For threads created with the threading module, the main thread joins all non-daemon threads on exit.
If you have some non-daemon thread which is waiting for a lock, and you do not arrange for the lock to be released, then the main thread will hang on exit.
The method wait() waits for a signal (which can come from anywhere), and not for a particular thread (so your question is incorrectly worded).
So as long as non-daemon thread is waiting (does not matter what), jvm would not exit.

Resources