Java: strange deadlock - multithreading

Question

I've got a deadlock in my application, but there is no obvious locking instance in the stack trace. How is this possible? Is this a bug?
jstack -l output
Full thread dump OpenJDK 64-Bit Server VM (19.0-b09 mixed mode):
"Attach Listener" daemon prio=10 tid=0x000000000120d000 nid=0x34a9 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"queue_1" prio=10 tid=0x0000000001043800 nid=0x2ecc waiting for monitor entry [0x00000000426f2000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.wordstat.AbstractWordstatEngine.startQueueProcessing(AbstractWordstatEngine.java:56)
at org.seosrv.engines.wordstat.WordstatEngineManager$EngineQueueXable.act(WordstatEngineManager.java:155)
at org.seosrv.engines.wordstat.WordstatEngineManager$EngineQueueXable.run(WordstatEngineManager.java:145)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"queue_2" prio=10 tid=0x0000000001043000 nid=0x2ecb waiting for monitor entry [0x00000000413a5000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.wordstat.AbstractWordstatEngine.startQueueProcessing(AbstractWordstatEngine.java:56)
at org.seosrv.engines.wordstat.WordstatEngineManager$EngineQueueXable.act(WordstatEngineManager.java:155)
at org.seosrv.engines.wordstat.WordstatEngineManager$EngineQueueXable.run(WordstatEngineManager.java:145)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_1" prio=10 tid=0x0000000001a7c800 nid=0xdb8 waiting for monitor entry [0x00000000433ff000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_2" prio=10 tid=0x0000000003f24800 nid=0xdb5 waiting for monitor entry [0x00000000431fd000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_3" prio=10 tid=0x0000000001578000 nid=0xdb2 waiting for monitor entry [0x00000000424f0000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_4" prio=10 tid=0x0000000003f20000 nid=0xdad waiting for monitor entry [0x0000000042ffb000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_5" prio=10 tid=0x0000000001091000 nid=0xda3 waiting for monitor entry [0x00000000428f4000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_6" prio=10 tid=0x0000000001592000 nid=0xd9d waiting for monitor entry [0x0000000042af6000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_7" prio=10 tid=0x00000000030f5000 nid=0xd98 waiting for monitor entry [0x0000000042efa000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_8" prio=10 tid=0x0000000001094000 nid=0xd92 waiting for monitor entry [0x0000000042bf7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_9" prio=10 tid=0x0000000001485800 nid=0xd5c waiting for monitor entry [0x0000000041e13000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_10" prio=10 tid=0x0000000003f26800 nid=0xd42 waiting for monitor entry [0x00000000432fe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_11" prio=10 tid=0x00000000032ed000 nid=0xd34 waiting for monitor entry [0x00000000425f1000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_12" prio=10 tid=0x00000000011b8000 nid=0xd05 waiting for monitor entry [0x00000000427f3000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_13" prio=10 tid=0x000000000104b800 nid=0xcb7 waiting for monitor entry [0x00000000422ee000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_14" prio=10 tid=0x00000000014f9800 nid=0xb3c waiting for monitor entry [0x00000000411f0000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_15" prio=10 tid=0x0000000001213000 nid=0xb25 waiting for monitor entry [0x00000000429f5000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_16" prio=10 tid=0x00000000032fa800 nid=0xabe waiting for monitor entry [0x00000000423ef000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_17" prio=10 tid=0x0000000003f22000 nid=0xaaf waiting for monitor entry [0x00000000430fc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"actor_18" prio=10 tid=0x0000000001595800 nid=0xa9f waiting for monitor entry [0x0000000042cf8000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.releaseConnectionId(ConnectionManagerImpl.java:29)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.releaseConnectionId(ConnectionUtils.java:19)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.act(SearchEngineContainer.java:352)
at org.seosrv.engines.search.SearchEngineContainer$PositionCheckerXable.run(SearchEngineContainer.java:266)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"queue_3" prio=10 tid=0x00000000014f2000 nid=0x890 waiting for monitor entry [0x0000000040b05000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:134)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"queue_4" prio=10 tid=0x0000000001039000 nid=0x88f waiting for monitor entry [0x000000004081b000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:134)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"queue_5" prio=10 tid=0x0000000001786000 nid=0x88e waiting for monitor entry [0x000000004071a000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:134)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"queue_6" prio=10 tid=0x0000000001f96000 nid=0x88d waiting for monitor entry [0x00000000421ed000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:134)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"queue_7" prio=10 tid=0x00000000021a3800 nid=0x88c waiting for monitor entry [0x0000000041d12000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
- waiting to lock <0x00007f9fd56f4ab8> (a org.seosrv.manager.ConnectionManagerImpl)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:134)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"queue_8" prio=10 tid=0x00000000016b9800 nid=0x88b waiting on condition [0x0000000040a04000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:136)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"DestroyJavaVM" prio=10 tid=0x0000000000cf8800 nid=0x86d waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Thread-4" prio=10 tid=0x0000000001495800 nid=0x88a runnable [0x0000000041c11000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x00007f9fd5878c78> (a sun.nio.ch.Util$1)
- locked <0x00007f9fd5878c90> (a java.util.Collections$UnmodifiableSet)
- locked <0x00007f9fd5878c00> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:307)
at java.lang.Thread.run(Thread.java:636)
Locked ownable synchronizers:
- None
"server-timer" daemon prio=10 tid=0x0000000001613000 nid=0x889 in Object.wait() [0x0000000040fcc000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f9fd5878d78> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:531)
- locked <0x00007f9fd5878d78> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:484)
Locked ownable synchronizers:
- None
"InternetCheckerThread" daemon prio=10 tid=0x0000000001310800 nid=0x884 waiting on condition [0x000000004032c000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.seosrv.utility.InternetCheckerThread.run(InternetCheckerThread.java:61)
Locked ownable synchronizers:
- None
"ConsoleThread" prio=10 tid=0x00000000015e3800 nid=0x883 runnable [0x0000000040ecb000]
java.lang.Thread.State: RUNNABLE
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:236)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
- locked <0x00007f9fd54b01c0> (a java.io.BufferedInputStream)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
- locked <0x00007f9fd56e8a48> (a java.io.InputStreamReader)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:153)
at java.io.BufferedReader.readLine(BufferedReader.java:316)
- locked <0x00007f9fd56e8a48> (a java.io.InputStreamReader)
at java.io.BufferedReader.readLine(BufferedReader.java:379)
at org.fs.jcs.JcsCliThread.run(JcsCliThread.java:97)
Locked ownable synchronizers:
- None
"WordstScheduleThread" prio=10 tid=0x000000000172d800 nid=0x882 waiting on condition [0x0000000041f68000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.seosrv.engines.wordstat.WordstatEngineManager.launchAndWait(WordstatEngineManager.java:104)
at org.seosrv.schedule.WordstatSchedule.fullCheck(WordstatSchedule.java:64)
at org.seosrv.schedule.WordstatSchedule.run(WordstatSchedule.java:100)
Locked ownable synchronizers:
- None
"PosScheduleThread" prio=10 tid=0x00000000012d1800 nid=0x881 waiting on condition [0x000000004197c000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.seosrv.engines.search.SearchEngineManager.launchAndWait(SearchEngineManager.java:121)
at org.seosrv.schedule.PositionSchedule.fullCheck(PositionSchedule.java:135)
at org.seosrv.schedule.PositionSchedule.run(PositionSchedule.java:326)
Locked ownable synchronizers:
- None
"net.sf.ehcache.CacheManager#45c1f5b2" daemon prio=10 tid=0x0000000001787000 nid=0x880 in Object.wait() [0x0000000040c97000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f9fd56e8e20> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:531)
- locked <0x00007f9fd56e8e20> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:484)
Locked ownable synchronizers:
- None
"Low Memory Detector" daemon prio=10 tid=0x0000000000d8e800 nid=0x876 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"CompilerThread1" daemon prio=10 tid=0x0000000000d89000 nid=0x875 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"CompilerThread0" daemon prio=10 tid=0x0000000000d87000 nid=0x874 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Signal Dispatcher" daemon prio=10 tid=0x0000000000d85800 nid=0x873 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Finalizer" daemon prio=10 tid=0x0000000000d60800 nid=0x870 in Object.wait() [0x00000000416d2000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f9fd54b0550> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
- locked <0x00007f9fd54b0550> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
Locked ownable synchronizers:
- None
"Reference Handler" daemon prio=10 tid=0x0000000000d5f000 nid=0x86f in Object.wait() [0x000000004049a000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f9fd54b0630> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x00007f9fd54b0630> (a java.lang.ref.Reference$Lock)
Locked ownable synchronizers:
- None
"VM Thread" prio=10 tid=0x0000000000d58000 nid=0x86e runnable
"VM Periodic Task Thread" prio=10 tid=0x0000000000d91800 nid=0x877 waiting on condition
JNI global references: 1608
ConnectionManagerImpl is a singleton, it's guaranteed that there is no explicit synchronization on it via synchronized (conManager) {...} within the hand-written code. Here is it's source:
public class ConnectionManagerImpl implements ConnectionManager {
private int maxCons = 30;
private int lastGrantedConnectionId = 0;
private final List <Integer> allConnections = new ArrayList <Integer>();
#Override
public synchronized int getConnectionId(){
if (allConnections.size() >= maxCons) return -1;
if (!InternetChecker.waitForInternet()) { //unsynch
return -1;
}
int newId = ++lastGrantedConnectionId;
allConnections.add(newId);
return newId;
}
#Override
public synchronized void releaseConnectionId(int connectionId){
if (!allConnections.remove(Integer.valueOf(connectionId))) {
LogUtils.logError(new InternalException("Finished thread never was registred: " + "networkId = " + connectionId + ", active network IDs: " + allConnections));
}
}
}
VPS configuration: CentOS 5 x64, OpenJDK 64-Bit 19.0-b09
App runned with
-XX:HeapDumpPath=path_here
-XX:+HeapDumpOnOutOfMemoryError
-XX:-PrintClassHistogram
-XX:-UseCompressedOops

Answer 1

I removed a InternetChecker.waitForInternet() call like Udi Cohen proposed and here is what I got:
Exception in thread "queue_1" java.lang.IllegalMonitorStateException
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:134)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
Exception in thread "queue_2" java.lang.IllegalMonitorStateException
at org.seosrv.manager.ConnectionManagerImpl.getConnectionId(ConnectionManagerImpl.java:20)
at org.seosrv.utility.ConnectionUtils.getConnectionId(ConnectionUtils.java:15)
at org.seosrv.engines.search.SearchEngineContainer.startQueueProcessing(SearchEngineContainer.java:134)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.act(SearchEngineManager.java:202)
at org.seosrv.engines.search.SearchEngineManager$EngineQueueXable.run(SearchEngineManager.java:189)
at java.lang.Thread.run(Thread.java:636)
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (synchronizer.cpp:1954), pid=15269, tid=1104439616
# guarantee(mid->header()->is_neutral()) failed: invariant
#
# JRE version: 6.0_20-b20
# Java VM: OpenJDK 64-Bit Server VM (19.0-b09 mixed mode linux-amd64 )
# Derivative: IcedTea6 1.9.8
# Distribution: CentOS release 5.6 (Final), package rhel-1.22.1.9.8.el5_6-x86_64
# An error report file with more information is saved as:
# /hs_err_pid15269.log
#
# If you would like to submit a bug report, please include
# instructions how to reproduce the bug and visit:
# http://icedtea.classpath.org/bugzilla
#
I've already noticed the IllegalMonitorStateException under Oracle JDK 1.6.0_27 for c3p0 thread, but I thought that was a c3p0 problem. Now i've tested it better and got similiar error with IBM SDK.
It seems like this is a virtualization bug. Thank you for your efforts.

Answer 2

(Disclaimer: my company develops JProfiler)
Obviously, the jstack output is missing something in this case.
I would recommend to consult a different tool that gets this information from the JVMTI profiling interface. In JProfiler, you get a view like this:

Answer 3

Do you have any specific reason for using OpenJDK?
If not you can switch to another JDK to make sure this is not a bug in the JDK implementation.