Reads:45685Replies:1
Java threads
This article has been “at the bottom of my closet” for a long time. Recently some colleagues from our company's technical community have posted several articles sharing their ideas on Java threads and I found more can be discussed about this topic. So I spent some time to finish this article. This article only focuses on thread models on the JVM and does not mean to reflect the thread models in real operating systems (Due to the limited length of this article, the thread dump structures won’t be discussed, nor will the comprehensive usage of tools during the optimization process, such as ps, perf.top, iostat, jstack, TDA plugin, and thread inspector. If you have any questions, please feel free to post a reply for discussion). I may consider conducting in-depth analysis on the thread/process models on the xUnix and Windows platforms and hope you will like it.
Okay. Let's get down to the business. Start with the following picture: Java threads have a total of six states, namely New, Runnable, Blocked, Waiting, Timed_Waiting and Terminated. We will focus on the Blocked and Timed_Waiting states. Blocked state: There are generally two preconditions for the thread to enter this state: first, waiting for monitor (intrinsic or external); second, using entry or reenter to synchronize code blocks. Here let's first look at the two queues in the Java thread model, as shown in the figure below: Each monitor can only be owned by one thread at a time, and the owner thread is the “active thread”, and all the other threads are “waiting threads” waiting in two queues, namely the “entry set” and the “wait set” respectively. The state of threads in the “entry set” is “Waiting for monitor entry”, while the state of threads waiting in the “wait set” is “in Object.wait()”. If you have used the ReentrantLock or ReentrantReadWriteLock classes improperly, your thread may be stuck in the Blocked state which is also a frequently-encountered case during our optimization. The solution is also very easy, just to locate the address of the thread to be locked and analyze whether thread starvation has occurred. The official documents provide detailed explanations on the Timed_Waiting state, that is, the thread will enter the state when you call the methods below. Thread.sleep Object.wait with timeout Thread.join with timeout LockSupport.parkNanos LockSupport.parkUntil Here we should pay attention to the LockSupport class. The class is a basic thread congestion primitive used to create locks and other sync classes, and an optimization for Thread.suspend and Thread.resume(). It is also an optimization targeting busy wait and preventing excessive spin (If you are interested in this topic, you can refer to the Reference document No 3). Okay. After briefing the several key thread states, we can now start to look at the thread stack through several cases: Case 1: Acceptor in the NIO "qtp589745448-36 Acceptor0 SelectChannelConnector@0.0.0.0:8161" prio=10 tid=0x00007f02f8eea800 nid=0x18ee runnable [0x00007f02e70b3000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241) - locked <0x00000000ec8ffde8> (a java.lang.Object) at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None The implementation in the source code is as follows: public void accept(int acceptorID) throws IOException 100 { 101 ServerSocketChannel server; 102 synchronized(this) 103 { 104 server = _acceptChannel; 105 } 106 107 if (server!=null && server.isOpen() && _manager.isStarted()) 108 { 109 SocketChannel channel = server.accept(); 110 channel.configureBlocking(false); 111 Socket socket = channel.socket(); 112 configure(socket); 113 _manager.register(channel); 114 } 115 } I want to emphasize on the thread stack that the NID (native LWP ID) refers to native lightweight process (that is, thread) ID. Case 2: Selector in the NIO "qtp589745448-35 Selector0" prio=10 tid=0x00007f02f8ee9800 nid=0x18ed runnable [0x00007f02e71b4000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:228) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:81) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000ec9006f0> (a sun.nio.ch.Util$2) - locked <0x00000000ec9006e0> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000ec9004c0> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569) at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None The code snippets are as follows: // If we should wait with a select 566 if (wait>0) 567 { 568 long before=now; 569 selector.select(wait); 570 now = System.currentTimeMillis(); 571 _timeout.setNow(now); 572 573 // If we are monitoring for busy selector 574 // and this select did not wait more than 1ms 575 if (__MONITOR_PERIOD>0 && now-before <=1) 576 { 577 // count this as a busy select and if there have been too many this monitor cycle 578 if (++_busySelects>__MAX_SELECTS) 579 { 580 // Start injecting pauses 581 _pausing=true; 582 583 // if this is the first pause 584 if (!_paused) 585 { 586 // Log and dump some status 587 _paused=true; 588 LOG.warn("Selector {} is too busy, pausing!",this); 589 } 590 } 591 } 592 } Case 3: Handlers for the MQTT protocol in ActiveMQ "ActiveMQ Transport Server Thread Handler: mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600" daemon prio=10 tid=0x00007f02f8ba6000 nid=0x18dc waiting on condition [0x00007f02ec824000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000faad0458> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.activemq.transport.tcp.TcpTransportServer$1.run(TcpTransportServer.java:373) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None Code snippets: @Override protected void doStart() throws Exception { if (useQueueForAccept) { Runnable run = new Runnable() { @Override public void run() { try { while (!isStopped() && !isStopping()) { Socket sock = socketQueue.poll(1, TimeUnit.SECONDS); if (sock != null) { handleSocket(sock); } } } catch (InterruptedException e) { LOG.info("socketQueue interuppted - stopping"); if (!isStopping()) { onAcceptError(e); } } } }; socketHandlerThread = new Thread(null, run, "ActiveMQ Transport Server Thread Handler: " + toString(), getStackSize()); socketHandlerThread.setDaemon(true); socketHandlerThread.setPriority(ThreadPriorities.BROKER_MANAGEMENT - 1); socketHandlerThread.start(); } super.doStart(); } Case 5: Simulate bank transfer and deposit. "withdraw" prio=10 tid=0x00007f3428110800 nid=0x2b6b waiting for monitor entry [0x00007f34155bb000] java.lang.Thread.State: BLOCKED (on object monitor) at com.von.thread.research.DeadThread.depositMoney(DeadThread.java:13) - waiting to lock <0x00000000d7fae540> (a java.lang.Object) - locked <0x00000000d7fae530> (a java.lang.Object) at com.von.thread.research.DeadThread.run(DeadThread.java:28) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None "deposit" prio=10 tid=0x00007f342810f000 nid=0x2b6a waiting for monitor entry [0x00007f34156bc000] java.lang.Thread.State: BLOCKED (on object monitor) at com.von.thread.research.DeadThread.withdrawMoney(DeadThread.java:21) - waiting to lock <0x00000000d7fae530> (a java.lang.Object) - locked <0x00000000d7fae540> (a java.lang.Object) at com.von.thread.research.DeadThread.run(DeadThread.java:29) at java.lang.Thread.run(Thread.java:724) Locked ownable synchronizers: - None Found one Java-level deadlock: ============================= "withdraw": waiting to lock monitor 0x00007f3400003620 (object 0x00000000d7fae540, a java.lang.Object), which is held by "deposit" "deposit": waiting to lock monitor 0x00007f3400004b20 (object 0x00000000d7fae530, a java.lang.Object), which is held by "withdraw" Java stack information for the threads listed above: =================================================== "withdraw": at com.von.thread.research.DeadThread.depositMoney(DeadThread.java:13) - waiting to lock <0x00000000d7fae540> (a java.lang.Object) - locked <0x00000000d7fae530> (a java.lang.Object) at com.von.thread.research.DeadThread.run(DeadThread.java:28) at java.lang.Thread.run(Thread.java:724) "deposit": at com.von.thread.research.DeadThread.withdrawMoney(DeadThread.java:21) - waiting to lock <0x00000000d7fae530> (a java.lang.Object) - locked <0x00000000d7fae540> (a java.lang.Object) at com.von.thread.research.DeadThread.run(DeadThread.java:29) at java.lang.Thread.run(Thread.java:724) Found 1 deadlock. Here it is a deadlock triggered by non-sequential locking. Well, it is about time to summarize the three cases deserving special attention during the optimization process (grep java.lang.Thread.State dump.bin | awk ‘{print $2$3$4$5}’ | sort | uniq -c): 1. Waiting for monitor entry – thread is in the Blocked state. Possible issues: Deadlock (sequential deadlock, starvation deadlock...) 2. Waiting on condition – sleeping or Timed_Waiting state. Possible issues: IO bottleneck 3. Object.wait – Timed_Waiting. To use wait & notifyAll, you need to be clear about their performance ceilings and limitations. In JCIP (Java Concurrency in Practice) we also recommend to use the advanced concurrency primitive AQS (Abstract Queued Synchronizer) provided by JUC (Java Util Concurrency) as much as possible. References: 1. http://architects.dzone.com/articles/how-analyze-java-thread-dumps 2. http://stackoverflow.com/questions/7698861/simple-java-example-runs-with-14-threads-why 3. http://docs.oracle.com/javase/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html 4. http://www.artima.com/insidejvm/ed2/threadsynch.html 5. http://stackoverflow.com/questions/37026/java-notify-vs-notifyall-all-over-again |
|
1st Reply#
Posted time:Jul 13, 2017 15:01 PM
Threads and Deadlock
A lock occurs when multiple processes try to access the same resource at the same time. A deadlock occurs when the waiting process is still holding on to another resource that the first needs before it can finish. A Java multithreaded program may suffer from the deadlock condition because the synchronized keyword causes the executing thread to block while waiting for the lock, or monitor , associated with the specified object. More on....Deadlock in Java |
|