Gordon
Assistant Engineer
Assistant Engineer
  • UID622
  • Fans1
  • Follows0
  • Posts52
Reads:1150Replies:0

Troubleshoot OSS Java SDK hanging

Created#
More Posted time:Nov 2, 2016 17:34 PM
Hanging means the program is still running, but it is stuck calling a method, with no returned values or thrown exceptions; the stuck time ranges from several seconds to several hours. When a Java program is hung, you should first use jstack to save the stack information of the Java program for future analysis and usage.
Hanging of a program calling the OSS Java SDK may fall into the following categories and their possible causes are provided.


Note:
• The jstack -l <pid> > js.txt can be used to save the PID stack information to the js.txt file. You can find the PID through the jps command.
• For packet capture tools, Wireshark is recommended for Windows systems. The capture filter can be set to
host <bucket-name>.<endpoint>, for example:
host my-bucket.oss-cn-hangzhou.aliyuncs.com`

Linux uses the tcpdump command for packet capture and the command format is as follows:
sudo tcpdump tcp -i <eth0> -t -s 0 -c 200000 and host <bucket>.<endpoint> -w <file.cap>

For example:
sudo tcpdump tcp -i eth0 -t -s 0 -c 200000 and host my-bucket.oss-cn-hangzhou.aliyuncs.com -w out.cap`

Then Wireshark is used to open file.cap to analyze the captured data.
• The InputStream obtained by getObject should be closed, as shown below:
OSSObject ossObject = ossClient.getObject(bucketName, key);
ossObject.getObjectContent().close();


• There is a bug in calling the getSimplifiedObjectMeta method in versions earlier than OSS Java SDK 2.2.3. The connection is not closed, leading to memory leakage. This issue is fixed in OSS Java SDK 2.2.3.
• The stack hangs at the PoolingHttpClientConnectionManager.leaseConnection method are as follows:
"main" prio=6 tid=0x000000000291e000 nid=0xc40 waiting on condition [0x0000000002dae000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000007d85697f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:138)
at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:306)
at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192)
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185)
at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at com.aliyun.oss.common.comm.DefaultServiceClient.sendRequestCore(DefaultServiceClient.java:113)
at com.aliyun.oss.common.comm.ServiceClient.sendRequestImpl(ServiceClient.java:123)
at com.aliyun.oss.common.comm.ServiceClient.sendRequest(ServiceClient.java:68)
at com.aliyun.oss.internal.OSSOperation.send(OSSOperation.java:94)
at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:146)
at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:113)
at com.aliyun.oss.internal.OSSObjectOperation.getObject(OSSObjectOperation.java:229)
at com.aliyun.oss.OSSClient.getObject(OSSClient.java:629)
at com.aliyun.oss.OSSClient.getObject(OSSClient.java:617)
at samples.HelloOSS.main(HelloOSS.java:49)


• The stack hangs in java.net.SocketOutputStream.socketWrite are as follows:
"main" prio=10 tid=0x0000000040112800 nid=0x2ad8 runnable [0x00000000418e1000]
java.lang.Thread.State: RUNNABLE
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
    at org.apache.http.impl.conn.LoggingOutputStream.write(LoggingOutputStream.java:77)
    at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:126)
    at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:138)
    at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:169)
    at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:115)
    at com.aliyun.oss.common.comm.RepeatableInputStreamEntity$NoAutoClosedInputStreamEntity.writeTo(RepeatableInputStreamEntity.java:127)
    at com.aliyun.oss.common.comm.RepeatableInputStreamEntity.writeTo(RepeatableInputStreamEntity.java:71)
    at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:158)
    at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:162)
    at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:237)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:122)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at com.aliyun.oss.common.comm.DefaultServiceClient.sendRequestCore(DefaultServiceClient.java:113)
    at com.aliyun.oss.common.comm.ServiceClient.sendRequestImpl(ServiceClient.java:121)
    at com.aliyun.oss.common.comm.ServiceClient.sendRequest(ServiceClient.java:67)
    at com.aliyun.oss.internal.OSSOperation.send(OSSOperation.java:92)
    at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:140)
    at com.aliyun.oss.internal.OSSOperation.doOperation(OSSOperation.java:111)
    at com.aliyun.oss.internal.OSSObjectOperation.writeObjectInternal(OSSObjectOperation.java:665)
    at com.aliyun.oss.internal.OSSObjectOperation.putObject(OSSObjectOperation.java:132)
    at com.aliyun.oss.OSSClient.putObject(OSSClient.java:528)
    at com.aliyun.oss.OSSClient.putObject(OSSClient.java:516)
    at com.aliyun.oss.OSSClient.putObject(OSSClient.java:522)
    at com.aliyun.oss.demo.HelloOSS.main(HelloOSS.java:25)
Guest