Performance Tuning-Java SDK tuning-Alibaba Cloud Developer Community

Environment preparation

• use Java 1.8 or later.
• View version run the java -version command to view the Java version

download SDK

• download directly from GitHub
• install the SDK and add dependencies to the Maven project (recommended method)

<dependency>
<groupId>com.aliyun.oss</groupId>
<artifactId>aliyun-sdk-oss</artifactId>
<version>2.8.3</version>
</dependency>

initialization parameters

log optimization parameters

generally, the log function is not enabled. Many users write slowly after logging on log4j is enabled. There are many ways to disable this feature on the Internet. You can find open-source processing documents to disable this feature, if you have configured the log4j property file, you need to configure the oss log level in that property file;

timeout parameter settings

in a weak network environment, it is best to set a longer timeout period to increase the number of retries to avoid Upload failures.

// 创建ClientConfiguration。ClientConfiguration是OSSClient的配置类，可配置代理、连接超时、最大连接数等参数。
ClientConfiguration conf = new ClientConfiguration();

// 设置OSSClient允许打开的最大HTTP连接数，默认为1024个。
conf.setMaxConnections(2048);
// 设置Socket层传输数据的超时时间，默认为50000毫秒。
conf.setSocketTimeout(10000);
// 设置建立连接的超时时间，默认为50000毫秒。
conf.setConnectionTimeout(10000);
// 设置从连接池中获取连接的超时时间（单位：毫秒），默认不超时。
conf.setConnectionRequestTimeout(1000);
// 设置连接空闲超时时间。超时则关闭连接，默认为60000毫秒。
conf.setIdleConnectionTime(10000);
// 设置失败请求重试次数，默认为3次。
conf.setMaxErrorRetry(5);

Network Streaming

when you use network streaming, the SDK first pulls the source file through the Internet. If the source file is pulled slowly, it directly affects the speed of writing to OSS. Try not to upload files through network streaming, and try to use local upload. If you want to use a network stream, make sure that the source file Network is smooth and the bandwidth is sufficient;

// Endpoint以杭州为例，其它Region请按实际情况填写。
String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
// 阿里云主账号AccessKey拥有所有API的访问权限，风险很高。强烈建议您创建并使用RAM账号进行API访问或日常运维，请登录 https://ram.console.aliyun.com 创建RAM账号。
String accessKeyId = "<yourAccessKeyId>";
String accessKeySecret = "<yourAccessKeySecret>";

// 创建OSSClient实例。
OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret);

// 上传网络流。
InputStream inputStream = new URL("https://www.aliyun.com/").openStream();
ossClient.putObject("<yourBucketName>", "<yourObjectName>", inputStream);

// 关闭OSSClient。
ossClient.shutdown();

host file descriptor

the connection configured by the SDK represents the maximum processing capability of the application layer, which is equivalent to the file descriptor of the host. If the FD set by the host is 1000, even if the FD set by the SDK application layer is 2000, the final result is less than 2,000, and a maximum of 1,000 connections can be processed;

code Optimization

java SDK supports resumable transmission based on shards. This allows you to add concurrency based on shards and supports resumable recording. When a network problem occurs, you do not need to retransmit data from a file every time, continue transmission from the position of the record breakpoint;

the recommended size of each part is 100m, 1m, 1000m, 10m, 100g, and 1g.

setPartSize set the part size. Default value: 1MB.

// Endpoint以杭州为例，其它Region请按实际情况填写。
String endpoint = "http://oss-cn-hangzhou.aliyuncs.com";
// 阿里云主账号AccessKey拥有所有API的访问权限，风险很高。强烈建议您创建并使用RAM账号进行API访问或日常运维，请登录 https://ram.console.aliyun.com 创建RAM账号。
String accessKeyId = "<yourAccessKeyId>";
String accessKeySecret = "<yourAccessKeySecret>";

// 创建OSSClient实例。
OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret);

ObjectMetadata meta = new ObjectMetadata();
// 指定上传的内容类型。
meta.setContentType("text/plain");

// 通过UploadFileRequest设置多个参数。
UploadFileRequest uploadFileRequest = new UploadFileRequest("<yourBucketName>","<yourObjectName>");

// 通过UploadFileRequest设置单个参数。
// 设置存储空间名称。
//uploadFileRequest.setBucketName("<yourBucketName>");
// 设置文件名称。
//uploadFileRequest.setKey("<yourObjectName>");
// 指定上传的本地文件。
uploadFileRequest.setUploadFile("<yourLocalFile>");
// 指定上传并发线程数，默认为1。
uploadFileRequest.setTaskNum(5);
// 指定上传的分片大小，范围为100KB~5GB，默认为文件大小/10000。
uploadFileRequest.setPartSize(1 * 1024 * 1024);
// 开启断点续传，默认关闭。
uploadFileRequest.setEnableCheckpoint(true);
// 记录本地分片上传结果的文件。开启断点续传功能时需要设置此参数，上传过程中的进度信息会保存在该文件中，如果某一分片上传失败，再次上传时会根据文件中记录的点继续上传。上传完成后，该文件会被删除。默认与待上传的本地文件同目录，为uploadFile.ucp。
uploadFileRequest.setCheckpointFile("<yourCheckpointFile>");
// 文件的元数据。
uploadFileRequest.setObjectMetadata(meta);
// 设置上传成功回调，参数为Callback类型。
uploadFileRequest.setCallback("<yourCallbackEvent>");

// 断点续传上传。
ossClient.uploadFile(uploadFileRequest);

// 关闭OSSClient。
ossClient.shutdown();

FAQ

the following section describes the issues that need to be noticed based on some common use cases. The cases are supplemented irregularly and all user data information has been processed.

SDK.ServerUnreachable : Speicified endpoint or uri is not valid

this problem means that the user is not connected to the Alibaba Cloud Gateway, which is generally related to the following reasons:

• the STS of concurrent requests on the client is too high in the code, and the ECS or local PC on the client is insufficient to handle the concurrency at that time, which reduces the concurrency of OSS.
• If a timeout occurs between the network and the server, you can perform packet capture verification.
• The STS SDK version and SDK core version of the user are not the latest. Replace it with the new SDK and test it

access NoSuchKey

case:java.lang.Exception: com.aliyun.oss.OSSException: 
Not Foundn[ErrorCode]: NoSuchKeyn

java SDK this problem occurs, the source file does not exist. For more information, see [404](https://yq.aliyun.com/articles/657166? spm=ata.13261165.0.0.181e32fbXDYxLN)

socketException

obviously, the error is that the socket is abnormal. This may be because the socket fails in the init phase and the request has not been sent to OSS. We recommend that the client check the following:

• whether the network jitter occurs at that time. You can run the ping -c 100 -s 1024-I 0.01 or mtr command to view the real-time network detection data.
• Indicates whether the number of socket connections on the host is full. You can use the netstat command to check the current number of connections;
• the size of the maxconnection parameter set in the SDK. If the number of connections exceeds the value set in maxconnection, a socket exception occurs.
• If there is no problem, you can only deploy tcpdump or wireshark to capture packets, reproduce the problem, and split the packets.

Use Java and web to upload data to OSS through STS

if the following error occurs, it may be related to cross-origin or OSS information configured in the code. You can follow these steps.

If you use java code, see

• indicates whether the host and bucket parameters are correct when the server generates the upload information.
• Indicates whether cross-origin files are configured for OSS.