All Products
Search
Document Center

Object Storage Service:probe

Last Updated:Nov 24, 2023

This topic describes how to use the probe command to detect exceptions of network connectivity between the local client and Object Storage Service (OSS), check upload and download bandwidths, and obtain the status of local symbolic links.

Important

Sample command lines in this topic are based on the 64-bit Linux system. For other systems, replace ./ossutil64 in the commands with the corresponding binary name. For more information, see ossutil.

Usage notes

You can run the probe command to detect exceptions of network connectivity between the local client and OSS and check the upload and download bandwidths. You can also run this command to check the status of a large number of symbolic links before you upload them.

  • Detect network exceptions

    ossutil checks the status of the network connection between the local client and OSS by uploading or downloading objects. To upload or download objects to the specified location when you detect network exceptions, we recommend that you specify the names of the actual objects. If you need only to detect network exceptions, run the command without specifying object names. ossutil generates temporary objects for connectivity detection and deletes the objects after connectivity detection is complete.

  • Check upload and download bandwidths

    When you run this command to check upload and download bandwidths, OSS provides a recommended number of concurrent tasks in uploads and downloads based on the CPU of your local devices and your network bandwidth. You can use the recommended number of concurrent tasks to maximize the usage of your network bandwidth.

  • Check the status of local symbolic links

    When you upload a large number of symbolic links, the upload is interrupted when one of the links is abnormal. We recommend that you check the status of local symbolic links before you upload them. If one of the symbolic links is abnormal, fix the issue before you upload the links.

After you run the probe command, you can view the procedure and result of each upload or download task.

  • Detect network exceptions by uploading or downloading objects

    • A check mark (√) indicates that the check result is positive. A cross sign (×) indicates that the check result is negative.

    • If the detection succeeds, ossutil returns the information about the upload or download task, including the size of the uploaded or downloaded object, the time used to perform the upload or download task, and the location to which the object is uploaded or downloaded.

    • If the detection fails, ossutil returns the error causes or error codes, which can be used to troubleshoot the errors.

      For more information about error codes, see Overview.

    • After a detection is complete, ossutil creates a log file named in the logOssProbe+DetectionCompletionTime.log format in the installation location. The log file contains detailed information about the probe command.

  • Check a specific item

    When you run the probe command to check the upload and download bandwidths or the status of local symbolic links, ossutil provides the check results and suggestions.

Check the network connection status by uploading an object and generate a report

ossutil checks the network connection status between the local client and a bucket by uploading an object to the bucket.

  • Command syntax

    ./ossutil64 probe {--upload [file_name]} {--bucketname bucket_name} [--object object_name] [--addr domain_name] [--upmode]

    The following table describes the parameters in the preceding command.

    Parameter

    Required

    Description

    --upload

    Yes

    Specifies the connectivity detection is performed by performing an upload.

    file_name

    No

    The full path of the local file that you want to upload to the bucket. If you leave this parameter unspecified, ossutil generates a temporary object and detects connectivity by uploading the object.

    --bucketname

    No

    The name of the bucket to which you want to check the connection.

    --object

    No

    The name of the object that you want to upload. If you specify this parameter in the command, ossutil stores the uploaded object in the specified bucket. If you do not specify this parameter in the command, ossutil deletes the uploaded object after the connectivity detection is complete.

    --addr

    No

    Specifies the network address whose connectivity you want to check. ossutil runs the ping command to check the network connectivity between your local computer and the specified address.

    Default value: www.aliyun.com.

    --upmode

    No

    The method used to upload the object.

    Valid values:

    • normal: This is the default value. The object is uploaded by using simple upload.

    • append: The object is uploaded by using append upload.

    • multipart: The object is uploaded by using multipart upload.

  • Examples

    • Check the network by uploading a random object and specifying the network address

      In this example, the bucket to which the object is uploaded is named examplebucket and the network address to check is aliyun.com. Sample command:

      ./ossutil64 probe --upload --bucketname examplebucket --addr aliyun.com

      Sample output:

      begin parse parameters and prepare file...[✓]
      begin network detection...[✓]
      begin upload file(normal)...[✓]
      
      *************************  upload result  *************************
      upload file:success
      upload file size:122880(byte)
      upload time consuming:245(ms)
      (only the time consumed by probe command)
      
      
      ************************* report log info*************************
      report log file:/root/logOssProbe20201201173031.log
    • Check network connectivity by uploading a specified object in default simple upload mode and delete the uploaded object after detection is complete

      In this example, the file named example.txt in the local root directory is uploaded to the bucket named examplebucket. Sample command:

      ./ossutil64 probe --upload example.txt --bucketname examplebucket 

      Sample output:

      begin parse parameters and prepare file...[✓]
      begin network detection...[✓]
      begin upload file(normal)...[✓]
      
      *************************  upload result  *************************
      upload file:success
      upload file size:7(byte)
      upload time consuming:224(ms)
      (only the time consumed by probe command)
      upload object is example.txt
      
      ************************* report log info*************************
      report log file:/root/logOssProbe20201201173841.log
    • Check network connectivity by uploading a specified object in append upload mode and store the uploaded object in the bucket after detection is complete

      In this example, the file named example.txt in the local root directory is uploaded to the bucket named examplebucket. Sample command:

      ./ossutil64 probe --upload example.txt --bucketname examplebucket --object example.txt --upmode append

      Sample output:

      begin parse parameters and prepare file...[✓]
      begin network detection...[✓]
      begin upload file(append)...[✓]
      
      *************************  upload result  *************************
      upload file:success
      upload file size:7(byte)
      upload time consuming:171(ms)
      (only the time consumed by probe command)
      
      upload object is example.txt
      
      ************************* report log info*************************
      report log file:/root/logOssProbe20201201174126.log

Check network connectivity by downloading an object using its URL and generate a report

ossutil checks network connectivity between the local client and a bucket by downloading an object using its URL from the bucket.

  • Command syntax

    ./ossutil64 probe {--download} {--url http_url} [--addr=domain_name] [file_name]

    The following table describes the parameters in the preceding command.

    Parameter

    Required

    Description

    --download

    Yes

    Specifies that the connectivity check is performed by downloading an object.

    --url

    Yes

    The URL of the object that you want to download. ossutil uses this URL to download the object to a local directory.

    • If the ACL of the object to download is public-read, set this parameter to the URL of the object. Example: https://examplebucket.oss-cn-beijing.aliyuncs.com/example.jpg.

    • If the ACL of the object to download is private, set this parameter to the signed URL of the object. The URL must be enclosed within double quotation marks (""). Example: "https://examplebucket.oss-cn-beijing.aliyuncs.com/example.jpg?Expires=1552015472&OSSAccessKeyId=TMP.******5r9f1FV12y8_Qis6LUVmvoSCUSs7aboCCHtydQ0axN32Sn-UvyY3AAAwLAIUarYNLcO87AKMEcE5O3A******oCFAQuRdZYyVFyqOW8QkGAN-bamUiQ&Signature=bIa4llbMbldrl7rwckr%2FXXvTtxw%3D".

    --addr

    No

    Specifies the network address whose connectivity you want to check. ossutil runs the ping command to check the network connectivity between your local computer and the specified address.

    Default value: www.aliyun.com.

    file_name

    No

    The local path to which the object is downloaded.

    • If you specify only the object name but do not specify a directory, the object is stored by using the specified name in the installation directory of ossutil.

    • If you specify only a directory but do not specify the object name, the object is stored by using its original name in the specified directory.

    • If you leave this parameter unspecified, the object is stored with its original name in the installation directory of ossutil.

  • Examples

    • Check network connectivity by downloading a specified object using its URL and rename the object

      In this example, the example.txt object whose ACL is public-read is downloaded from https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt to a local directory and renamed /localfile/test.txt. Sample command:

      ./ossutil64 probe --download --url https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt  /localfile/test.txt

      Sample output:

      begin parse parameters and prepare object...[✓]
      begin network detection...[✓]
      begin download file...[✓]
      
      *************************  download result  *************************
      download file:success
      download file size:57374182(byte)
      download time consuming:1246(ms)
      (only the time consumed by probe command)
      
      download file is /localfile/test.txt
      
      ************************* report log info*************************
      report log file:/root/logOssProbe20201202171639.log
                                      
    • Check network connectivity by downloading a specified object using its URL and specifying the network address for a ping test

      In this example, the example.txt object whose ACL is public-read is downloaded from https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt to a local directory and the network address for a ping test is aliyun.com. Sample command:

      ./ossutil64 probe --download --url https://examplebucket.oss-cn-beijing.aliyuncs.com/example.txt --addr aliyun.com

      Sample output:

      begin parse parameters and prepare object...[✓]
      begin network detection...[✓]
      begin download file...[✓]
      
      *************************  download result  *************************
      download file:success
      download file size:57374182(byte)
      download time consuming:1344(ms)
      (only the time consumed by probe command)
      
      download file is /root/example.txt
      
      ************************* report log info*************************
      report log file:/root/logOssProbe20201202172232.log                                

Check network connectivity by downloading a specified object and generate a report

ossutil checks network connectivity between the local client and a bucket by downloading an object from the bucket.

  • Command syntax

    ./ossutil64 probe {--download} {--bucketname bucket_name} [--object object_name] [--addr domain_name] [file_name]

    The following table describes the parameters in the preceding command.

    Parameter

    Required

    Description

    --download

    Yes

    Specifies that the connectivity check is performed by downloading an object.

    --bucketname

    Yes

    The name of the bucket to which you want to check the connection.

    --object

    No

    The name of the object that you want to download. If you specify this parameter in the command, ossutil downloads the specified object to a local directory. If you do not specify this parameter in the command, ossutil generates a temporary object, uploads the object to the specified bucket, and then downloads the object. After the connectivity check is complete, ossutil deletes the temporary object from the bucket.

    --addr

    No

    Specifies the network address whose connectivity you want to check. ossutil runs the ping command to check the network connectivity between your local computer and the specified address.

    Default value: www.aliyun.com.

    file_name

    No

    The local path to which the object is downloaded.

    • If you specify only the object name but do not specify a directory, the object is stored by using the specified name in the installation directory of ossutil.

    • If you specify only a directory but do not specify the object name, the object is stored by using its original name in the specified directory.

    • If you leave this parameter unspecified, the object is stored with its original name in the installation directory of ossutil.

  • Examples

    • Check network connectivity by downloading a specified object and rename the object

      In this example, the /ossfolder/example.txt object is downloaded from the examplebucket bucket and renamed /localfolder/text.txt. Sample command:

      ./ossutil64 probe --download --bucketname examplebucket --object /ossfolder/example.txt /localfolder/text.txt

      Sample output:

      begin parse parameters and prepare object...[✓]
      begin network detection...[✓]
      begin download file...[✓]
      
      *************************  download result  *************************
      download file:success
      download file size:57374182(byte)
      download time consuming:1108(ms)
      (only the time consumed by probe command)
      
      download file is /localfolder/text.txt
      
      ************************* report log info*************************
      report log file:/root/logOssProbe20201202173032.log
    • Check the network connection status by downloading a temporary object and specifying the address of the network for a ping test

      In this example, a temporary object is downloaded from the examplebucket bucket. The network address to check is aliyun.com. Sample command:

      ./ossutil64 probe --download --bucketname examplebucket --addr aliyun.com

      Sample output:

      begin parse parameters and prepare object...[✓]
      begin network detection...[✓]
      begin download file...[✓]
      
      *************************  download result  *************************
      download file:success
      download file size:122880(byte)
      download time consuming:516(ms)
      (only the time consumed by probe command)
      
      download file is /root/oss-test-probe-1606902911701892000-3ifwj9t0ln
      
      ************************* report log info*************************
      report log file:/root/logOssProbe20201202173512.log

Check a specific item

You can run the probe command to check a specific item, including the status of local symbolic links and the upload and download bandwidths. ossutil also returns the names of abnormal symbolic links and a recommended number of concurrent tasks in uploads and downloads based on the check results.

  • Command syntax

    ./ossutil64 probe {--probe-item item_value} {--bucketname bucket-name} [--object object_name][--parallel <value>][--part-size <value>][--runtime <value>]

    The following table describes the parameters in the preceding command.

    Parameter

    Required

    Description

    --probe-item

    Yes

    The item that you want to check.

    Valid values:

    • cycle-symlink: ossutil checks whether local abnormal symbolic links exist.

    • upload-speed: ossutil checks the upload bandwidth.

    • download-speed: ossutil checks the download bandwidth.

    • download-time: ossutil checks the time required for a download.

    --bucketname

    Yes only when the value of --probe-item is not cycle-symlink

    The name of the bucket.

    --object

    Yes only when the value of --probe-item is download-speed

    The path of the object that you want to download. The object must exist and be larger than 5 MB in size. Example: ossfolder/example.txt.

    --parallel

    No

    The number of concurrent tasks for an operation on a single object. Default value: 1.

    Note

    This parameter takes effect only when the value of --probe-item is download-time.

    --part-size

    No

    The part size. Unit: bytes. By default, ossutil determines the part size based on the object size.

    Valid values: 1 to 9223372036854775807.

    Note

    This parameter takes effect only when the value of --probe-item is download-time.

    --runtime

    No

    The maximum execution time. If the maximum execution time is exceeded, the command execution is interrupted.

    Note

    This parameter takes effect only when the value of --probe-item is upload-speed or download-speed.

  • Examples

    • Check whether abnormal symbolic links exist in the localfolder directory of the local root directory

      Sample command:

      ./ossutil64 probe --probe-item cycle-symlink /root/localfolder

      Sample output:

      Error: stat /root/localfolder/example.jpg: no such file or directory

      The output shows that the example.jpg symbolic link is abnormal.

    • Check the upload bandwidth

      In this example, ossutil uploads a temporary object to a bucket named examplebucket and returns a recommended number of concurrent upload tasks based on the hardware specifications of the current device and the upload bandwidth.

      Sample command:

      ./ossutil64 probe --probe-item upload-speed --bucketname examplebucket

      Sample output:

      cpu core count:2
      parallel:2,average speed:679.72(KB/s),current speed:1344.00(KB/s),max speed:1440.00(KB/s))
      parallel:3,average speed:643.31(KB/s),current speed:704.00(KB/s),max speed:1632.00(KB/s))
      parallel:4,average speed:646.62(KB/s),current speed:512.00(KB/s),max speed:1600.00(KB/s))
      
      suggest parallel is 2, max average speed is 679.72(KB/s)

      The output shows that the CPU of the device has two cores, the maximum average upload bandwidth is 679.72 KB/s, and the recommended number of concurrent upload tasks is 2.

    • Check the download bandwidth

      In this example, ossutil downloads the example.txt object from the examplebucket bucket and recommends the number of concurrent download tasks based on the hardware performance and download bandwidth.

      Sample command:

      ./ossutil64 probe --probe-item download-speed --bucketname examplebucket --object example.txt

      Sample output:

      cpu core count:2
      parallel:2,average speed:12524.93(KB/s),current speed:12288.63(KB/s),max speed:14302.25(KB/s)
      parallel:3,average speed:12564.45(KB/s),current speed:12144.39(KB/s),max speed:14484.24(KB/s)
      parallel:4,average speed:12545.21(KB/s),current speed:12766.58(KB/s),max speed:13534.42(KB/s)
      
      suggest parallel is 3, max average speed is 12564.45(KB/s)

      The output shows that the CPU of the device has two cores, the maximum average download bandwidth is 12,564.45 KB/s, and concurrent download tasks based on the hardware performance and download bandwidth.

    • Check the download bandwidth and specify the maximum execution time

      In this example, ossutil downloads the example.zip object from the examplebucket bucket and returns a recommended number of concurrent download tasks based on the hardware performance and download bandwidth, with a maximum execution time specified.

      ./ossutil64 probe --probe-item download-speed --bucketname examplebucket --object  example.zip --runtime 2

      Sample output:

      cpu core count:8
      parallel:8,average speed:10610.66(KB/s),current speed:11245.14(KB/s),max speed:11470.41(KB/s)
      parallel:9,average speed:9915.97(KB/s),current speed:10763.22(KB/s),max speed:11401.97(KB/s))
      parallel:10,average speed:9969.66(KB/s),current speed:11213.43(KB/s),max speed:11384.52(KB/s))
      parallel:11,average speed:10298.34(KB/s),current speed:10522.27(KB/s),max speed:11282.21(KB/s)
      parallel:12,average speed:10454.31(KB/s),current speed:9725.25(KB/s),max speed:11426.21(KB/s))
      parallel:13,average speed:9531.86(KB/s),current speed:8634.53(KB/s),max speed:11220.90(KB/s)))
      parallel:14,average speed:9329.45(KB/s),current speed:8542.94(KB/s),max speed:10505.16(KB/s))
      parallel:15,average speed:10193.00(KB/s),current speed:10770.35(KB/s),max speed:10971.39(KB/s)
      parallel:16,average speed:10637.59(KB/s),current speed:10483.77(KB/s),max speed:11179.55(KB/s)
      
      suggest parallel is 16, max average speed is 10637.59(KB/s)
      
      run download-speed  2 seconds with parallel 16
      parallel:16,average speed:9197.00(KB/s),current speed:9849.30(KB/s),max speed:9849.30(KB/s)
      280.345640(s) elapsed
    • Check the time consumed by a download

      In this example, ossutil downloads the example.txt object from the examplebucket bucket and returns the time consumed by the download task. Sample command:

      ./ossutil64 probe --probe-item download-time --bucketname examplebucket --object example.txt

      Sample output:

      downloading average speed:6178.75(KB/s),current speed:14446.02(KB/s),max speed:14446.02(KB/s)
      download-speed part-size:-1, parallel:1 total bytes:31707929, cost:4.918 s, avg speed:7741.00(kB/s)
      
      5.206202(s) elapsed
    • Check the time consumed by a download and set the part size

      In this example, ossutil downloads the example.zip object from the examplebucket bucket with a part size configured and returns the time consumed by the download task.

      ./ossutil64 probe --probe-item download-time --bucketname examplebucket --object example.zip --part-size 10000000

      Sample output:

      downloading average speed:9229.00(KB/s),current speed:18458.57(KB/s),max speed:18458.57(KB/s)
      download-speed part-size:10000000, parallel:1 total bytes:31707929, cost:3.230 s, avg speed:10321.33(kB/s)
      
      3.569254(s) elapsed
    • Check the time consumed by a download and configure the concurrency level

      In this example, ossutil downloads the example.zip object from the examplebucket bucket with a part size configured and returns the time consumed by the download task.

      ./ossutil64 probe --probe-item download-time --bucketname examplebucket --object example.zip --parallel 3

      Sample output:

      downloading average speed:9155.00(KB/s),current speed:18310.68(KB/s),max speed:18310.68(KB/s)
      download-speed part-size:-1, parallel:3 total bytes:31707929, cost:3.121 s, avg speed:10321.33(kB/s)
      
      3.424191(s) elapsed