All Products
Search
Document Center

Object Storage Service:Form upload

Last Updated:Apr 12, 2024

Form upload allows users to directly upload objects to Object Storage Service (OSS) by using standard HTML forms from web applications. This way, after an object is selected on the front-end page, the browser initiates a PostObject request to directly upload the object to the OSS server without using the website server. This reduces the workload on the website server and improves the efficiency and stability of object upload.

Limits

The size of the object that you want to upload by using form upload cannot exceed 5 GB.

Scenarios

Form upload is widely used in web applications, including but not limited to the following scenarios:

  • User data upload: Users upload an avatar, ID card photo, or other identity verification materials when they register an account. They upload a new avatar or background image when they modify information in the personal center.

  • Object sharing and storage: Users upload objects in various formats, such as documents, images, audio files, and video files, to Alibaba Cloud for storage and sharing by using forms on online disks and collaborative working platforms.

  • Content creation and publishing: Users write articles on platforms, such as blogs, forums, and Q&A communities, and upload images and attachments as content supplements by using forms.

  • E-commerce management: Merchants upload information, such as commodity pictures, detailed description documents, and qualifications, in the background of e-commerce platforms. Consumers upload invoice requirements or other supporting materials during the purchase process.

  • Online education platform: Students upload homework or projects, such as documents, PPTs, and video files. Teachers upload teaching materials and courseware.

  • Recruitment website: Job seekers upload resumes, portfolios and other materials. Enterprises upload materials, such as their company logos and recruitment documents, when they post positions.

  • Questionnaires and feedback: Users upload additional evidence or explanatory documents when they fill out the online questionnaires.

  • Software development collaboration: Developers upload code files or project documents from code hosting platforms such as GitHub and GitLab.

Solution

The client requests the signature and POST policy required by a PostObject request from the application server. Then, the client can use the signature and POST policy to upload objects without using OSS SDKs. You can use the policy generated by the server to limit the attributes of an object that can be uploaded, such as the object size and type. This solution is suitable for uploading objects by using HTML forms. This solution does not support multipart upload and resumable upload. For more information, see PostObject.

The following figure shows how to upload an object from a client to OSS by using the signature and POST policy that are obtained from the application server.

image
  1. The client requests information, such as a signature and POST policy, from the application server.

  2. The application server generates and returns information, such as the signature and POST policy, to the client.

  3. The client uses information, such as the signature and POST policy, to call the PostObject operation to upload the object to OSS by using an HTML form.

  4. OSS returns a success response to the client.

Procedure

  1. The server generates information, such as the signature and POST policy.

    • Demo project

    • Examples

      The following sample code provides an example on how to obtain a signature and POST policy from the application server:

      import os
      from hashlib import sha1 as sha
      import json
      import base64
      import hmac
      import datetime
      import time
      
      # Configure the OSS_ACCESS_KEY_ID environment variable. 
      access_key_id = os.environ.get('OSS_ACCESS_KEY_ID')
      # Configure the OSS_ACCESS_KEY_SECRET environment variable. 
      access_key_secret = os.environ.get('OSS_ACCESS_KEY_SECRET')
      # Replace <YOUR_BUCKET> with the name of the bucket. 
      bucket = '<YOUR_BUCKET>'
      # Set host to a value that is in the BucketName.Endpoint format. Replace <YOUR_BUCKET> with the name of the bucket. Replace <YOUR_ENDPOINT> with the endpoint of the region in which the bucket is located. 
      host = 'https://<YOUR_BUCKET>.<YOUR_ENDPOINT>'
      # Specify the prefix in the name of the object that you want to upload to OSS. 
      upload_dir = 'user-dir-prefix/'
      # Specify the validity period. Unit: seconds. 
      expire_time = 3600
      
      
      def generate_expiration(seconds):
          """
          The expiration time is calculated based on the specified validity period. Unit: seconds. 
          :param seconds: the validity period (seconds). 
          :return: the time string in the ISO 8601 standard. Example: 2014-12-01T12:00:00.000Z. 
          """
          now = int(time.time())
          expiration_time = now + seconds
          gmt = datetime.datetime.utcfromtimestamp(expiration_time).isoformat()
          gmt += 'Z'
          return gmt
      
      
      def generate_signature(access_key_secret, expiration, conditions, policy_extra_props=None):
          """
          Generate a signature string. 
          :param access_key_secret: the AccessKey secret that has the permissions to access the bucket. 
          :param expiration: the time when the signature expires. Specify the time in the ISO 8601 standard in the yyyy-MM-ddTHH:mm:ssZ format. Example: 2014-12-01T12:00:00.000Z. 
          :param conditions: the policy conditions that can be used to limit the values that you can specify during form upload. 
          :param policy_extra_props: the additional policy parameters. You can pass additional parameters as a dict. 
          :return: the signature string. 
          """
          policy_dict = {
              'expiration': expiration,
              'conditions': conditions
          }
          if policy_extra_props is not None:
              policy_dict.update(policy_extra_props)
          policy = json.dumps(policy_dict).strip()
          policy_encode = base64.b64encode(policy.encode())
          h = hmac.new(access_key_secret.encode(), policy_encode, sha)
          sign_result = base64.b64encode(h.digest()).strip()
          return sign_result.decode()
      
      def generate_upload_params():
          policy = {
              # Specify the validity period of the policy. 
              "expiration": generate_expiration(expire_time),
              # Specify the policy conditions. 
              "conditions": [
                  # If success_action_redirect is not specified, HTTP status code 204 is returned after the object is uploaded. 
                  ["eq", "$success_action_status", "200"],
                  # The value of the form field must start with the specified prefix. For example, if the value of the key form field starts with user/user1, the condition is ["starts-with", "$key", "user/user1"]. 
                  ["starts-with", "$key", upload_dir],
                  # Specify the minimum and maximum sizes of the object that can be uploaded. Unit: bytes. 
                  ["content-length-range", 1, 1000000],
                  # Set the type of the object that you want to upload to the specified image type.
                  ["in", "$content-type", ["image/jpg", "image/png"]]
              ]
          }
          signature = generate_signature(access_key_secret, policy.get('expiration'), policy.get('conditions'))
          response = {
              'policy': base64.b64encode(json.dumps(policy).encode('utf-8')).decode(),
              'ossAccessKeyId': access_key_id,
              'signature': signature,
              'host': host,
              'dir': upload_dir
              # Specify additional parameters.
          }
          return json.dumps(response)
  2. The client uses information, such as the signature and POST policy, to call the PostObject operation to upload the object to OSS by using an HTML form.

    JavaScript is used in the following example:

    <!DOCTYPE html>
    <html lang="zh-CN">
    <head>
      <meta charset="UTF-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title> Upload an object to OSS</title>
    </head>
    <body>
      <div class="container">
        <form>
          <div class="mb-3">
            <label for="file" class="form-label">Select the object</label>
            <input type="file" class="form-control" id="file" name="file" required>
          </div>
          <button type="submit" class="btn btn-primary">Upload</button>
        </form>
      </div>
      <script type="text/javascript">
          const form = document.querySelector('form');
          const fileInput = document.querySelector('#file');
          form.addEventListener('submit', (event) => {
            event.preventDefault();
            let file = fileInput.files[0];
            let filename = fileInput.files[0].name;
            fetch('/get_post_signature_for_oss_upload', { method: 'GET' })
              .then(response => response.json())
              .then(data => {
                const formData = new FormData();
                formData.append('name',filename);
                formData.append('policy', data.policy);
                formData.append('OSSAccessKeyId', data.ossAccessKeyId);
                formData.append('success_action_status', '200');
                formData.append('signature', data.signature);
                formData.append('key', data.dir + filename);
                // file must be the last form field. No particular order is required for other form fields. 
                formData.append('file', file);
                fetch(data.host, { method: 'POST', body: formData},).then((res) => {
                  console.log(res);
                  alert ('Object Uploaded');
                });
              })
              .catch(error => {
                console.log('Error occurred while getting OSS upload parameters:', error);
              });
          });
      </script>
    </body>
    </html>
    

Usage notes

Data security

Methods to prevent existing objects from being overwritten by uploaded objects that have the same names

By default, OSS overwrites existing objects with the uploaded objects that have the same names. You can use the following methods to prevent the existing objects from being unexpectedly overwritten:

  • Enable versioning for the bucket

    If you enable versioning for a bucket, objects that are overwritten in the bucket are saved as previous versions. You can recover the previous versions of the objects at any time. For more information, see Overview.

  • Include the x-oss-forbid-overwrite parameter in the upload request

    Add the x-oss-forbid-overwrite parameter to the header of the upload request and set the parameter to true. If you upload an object that has the same name as an existing object in OSS, the object fails to be uploaded and the FileAlreadyExists error code is returned. If you do not add this parameter to the request header or if you set this parameter to false, the uploaded object that has the same name as the existing object overwrites the existing object.

Authorized upload

  • OSS allows you to configure access control lists (ACLs) for buckets and objects. This way, third-party users who are not granted the required permissions cannot upload data to your bucket. For more information, see Overview.

  • You can use a signed URL to grant a third-party user the permissions to upload a specific object. This way, the third-party user can upload data without access credentials or authorization. Then, OSS stores the uploaded data as an object in a specific bucket. For more information, see Upload local files with signed URLs.

Lower PUT request fees

If you want to upload a large number of objects and set the storage classes of the objects to Deep Cold Archive, you are charged high PUT request fees. We recommend that you set the storage classes of the objects to Standard when you upload the objects, and configure lifecycle rules to convert the storage classes of the Standard objects to Deep Cold Archive. This reduces PUT request fees.

Object upload to a bucket for which OSS-HDFS is enabled

To maintain OSS-HDFS stability and prevent data loss, do not upload an object to the .dlsdata/ directory of a bucket for which OSS-HDFS is enabled by using methods that are not supported by OSS-HDFS.

Performance tuning of object upload

If you upload a large number of objects and the names of the objects contain sequential prefixes, such as timestamps and letters, multiple object indexes may be stored in a single partition. In this case, if you send a large number of requests to query these objects, latency may increase. We recommend that you use random prefixes instead of sequential prefixes to specify object names when you upload a large number of objects. For more information, see OSS performance and scalability best practices.