×
Community Blog What is Object Storage? Object vs File vs Block

What is Object Storage? Object vs File vs Block

Object storage is a computer data storage architecture that manages data as objects. This is in contrast to other storage architectures (such as file systems) and data as blocks in sectors and tracks.

Alibaba Cloud Object Storage Service (OSS) is an encrypted storage service that allows you to safely save, back up, and archive any volume of data in the cloud. OSS is cost-effective, highly secure while being a very reliable solution for any use. It's possible to interact with it using a restful API.

In this article, we'll show how to do some basic OSS operations into a spring-boot application using an SDK interface provided by Alibaba Cloud wrapped on top of a spring dependency.

Concepts

Let's grab some important concepts around the OSS service:

  1. Storage Class: OSS provides three storage classes: Standard, Infrequent Access, and Archive. These storage classes cover various data storage scenarios from hot data to cold data. For more information, see Introduction to storage classes.
  2. Bucket: A bucket is a container for objects stored in OSS. Every object is contained in a bucket.
  3. Objects: which are files, are the fundamental entities stored in OSS. An object is defined by some metadata, data and key. The key is a unique object field present in a bucket.
  4. Region: represents the physical location of an OSS data center. You can choose the region that has the least latency while being affordable for you.
  5. Endpoint: it's nothing more than the domain name used to access the OSS. OSS provides external services through HTTP RESTful APIs. Different regions use different endpoints. Let's note that each region has its endpoint.
  6. AccessKey: An AccessKey (AK) is composed of an AccessKeyId and an AccessKeySecret.

More details of OSS can be found here.

Dependencies

The following Maven dependency is used to make the whole logic possible, it's the spring wrapper for the SDK interface:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-alicloud-oss</artifactId>
    <version>0.2.1.RELEASE</version>
</dependency>

The most recent version at the time of writing this article is the 0.2.1.RELEASE accessible on Maven Central Repository.

You can refer to Spring Boot with Alibaba Cloud Object Storage Service and learn the following steps.

Related Blogs

How to Automate Backups with Alibaba Cloud Object Storage Service

We can automatically upload backups to Alibaba Cloud OSS with Minio Client and cronjob. Alibaba Cloud OSS is a suitable backup storage solution.

Offsite backups are an important security measure. They allow restoring data in case of hardware failure, accidental deletion, or any other catastrophic event. Automating backups improves the reliability of the backup process and ensures recent data gets backed up regularly.

We will use the Alibaba Cloud Object Storage Service (OSS) as an offsite backup storage solution. Data will be backed up by a simple bash script which gets executed regularly by a cronjob. Minio Client will be used to transfer the backups to OSS.

Alibaba Cloud OSS is a suitable backup storage solution. Rarely accessed objects such as backups can be stored reliably, cheaply and securely. The first 5 GB of storage can be used completely free of charge.

Prerequisites

Make sure you have an Alibaba Cloud account before you start. Sign up now to receive $ 300 in free credit. Create an ECS server or connect to an existing one with SSH to follow along. All you need is a Linux-based system.

Step 0 – Add a New User (Optional)

It is a good idea to use an account with limited system access for the execution of the backup script. Doing so increases security because an attacker will have limited possibilities if the user which executes the backup script gets compromised. Run the following command to add a new user with the name backup.

jsudo adduser backup

Limit the new user's system rights as much as possible without restricting it's ability to upload backups to OSS.

Step 1 – Create OSS Bucket

Next, we will create an OSS bucket for the backups. A bucket is a namespace for a collection of objects. Each object is uniquely identified by the bucket name and the object key.

First, navigate to the Alibaba Cloud Console. Open the Products dropdown from the main navigation bar and click on the Object Storage Service link in the Storage & CDN section. You will be asked to accept the OSS Terms of Service if you have not used OSS before. Click on Create Bucket to create a bucket for the backup objects.

Pangu – The High Performance Distributed File System by Alibaba Cloud

In this article, we discuss the core features and advantages of Pangu, the distributed file system developed by Alibaba Cloud, and how it proves to be the optimum storage system for cloud storage.

Pangu is a high-reliability, high-availability, and high-performance distributed file system developed by Alibaba Cloud. It has a history of nearly ten years. As a unified storage core of Alibaba Cloud, Pangu 1.0 stably and efficiently supported the rapid development of multiple business lines of Alibaba Cloud, including ECS, NAS, OSS, Table Store, MaxCompute, and AnalyticDB. In recent years, Alibaba Cloud redesigned and implemented the second-generation storage engine Pangu 2.0. It provides better storage services not only for Alibaba Cloud but also for businesses of Alibaba Group and Ant Financial. There are two reasons for the upgrade: hardware improvements and demands from the business.

Rapid Development of Underlying Hardware

In recent years, the performance of distributed storage-related hardware has dramatically improved.

The storage media has evolved from HDD to SATA SSD and NVMe SSD, and there is an improvement of two orders of magnitude in IOPS. Similarly, there is a reduction in latency by two orders of magnitude. The 4-KB sequential write performance has got reduced to 7 µs, and the Optane memory at the memory interface is about to enter the era of nanoseconds. Moreover, the network has rapidly developed from GE and 10GE to 25GE, 40GE, 50GE, or even 100GE. A hop takes as short as five µs.

The rapid development of the underlying hardware brings both opportunities and severe challenges to Pangu. To keep the leading capability in the industry, Pangu must make full use of the hardware.

Pressure from the Upper-Layer Business

Currently, an increasing number of businesses are getting connected to Pangu. However, as enterprises have different requirements for storage, adequately supporting businesses has become an enormous pressure for Pangu.

Firstly, separation of storage and computing is Alibaba Cloud's strategy, which entails a storage cluster that provides the storage service for multiple computing clusters. It poses higher performance requirements for storage clusters. Serving the most computing clusters with the least storage clusters is the goal of Pangu in the future.

Secondly, after separation of storage and computing, it is expected that computing clusters use the same storage cluster namespace if possible so that the internal architecture and control of the storage cluster are transparent to users. In Pangu 1.0, data nodes have already supported auto-scaling. However, metadata nodes still rely on particular models with high configuration, which restricts the node size and number of files supported by a file system. Also, the centralized metadata servers become the most vulnerable component of the system. Faults of some metadata servers may cause the failure of the entire storage cluster.

Thirdly, different businesses require different types of access modes. Compatibility with sophisticated open interfaces is necessary for the expansion of the access range. The demand for supporting HDFS access rises, and many application vendors have posed for supporting POSIX access.

Finally, the market competition is increasingly fierce. As a unified storage platform of Alibaba Group, Pangu must use technologies to minimize the cost and win the initiative for the business.

To meet these absolute and high requirements, Pangu must set up new objectives in dimensions such as the architecture design and engineering practice to achieve breakthroughs.

Apsara Block Storage A Storage Solution for Enterprises

Apsara Block Storage is an elastic, reliable, and distributed block storage service tailored for leading enterprises in the market.

Developed by the elastic computing block storage team of Alibaba Cloud, Apsara Block Storage is a distributed block storage product tailored for leading enterprises in the market. It is an elastic and reliable cloud storage service platform that provides high performance and supports large-scale horizontal scale-out. Apsara Block Storage can be connected to various types of computing platforms, including Alibaba Cloud ECS, ZStack, OpenStack, Kubernetes, and physical servers, so it can satisfy customers' demand for agile access to massive storage resources in the cloud era. It is applicable to cloud storage business scenarios for offline data centers of government agencies, financial institutions, operators, and large manufacturing enterprises. Apsara Block Storage aims to compete with distributed storage products of mainstream players in the traditional enterprise storage market, such as EMC and Huawei. Alibaba Cloud will officially release Apsara Block Storage in the near future, marking Alibaba Cloud's foray into the global enterprise storage market. Alibaba Cloud will become one of the few service providers in the world that are capable of providing enterprise-oriented professional storage services for both public and private cloud markets.

Why Does Alibaba Cloud Provide Block Storage Services Independently?

Customer Pain Points and Industry Trends

With a wider and deeper application of cloud computing, big data, mobile Internet, and artificial intelligence (AI) in all walks of life, digital transformation has become an important driving force for customers' business innovation and agile development. Traditional centralized and siloed IT system architecture can no longer meet the increasingly complex business development requirements, especially when it comes to flexible use of storage resource.

Related Courses

Operate and Manage Object Storage on the Cloud

This course explains the basic concepts of OSS. It also includes demos such as bucket management, object storage operations, image processing, website hosting and monitoring and introduces the OSS related security features.

Introduction to Network Attached Storage

The File Storage in the Cloud course is designed to provide you with the fundamental knowledge to plan, configure and administer Network Attached Storage (NAS) in Enterprise enviroment. In this course we will discuss deployment best practices and use cases of Network Attached Storage (NAS).

Related Documentation

Get started with Object Storage Service

Alibaba Cloud Object Storage Service (OSS) provides you with network-based data access services. OSS allows you to store and retrieve a variety of unstructured data files, including text files, images, audio files, and video files, over the network at any time.

Before you use OSS, we recommend that you familiarize with OSS usage limits. For more information, see Limits.

OSS uploads data files as objects to a bucket. You can perform the following operations in OSS:

  1. Create one or multiple buckets and upload one or multiple objects to each bucket.
  2. Share or download an object by using its URL assigned by OSS.
  3. Configure the ACL for a bucket or object by modifying its attributes or metadata.
  4. Perform basic and advanced OSS operations in the OSS console or by using various convenient tools and SDKs.

Apsara File Storage NAS Capacity - NAS

This topic describes the features, specifications, and scenarios of NAS Capacity. You can select this type of NAS file system based on your business requirements.

NAS Capacity uses SATA hard disk drives (SATA HDDs) as the storage devices, and provides high-performance storage space at low costs. NAS Capacity is a file sharing solution. It is applicable to cost-efficient scenarios that require high throughput and scalability. You can adopt this cost-efficient solution if you do not need to perform frequent read/write operations and do not have high requirements for response latency.

What is block storage? - Elastic Compute Service

Block Storage is a high-performance, low-latency block storage service for Alibaba Cloud ECS. It supports random read and write operations. A Block Storage device is similar to a physical disk. You can format a Block Storage device and create a file system on it to meet the data storage requirements of your business.

Block Storage devices

Alibaba Cloud provides a variety of Block Storage devices for ECS instances, such as cloud disks based on a distributed storage architecture, and local disks located on the physical machines where the ECS instances are hosted.

Related Market Products

Operate and Manage Object Storage on the Cloud

Quickly learn the concepts and features of OSS and how to use OSS console to perform common operations.

Distributed Storage Introduction

Through this course, you will learn the concepts of distributed storage, the types of common distributed file systems, and the high availability and scalability features.

Related Products

Object Storage Service

An encrypted and secure cloud storage service that can store, process, and access massive amounts of data from anywhere in the world

Hybrid Cloud Storage Array

Hybrid Cloud Storage Array (HCSA) is an enterprise-level storage array that integrates with the Alibaba Cloud storage gateway. You can use and manage cloud storage resources (blocks, files, and objects) just as you would with local storage. Cloud Cache, Cloud Sync, Cloud Tiering, and Cloud Backup all make local storage migrate to the cloud seamlessly.

0 0 0
Share on

Alibaba Clouder

2,097 posts | 487 followers

You may also like

Comments