Storage Concepts
The S1 system is a general-purpose single storage engine that can manage storage resources for many different use cases. It combines cutting-edge storage technology with maximum flexibility, providing customized storage options at the logical volume level. S1 has its own storage architecture that may differ from other storage systems. The following provides an overview of S1 storage concepts.
Pools
Storage media in your StorONE system is grouped into drive pools. You create pools and then assign drives to those pools. Pools can only contain drives of the same media type (HDD, SSD, or NVMe). For example, if you have SSD drives and HDD drives, you would create one or more pools of SSD drives, and one or more pools of HDD drives, depending on your requirements.
Virtual Storage Containers (VSCs, also referred to as volumes) use drive pools as the underlying storage media. For example, you might create an all-flash VSC using a single SSD or NVMe pool, and a hybrid VSC using an SSD pool for the upper tier, and an HDD pool for the lower tier.
Applications
An application instance is simply a namespace for VSCs. All VSCs are mapped to one (and only one) application instance. However, an application instance may contain more than one VSC.
Virtual Storage Containers
A Virtual Storage Container (VSC, also referred to as a volume) is the basic unit of storage within the StorONE system. All data access is provided by the VSC. The first step in provisioning new storage is to create a new volume. You can then specify whether the volume will be used for file, block, or object storage.
When creating a new volume, you specify the drive pools that the volume will use for its underlying storage. This allows you to customize the VSC to your use case requirements. For example, for a virtualization use case, you might create an all-flash volume using a single SSD or NVMe pool for high performance with random I/O. For a backup use case, you might create a hybrid volume with a small SSD upper tier for fast ingest performance and a large HDD lower tier for lower cost capacity.
If a volume is low on space, you can increase the capacity of the volume. If the underlying drive pools are low on space, you can add drives to the pools and then increase the volume capacity.
You can also configure data resilience and performance characteristics at the VSC level using the erasure coding N and K values, where N is the number of drives used for the data fragments and K is the number of drives used for the parity fragments. This allows you to trade off performance and storage overhead depending on the use case for the volume.
Consistency groups
A consistency group is a basic logical storage entity that consists of one or more VSCs. Every VSC (also referred to as a volume) is mapped to a consistency group. When you create a volume, you can map it to an existing consistency group. If you do not specify an existing consistency group, the S1 system creates a new consistency group automatically.
By grouping volumes together into a consistency group, you can ensure that the data on all the volumes is consistent at a point in time. This is especially important in applications where multiple volumes may be accessed and modified simultaneously. For example, if a database and its transaction log are on different volumes, it is essential that the database volume and the log volume are consistent. Snapshots are performed on all volumes in a consistency group simultaneously, so that if you need to restore an application that uses multiple volumes, the restored volumes are in sync with each other.
Filesystems
A VSC (or volume) is simply a storage space, similar to a logical volume on your local machine. To use the storage space within these volumes, you must create a filesystem on them, just as you would need to partition and create a filesystem (such as NTFS, fat32, ext2, or ext4) on a physical disk in order to use it.
There are generally two options for formatting a filesystem on your volumes:
- Using the S1 system: This involves using either the command-line interface or a graphical-user interface to create a new filesystem.
- Mapping your volume: This option involves connecting a block volume to a remote machine via a storage-area network (SAN). This allows the remote machine to access the disk as if it were a local disk and format it in the same way it would format a local disk.
Storage options
The StorONE system provides three options for provisioning storage.
The following sections cover these in more detail.
Object storage architecture
Object storage stores data as individual units called objects. Each object contains all of the data associated with it and is assigned a unique identifier. The objects are stored in a flat address space, rather than in a hierarchical file system.
When to use object storage architecture
Object storage is a type of data storage architecture that is designed for storing and accessing large volumes of data, such as video, audio, documents, and other types of binary data. It is a good choice for storing data that does not need to be accessed frequently and does not require the low latency or high performance of a traditional block or file storage system.
Some common use cases for object storage include:
- Backup and archive: Object storage can be used to store large amounts of data that need to be retained for a long period of time, such as backups and archives.
- Media and entertainment: Object storage is often used to store and distribute media content, such as videos, music, and images.
- Scientific research: Object storage can be used to store and manage large volumes of data generated by scientific research projects.
- Cloud storage: Object storage is often used as the underlying storage layer for cloud-based applications and services.
Object storage is well-suited where data durability and scalability are important considerations.
File storage
File storage architecture stores files hierarchically, with folders and files arranged in a tree-like structure. Each file has a unique name and is stored in a specific location within the file system.
When to use file storage architecture
File storage is typically accessed over a network using a protocol such as Network File System (NFS) or Server Message Block (SMB).
Some common use cases for file storage include:
- Collaboration and sharing: File storage can be used to store and share files with team members or other users, allowing multiple people to access and edit the same files.
- File servers: File storage can be used to store and manage files on a centralized server, providing a single point of access for users.
- Network-attached storage (NAS): File storage can be used to store and access files on a device that is attached to a NAS.
- Home directories: Home directories are commonly used in organizations to provide each user with a designated space to store their personal files, making it easier for users to access and share their files with others. File storage can be used to store and manage personal files, such as documents, photos, and music. In general, file storage is a good choice for frequent access data that requires low latency. File storage is a suitable solution for scenarios that prioritize data organization and access control, particularly when multiple users require access to and editing capabilities for the same files.
Block storage architecture
Block storage architecture organizes data into fixed-size blocks, typically ranging in size from 512 bytes to 64 kilobytes.
Block storage is typically used to provision raw storage to a client that then formats and uses the storage as if it were a local block device.
One of the main benefits of block storage is its high performance, as data can be accessed quickly and directly. However, a block storage system (SAN) can be more expensive to implement and maintain compared to other storage architectures, and is particularly sensitive to network latency. Dedicated and isolated network equipment is strongly recommended when using block storage.
When to use block storage architecture
Block storage is a type of data storage architecture that is designed for storing and accessing data at the block level, rather than as files or objects. It is typically used to store data that requires fast, low-latency access, such as databases, virtual machines (VMs), and other types of transactional data.
Some common use cases for block storage include:
- Databases: Block storage is often used to store and access data for transactional databases, such as MySQL, Oracle, and Microsoft SQL Server.
- Virtual machines (VMs): Block storage is often used to store VM images, which are used to create and run virtual machines in a cloud or on-premise environment.
- Applications with high performance requirements: Block storage can be used to store data for applications that require fast, low-latency access to data, such as financial trading systems and online gaming platforms.
- Cloud storage: Block storage is often used as the underlying storage layer for cloud-based applications and services.
In general, block storage is a good choice for storing and accessing data that requires fast, low-latency access, and is well-suited for use cases where performance and scalability are more important considerations than cost.
No Comments