When containers began out, they had been meant to be ephemeral – stateless, disposable and data-light. However that’s all modified. As Gartner notes, use circumstances for containers have advanced to incorporate analytics and synthetic intelligence (AI) processing, and by 2028, it predicts 15% of on-premise manufacturing workloads will run in containers. That’s a 300% enhance since 2022.
Now, whereas containers themselves retain all the advantages of ephemerality – quickly reproducing, then dying again simply as shortly to account for workload spikes – the storage connected to them can’t reside by the identical guidelines.
As enterprises transfer from proofs of idea to operating an enormous chunk of manufacturing workloads in containers, the storage layer has grow to be a pivot level. Whereas the early days had been targeted on easy net scaling, containers have now moved into the realm of mission-critical databases, huge information science pipelines, and the power-hungry world of generative AI (GenAI).
The problem lies in navigating key selections resembling file versus block versus object storage, CSI versus container-native storage, and whether or not to go for a devoted container storage platform.
Containerisation is light-weight virtualisation
Containerisation is a light-weight type of virtualisation. Not like conventional digital machines (VMs) that require a hypervisor and a full visitor working system (OS), containers share the host server’s OS. This makes them lighter, quicker to scale and extra transportable. They’re constructed on microservices ideas that break monolithic purposes into discrete, software programming interface (API)-linked parts in a means that aligns with DevOps methodologies.
Whereas a number of orchestrators exist (for instance, Docker Swarm and OpenShift), Kubernetes is the market chief. It manages the cluster of nodes, which is the place pods run the containers. Clusters are teams of nodes managed by a management aircraft, which is the place we discover the API server, a scheduler for pod placement, a controller to keep up the specified state, and etcd for storage configuration.
As initially conceived, container storage was ephemeral, and information vanished when a pod was deleted. So, to help enterprise purposes, Kubernetes developed persistent volumes (PV), that are connected to a cluster and decouple storage from compute to permit purposes to stay transportable whereas sustaining entry to information.
CSI vs container-native storage
Container Storage Interface (CSI) is a typical that enables storage suppliers – greater than 130 drivers can be found – to show their techniques to Kubernetes. CSI permits Kubernetes to set off superior information providers resembling snapshots, cloning and automatic provisioning throughout block, file and object storage in on-premise and cloud environments.
Container-native storage doubtlessly has the benefit of portability – on-premise, within the cloud, and so forth, by advantage of the virtualisation inherent – whereas CSI is extra more likely to tie a deployment to deployed storage arrays
CSI is basically a “dealer”. It’s an industry-standard API that acts as a intermediary, permitting Kubernetes to speak to exterior storage arrays. For instance, when a developer requests storage by way of a persistent quantity declare (PVC), the CSI driver tells the exterior storage field to carve out a chunk of capability and plug it into the container. The benefit is that you simply get to make use of the costly, dependable enterprise storage you already personal, however the storage remains to be “outdoors” the cluster, and should you transfer containers to a special cloud or datacentre, that exterior {hardware} may not be there.
In the meantime, container-native storage is storage that lives contained in the Kubernetes cluster. It’s often deployed as a set of containers itself. It takes specified drives connected to Kubernetes nodes and swimming pools them collectively into one large digital useful resource.
Container-native storage doubtlessly has the benefit of portability – on-premise, within the cloud, and so forth, by advantage of the virtualisation inherent – whereas CSI is extra more likely to tie a deployment to deployed storage arrays.
Container-native storage is location impartial, so you’ll be able to run the identical setup on-premise or within the cloud. However it could eat central processing unit (CPU) and random entry reminiscence (RAM) out of your Kubernetes nodes to handle the info, which can be a priority.
Do we want containers to be that transportable?
CSI affords connection to big-iron absolutely featured storage, and container-native storage holds the promise of versatile deployment, portability, and so forth. However is portability that necessary? Eric Phenix, who leads the engineering observe at analysts GigaOm, says not.
“Containers provide a compute abstraction layer that enables the applying to be infrastructure agnostic, reasonably than an answer that’s designed to make purposes extra transportable,” he says.
Phenix argues that whereas containers make the code agnostic, deployment is one other matter. “Until an organization is particularly a customer-facing instanced PaaS [platform as a service] the place they should run on each cloud, I don’t see the necessity to run the identical workload on a number of clouds. As soon as issues are deployed, they’re at all times messy emigrate,” he says.
And this “messiness” is sort of at all times an information downside, in response to Phenix. Whereas the container picture can transfer in seconds, the multi-terabyte persistent quantity connected to it can’t.
James Brown, an analyst at GigaOm, factors out that container-native storage is basically software-defined storage and brings its personal lock-ins. “Closely built-in, container-native provider platforms threat changing {hardware} lock-in with software program lock-in. Tying your structure to proprietary in-cluster storage options creates huge migration hurdles, successfully breaking the core portability promise of Kubernetes,” he says.
So, the selection right here comes down to only how transportable you want issues to be. Enterprises usually use a hybrid method: CSI to hook up with huge, high-performance arrays for his or her heaviest databases; container-native storage for contemporary, distributed apps that want to have the ability to transfer and not using a “messy” information migration.
In 2026, selecting the proper storage protocol for container storage is all about enjoying in a “blended economic system”, with a Kubernetes cluster in a position to pull from all three codecs concurrently.
Block for prime efficiency
Block storage presents information as a uncooked, unformatted quantity – like a bodily onerous drive – that’s connected to a single node at a time. In Kubernetes, that is sometimes dealt with by way of persistent volumes utilizing the ReadWriteOnce (RWO) entry mode.
Block storage may be in on-premise arrays or within the cloud, resembling in Amazon Elastic Block Retailer (EBS), Google Persistent Disk, or Microsoft Azure Disk.
Block storage affords the bottom latency and highest enter/output operations per second (IOPS) as a result of there is no such thing as a filesystem overhead between the applying and the storage. That makes it ideally suited for databases the place small, frequent updates occur at particular areas inside information.
In relation to the cons, most block storage can’t be mounted to a number of pods throughout completely different nodes concurrently, and scaling often requires resizing the amount and increasing the filesystem. Block storage is mostly the costliest, too.
File for listing entry
File storage supplies a shared hierarchical namespace (folders and information) accessible over a community. In Kubernetes, it’s the main option to obtain ReadWriteMany, permitting a number of pods on completely different nodes to learn and write to the identical information.
Additionally it is out there in on-premise storage or cloud providers resembling Amazon Elastic File System (EFS), Microsoft Azure Recordsdata and Google Filestore.
File entry is completely fitted to horizontal scaling of net servers the place all pods want entry to the identical belongings, and most legacy purposes are constructed to learn/write to a typical listing construction. In comparison with block entry, community protocols like NFS or SMB introduce extra latency, and at giant scales (tens of millions of information), traversing deep listing bushes can grow to be extraordinarily sluggish. In the meantime, dealing with concurrent writes throughout many pods can result in file locking conflicts if not managed fastidiously.
Object for sizeable datastores
Object storage manages information as discrete objects in a flat namespace and is accessed by way of APIs (for instance, S3 or Swift) reasonably than being “mounted” like a disk. It’s the cloud-native storage protocol, although it could run on-site, too. Examples embody Amazon Easy Storage Service (S3), MinIO, Google Cloud Storage and Ceph RGW. Object storage can retailer petabytes of information with out worrying about partition limits or disk sizes, and is often the most cost effective possibility for large-scale unstructured information (logs, photographs, backups).
Object storage is good for contemporary “cloud-native” apps that speak on to storage by way of HTTP/HTTPS, bypassing the OS kernel fully.
On the detrimental facet, object storage is mostly the slowest for transactional work with excessive throughput however greater latency than block or file. In the meantime, you’ll be able to’t “edit” a single line in a file; it’s essential to re-upload the complete object to alter it.
Storage protocol decision-making
In abstract, block storage is dear however the most effective performing, file storage is more cost effective however with scale restrictions, and object storage is nice for big capability but additionally lags in efficiency phrases. So, which one to decide on? It’s a case of horses for programs, in response to Tony Lock, director of engagement and distinguished analyst at Freeform Dynamics.
“In a super world, the selection of underlying storage – block, file or object – will probably rely on what the app is, the place the organisation needs to run it, and what its traits are when it comes to dimension, variety of containers, latency necessities, safety, location, value, and so on,” he says.
In the meantime, Whit Walters, subject chief know-how officer at GigaOm, believes S3 is profitable the battle, however block has its place. He says: “The actual story is protocol bifurcation inside AI pipelines. Object storage dominates the ingestion and information lake tier, providing exabyte-scale horizontal scaling with wealthy, customisable metadata that allows semantic discovery natively on the storage layer.
“Block storage nonetheless owns the inference scorching path the place vector databases demand 500,000+ IOPS, nonetheless.
“The rising development to observe is COSI, the Container Object Storage Interface, which goals to make object storage buckets first-class Kubernetes sources with standardised, declarative lifecycle administration.”
CSI vs container-native in storage provider container platform
All the massive storage suppliers present some type of platform or wrapper for container storage. These embody Dell’s Container Storage Modules, HPE’s Ezmeral Runtime Enterprise, the Hitachi Kubernetes Service (HKS), NetApp’s Astra and Everpure’s Portworx.
What all of them have in frequent is a method of managing container storage – and in some circumstances, information safety and extra. The place they differ below the hood is that the majority are primarily based round CSI, so they supply a layer from which to handle CSI drivers to their storage.
CSI connectivity could be higher suited to bigger, extra static environments, whereas container-native options may be greatest for extra dynamic units of workloads
Some differ in that they supply their administration performance from inside Kubernetes. Everpure’s Portworx, for instance, lives fully inside Kubernetes however makes use of CSI as a “handshake” with exterior storage.
In the meantime, HPE Ezmeral additionally runs in Kubernetes however accesses information by way of the CSI driver. NetApp’s Astra Datastore was container-native in an analogous option to Portworx, however was discontinued in 2023.
Whereas all the important thing storage suppliers provide merchandise that may handle storage for containers, you’ll want to examine the extent to which these are container-native or depending on CSI. As talked about, CSI connectivity could be higher suited to bigger, extra static environments, whereas container-native options may be greatest for extra dynamic units of workloads.
GigaOm’s Walters places a finer level on it: “The Kubernetes tax is actual, however it’s a trade-off. Container-native platforms run replication, dedupe and encryption on employee nodes. Ceph alone carries a 2-10% baseline CPU penalty per node only for cluster quorum, and that spikes onerous throughout reproduction rebuilds.
“In GPU [graphics processing unit]-dense AI environments, the place each cycle counts, offloading that work to devoted array ASICs [application-specific integrated circuits] by way of a complicated CSI mannequin retains compute nodes clear. However in multicloud or edge eventualities with out devoted arrays, that CPU tax buys you topology-aware placement and self-healing automation that’s genuinely onerous to copy in any other case.”
There might also be efficiency concerns when it comes to rivalry for sources, in addition to questions on how they’re administered.
In the direction of autonomous, agentic storage
As we glance in the direction of 2027, the main focus is shifting from handbook provisioning to policy-driven storage.
The last word purpose is a system the place the storage “senses” workload necessities. For instance, if an AI coaching container spins up, the system mechanically provisions high-throughput file storage, or if a database scales up, it will get low-latency block storage.
