Advancements in NVMe (Non-Volatile Memory Express), the storage protocol designed for flash, are revolutionising data storage. According to G2M Research, the NVMe market will grow to $60 billion by 2021, with 70 percent of all-flash arrays being based on the protocol by 2020. NVMe, acting like steroids for flash-based storage infrastructures, dynamically and dramatically accelerates data delivery. The rising NVMe trend is good news for the rapidly evolving media & entertainment sector, where HDR technology has disrupted an industry pushing for the continuous development and improvement of motion picture viewing experiences. Video resolution is accordingly increasing at a rapid pace, along with the realisation of applications enabling real-time end-user experiences. Now more than ever, M&E (Media and Entertainment) workloads demand storage solutions capable of processing substantial amounts of data in an instant.

M&E industry storage challenges

M&E technology innovation is having a massive impact on video post-production storage infrastructure requirements, with the default 4K resolution video soon to be replaced by 8K, and with 16K on the horizon. This places enormous pressure on the shoulders of production and post-production teams which must produce, edit, finish, and deliver high-quality content, often utilising legacy systems unequipped to deal with the demand. When a studio upgrades from 2K resolution at 30 frames per second (fps) to 4K at 60 fps, it needs to be prepared to store and stream eight times as much data.

With this evolution of HDR technology, ushering in an age of more advanced cameras and enhanced visual effects, the industry is struggling to process and manage the increasing volumes of data and the bandwidth required to accommodate high frame rates. Despite the use of already fast flash storage, the broadcast industry is experiencing latency issues such as lagging ingest speeds to workstations from legacy Fibre Channel storage arrays. Insufficient performance is negatively impacting post-production workflows, as production teams struggle to contend with dropped frames.

Studios urgently need to engage future-fit storage capability with scale, capacity and performance, to enable production teams to efficiently and cost-effectively manage their increasing data sets. Could NVMe be the answer to M&E’s most intractable data storage challenges?

The origins of NVMe

NVMe is not, as it is often thought, a storage solution in itself. Rather, it is a high-speed communications standard or protocol, designed specifically for Non-Volatile Memory (NVM) such as flash or Solid State Drives (SSDs). It substantially boosts the performance of flash and SSD media, making it around 100 times faster than mechanical disk drives.

The migration to flash technology from traditional mechanical Hard Disk Drives (HDDs) to SSDs has resulted in impressive performance, capacity and reliability improvements (1 millisecond latencies compared to 20 milliseconds with traditional spinning media). However, it is now the traditional communication protocols (such as SAS and SATA) between the faster storage and the rest of the IT environment that restrict the full performance advantage of flash arrays.

An NVMe drive is simply a flash drive or SSD that uses the NVMe protocol in contrast to one of the much slower legacy protocols.

The promise of NVMe capability

NVMe’s power essentially lies in its ability to enhance the communication between the IT system and storage solutions. Traditional communication protocols used with HDDs are based on a single queue with 32 commands; however flash and SSDs have the capacity to process much more than this. NVMe was built specifically for use with flash and SSDs, and flaunts 64,000 queues and 64,000 commands per queue. Thus, NVMe provides a performance injection for already-fast flash, enabling instant connection across thousands of parallel command queues. The performance and hence productivity gains are undeniable.

The NVMe protocol was designed to enhance the low latency and high performance of NVM storage media. It reduces the former by making the commands much more efficient, smartly streamlining them so that the flash technology only needs to read the commands required for data storage and retrieval, thus, substantially increasing performance.

In terms of cost, utilising NVMe also results in lower power consumption and smaller data footprints, lowering the Total Cost of Ownership (TCO).

NVMe development and growth

The development of NVMe technology has been underway for over a decade, even before flash was fully embraced. As the price of flash storage devices continues to fall, adoption rates are rising: NVMe is the next logical step. It is poised to replace the less cost-effective and reliable existing interface standards such as SAS and SATA controller interfaces for disk used for transferring data between host systems and storage devices in datacentres today.

With its innate low latency capability perfect for handling M&E, supercomputing and enterprise workloads, NVMe technology has reached a level of maturity now placing it in a leading position. As we move closer towards an all-flash datacentre, new solutions are stepping in to exploit NVMe drives and protocols.

Innovative solutions leveraging NVMe

Cutting edge storage solutions offer software that can enable companies to maximise utilisation of NVMe flash drives that are 'pooled' across the network, providing editing workstations access to distributed high performance and low latency storage. This new approach provides centralised storage at high speeds, removing the need to copy or move data between workstations and arrays, preserving workflows.

For the M&E industry, solutions leveraging NVMe in this way could be the holy grail to future-proofing data storage. NVMe can empower organisations to scale, increase capacity, and achieve optimum data performance with workflow protection, enabling the production of high-quality content and increasingly impressive viewer experiences.