As we work to build more sustainable IT, we need to revisit some of our assumptions, including data compression. The reality is every organization’s path is different and personal.
Myth #1 –There’s just one path to data compression
Enterprises often believe there is one path. They may think that it is done exclusively in software on the host CPU. Because the CPU does the processing, there is the risk of a performance penalty under load, making it a non-starter for critical performance workloads.
In the same way, the data pipeline within your organization is unique and tailored to your requirements, and architecting how data flows offers plenty of options. Data compression can be done in many ways, and the outcomes of choosing how and where compression should be processed can lead to benefits that cascade throughout the architecture. For example, there is untapped potential in Flash technology that is currently bottlenecked by the effects of write-amplification. By performing compression directly on the SSD in hardware, latency consistency, capacity, and endurance can all be multiplied several times while offloading CPUs and GPUs.
Mythbusting strategy: Consider data compression early and evaluate all the workloads that could benefit from it from both performance and lifespan perspectives. There is an approach that can work for your unique needs.
Myth #2 – Data compression can’t solve business problems
How can you improve the overall cost of ownership of your infrastructure? How can you increase storage and performance while decreasing power consumption? How can you make the data center more sustainable? When organizations try to solve these sorts of problems, it may not immediately leap to mind as the answer.
Data compression doesn’t get more attention because organizations simply aren’t thinking about it as a problem-solving tool. This becomes clear when you look at search trends related to data and see that “enterprise data compression” is orders of magnitude lower down the results than something like “data management.”
This is unfortunate because it is an easy, fundamental technology that can significantly address these issues if done in a modern way. Nearly all data is somewhat compressible; even a small compression ratio can significantly alleviate business burdens.
Mythbusting strategy: It can solve many business problems when done right. Compression is best done in hardware to maximize the benefits, close to where data resides. Build data considerations early into the design of your data pipeline.
Myth #3 – Compression results in performance penalties
Contrary to long-standing wisdom, compression avoids performance penalties when done in hardware. It acts as an “accelerator” for applications by freeing up bottlenecks in the overall systems. Data compression can optimize flash by compressing data during writes and decompressing it on reads without any host action.
Mythbusting strategy: Be thoughtful about the technology and how to apply it. Look beyond capacity to see how compression can extend the life of your hardware and reduce your power consumption.