Kafka Performance Gains With CSD Transparent Compression
Kafka Use Case Overview
The purpose of the test outlined in this paper was to conduct a comparison of the performance and latency of the ScaleFlux CSD 2000, with transparent compression, in an Apache Kafka environment vs. a competing and industry-leading NVMe TLC drive withhost-based LZ4 compression enabled.
Compared to standard NVMe SSDs, transparent compression is a unique feature the ScaleFlux CSD 2000 uses to enable cost savings and performance benefits. The CSD 2000 uses In-Drive Data Path Compression to increase usable capacities and improve I/O performance without burdening the CPU with compression. This compression function is integrated directly into the controller on the CSD 2000.