Today, April 16th, 2016, Quobyte announced the release of a TensorFlow Filesystem Plug-in that allows TensorFlow applications to integrate directly with Quobyte without passing through the operating system kernel. Quobyte was founded in 2013 and provides software-defined storage for a variety of customers including Yahoo! Japan. TensorFlow is a widely used open-source software library commonly used in machine learning applications.
Today, April 16th, 2016, Quobyte announced the release of a TensorFlow Filesystem Plug-in that allows TensorFlow applications to integrate directly with Quobyte without passing through the operating system kernel. Quobyte was founded in 2013 and provides software-defined storage for a variety of customers including Yahoo! Japan. TensorFlow is a widely used open-source software library commonly used in machine learning applications.
Quobyte’s new TensorFlow Filesystem Plug-In runs on Linux systems. Quobyte hopes that the Plug-In will allow older legacy Linux servers to use their technology in machine learning applications. They are claiming that their storage system can be used in all stages of machine learning and provides throughputs of up to 10GB/s to each client. The plug-in integrates with Quobyte to help fully leverage all provisioned system resources by optimizing GPU usage to reduce latency and CPU loading, and Quobyte is claiming it provides up to a 30% boost in throughput for TensorFlow users.
Additional benefits of Quobyte’s TensorFlow File System Plug-in include:
- The ability to leverage HDD and SSD to get the best price-performance ratio without cumbersome tiering
- Prefetching of training data can deliver substantial performance improvement. Much machine-generated data uses a sequential naming convention that makes it ideal for prefetching.
- Infinite scalability that allows users to grow storage in terms of throughput and capacity when they need it. As ML project requirements change – oftentimes more quickly than anticipated – the Quobyte installation will adapt. Disks or servers can be quickly and easily added when needed to provide more capacity or performance without any interruption to applications or services.
- Multi-tenancy that provides additional security by allowing users to define isolated namespaces and physical separation of data/workloads inside the same cluster. Administrators can further isolate tenants by controlling to which physical hardware they have access in order to ensure performance and that data is not accessible to any unauthorized users on the network.
Availability
Immediately
Sign up for the StorageReview newsletter
