User:GoToBedEarly/sandbox

Alluxio is an open source virtual distributed file system. Alluxio (initially named Tachyon) was developed in a doctoral thesis at the University of California, Berkeley AMPLab with grant funding from DARPA. Alluxio is an in-memory data layer between applications and data storage systems. The software is published under the Apache License.

Overview
The motivation for creating Alluxio stemmed from other research projects at AMPLab, notably Apache Mesos and Apache Spark which focused on the compute and data layers, respectively. Haoyuan Li, then a Ph.D. student working on distributed systems, identified the need for innovation at the data layer. Li developed the first version of Alluxio with the goal of creating technology that simplifies the way application frameworks connect to diaparate and heterogeneous storage systems. Alluxio is now used commercially in cloud-based big data environments for applications such as analytics processing and machine learning. Common use cases include:


 * Improving application performance by caching frequently accessed data or caching data locally from remote sources
 * Unifying data from multiple storage systems and/or locations
 * Providing shared access to a single data set for multiple application frameworks
 * Simplifying data access in hybrid cloud environments

History

 * In 2012 Haoyuan Li (also known as “HY”) was a Ph.D. student focused on distributed systems and the University of California, Berkeley AMPLab when he developed the first version of Alluxio (then known as the Tachyon project). Tachyon was incorporated into the Berkeley Data Analytics Stack
 * In 2013 Alluxio was initially released under the Apache open source license. The current source code repository can be found on GitHub.
 * In November 2014 the first academic paper on Alluxio, “Tachyon: Reliable Memory Speed Storage for Cluster Computing Frameworks” was published at SOCC
 * In March 2015 Tachyon Nexus (later renamed Alluxio, Inc.) was founded to provide ongoing development and commercial support for Alluxio.
 * In February 2016 Tachyon was renamed Alluxio and open source version 1.0 was released.
 * In July 2018 version 1.8 was released
 * Key Alluxio project members and contributors:
 * Haoyuan (“HY”) Li, creator
 * Founding members: Bin Fan, Yupeng Fu, Calvin Jia, Gene Pang

Technology
Alluxio is a virtual distributed file system that creates a shared in-memory data layer between compute and storage. The software acts as an abstraction layer that presents a set of disparate data stores (file or object) as a single file system, providing standard APIs and consistent semantics for applications. The solution integrates three primary innovations:


 * Unified Namespace: also referred to as a global namespace, the Alluxio file system aggregates disparate data sources regardless of location. Data sources, stored in any file- or object-based file system, are virtualized and appear as a single namespace that can be mounted and accessed via the Alluxio file system.
 * API Translation: Alluxio converts from client-side interface to native storage interface via server-side API translation.
 * Intelligent Cache Management: Configuration settings and user-defined policies establish the framework for cache management (data fetching and replacement), resource utilization across media (DRAM, SSD, HDD), data placement for performance and reliability, and data consistency with persistent storage.

Editions
Alluxio Open Source (AOS) is available for free download with no restrictions on the number of nodes to deploy on or duration of use. Alluxio source code can be downloaded from GitHub. With over 800 contributors as of August 2018, Alluxio is one of the most popular open source big data projects in the world. Alluxio Community Edition (ACE) is a free edition based on Alluxio Open Source (AOS) and supports Alluxio Manager, a graphical user interface (GUI) for system management.

Alluxio Enterprise Edition (AEE) is a commercially supported edition available via subscription. AEE includes AOS plus additional enterprise features.