How Tachyon Works
Tachyon's high speed data transfer platform enables data to be shared between Internet-connected hosts, such as browsers, phones, servers and IoT (Internet of Things) devices faster than is typically possible using traditional file sharing methods. The Tachyon platform provides developers with enterprise grade transfer and synchronization by delivering a highly-available service, bandwidth utilization guarantees, and data availability at the edge.
At its core, Tachyon is a cloud-based distributed file system that ensures any device uploading data to Tachyon will be available for download by any other devices as quickly as possible. But it is more than that. Tachyon makes it possible for developers to build apps and infrastructure that can share data as quickly as possible, without the worries of managing scale, latency, data durability, integrity and storage, connection recovery, host interoperability, network outages, encryption, security and authentication, throttling, and denial of service attacks, to name a few.
Tachyon is built as a distributed file system, with Coordination nodes and Data nodes.
Coordination nodes handle user authentication, command parsing, replication across data nodes, data indexing, and broker connections between Tachyon hosts and Data nodes. The coordination nodes are also responsible for relaying information about Filesystems to the Administration Dashboard, where users can see active uploads and downloads, upload and download data through a web-based interface, and browse the existing data in a Filesystem.
Data nodes primarily store data, and are designed to work on commodity hardware across high bandwidth wide area networks. Data nodes are deployed on an edge network, and Tachyon utilizes latency based routing to determine the closest data node to a host. Tachyon aims to have a latency of at most 20ms between a host and the closest data node. Data transfer between hosts and data nodes, and between data nodes themselves, happen over an open source application level UDP protocol called UDT.
A Unified Virtual Filesystem
The Tachyon platform organizes the data within any application into named Filesystems. Hosts connected to Tachyon have a full range of Unix-like functionality available to them to perform on the Filesystem, including uploading, downloading, making directories, listing files, and removing files and folders. Data is always saved to a Filesystem, and multiple Filesystems can be used to organize data across different projects, applications, customers, or users.
Filesystems are uniquely identified by a UUID string called a Filesystem ID, coupled with a 20 character API key, that allows operations on the filesystem. Hosts are completely decoupled, and can perform operations on a Filesystem simultaneously, subject to a write lock on files and folders.
Storage-Agnostic Caching (Coming soon!)
The Tachyon platform also supports the concept of Storage, which acts as a connector between any number third party storage services and the Tachyon Filesystem. By leveraging Storage, hosts can make multiple storage services available in a unified namespace through a single Filesystem object.
Realtime Data Sharing with Streaming and Syncing (Coming soon!)
Through the use of Stream and Sync, Tachyon also supports the ability to share data over wide area networks in realtime. Sync allows any number of hosts to synchronize changes made in their local storage medium with other hosts using file watching semantics, while Stream allows hosts to stream data to one or more other hosts using a many-to-many pub-sub model, bypassing storage mediums for better performance.