Seaweed FS
SeaweedFS is a simple and highly scalable distributed file system to store and serve billions of files fast! SeaweedFS object store has O(1) disk seek and SeaweedFS File...
What is Seaweed FS?
SeaweedFS is a simple and highly scalable distributed file system. There are two objectives:
to store billions of files! to serve the files fast!
SeaweedFS started as an Object Store to handle small files efficiently. Instead of managing all file metadata in a central master, the central master only manages file volumes, and it lets these volume servers manage files and their metadata. This relieves concurrency pressure from the central master and spreads file metadata into volume servers, allowing faster file access (O(1), usually just one disk read operation).
SeaweedFS can transparently integrate with the cloud. With hot data on local cluster, and warm data on the cloud with O(1) access time, SeaweedFS can achieve both fast local access time and elastic cloud storage capacity, without any client side changes.
There is only 40 bytes of disk storage overhead for each file's metadata. It is so simple with O(1) disk reads that you are welcome to challenge the performance with your actual use cases.
SeaweedFS started by implementing Facebook's Haystack design paper. Also, SeaweedFS implements erasure coding with ideas from f4: Facebook’s Warm BLOB Storage System
On top of the object store, optional Filer can support directories and POSIX attributes. Filer is a separate linearly-scalable stateless server with customizable metadata stores, e.g., MySql, Postgres, Mongodb, Redis, Cassandra, Elastic Search, LevelDB, MemSql, TiDB, Etcd, CockroachDB, etc.
Seaweed FS Screenshots
Seaweed FS Features
Seaweed FS information
- Developed byChris Lu
- LicensingOpen Source (Apache-2.0) and Free product.
- Written in
- RatingAverage rating of 5
- Alternatives14 alternatives listed
Supported Languages
- English
GitHub repository
- 16,850 Stars
- 1,936 Forks
- 173 Open Issues
- Updated
Comments and Reviews
Tags
- Cloud Storage
- File Encryption
- ttl
- cluster
- fuse
- hadoop-alternative
- posix
- File Upload
Recent user activities on Seaweed FS
nopro404 thinks MinIO is an alternative to Seaweed FS
remram44 thinks Ceph is an alternative to Seaweed FS- filiphanes reviewed Seaweed FSfiIt's fast, simple deployment, configuration, management. Efficient small files and large files too.
It's fast, simple deployment, configuration, management. Efficient small files and large files too.
Simple, fast, supports many APIs