distributed - Any key-value storages with emphasis on compression? -
are there key-value storages fit following criteria?
- are open-source
- persistent file storage
- have replication , oplog
- have configurable compression usable storing 10-100 megabytes of raw text per second
- work on windows , linux
desired interface should contain @ least:
- store record text or numeric id
- retrieve record id
wiredtiger support different kind of compression:
compression considerations
wiredtiger compresses data @ several stages preserve memory , disk space. applications can configure these different compression algorithms tailor requirements between memory, disk , cpu consumption. compression algorithms other block compression work modifying how keys , values represented, , hence reduce data size in-memory , on-disk. block compression on other hand compress data in binary representation while saving on disk.
configuring compression may change application throughput. example, in applications using solid-state drives (where i/o less expensive), turning off compression may increase application performance reducing cpu costs; in applications i/o costs more expensive, turning on compression may increase application performance reducing overall number of i/o operations.
wiredtiger uses internal algorithms compress amount of data stored not configurable, on. example, run-length reduces size requirement storing sequential, duplicate values in store single time (with associated count).
wiredtiger support different kind of compression:
- key prefix
- dictionary
- huffman
- and block compression support among other things lz4, snappy, zlib , zstd.
have @ documentation full cover of subject.
Comments
Post a Comment