distributed - Any key-value storages with emphasis on compression? -


are there key-value storages fit following criteria?

  • are open-source
  • persistent file storage
  • have replication , oplog
  • have configurable compression usable storing 10-100 megabytes of raw text per second
  • work on windows , linux

desired interface should contain @ least:

  • store record text or numeric id
  • retrieve record id

wiredtiger support different kind of compression:

compression considerations

wiredtiger compresses data @ several stages preserve memory , disk space. applications can configure these different compression algorithms tailor requirements between memory, disk , cpu consumption. compression algorithms other block compression work modifying how keys , values represented, , hence reduce data size in-memory , on-disk. block compression on other hand compress data in binary representation while saving on disk.

configuring compression may change application throughput. example, in applications using solid-state drives (where i/o less expensive), turning off compression may increase application performance reducing cpu costs; in applications i/o costs more expensive, turning on compression may increase application performance reducing overall number of i/o operations.

wiredtiger uses internal algorithms compress amount of data stored not configurable, on. example, run-length reduces size requirement storing sequential, duplicate values in store single time (with associated count).

wiredtiger support different kind of compression:

  • key prefix
  • dictionary
  • huffman
  • and block compression support among other things lz4, snappy, zlib , zstd.

have @ documentation full cover of subject.


Comments

Popular posts from this blog

android - MPAndroidChart - How to add Annotations or images to the chart -

javascript - Add class to another page attribute using URL id - Jquery -

firefox - Where is 'webgl.osmesalib' parameter? -