Bitalos

# Introduction

  • Bitalos, produced by Zuoyebang platform technical team, is a high-performance distributed storage system, core engine based on bitalosdb (self-developed high-performance KV storage engine), compatible with Redis protocol. As an alternative to Redis, it stores data with low-cost hard disk instead of expensive memory, takes full advantage of multi-core and provides excellent single-core performance, which can significantly reduce service costs.

  • Bitalos contains three main projects: dashboard (visual management platform), stored (storage service), and proxy (proxy service). Current open-source version is stable, and provides a complete industrial grade solution.

# Creator

  • Author: Xu Ruibo, joined Zuoyebang in December 2018 (working till now), is responsible for live class middle platform and Zuoyebang platform, and leads the storage technology team to develop Bitalos from 0 to 1.

  • Contributors：Xing Fu, Lu Wenwei, Liu Fang, Li Jingchen

# Practice (Zuoyebang)

  • Application: carry the KV storage traffic in Zuoyebang

  • Peak traffic: QPS（30 Million），Network（8000Gbps）

  • Storage data: 1PB

  • Average reading time: 100μs, average writing time: 150μs

  • Availability: 99.999%

  • Stability: Since V1.0 was released in 2019, there have been zero online incidents

# Key Technology

  • Compatible with Redis protocol, low integration cost. Supports most commands, including LUA, distributed transactions.

  • Efficient horizontal scaling, supporting up to 1024 shards and enabling shard expansion in seconds.

  • Multi-cloud disaster recovery, supporting multi-data room or multi-cloud deployment & management, with a complete downgrade and disaster recovery solution.

  • Multi-master architecture (supported in Enterprise Edition), based on CRDT, optimized data synchronization and consistency strategies, ensuring adaptive conflict resolution during multi-master writes and achieving eventual consistency.

  • High-performance data consistency architecture, based on Bitalos-Raft, with deeply optimized Raft protocol, which greatly improves write performance, and features more stable election strategy and data synchronization process.

    • High-performance log engine: bitaloslog, featuring log indexing based on bitalostree, supporting high-throughput write operations.

  • High-performance kernel equipped with self-developed KV engine: bitalosdb.

    • High-performance compressed indexing technology: bitalostree, a B+ tree based on ultra-large pages, featuring innovative index compression that eliminates write amplification in B+ trees and maximizes read performance.

    • High-performance KV index, vector computation implemented based on ASM assembly, with significantly improved performance.

    • High-performance K-KV index, multi-level vector index based on bitalostree, balancing index compression ratio and retrieval performance.

    • High-performance KV separation technology: bithash, based on a compact index structure, with O(1) retrieval efficiency and capable of independent GC.

    • High-performance storage structure that compresses Redis composite data types, significantly reducing I/O costs and improving system throughput.

# Technology accumulation(bitalosearch)

  • High performance distributed search & analysis engine, SQL protocol, focusing on AP scenarios, and has certain TP capabilities. It is being practiced internally, and the open source plan is to be determined

  • Compared to elasticsearch, bitalosearch has significant cost advantages. Hard disk consumption is saved 30%; data writing performance is improved by 25%; for complex analysis logic, query performance is improved by 20% to 500%

ZUOYEBANG