DistributedSystem-OpenSource-List

Distributed System OpenSource List

Distributed ID

Distributed Coordination

Job Scheduler

  • 2017-Elastic-Job #Project#: Elastic-Job is a distributed scheduled job solution. Elastic-Job is composited from 2 independent sub projects: Elastic-Job-Lite and Elastic-Job-Cloud.

  • Disque #Project#: Disque is a distributed message broker.

  • Ofelia #Project#: Ofelia is a modern and low footprint job scheduler for docker environments, built on Go. Ofelia aims to be a replacement for the old fashioned cron.

  • TaskBotJS #Project#: The best JavaScript/TypeScript job processing framework on the planet.

  • SIA #Project#: SIA 是我们公司基础开发平台 Simple is Awesome 的简称,SIA-TASK(微服务任务调度平台)是其中的一项重要产品,SIA-TASK 契合当前微服务架构模式,具有跨平台,可编排,高可用,无侵入,一致性,异步并行,动态扩展,实时监控等特点。

Distributed Lock

Distributed Consensus

  • liferaft #Project#: Consensus protocol based on raft, but only for saving lifes and works in node as well as in browers.

Distributed Transaction

  • Seata #Project#: Seata is an easy-to-use, high-performance, java based, open source distributed transaction solution.

Distributed Storage

Distributed FS

Object Storage

  • Image Server #Project#: An image server toolkit in Go (Golang)

  • Pavlov Match #Project#:Scalable reverse image search

  • Blb #Project#: Blb is a distributed object storage system designed for use on bare metal with spinning disks in cluster computing environments.

  • BFS

  • TFS

  • fastDFS

  • MogileFS

  • 2019-MinIO #Project#: MinIO is a high performance object storage server compatible with Amazon S3 APIs.

Block Storage

DSL

  • Apache Beam #Project#: Apache Beam is a unified model for defining both batch and streaming data-parallel processing pipelines, as well as a set of language-specific SDKs for constructing pipelines and Runners for executing them on distributed processing backends, including Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow.

Message Oriental Middleware

  • 2017-Sandglass #Project#: Sandglass is a distributed, horizontally scalable, persistent, time sorted message queue.

  • 2018-PhxQueue #Project#: PhxQueue 是微信开源的一款基于 Paxos 协议实现的高可用、高吞吐和高可靠的分布式队列,保证 At-Least-Once Delivery,在微信内部广泛支持微信支付、公众平台等多个重要业务。

  • 2018-Jocko #Project#: Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

  • 2018-QMQ #Project#: QMQ 是去哪儿网内部广泛使用的消息中间件,自 2012 年诞生以来在去哪儿网所有业务场景中广泛的应用,包括跟交易息息相关的订单场景; 也包括报价搜索等高吞吐量场景。

  • 2018-NATS #Project#: High-Performance server for NATS, the cloud native messaging system.

  • 2018-Waltz #Project#: Waltz is a quorum-based distributed write-ahead log for replicating transactions.

Processing Engine

  • hazelcast-jet #Project#: A general purpose distributed data processing engine, built on top of Hazelcast.

  • Flink #Project#: Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities.

  • Wallaroo #Project#: Wallaroo is a fast, elastic data processing engine that rapidly takes you from prototype to production by eliminating infrastructure complexity.

Edge Computing | 边缘计算

  • OpenEdge #Project#: Extend cloud computing, data and service seamlessly to edge devices.