DistributedCompute-OpenSource-List

Distributed System OpenSource List

DSL

  • Apache Beam #Project#: Apache Beam is a unified model for defining both batch and streaming data-parallel processing pipelines, as well as a set of language-specific SDKs for constructing pipelines and Runners for executing them on distributed processing backends, including Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow.

Message Oriental Middleware

  • 2017-Sandglass #Project#: Sandglass is a distributed, horizontally scalable, persistent, time sorted message queue.
  • 2018-PhxQueue #Project#: PhxQueue 是微信开源的一款基于 Paxos 协议实现的高可用、高吞吐和高可靠的分布式队列,保证 At-Least-Once Delivery,在微信内部广泛支持微信支付、公众平台等多个重要业务。