Database-OpenSource-List

返回目录

Database OpenSource List | 数据库相关开源工具索引

Universal SQL Tools

Client

GUI

  • SQL Fiddle #Project#: A tool for easy online testing and sharing of database problems and their solutions.

  • Dbeaver #Project#: Free multi-platform database tool for developers, SQL programmers, database administrators and analysts.

  • HeidiSQL #Project#: HeidiSQL is free software, and has the aim to be easy to learn. "Heidi" lets you see and edit data and structures from computers running one of the database systems MariaDB, MySQL, Microsoft SQL or PostgreSQL.

CMD

  • 2018-q #Project#: q is a command line tool that allows direct execution of SQL-like queries on CSVs/TSVs (and any other tabular text files).

  • usql #Project#: usql is a universal command-line interface for working with SQL databases.

Editor

  • 2014-sqlpad #Project#: Web-based SQL editor run in your own private cloud. Supports MySQL, Postgres, SQL Server, Vertica, Crate, Presto, SAP HANA, and Cassandra.

  • 2016-Falcon #Project#: Free, open-source SQL client for Windows and Mac 🦅

Analysis & Execution

  • queryparser #Project#: Parsing and analysis of Vertica, Hive, and Presto SQL. Queryparser, an Open Source Tool for Parsing and Analyzing SQL

  • sql-parser #Project#: Light and fast sql parser on browser.

  • 2013-druid #Project#: 阿里巴巴数据库事业部出品,为监控而生的数据库连接池。

  • Apache Calcite #Project#: Industry-standard SQL parser,validator and JDBC driver.

  • 2017-SQLAdvisor #Project#: SQLAdvisor 是由美团点评公司技术工程部 DBA 团队(北京)开发维护的一个分析 SQL 给出索引优化建议的工具。它基于 MySQL 原生态词法解析,结合分析 SQL 中的 where 条件、聚合条件、多表 Join 关系 给出索引优化建议。目前 SQLAdvisor 在美团点评内部广泛应用,公司内部对 SQLAdvisor 的开发全面转到 github 上,开源和内部使用保持一致。

  • 2018-sqler #Project#: write APIs using direct SQL queries with no hassle, let's rethink about SQL.

  • Zetasql #Project#: ZetaSQL defines a language (grammar, types, data model, and semantics) as well as a parser and analyzer.

Migration & Transfer

Benchmark

Relational Database

MySQL

  • Freno #Project#: Cooperative, highly available throttler service: clients use freno to throttle writes to a resource.

Tools

HA

Sharding

  • Sharding-Sphere #Project#: Sharding-Sphere is an open-source ecosystem consisted of a set of distributed database middleware solutions, including 3 independent products, Sharding-JDBC, Sharding-Proxy & Sharding-Sidecar (todo).

  • 2015-KingShard #Project#: kingshard is a high-performance proxy for MySQL powered by Go.

  • Atlas #Project#: A high-performance and stable proxy for MySQL, it is developed by Qihoo's DBA and infrastructure team.

  • 2017-Vitess #Project#: Vitess is a database clustering system for horizontal scaling of MySQL.

  • 2019-Gaea #Project#: Gaea is a mysql proxy, it's developed by xiaomi b2c-systech team.

Key-Value Database

  • LevelDB #Project#: LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

  • RocksDB #Project#: A library that provides an embeddable, persistent key-value store for fast storage.

  • Badger #Project#: Fast key-value DB in Go.

Redis

  • RedisGraph #Project#: A High Performance In-Memory Graph Database as a Redis Module.

  • 2018-RDR #Project#: RDR(redis data reveal) is a tool to parse redis rdbfile. Comparing to redis-rdb-tools, RDR is implemented by golang, much faster (5GB rdbfile takes about 2mins on my PC).

  • 2018-Tidis #Project#: Tidis is a Distributed NoSQL database, providing a redis-protocal api(string,list,hash,set,sorted-set), written in Go.

  • 2018-KeyDB #Project#: A Multithreaded Fork of Redis

Document Database

Search Engine

  • search-index #Project#: A persistent, network resilient, full text search library for the browser and Node.js.

MongoDB

Graph Database

  • Cayley #Project#: Cayley is an open-source graph inspired by the graph database behind Freebase and Google's Knowledge Graph.

  • Neo4j #Project#: Neo4j is the world’s leading Graph Database. It is a high performance graph store with all the features expected of a mature and robust database, like a friendly query language and ACID transactions.

  • Dgraph #Project#: Dgraph is a horizontally scalable and distributed graph database, providing ACID transactions, consistent replication and linearizable reads.

TimeSeries

  • 2013-InfluxDB #Project#: InfluxDB is an open source time series database with no external dependencies. It's useful for recording metrics, events, and performing analytics.

  • 2017-timescaledb #Project#: An open-source time-series database optimized for fast ingest and complex queries. Engineered up from PostgreSQL, packaged as an extension.

  • 2018-M3 monorepo #Project#: Distributed TSDB, Aggregator and Query Engine, Prometheus Sidecar, Metrics Platform

  • VictoriaMetrics #Project#: VictoriaMetrics is fast, cost-effective and scalable time series database.

  • TDengine #Project#: TDengine is an open-sourced big data platform under GNU AGPL v3.0, designed and optimized for the Internet of Things (IoT), Connected Cars, Industrial IoT, and IT Infrastructure and Application Monitoring.

  • 2019-LinDB #Project#: LinDB is a scalable, high performance, high availability distributed time series database.

NewSQL

  • FoundationDB #Project#: FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers.

Datawarehouse

OLAP

  • 2016-ClickHouse #Project#: ClickHouse is an open source column-oriented database management system capable of real time generation of analytical data reports using SQL queries.

  • 2017-Druid #Project#: Apache Druid (incubating) is a high performance analytics data store for event-driven data.

  • 2017-Mondrian #Project#: Mondrian is an Online Analytical Processing (OLAP) server that enables business users to analyze large quantities of data in real-time.

  • Pinot #Project#: Pinot is a realtime distributed OLAP datastore, which is used at LinkedIn to deliver scalable real time analytics with low latency.

OLAP Browser

  • 2015-Metabase #Project#: The simplest, fastest way to get business intelligence and analytics to everyone in your company.

  • 2016-Redash #Project#: Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

  • Saiku #Project#: Saiku Analytics - The Worlds Greatest Open Source OLAP Browser

  • CBoard #Project#: An easy to use, self-service open BI reporting and BI dashboard platform.

  • Apache Superset #Project#: Apache Superset (incubating) is a modern, enterprise-ready business intelligence web application

  • Metatron Discovery #Project#: Metatron Discovery is an end-to-end big data self discovery solution. To learn more about it, visit our web site. Check our blog for upcoming events and development news.

  • Poli #Project#: An easy-to-use BI server built for SQL lovers. Power data analysis in SQL and gain faster business insights.

  • Cube.js #Project#: Open Source Analytics Framework.

  • 2015-Caravel #Project#: Apache Superset (incubating) is a modern, enterprise-ready business intelligence web application

  • 2016-Redash #Project#: Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

MPP

  • Presto #Project#: Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes.

Data Aggregation

ETL

  • awesome-etl #Project#: A curated list of awesome ETL frameworks, libraries, and software.

  • DataX #Project#: 阿里巴巴集团内被广泛使用的离线数据同步工具/平台

  • dbt #Project#: dbt (data build tool) enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

Data Pipeline

  • Debezium #Project#: Debezium is a distributed platform that turns your existing databases into event streams, so applications can see and respond immediately to each row-level change in the databases.

  • Canal #Project#: 阿里巴巴 mysql 数据库 binlog 的增量订阅&消费组件 。阿里云 DRDS( https://www.aliyun.com/product/drds )、阿里巴巴 TDDL 二级索引、小表复制 powerd by canal.

  • Otter #Project#: 阿里巴巴分布式数据库同步系统(解决中美异地机房)