Apache Kylin
 | This article needs additional citations for verification. (September 2018) |
 | |
| Developer(s) | Apache Kylin Committee |
|---|---|
| Initial release | June 10, 2015[1] |
| Stable release | |
| Repository | Kylin Repository |
| Written in | Java |
| License | Apache License 2.0 |
| Website | kylin |
Apache Kylin is an open source distributed analytics engine designed to provide a SQL interface and multi-dimensional analysis (OLAP) on Hadoop and Alluxio supporting extremely large datasets.
It was originally developed by eBay, and is now a project of the Apache Software Foundation.[3]
History
The Kylin project was started in 2013, in eBay's R&D in Shanghai, China. In Oct 2014, Kylin v0.6 was open sourced on github.com with the name "KylinOLAP".[4]
In November 2014, Kylin joined Apache Software Foundation incubator.
In December 2015, Apache Kylin graduated to be a Top Level Project.[3]
In March 2016, Kyligence, Inc. was founded by the creators of Apache Kylin.[5][6] Kyligence provides a commercial analytics platform based on Apache Kylin for on-premise and cloud-based datasets.[7]
Architecture
Apache Kylin is built on top of Apache Hadoop, Apache Hive, Apache HBase, Apache Parquet, Apache Calcite, Apache Spark and other technologies.[8] These technologies enable Kylin to easily scale to support massive data loads.[9]
Kylin has the following core components:[10][8]
- REST Server: Receive and response user or API requests
- Metadata: Persistent and manage system, especially the cube metadata;
- Query Engine: Parse SQL queries to execution plan, and then talk with storage engine;
- Storage Engine: Pushdown and scan underlying cube storage (default in HBase);
- Job Engine: Generate and execute MapReduce or Spark job to build source data into cube;
Users
Apache Kylin has been adopted by many companies as their OLAP platform in production. Typical users includes eBay, Meituan, XiaoMi, NetEase, Beike, Yahoo! Japan.
Roadmap
Apache Kylin roadmap (from Kylin website[11]):
- Hadoop 3.0 support (Erasure Coding) - completed (v2.5)
- Fully on Spark Cube engine - completed (v2.5)
- Connect more data sources (MySQL, Oracle, SparkSQL, etc) - completed (v2.6)
- Real-time analytics with Lambda Architecture - completed (v3.0)
- Cloud-native storage (Parquet) - In progress (v4.0.0-alpha)
- Ad hoc queries without Cubing
References
- ^ "Previous Release". v0.7.1-incubating (First Apache Release). Retrieved 15 June 2019.
- ^ a b "Apache Kylin - Release Notes". Retrieved 27 September 2022.
- ^ a b Apache Software Foundation. "The Apache Software Foundation Announces Apache Kylin as a Top-Level Project", 8 December 2015
- ^ "Announcing Kylin: Extreme OLAP Engine for Big Data". www.ebayinc.com. 2014-10-20. Retrieved 2018-11-08.
- ^ "Apache Kylin Through the Eyes of the Founders - Part One". Kyligence. 2020-06-12. Retrieved 2020-09-30.
- ^ "Big Data Analytics Platform | Learn More About Kyligence". Kyligence. Retrieved 2020-09-30.
- ^ "Big Data Analytics Platform: Apache Kylin vs. Kyligence". Kyligence. Retrieved 2020-09-30.
- ^ a b "Apache Kylin | Analytical Data Warehouse for Big Data". kylin.apache.org. Retrieved 2020-09-30.
- ^ Knorr, Eric (2016-03-07). "What eBay looks like under the hood". InfoWorld. Retrieved 2020-09-30.
- ^ "Apache Kylin Adds Real-time OLAP". www.i-programmer.info. Retrieved 2020-09-30.
- ^ Kylin, Apache. "Apache Kylin | Development Quick Guide". kylin.apache.org. Retrieved 2020-09-30.
 | This business software article is a stub. You can help Wikipedia by expanding it. |
 | This database software-related article is a stub. You can help Wikipedia by expanding it. |