TianHang Tang Track 3: Applications https://open.mi.com/conference/hbasecon-asia-2019 THE COMMUNITY EVENT FOR APACHE HBASE™ July 20th, 2019 - Sheraton Hotel, Beijing, China https://hbase.apache.org/hbaseconasia-2019/

2. HBase at DiDi
TianHang Tang

3. Content
1.Usage Scenario On HBase
2.GeoMesa Applications On HBase
3.Improvements Of ZK
4.Improvements Of HBase & Phoenix

4. Usage Scenario On
HBase

6. 02
01
0305
06
04
Clusters
Chinese clusters: 7
Internationalized clusters: 4
Buiness Line
Full coverage
Projects
200+
TPS
1kw+/sec
Amount
PB level
HBase Tables
1000+
Overview

9. Pain points
• The feature dimension continues to change
• Support feature values of different road link versions
• Support for increasing time series features
• Supports querying according to KEY index, and supports SCAN according to KEY range
• Good scalability and query performance
• Feature name, map version, time slice, three dimensions cross
HBase
• Dynamic columns
• Multi-version
• Get and Scan
• Linear expansion,region balance

11. GeoMesa Applications
On HBase

12. Server HBase
HDFS
GeoMesa
Client
HBase Client
GeoMesa Client
Create Table/ Insert/
Query/Modify/…
GeoMesa is an open source suite of tools that
enables large-scale geospatial querying and
analytics on distributed computing systems.
GeoMesa
• Out of the box
• Text language support
• Horizontal expansion
• Hadoop ecology

18. Research
First User
MR，Spark，
Python Spark…
StreamingSQL
ingest
Precipitation
access
process
2018.04 2018.08 2018.11 2019.03 2019.05

19. Improvements Of
ZooKeeper

21. ZOOKEEPER-832
If the hbase client is not
restarted, the client will cache
the id of the old server zk. This
id is larger than the client zk id,
resulting in rejection of many
attempts when accessing the
client zk cluster.
HBASE-20159
Support using separate ZK
quorums for client.

23. Improvements Of
HBase & Phoenix

24. 1. HBASE-21964 :
unset Quota by Throttle Type
2. HBASE-22620 :
when a cluster open replication,regionserver will not clean up the
walLog references on zk due to no wal entry need to be replicated
3. PHOENIX-5242 :
concurrent tasks 14000 to 32

25. Quota experience
1. NUMBER type Semantic : RPC
2. ThrottlingException -> DoNotRetryIOException
3. HBASE-21592 : quota.addGetResult(r) throws NPE
4. HBASE-21964

26. Full Link Monitor

27. Thanks！
tangtianhang@didiglobal.com