doris数据库详解（生产Doris系列之集群部署完整版）

1.1 查看centos系统 7.1及以上 [root@ruozedata001 ~]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) [root@ruozedata001 ~]# 1.2 JDK版本在1.8及以上 [root@ruozedata001 ~]# mkdir /usr/java [root@ruozedata001 ~]# tar -xzvf jdk-8u181-Linux-x64.gz -C /usr/java/ 切记必须修正所属⽤户及⽤户组 [root@ruozedata001 ~]# chown -R root:root /usr/java/jdk1.8.0_181 配置 [root@ruozedata001 ~]# vi /etc/proFile export JAVA_HOME=/usr/java/jdk1.8.0_181 export PATH=/usr/java/jdk1.8.0_181/bin:${PATH} 生效 [root@ruozedata001 ~]# source /etc/profile [root@ruozedata001 ~]# which java /usr/java/jdk1.8.0_181/bin/java [root@ruozedata001 ~]# 1.3 GCC版本在4.8.2 及以上 [root@ruozedata001 ~]# gcc --version -bash: gcc: command not found 如果没有gcc命令，执行以下命令查看版本 [root@ruozedata001 ~]# ll /usr/libexec/gcc/x86_64-redhat-linux/ total 4 drwxr-xr-x. 2 root root 4096 Feb 3 2021 4.8.2 lrwxrwxrwx 1 root root 5 Dec 7 2020 4.8.5 -> 4.8.2 1.4 文件系统最好是ext4 在安装操作系统的时候，请选择ext4文件系统。 [root@ruozedata001 ~]# df -T Filesystem Type 1K-blocks Used Available Use% Mounted on devtmpfs devtmpfs 8124216 0 8124216 0% /dev tmpfs tmpfs 8133996 0 8133996 0% /dev/shm tmpfs tmpfs 8133996 8844 8125152 1% /run tmpfs tmpfs 8133996 0 8133996 0% /sys/fs/cgroup /dev/vda2 ext4(这里) 37024300 23522844 11614224 67% / tmpfs tmpfs 1626800 0 1626800 0% /run/user/0 cm_processes tmpfs 8133996 6812 8127184 1% /run/cloudera-scm-agent/process 1.5 时钟同步 Doris 的元数据要求时间精度要小于5000ms，所以所有集群所有机器要进行时钟同步，避免因为时钟问题引发的元数据不一致导致服务出现异常。 #所有节点安装ntp [root@ruozedata001 ~]# yum install -y ntp #选取ruozedata001为ntp的主节点 [root@ruozedata001 ~]# vi /etc/ntp.conf #time server 0.asia.pool.ntp.org server 1.asia.pool.ntp.org server 2.asia.pool.ntp.org server 3.asia.pool.ntp.org #当外部时间不可⽤时，可使⽤本地硬件时间 server 127.127.1.0 iburst local clock #允许哪些⽹段的机器来同步时间 restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap #开启ntpd及查看状态 [root@ruozedata001 ~]# systemctl start ntpd [root@ruozedata001 ~]# systemctl status ntpd ● ntpd.service - Network Time Service Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: d isabled) Active: active (running) since Sat 2019-05-11 10:15:00 CST; 11min ago Main PID: 18518 (ntpd) CGroup: /system.slice/ntpd.service !"18518 /usr/sbin/ntpd -u ntp:ntp -g May 11 10:15:00 ruozedata001 systemd[1]: Starting Network Time Service... May 11 10:15:00 ruozedata001 ntpd[18518]: proto: precision = 0.088 usec May 11 10:15:00 ruozedata001 ntpd[18518]: 0.0.0.0 c01d 0d kern kernel time sync en abled May 11 10:15:00 ruozedata001 systemd[1]: Started Network Time Service. #验证 [root@ruozedata001 ~]# ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== LOCAL(0) .LOCL. 10 l 726 64 0 0.000 0.000 0.000 #其他从节点停⽌禁⽤ntpd服务 [root@ruozedata002 ~]# systemctl stop ntpd [root@ruozedata002 ~]# systemctl disable ntpd Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service. [root@ruozedata002 ~]# /usr/sbin/ntpdate ruozedata001 11 May 10:29:22 ntpdate[9370]: adjust time server 192.168.1.141 offset 0.000867 sec #每天凌晨同步ruozedata001节点时间 [root@ruozedata002 ~]# crontab -e 00 00 * * * /usr/sbin/ntpdate ruozedata001 [root@ruozedata003 ~]# systemctl stop ntpd [root@ruozedata003 ~]# systemctl disable ntpd Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service. [root@ruozedata003 ~]# /usr/sbin/ntpdate ruozedata001 11 May 10:29:22 ntpdate[9370]: adjust time server 192.168.1.141 offset 0.000867 sec #每天凌晨同步ruozedata001节点时间 [root@ruozedata003 ~]# crontab -e 00 00 * * * /usr/sbin/ntpdate ruozedata001 [root@ruozedata004 ~]# systemctl stop ntpd [root@ruozedata004 ~]# systemctl disable ntpd Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service. [root@ruozedata003 ~]# /usr/sbin/ntpdate ruozedata001 11 May 10:29:22 ntpdate[9370]: adjust time server 192.168.1.141 offset 0.000867 sec #每天凌晨同步ruozedata001节点时间 [root@ruozedata004 ~]# crontab -e 00 00 * * * /usr/sbin/ntpdate ruozedata001 1.6 swap关闭 Linux交换分区会给Doris带来很严重的性能问题，需要在安装之前禁用交换分区 不重启电脑，禁用启用swap，立刻生效 [root@ruozedata001 ~]# free -m total used free shared buff/cache available Mem: 15886 10771 728 16 4387 4851 Swap: 4095 0 4095 [root@ruozedata001 ~]# swapoff -a [root@ruozedata001 ~]# free -m total used free shared buff/cache available Mem: 15886 10769 730 16 4386 4853 Swap: 0 0 0 重启，永久禁用Swap swap这行前面加上#注释掉 [root@ruozedata001 ~]# vi /etc/fstab # # /etc/fstab # Created by anaconda on Tue Sep 12 00:20:17 2017 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # UUID=51c8bb02-2d2a-4f2f-b823-f8c974300184 / ext3 defaults 1 1 #UUID=4104bb80-c9e6-440c-a9e0-5606951284f9 swap swap defaults 0 0 重启生效，查看 [root@ruozedata001 ~]# reboot [root@ruozedata001 ~]# free -m 1.7 系统优化参数 设置系统最大打开文件句柄数和进程数 echo "* soft nofile 960000" >> /etc/security/limits.conf echo "* hard nofile 960000" >> /etc/security/limits.conf echo "* soft nproc 960000" >> /etc/security/limits.conf echo "* hard nproc 960000" >> /etc/security/limits.conf 重启生效 [root@ruozedata001 ~]# reboot [root@ruozedata001 ~]# ulimit -n 960000 [root@ruozedata001 ~]#

wget https://archive.apache.org/dist/doris/1.1/1.1.0-rc05/apache-doris-1.1.0-bin-x86-jdk8.tar.gz

3.1 解压，重命名 [root@ruozedata001 ~]# tar -xzvf apache-doris-1.1.0-bin-x86-jdk8.tar.gz -C /opt/ [root@ruozedata001 ~]# cd /opt/ [root@ruozedata001 opt]# ln -s apache-doris-1.1.0-bin-x86-jdk8 doris [root@ruozedata001 opt]# ll total 16 drwxr-xr-x 7 1008 1008 4096 Jul 26 11:04 apache-doris-1.1.0-bin-x86-jdk8 drwxr-xr-x 8 cloudera-scm cloudera-scm 4096 Jul 25 15:28 cloudera drwxr-xr-x 3 root root 4096 Jul 25 15:02 cloudera-manager lrwxrwxrwx 1 root root 31 Jul 26 11:04 doris -> apache-doris-1.1.0-bin-x86-jdk8 drwxr-xr-x. 2 root root 4096 Oct 31 2018 rh [root@ruozedata001 opt]# cd doris 3.2 创建meta文件夹及编辑fe参数文件 [root@ruozedata001 doris]# mkdir meta [root@ruozedata001 doris]# cd fe/conf/ [root@ruozedata001 conf]# vi fe.conf #元数据路径 meta_dir = /opt/doris/meta #当前机器的IP priority_networks = 192.168.1.141/24 3.3 scp整个文件夹到另外2个机器 [root@ruozedata001 opt]# scp -r doris ruozedata002:/opt/ [root@ruozedata001 opt]# scp -r doris ruozedata003:/opt/ [root@ruozedata001 opt]# scp -r doris ruozedata004:/opt/ 3.4 ruozedata002机器修改fe配置文件 [root@ruozedata002 ~]# cd /opt/doris/fe/conf [root@ruozedata002 conf]# vi fe.conf #元数据路径 meta_dir = /opt/doris/meta #当前机器的IP priority_networks = 192.168.1.142/24 3.5 ruozedata003机器修改fe配置文件 [root@ruozedata003 ~]# cd /opt/doris/fe/conf [root@ruozedata003 conf]# vi fe.conf #元数据路径 meta_dir = /opt/doris/meta #当前机器的IP priority_networks = 192.168.1.143/24 3.6 ruozedata004机器修改fe配置文件 [root@ruozedata004 ~]# cd /opt/doris/fe/conf [root@ruozedata004 conf]# vi fe.conf #元数据路径 meta_dir = /opt/doris/meta #当前机器的IP priority_networks = 192.168.1.144/24 --------------------------------------------------------- 3.6 ruozedata001~ruozedata004配置be [root@ruozedata001 doris]# mkdir data01 data02 data03 #3块磁盘 [root@ruozedata001 doris]# vi be/conf/be.conf #当前机器的IP priority_networks = 192.168.1.141/24 #数据存储目录 storage_root_path = ${DORIS_HOME}/data01;${DORIS_HOME}/data02;${DORIS_HOME}/data03 [root@ruozedata002 doris]# mkdir data01 data02 data03 #3块磁盘 [root@ruozedata002 doris]# vi be/conf/be.conf #当前机器的IP priority_networks = 192.168.1.142/24 #数据存储目录 storage_root_path = ${DORIS_HOME}/data01;${DORIS_HOME}/data02;${DORIS_HOME}/data03 [root@ruozedata003 doris]# mkdir data01 data02 data03 #3块磁盘 [root@ruozedata003 doris]# vi be/conf/be.conf #当前机器的IP priority_networks = 192.168.1.143/24 #数据存储目录 storage_root_path = ${DORIS_HOME}/data01;${DORIS_HOME}/data02;${DORIS_HOME}/data03 [root@ruozedata004 doris]# mkdir data01 data02 data03 #3块磁盘 [root@ruozedata004 doris]# vi be/conf/be.conf #当前机器的IP priority_networks = 192.168.1.144/24 #数据存储目录 storage_root_path = ${DORIS_HOME}/data01;${DORIS_HOME}/data02;${DORIS_HOME}/data03

第一台作为master [root@ruozedata001 ~]# start_fe.sh --daemon 客户端登录，添加FOLLOWER和OBSERVER节点 [root@ruozedata001 ~]# /usr/local/MySQL/bin/mysql -h ruozedata001 -P 9030 -uroot #直接回车 刚开始无密码即可登录，后续可设置 Welcome to the mysql monitor. Commands end with ; or \g. Your MySQL connection id is 0 Server version: 5.7.37 Doris version 1.1.0-rc05-Unknown Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. #添加 mysql> ALTER SYSTEM ADD FOLLOWER "ruozedata002:9010"; mysql> ALTER SYSTEM ADD FOLLOWER "ruozedata003:9010"; mysql> ALTER SYSTEM ADD OBSERVER "ruozedata004:9010"; #查看 mysql> show proc '/frontends'; ---------------------------------- --------------- -------------- ------------- ---------- ----------- --------- ---------- ---------- ----------- ------- ------- ------------------- --------------------- ---------- -------------------------------------------------------------------- -------------------- ------------------ | Name | IP | HostName | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected | ---------------------------------- --------------- -------------- ------------- ---------- ----------- --------- ---------- ---------- ----------- ------- ------- ------------------- --------------------- ---------- -------------------------------------------------------------------- -------------------- ------------------ | 192.168.1.141_9010_1658812495496 | 192.168.1.141 | ruozedata001 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 80066555 | true | true | 322 | 2022-07-26 14:32:12 | true | | 1.1.0-rc05-Unknown | Yes | | 192.168.1.142_9010_1658817055629 | 192.168.1.142 | ruozedata002 | 9010 | 8030 | 0 | 0 | FOLLOWER | false | 80066555 | false | false | 0 | NULL | true | java.net.ConnectException: Connection refused (Connection refused) | NULL | No | | 192.168.1.143_9010_1658817055629 | 192.168.1.143 | ruozedata003 | 9010 | 8030 | 0 | 0 | FOLLOWER | false | 80066555 | false | false | 0 | NULL | true | java.net.ConnectException: Connection refused (Connection refused) | NULL | No | | 192.168.1.144_9010_1658817081961 | 192.168.1.144 | ruozedata004 | 9010 | 8030 | 0 | 0 | OBSERVER | false | 80066555 | false | false | 0 | NULL | false | java.net.ConnectException: Connection refused (Connection refused) | NULL | No | ---------------------------------- --------------- -------------- ------------- ---------- ----------- --------- ---------- ---------- ----------- ------- ------- ------------------- --------------------- ---------- -------------------------------------------------------------------- -------------------- ------------------

[root@ruozedata002 ~]# start_fe.sh --helper ruozedata001:9010 --daemon [root@ruozedata003 ~]# start_fe.sh --helper ruozedata001:9010 --daemon [root@ruozedata004 ~]# start_fe.sh --helper ruozedata001:9010 --daemon --helper 参数仅在 follower 和 observer 第一次启动时才需要。 查看这2台的fe状态 mysql> show proc '/frontends'; ---------------------------------- --------------- -------------- ------------- ---------- ----------- --------- ---------- ---------- ----------- ------ ------- ------------------- --------------------- ---------- -------- -------------------- ------------------ | Name | IP | HostName | EditLogPort | HttpPort | QueryPort | RpcPort | Role | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat | IsHelper | ErrMsg | Version | CurrentConnected | ---------------------------------- --------------- -------------- ------------- ---------- ----------- --------- ---------- ---------- ----------- ------ ------- ------------------- --------------------- ---------- -------- -------------------- ------------------ | 192.168.1.141_9010_1658812495496 | 192.168.1.141 | ruozedata001 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | true | 80066555 | true | true | 387 | 2022-07-26 14:35:47 | true | | 1.1.0-rc05-Unknown | Yes | | 192.168.1.142_9010_1658817055629 | 192.168.1.142 | ruozedata002 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | false | 80066555 | true | true | 385 | 2022-07-26 14:35:47 | true | | 1.1.0-rc05-Unknown | No | | 192.168.1.143_9010_1658817055629 | 192.168.1.143 | ruozedata003 | 9010 | 8030 | 9030 | 9020 | FOLLOWER | false | 80066555 | true | true | 385 | 2022-07-26 14:35:47 | true | | 1.1.0-rc05-Unknown | No | | 192.168.1.144_9010_1658817081961 | 192.168.1.144 | ruozedata004 | 9010 | 8030 | 9030 | 9020 | OBSERVER | false | 80066555 | true | true | 385 | 2022-07-26 14:35:47 | false | | 1.1.0-rc05-Unknown | No | ---------------------------------- --------------- -------------- ------------- ---------- ----------- --------- ---------- ---------- ----------- ------ ------- ------------------- --------------------- ---------- -------- -------------------- ------------------ 3 rows in set (0.02 sec) mysql>

mysql> ALTER SYSTEM ADD BACKEND "ruozedata001:9050"; mysql> ALTER SYSTEM ADD BACKEND "ruozedata002:9050"; mysql> ALTER SYSTEM ADD BACKEND "ruozedata003:9050"; mysql> ALTER SYSTEM ADD BACKEND "ruozedata004:9050";

[root@ruozedata001 ~]# start_be.sh --daemon [root@ruozedata002 ~]# start_be.sh --daemon [root@ruozedata003 ~]# start_be.sh --daemon [root@ruozedata004 ~]# start_be.sh --daemon 客户端登录查看 mysql> show proc '/backends'; ----------- ----------------- --------------- -------------- --------------- -------- ---------- ---------- --------------------- --------------------- ------- ---------------------- ----------------------- ----------- ------------------ --------------- --------------- --------- ---------------- -------------------------- -------- -------------------- ------------------------------------------------------------------------------------------------------------------------------- | BackendId | Cluster | IP | HostName | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | Tag | ErrMsg | Version | Status | ----------- ----------------- --------------- -------------- --------------- -------- ---------- ---------- --------------------- --------------------- ------- ---------------------- ----------------------- ----------- ------------------ --------------- --------------- --------- ---------------- -------------------------- -------- -------------------- ------------------------------------------------------------------------------------------------------------------------------- | 11001 | default_cluster | 192.168.1.141 | ruozedata001 | 9050 | 9060 | 8040 | 8060 | 2022-07-26 15:13:05 | 2022-07-26 15:15:06 | true | false | false | 0 | 0.000 | 31.134 GB | 105.927 GB | 70.61 % | 70.61 % | {"location" : "default"} | | 1.1.0-rc05-Unknown | {"lastSuccessReportTabletsTime":"2022-07-26 15:14:13","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | | 11002 | default_cluster | 192.168.1.142 | ruozedata002 | 9050 | 9060 | 8040 | 8060 | 2022-07-26 15:14:57 | 2022-07-26 15:15:06 | true | false | false | 0 | 0.000 | 61.977 GB | 105.927 GB | 41.49 % | 41.49 % | {"location" : "default"} | | 1.1.0-rc05-Unknown | {"lastSuccessReportTabletsTime":"2022-07-26 15:15:02","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | | 11003 | default_cluster | 192.168.1.143 | ruozedata003 | 9050 | 9060 | 8040 | 8060 | 2022-07-26 15:13:54 | 2022-07-26 15:15:06 | true | false | false | 0 | 0.000 | 61.979 GB | 105.927 GB | 41.49 % | 41.49 % | {"location" : "default"} | | 1.1.0-rc05-Unknown | {"lastSuccessReportTabletsTime":"2022-07-26 15:14:59","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | | 11004 | default_cluster | 192.168.1.144 | ruozedata004 | 9050 | 9060 | 8040 | 8060 | 2022-07-26 15:13:54 | 2022-07-26 15:15:06 | true | false | false | 0 | 0.000 | 61.979 GB | 105.927 GB | 41.49 % | 41.49 % | {"location" : "default"} | | 1.1.0-rc05-Unknown | {"lastSuccessReportTabletsTime":"2022-07-26 15:14:59","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false} | ----------- ----------------- --------------- -------------- --------------- -------- ---------- ---------- --------------------- --------------------- ------- ---------------------- ----------------------- ----------- ------------------ --------------- --------------- --------- ---------------- -------------------------- -------- -------------------- ------------------------------------------------------------------------------------------------------------------------------- 3 rows in set (0.00 sec)

open http://ruozedata001:8030 admin/无密码

注: 外网，先去安全组开放8030端口号