从零开始学习MySQL调试跟踪_1)

GreatSQL是MySQL的国产分支版本,使用上与MySQL一致.

编译GreatSQL

安装gdb

开始调试GreatSQL源码

有时为了跟踪故障需要调试MySQL/GreatSQL源码,本文介绍如何在Linux下构建MySQL/GreatSQL源码调试环境.

在这之前,我也是一名小白,一起从零开始探索吧.

1. 编译GreatSQL

查看系统环境：

$ cat /etc/system-release

CentOS Linux release ⑧④2105

Source Code

此时此刻呢,参考文章 在Linux下源码编译安装GreatSQL 构建好编译环境.然后开始编译GreatSQL源码,编译参数中增加/修改debug相关选项,这样编译后得到的二进制文件才能支持调试模式,例如：

$ cd /opt/greatsql-⑧0.25-16
$ mkdir -p build
$ cd build
$ cmake3 .. \
-DBOOST_INCLUDE_DIR=/opt/boost_73_0 \
-DLOCAL_BOOST_DIR=/opt/boost_73_0 \
-DCMAKE_INSTALL_PREFIX=/usr/local/GreatSQL-⑧0.25-16-Linux-glibc2.28-x86_64 \
-DWITH_ZLIB=bundled \
-DWITH_NUMA=ON \
-DCMAKE_EXE_LINKER_FLAGS="-ljemalloc" \
-DBUILD_CONFIG=mysql_release \
-DWITH_TOKUDB=OFF \
-DWITH_ROCKSDB=OFF \
-DMAJOR_VERSION=8 \
-DMINOR_VERSION=0 \
-DPATCH_VERSION=25 \
-DWITH_UNIT_TESTS=OFF \
-DWITH_NDBCLUSTER=OFF \
-DWITH_SSL=system \
-DWITH_SYSTEMD=ON \
-DWITH_LDAP=OFF \
-DWITH_AUTHENTICATION_LDAP=OFF \
-DWITH_DEBUG=1 \
-DCMAKE_BUILD_TYPE=Debug \
 make -j8 VERBOSE=1  make install

主要是增加两个参数 -DWITH_DEBUG=1 和 -DCMAKE_BUILD_TYPE=Debug,注意不要有参数 -DCMAKE_BUILD_TYPE=RelWithDebInfo.

编译完成后,即可得到包含debug功能的GreatSQL二进制文件,执行下面的命令检查：

$ cd /usr/local/GreatSQL-⑧0.25-16-Linux-glibc2.28-x86_64
$ ./bin/mysqld-debug --verbose --version

/usr/local/GreatSQL-⑧0.25-16-Linux-glibc2.28-x86_64/bin/mysqld-debug  Ver ⑧0.25-16-debug for Linux on x86_64 (Source distribution)

可以看到,输出的结果中包含 debug 关键字,这就表示成功了.

2. 安装gdb

直接执行yum安装gdb即可：

$ yum install -y gdb
$ gdb --version
GNU gdb (GDB) Red Hat Enterprise Linux 9.2-④el8
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3◆: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

gdb常用的调试相关指令有以下几个：

③ 开始调试GreatSQL源码

第一次运行gdb准备调试时,可能会提示类似下面的信息

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
0x00007ffb358ada41 in poll () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install keyutils-libs-1.⑤10-9.el⑧x86_64 ...

这表示缺少一些相关的debuginfo包,可以根据提示内容补充安装,例如：

dnf debuginfo-install keyutils-libs-1.⑤10-9.el⑧x86_64 ...

如果提示找不到这些安装包：

Could not find debuginfo package for the following installed packages: keyutils-libs-1.⑤10-9.el⑧x86_64 ...

可以检查yum配置文件 /etc/yum.repos.d/CentOS-Linux-Debuginfo.repo,确认是否设置了 enable = 1,例如：

# CentOS-Linux-Debuginfo.repo
#
# All debug packages are merged into a single repo, split by basearch, and are
# not signed.

[debuginfo]
name=CentOS Linux $releasever - Debuginfo
baseurl=http://debuginfo.centos.org/$releasever/$basearch/
gpgcheck=1
enabled=1    #<---这里要设置1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

$ tar zxf PATH/greatsql-⑧0.25-1⑥tar.gz -C /opt/

此时此刻呢,演示如何跟踪调试.

先初始化GreatSQL数据文件,然后再启动GreatSQL服务进程：

# 初始化GreatSQL
$ cd /usr/local/GreatSQL-⑧0.25-16-Linux-glibc2.28-x86_64
$ ./bin/mysqld-debug --no-defaults --initialize-insecure --user=mysql --datadir=./data

# 启动GreatSQL
$ ./bin/mysqld-debug --no-defaults --user=mysql --datadir=./data1 

# 查看进程pid
$ ps -ef | grep mysqld
...
mysql    2644322 2542442  3 14:38 pts/7    00:00:01 ./bin/mysqld-debug --no-defaults --user=mysql --datadir=./data1

# 在另一个终端（终端#2）,连入GreatSQL
$ mysql -S/tmp/mysql.sock
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 7
Server version: ⑧0.25-16-debug Source distribution
...
mysql>\s
...
Server version:         ⑧0.25-16-debug Source distribution
...

启动gdb,准备调试跟踪GreatSQL,我们分别演示几种不同方式.

③1 利用gdb设置断点

在 终端#1 中启动gdb,并挂接GreatSQL进程,准备跟踪

$ gdb -p 2644322
GNU gdb (GDB) Red Hat Enterprise Linux 9.2-④el8
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3◆: GNU GPL version 3 or later 
...
Attaching to process 2644322
[New LWP 2643482]
[New LWP 2643483]
[New LWP 2643484]
...
[New LWP 2643522]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
--Type  for more, q to quit, c to continue without paging--  #<-- 这里按下回车,即可进入
0x00007fb7ae93ba41 in __GI___poll (fds=0x7fb7ae229140, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
29        return SYSCALL_CANCEL (poll, fds, nfds, timeout);
(gdb)
(gdb) p mysql_sysvar_version  #<-- 打印变量,查看GreatSQL版本号
$1 = {flags = 68101, name = 0x7f10d1c6cc90 "innodb_version", comment = 0x6c47f92 "InnoDB version", check = 0x37dd9e2
 , update = 0x37ddeb0 ,
  value = 0x7e7c768 , def_val = 0x6c38440 "⑧0.25-15"}
(gdb) 
(gdb) 
(gdb) b mysql_execute_command  #<--- 输入指令"b dispatch_command"设置断点,意为当GreatSQL程序运行到这个函数时,就会停下来
Breakpoint 3 at 0x379c3f2: file /opt/greatsql-⑧0.25-16/sql/sql_parse.cc, line 287⑤
(gdb)

mysql> select 'debug' from dual;

回到 终端#1,继续调试：

(gdb) 
(gdb) bt  #<-- 打印函数调用信息
#0  dispatch_command (thd=0x7f10a3a0b000, com_data=0x7f10d12a7370, command=COM_QUERY) at /opt/greatsql-⑧0.25-16/sql/sql_parse.cc:1605
#1  0x0000000003797c48 in do_command (thd=0x7f10a3a0b000) at /opt/greatsql-⑧0.25-16/sql/sql_parse.cc:1388
#2  0x0000000003991168 in handle_connection (arg=0x7f10d1f9d120) at /opt/greatsql-⑧0.25-16/sql/conn_handler/connection_handler_per_thread.cc:307
#3  0x00000000052e4b22 in pfs_spawn_thread (arg=0x7f10e8a45660) at /opt/greatsql-⑧0.25-16/storage/perfschema/pfs.cc:2899
#4  0x00007f10eb1e917a in start_thread (arg=) at pthread_create.c:479
#5  0x00007f10e9128dc3 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)
(gdb) p thd->m_query_string  #<-- 打印SQL语句
$14 = {str = 0x7f10a3a0e828 "select 'debug' from dual", length = 24}
(gdb)
(gdb) c  #<-- 继续执行,终端#2里被阻塞的SQL语句就可以执行了
Continuing.

mysql> select 'debug' from dual;
◆-------◆
| debug |
◆-------◆
| debug |
◆-------◆
1 row in set (12 min 11.55 sec)

Thread 39 "mysqld-debug" hit Breakpoint 1, dispatch_command (thd=0x7f10a3a0b000, com_data=0x7f10d12a7370, command=COM_QUIT)
at /opt/greatsql-⑧0.25-16/sql/sql_parse.cc:1605
1605      bool error = false;
(gdb)

如果不想继续跟踪调试了,只需输入指令 q 或 quit 即可退出gdb.

(gdb) quit
A debugging session is active.

    Inferior 1 [process 2644322] will be detached.

Quit anyway? (y or n) y
Detaching from program: /usr/local/GreatSQL-⑧0.25-16-Linux-glibc2.28-x86_64/bin/mysqld-debug, process 2644322
[Inferior 1 (process 2644322) detached]

③2 使用 Trace 文件调试

mysql> SET SESSION debug = 'debug_options';

变量 debug 支持多种设置模式：

debug_options = field_1:field_2:...:field_N
field = [◆|-]flag[,modifier,modifier,...,modifier]

◆, - 表示从当前debug值添加或者减少某些选项.

flag相关可选项如下：

使用案例1（精简模式）

# 设置debug选项
mysql> set session debug='d:t:o,/tmp/mysqld.trace';

# 执行SQL请求
mysql> select 'debug' from dual;
◆-------◆
| debug |
◆-------◆
| debug |
◆-------◆
1 row in set (0.00 sec)

查看生成的trace文件：

$ cat /tmp/mysqld.trace
...
>do_command
| >THD::clear_error
| Diagnostics_area::reset_diagnostics_area
| my_net_set_read_timeout
| | enter: timeout: 28800
| | >vio_socket_timeout
| | vio_is_blocking
| net_read_raw_loop
| | >vio_read
| | | >vio_is_blocking
| | | vio_io_wait
| | | 

mysql> set session debug='d:t:L:F:o,/tmp/mysqld.trace';
mysql> select 'debug' from dual;
...

$ cat /tmp/mysqld.trace
...
  sql_parse.cc: do_command
   sql_class.h:  3287: | >THD::clear_error
   sql_class.h: | Diagnostics_area::reset_diagnostics_area
  sql_error.cc: | my_net_set_read_timeout
   net_serv.cc:  2247: | | enter: timeout: 28800
  viosocket.cc:   380: | | >vio_socket_timeout
  viosocket.cc: | | vio_is_blocking
  viosocket.cc: | net_read_raw_loop
  viosocket.cc:   169: | | >vio_read
  viosocket.cc:   373: | | | >vio_is_blocking
  viosocket.cc: | | | vio_io_wait
  viosocket.cc: | | |