解决 MySQL 分页数据错乱重复

前言

一天，小明很着急地在通讯工具上说：这边线上出现了个奇怪的问题，麻烦 DBA 大大鉴定下，执行语句 select xx from table_name wheere xxx order by 字段A limit offset;，
表数据总共 48 条，分页数量正常，但出现了结果混杂的情况，第一页的数据出现在了第二页上；如果 order by 字段B 就不会出现这个现象，怎么会这样呢！

其实，这个问题很简单，如果你有仔细阅读官档的话。~^_^~

我们先来看看官档是怎么说的：

If multiple rows have identical values in the ORDER BY columns, the server is free to return those rows in any order, and may do so differently depending on the overall execution plan. In other words, the sort order of those rows is nondeterministic with respect to the nonordered columns.

One factor that affects the execution plan is LIMIT, so an ORDER BY query with and without LIMIT may return rows in different orders.

问题重现

本次实验使用社区版 MySQL 5.6.26（因为小明出现问题的环境就是这个版本O(∩_∩)O~），下面先创建实验环境和初始化测试数据：

root@localhost [(none)]>select @@version;+------------+| @@version  |+------------+| 5.6.26-log |+------------+1 row in set (0.00 sec)root@localhost [(none)]>show variables like "sql_mode";+---------------+------------------------+| Variable_name | Value                  |+---------------+------------------------+| sql_mode      | NO_ENGINE_SUBSTITUTION |+---------------+------------------------+1 row in set (0.00 sec)root@localhost [(none)]>create database glon_ho;Query OK, 1 row affected (0.04 sec)root@localhost [(none)]>use glon_hoDatabase changedroot@localhost [glon_ho]>create table glon(      ->     id int not null auto_increment primary key,      ->     name varchar(20) not null,      ->     create_time datetime not null,      ->     age tinyint unsigned default 18      -> );Query OK, 0 rows affected (0.01 sec)root@localhost [glon_ho]>INSERT INTO `glon` VALUES (1, 'Eason Chan', '2017-05-02 08:10:10', 19),(2, 'Glon Ho', '2017-05-03 12:10:10', 18),(3, '赵敏', '2017-05-03 14:10:10', 17),(4, 'Jacky Cheung', '2017-05-02 14:00:00', 22),(5, '周芷若', '2017-05-02 14:00:00', 16),(6, 'Andy Lau', '2017-05-02 14:00:00', 50),(7, '至尊宝', '2017-05-02 14:00:00', 20),(8, '刘三姐', '2017-05-02 14:00:00', 19);Query OK, 8 rows affected (0.01 sec)Records: 8  Duplicates: 0  Warnings: 0root@localhost [glon_ho]>select * from glon;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  1 | Eason Chan   | 2017-05-02 08:10:10 |   19 ||  2 | Glon Ho      | 2017-05-03 12:10:10 |   18 ||  3 | 赵敏         | 2017-05-03 14:10:10 |   17 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  5 | 周芷若       | 2017-05-02 14:00:00 |   16 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 ||  7 | 至尊宝       | 2017-05-02 14:00:00 |   20 ||  8 | 刘三姐       | 2017-05-02 14:00:00 |   19 |+----+--------------+---------------------+------+8 rows in set (0.00 sec)

这里创建了一个 glon 表，字段有自增 id, 姓名 name, 年龄 age, 及用户注册时间 create_time。

接着来复现问题：

• 根据用户注册时间 create_time 来排序：

root@localhost [glon_ho]>select * from glon ORDER BY create_time limit 0, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  1 | Eason Chan   | 2017-05-02 08:10:10 |   19 ||  8 | 刘三姐       | 2017-05-02 14:00:00 |   19 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time limit 4, 4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+|  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 ||  2 | Glon Ho   | 2017-05-03 12:10:10 |   18 ||  3 | 赵敏      | 2017-05-03 14:10:10 |   17 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)

可以看到两次查询结果中都出现了 id 为 8 的刘三姐，从上面初始化数据来看，总共有 8 条数据，现在不但分页出现重复数据，还丢了一条！

问题确实重现了，不过先不急，我们再来试多几组其他的排序方式。

• create_time 和 age 组合排序

root@localhost [glon_ho]>select * from glon ORDER BY create_time,age limit 0, 4;+----+------------+---------------------+------+| id | name       | create_time         | age  |+----+------------+---------------------+------+|  1 | Eason Chan | 2017-05-02 08:10:10 |   19 ||  5 | 周芷若     | 2017-05-02 14:00:00 |   16 ||  8 | 刘三姐     | 2017-05-02 14:00:00 |   19 ||  7 | 至尊宝     | 2017-05-02 14:00:00 |   20 |+----+------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time,age limit 4, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 ||  2 | Glon Ho      | 2017-05-03 12:10:10 |   18 ||  3 | 赵敏         | 2017-05-03 14:10:10 |   17 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)

• create_time 和 id 组合排序

root@localhost [glon_ho]>select * from glon ORDER BY create_time,id limit 0, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  1 | Eason Chan   | 2017-05-02 08:10:10 |   19 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  5 | 周芷若       | 2017-05-02 14:00:00 |   16 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time,id limit 4, 4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+|  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 ||  2 | Glon Ho   | 2017-05-03 12:10:10 |   18 ||  3 | 赵敏      | 2017-05-03 14:10:10 |   17 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)

• 主键 id 排序

root@localhost [glon_ho]>select * from glon ORDER BY id limit 0, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  1 | Eason Chan   | 2017-05-02 08:10:10 |   19 ||  2 | Glon Ho      | 2017-05-03 12:10:10 |   18 ||  3 | 赵敏         | 2017-05-03 14:10:10 |   17 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY id limit 4, 4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+|  5 | 周芷若    | 2017-05-02 14:00:00 |   16 ||  6 | Andy Lau  | 2017-05-02 14:00:00 |   50 ||  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)

看到，后面的几组排序方式都没有再出现问题了，结合官档，我们知道 order by 排序的时候，如果排序字段中有多行相同的列值，则排序结果是不确定的。所以后面的几组组合形式的排序或者是主键 id 的排序，因为唯一性高，所以排序是确定的，不会出现结果混乱的问题。

那是不是可以就此结束了呢，no way， 我们再来看下面的实验，继续巩固一下：

• 根据年龄 age 来排序：

root@localhost [glon_ho]>select * from glon ORDER BY age limit 0, 4;+----+------------+---------------------+------+| id | name       | create_time         | age  |+----+------------+---------------------+------+|  5 | 周芷若     | 2017-05-02 14:00:00 |   16 ||  3 | 赵敏       | 2017-05-03 14:10:10 |   17 ||  2 | Glon Ho    | 2017-05-03 12:10:10 |   18 ||  1 | Eason Chan | 2017-05-02 08:10:10 |   19 |+----+------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY age limit 4, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  8 | 刘三姐       | 2017-05-02 14:00:00 |   19 ||  7 | 至尊宝       | 2017-05-02 14:00:00 |   20 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)

咦，这个排序也只是根据一个字段 age 来排序，怎么就没有出问题呢？不急，还有招：

root@localhost [glon_ho]>insert into glon values (9,'乔峰','2017-05-03 13:10:10',22),(10,'段誉','2017-05-03 15:10:10',19),(11,'郭靖','2017-05-03 17:10:10',20),(12,'黄蓉','2017-05-03 08:10:10',19);Query OK, 4 rows affected (0.01 sec)Records: 4  Duplicates: 0  Warnings: 0root@localhost [glon_ho]>select * from glon;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  1 | Eason Chan   | 2017-05-02 08:10:10 |   19 ||  2 | Glon Ho      | 2017-05-03 12:10:10 |   18 ||  3 | 赵敏         | 2017-05-03 14:10:10 |   17 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  5 | 周芷若       | 2017-05-02 14:00:00 |   16 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 ||  7 | 至尊宝       | 2017-05-02 14:00:00 |   20 ||  8 | 刘三姐       | 2017-05-02 14:00:00 |   19 ||  9 | 乔峰         | 2017-05-03 13:10:10 |   22 || 10 | 段誉         | 2017-05-03 15:10:10 |   19 || 11 | 郭靖         | 2017-05-03 17:10:10 |   20 || 12 | 黄蓉         | 2017-05-03 08:10:10 |   19 |+----+--------------+---------------------+------+12 rows in set (0.00 sec)

我又给 glon 表新增了几条数据，然后再来看看：

root@localhost [glon_ho]>select * from glon ORDER BY create_time limit 0, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  1 | Eason Chan   | 2017-05-02 08:10:10 |   19 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  5 | 周芷若       | 2017-05-02 14:00:00 |   16 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time limit 4, 4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+|  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 || 12 | 黄蓉      | 2017-05-03 08:10:10 |   19 ||  2 | Glon Ho   | 2017-05-03 12:10:10 |   18 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time limit 8, 4;+----+--------+---------------------+------+| id | name   | create_time         | age  |+----+--------+---------------------+------+|  9 | 乔峰   | 2017-05-03 13:10:10 |   22 ||  3 | 赵敏   | 2017-05-03 14:10:10 |   17 || 10 | 段誉   | 2017-05-03 15:10:10 |   19 || 11 | 郭靖   | 2017-05-03 17:10:10 |   20 |+----+--------+---------------------+------+4 rows in set (0.00 sec)

根据 create_time 排序，没有问题了，再来：

root@localhost [glon_ho]>select * from glon ORDER BY age limit 0, 4;+----+------------+---------------------+------+| id | name       | create_time         | age  |+----+------------+---------------------+------+|  5 | 周芷若     | 2017-05-02 14:00:00 |   16 ||  3 | 赵敏       | 2017-05-03 14:10:10 |   17 ||  2 | Glon Ho    | 2017-05-03 12:10:10 |   18 ||  1 | Eason Chan | 2017-05-02 08:10:10 |   19 |+----+------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY age limit 4, 4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+| 12 | 黄蓉      | 2017-05-03 08:10:10 |   19 || 10 | 段誉      | 2017-05-03 15:10:10 |   19 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 ||  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY age limit 8, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  7 | 至尊宝       | 2017-05-02 14:00:00 |   20 ||  9 | 乔峰         | 2017-05-03 13:10:10 |   22 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)

可以看到根据年龄 age 排序，问题出现了。

然后在看看组合的排序：

root@localhost [glon_ho]>select * from glon ORDER BY create_time,id limit 0, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  1 | Eason Chan   | 2017-05-02 08:10:10 |   19 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  5 | 周芷若       | 2017-05-02 14:00:00 |   16 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time,id limit 4, 4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+|  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 || 12 | 黄蓉      | 2017-05-03 08:10:10 |   19 ||  2 | Glon Ho   | 2017-05-03 12:10:10 |   18 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time,id limit 8, 4;+----+--------+---------------------+------+| id | name   | create_time         | age  |+----+--------+---------------------+------+|  9 | 乔峰   | 2017-05-03 13:10:10 |   22 ||  3 | 赵敏   | 2017-05-03 14:10:10 |   17 || 10 | 段誉   | 2017-05-03 15:10:10 |   19 || 11 | 郭靖   | 2017-05-03 17:10:10 |   20 |+----+--------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time,age limit 0, 4;+----+------------+---------------------+------+| id | name       | create_time         | age  |+----+------------+---------------------+------+|  1 | Eason Chan | 2017-05-02 08:10:10 |   19 ||  5 | 周芷若     | 2017-05-02 14:00:00 |   16 ||  8 | 刘三姐     | 2017-05-02 14:00:00 |   19 ||  7 | 至尊宝     | 2017-05-02 14:00:00 |   20 |+----+------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time,age limit 4, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 || 12 | 黄蓉         | 2017-05-03 08:10:10 |   19 ||  2 | Glon Ho      | 2017-05-03 12:10:10 |   18 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY create_time,age limit 8, 4;+----+--------+---------------------+------+| id | name   | create_time         | age  |+----+--------+---------------------+------+|  9 | 乔峰   | 2017-05-03 13:10:10 |   22 ||  3 | 赵敏   | 2017-05-03 14:10:10 |   17 || 10 | 段誉   | 2017-05-03 15:10:10 |   19 || 11 | 郭靖   | 2017-05-03 17:10:10 |   20 |+----+--------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY age,id limit 0, 4;+----+------------+---------------------+------+| id | name       | create_time         | age  |+----+------------+---------------------+------+|  5 | 周芷若     | 2017-05-02 14:00:00 |   16 ||  3 | 赵敏       | 2017-05-03 14:10:10 |   17 ||  2 | Glon Ho    | 2017-05-03 12:10:10 |   18 ||  1 | Eason Chan | 2017-05-02 08:10:10 |   19 |+----+------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY age,id limit 4, 4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+|  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 || 10 | 段誉      | 2017-05-03 15:10:10 |   19 || 12 | 黄蓉      | 2017-05-03 08:10:10 |   19 ||  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon ORDER BY age,id limit 8, 4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+| 11 | 郭靖         | 2017-05-03 17:10:10 |   20 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  9 | 乔峰         | 2017-05-03 13:10:10 |   22 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)

思考

既然排序不定，那么给排序字段加上索引会不会有用呢？

root@localhost [glon_ho]>alter table glon add index ix_age(age);Query OK, 0 rows affected (0.02 sec)Records: 0  Duplicates: 0  Warnings: 0root@localhost [glon_ho]>show create table glon\G*************************** 1. row ***************************       Table: glonCreate Table: CREATE TABLE `glon` (  `id` int(11) NOT NULL AUTO_INCREMENT,  `name` varchar(20) NOT NULL,  `create_time` datetime NOT NULL,  `age` tinyint(3) unsigned DEFAULT '18',  PRIMARY KEY (`id`),  KEY `ix_age` (`age`)) ENGINE=InnoDB AUTO_INCREMENT=13 DEFAULT CHARSET=utf81 row in set (0.00 sec)root@localhost [glon_ho]>select * from glon order by age limit 0,4;+----+------------+---------------------+------+| id | name       | create_time         | age  |+----+------------+---------------------+------+|  5 | 周芷若     | 2017-05-02 14:00:00 |   16 ||  3 | 赵敏       | 2017-05-03 14:10:10 |   17 ||  2 | Glon Ho    | 2017-05-03 12:10:10 |   18 ||  1 | Eason Chan | 2017-05-02 08:10:10 |   19 |+----+------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon order by age limit 4,4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+| 12 | 黄蓉      | 2017-05-03 08:10:10 |   19 || 10 | 段誉      | 2017-05-03 15:10:10 |   19 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 ||  7 | 至尊宝    | 2017-05-02 14:00:00 |   20 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from glon order by age limit 8,4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  7 | 至尊宝       | 2017-05-02 14:00:00 |   20 ||  9 | 乔峰         | 2017-05-03 13:10:10 |   22 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)

也可以不在 order by 后面加多一列增加唯一性，可以改写成下面的形式：

root@localhost [glon_ho]>select * from (select distinct g.* from glon g order by age) t limit 0,4;+----+------------+---------------------+------+| id | name       | create_time         | age  |+----+------------+---------------------+------+|  5 | 周芷若     | 2017-05-02 14:00:00 |   16 ||  3 | 赵敏       | 2017-05-03 14:10:10 |   17 ||  2 | Glon Ho    | 2017-05-03 12:10:10 |   18 ||  1 | Eason Chan | 2017-05-02 08:10:10 |   19 |+----+------------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from (select distinct g.* from glon g order by age) t limit 4,4;+----+-----------+---------------------+------+| id | name      | create_time         | age  |+----+-----------+---------------------+------+| 10 | 段誉      | 2017-05-03 15:10:10 |   19 ||  8 | 刘三姐    | 2017-05-02 14:00:00 |   19 || 12 | 黄蓉      | 2017-05-03 08:10:10 |   19 || 11 | 郭靖      | 2017-05-03 17:10:10 |   20 |+----+-----------+---------------------+------+4 rows in set (0.00 sec)root@localhost [glon_ho]>select * from (select distinct g.* from glon g order by age) t limit 8,4;+----+--------------+---------------------+------+| id | name         | create_time         | age  |+----+--------------+---------------------+------+|  7 | 至尊宝       | 2017-05-02 14:00:00 |   20 ||  9 | 乔峰         | 2017-05-03 13:10:10 |   22 ||  4 | Jacky Cheung | 2017-05-02 14:00:00 |   22 ||  6 | Andy Lau     | 2017-05-02 14:00:00 |   50 |+----+--------------+---------------------+------+4 rows in set (0.00 sec)

总之，如果发生了，最简单的方法就是在排序列（如 create time）上加索引，然后在 order by 上明示 primary key，这个问题就非常圆满的解决了。