seata 提交全局事务失败 Unable to commit against JDBC Connection

• 前言 

将seata部署到生产环境中出现问题，业务代码正常执行结束后，应该提交全局事务，但是日志却打出 rollback status: Rollbacked。简略的异常信息如下

15:48:13.947 [http-nio2-8821-exec-70] INFO  i.s.t.a.DefaultGlobalTransaction - [rollback,178] - [10.000.000.00:8091:93009917309542400] rollback status: Rollbacked
15:48:13.950 [http-nio2-8821-exec-70] ERROR c.x.c.j.r.GloablExceptionHandler - [handleException,19] - 服务器异常
org.springframework.orm.jpa.JpaSystemException: Unable to commit against JDBC Connection; nested exception is org.hibernate.TransactionException: Unable to commit against JDBC Connection
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:351)
        at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:253)
        at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:98)
        at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:532)
        ... 85 common frames omitted
Caused by: io.seata.rm.datasource.exec.LockConflictException: get global lock fail, xid:10.000.00.00:8091:93009917309542400, lockKeys:scan_code_record:7df68bd5ea30425abb8022bde5f0e296;sold_sc_code:cadf510822f244169927f0c7fa0ecd30;sold_receipt:16c74a3a11de4f84b12c1a6cec13a1d6;scan_code_product:e0c05a6a891f41c7853cc77b6237003b;sold_note:4ae09914f1b34fde9b8ae02c1fb5aa5f
        at io.seata.rm.datasource.ConnectionProxy.recognizeLockKeyConflictException(ConnectionProxy.java:155)
        at io.seata.rm.datasource.ConnectionProxy.processGlobalTransactionCommit(ConnectionProxy.java:221)
        at io.seata.rm.datasource.ConnectionProxy.doCommit(ConnectionProxy.java:199)
        at io.seata.rm.datasource.ConnectionProxy.lambda$commit$0(ConnectionProxy.java:184)
        at io.seata.rm.datasource.ConnectionProxy$LockRetryPolicy.execute(ConnectionProxy.java:292)
        at io.seata.rm.datasource.ConnectionProxy.commit(ConnectionProxy.java:183)
        at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:81)
        ... 88 common frames omitted

•  解决过程：

看到日志中的出现“jpa”相关的信息，但是项目中没有使用任何JPA的框架，所以排除项目自身的问题。

因为引入了seata框架，再加上最后的异常信息也指向了seata，群友讨论怀疑是竞争锁导致的。所以修改seata服务的参数 

client.rm.lock.retryInterval 20   校验或占用全局锁重试间隔    默认10，单位毫秒
client.rm.lock.retryTimes  60   校验或占用全局锁重试次数    默认30

然后重启应用，依旧出现刚刚的异常。

想起查看一下 seata的日志，默认在 home/logs 下面

2021-01-14 16:38:28.243 ERROR --- [  ServerHandlerThread_1_19_500] i.s.s.s.db.lock.LockStoreDataBaseDAO     : Global lock batch acquire error: Data truncation: Data too long for column 'row_key' at row 1
==>
java.sql.BatchUpdateException: Data truncation: Data too long for column 'row_key' at row 1
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.cj.util.Util.handleNewInstance(Util.java:192)
        at com.mysql.cj.util.Util.getInstance(Util.java:167)
        at com.mysql.cj.util.Util.getInstance(Util.java:174)

发现出现了 row_key超长的报错信息。查看表结构 

-- the table to store lock data
CREATE TABLE IF NOT EXISTS `lock_table`
(
    `row_key`        VARCHAR(128) NOT NULL,
    `xid`            VARCHAR(96),
    `transaction_id` BIGINT,
    `branch_id`      BIGINT       NOT NULL,
    `resource_id`    VARCHAR(256),
    `table_name`     VARCHAR(32),
    `pk`             VARCHAR(36),
    `gmt_create`     DATETIME,
    `gmt_modified`   DATETIME,
    PRIMARY KEY (`row_key`),
    KEY `idx_branch_id` (`branch_id`)
) ENGINE = InnoDB
  DEFAULT CHARSET = utf8;

使用官方提供的sql建表，按说不应该出现问题。当时异常信息如此明显，于是尝试将长度改为255。

然后再次执行方法，问题解决。

• 深扒原理

扒seata源码，找到 row_key的生成策略如下

protected List<LockDO> convertToLockDO(List<RowLock> locks) {
        List<LockDO> lockDOs = new ArrayList<>();
        if (CollectionUtils.isEmpty(locks)) {
            return lockDOs;
        }
        for (RowLock rowLock : locks) {
            LockDO lockDO = new LockDO();
            lockDO.setBranchId(rowLock.getBranchId());
            lockDO.setPk(rowLock.getPk());
            lockDO.setResourceId(rowLock.getResourceId());
            lockDO.setRowKey(getRowKey(rowLock.getResourceId(), rowLock.getTableName(), rowLock.getPk()));
            lockDO.setXid(rowLock.getXid());
            lockDO.setTransactionId(rowLock.getTransactionId());
            lockDO.setTableName(rowLock.getTableName());
            lockDOs.add(lockDO);
        }
        return lockDOs;
    }

因为 row_key 的生成策略是 资源名 + 表名 + 主键，比如

jdbc:mysql://116.62.62.62/seata-storage

生成环境使用连接比较长，再拼接上 表名 + 主键 所以出现了 row_key 超长的情况。