[Bug] jdbc insert into value 每个批次记录数太大会报错

[bw-chuxin]

Search before asking

• I had searched in the issues and found no similar issues.

Version

2.1.2

What's Wrong?

jdbc批量写入开启useServerPrepStmts=true和rewriteBatchedStatements=true后如果批量写入的参数总数超过65535,jdbc驱动会报如下错误

java.lang.ArrayIndexOutOfBoundsException: 2464
	at com.mysql.cj.NativeQueryBindings.getBinding(NativeQueryBindings.java:191)
	at com.mysql.cj.NativeQueryBindings.setFromBindValue(NativeQueryBindings.java:198)
	at com.mysql.cj.jdbc.ClientPreparedStatement.setOneBatchedParameterSet(ClientPreparedStatement.java:591)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchWithMultiValuesClause(ClientPreparedStatement.java:675)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchInternal(ClientPreparedStatement.java:409)
	at com.mysql.cj.jdbc.StatementImpl.executeBatch(StatementImpl.java:795)
	at com.dlink.App.main(App.java:44)

实际原因是 doris服务端在处理COM_STMT_PREPARE命令时会从sql中解析出参数的个数，在应答的时候会将参数个数返回去，由于应答中报文中参数个数的的长度时2个字节，当参数个数超过65535时，数据会溢出，导致jdbc驱动接收到的长度不是真实的长度，所以jdbc构造出的NativeQueryBindings长度不合法导致操作报错。

What You Expected?

应该给出一个更加合理的提示，目前这个报错不太友好，会误导使用者。

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[cambyzju]

能提供一个，可以直接复制粘贴就能使用的复现步骤吗？

[bw-chuxin]

doris建表语句

CREATE TABLE `test1` (
  `id` VARCHAR(32) NOT NULL,
  `dept_id` VARCHAR(32) NOT NULL,
  `name` VARCHAR(100) NULL,
  `create_time` DATETIME NULL
) ENGINE=OLAP
UNIQUE KEY(`id`, `dept_id`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`id`) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
); 

mysql 驱动

<dependency>
      <groupId>com.mysql</groupId>
      <artifactId>mysql-connector-j</artifactId>
      <version>8.0.31</version>
    </dependency>

测试java代码
jdbcurl

jdbc:mysql://ip:端口/数据库?useServerPrepStmts=true&useLocalSessionState=true&rewriteBatchedStatements=true&cachePrepStmts=true&prepStmtCacheSqlLimit=9999&prepStmtCacheSize=9999999

public static void main( String[] args ) throws SQLException, IOException {
        String jdbcUrl = args[0];
        String username = args[1];
        String password = args[2];

        Properties props = new Properties();
        props.put("user", username);
        props.put("password", password);


        try (Connection conn = DriverManager.getConnection(jdbcUrl, props)) {
            conn.createStatement().execute("set group_commit = async_mode;");

            String query = "insert into test1" + " values(?, ?, ?,?)";
            try (PreparedStatement stmt = conn.prepareStatement(query)) {
                for (int j = 0; j < 10; j++) {
                    long start = System.currentTimeMillis();
                    // 10 rows per insert
                    for (int i = 0; i < 17000 + j; i++) {
                        stmt.setInt(1, i);
                        stmt.setString(2, i +"");
                        stmt.setString(3, "name" + i);
                        stmt.setTimestamp(4, Timestamp.valueOf(LocalDateTime.now()));
                        stmt.addBatch();
                    }
                    int[] result = stmt.executeBatch();
                    long end = System.currentTimeMillis();
                    System.out.println("insert " + result.length + " rows" + " cost time: " + (end - start) + "ms");
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

报错信息

java.lang.ArrayIndexOutOfBoundsException: 2464
	at com.mysql.cj.NativeQueryBindings.getBinding(NativeQueryBindings.java:191)
	at com.mysql.cj.NativeQueryBindings.setFromBindValue(NativeQueryBindings.java:198)
	at com.mysql.cj.jdbc.ClientPreparedStatement.setOneBatchedParameterSet(ClientPreparedStatement.java:591)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchWithMultiValuesClause(ClientPreparedStatement.java:675)
	at com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchInternal(ClientPreparedStatement.java:409)
	at com.mysql.cj.jdbc.StatementImpl.executeBatch(StatementImpl.java:795)
	at com.dlink.App.main(App.java:44)