[Btree index] The splits can not be filtered for transactional partition table

Software Environment:

openLooKeng version (source or binary):1.1.0
OS platform & distribution (eg., Linux Ubuntu 16.04):Linux
Java version:1.8

Describe the current behavior

Describe the expected behavior

Steps to reproduce the issue

1.Create table

CREATE TABLE hive.index_test.testorders4 (
orderkey bigint,
orderstatus varchar,
totalprice double,
orderdate date
) WITH (partitioned_by = ARRAY['orderdate'], transactional = true);

2.Insert data into table
输入图片说明

3.Create btree index

create index idx1 using btree on hive.index_test.testorders4(orderkey) with (level=partition);

4.Query data 2 times

select * from hive.index_test.testorders4 where orderkey=10;

【Result】

Step4 the split do not reduce，and there throw some exception from server.log
输入图片说明

From the log, it may not find the index file in HDFS.

Read index from HDFS
输入图片说明

Related log/screenshots

Special notes for this issue

Test Version

openlooKeng1.4.0

Env

2CN+3WN

Test step

lk:test> CREATE TABLE testorders4 (
      -> orderkey bigint,
      -> orderstatus varchar,
      -> totalprice double,
      -> orderdate date
      -> ) WITH (partitioned_by = ARRAY['orderdate'], transactional = true);
CREATE TABLE
lk:test> insert into testorders4 values(11,varchar'BAD',double'100.15',date'2019-05-20');
INSERT: 1 row

Query 20211018_062106_00095_4rik7, FINISHED, 5 nodes
Splits: 102 total, 102 done (100.00%)
0:02 [0 rows, 0B] [0 rows/s, 0B/s]

lk:test> insert into testorders4 values(10,varchar'SUCCESS',double'125.15',date'2019-05-17');
INSERT: 1 row

Query 20211018_062137_00096_4rik7, FINISHED, 5 nodes
Splits: 102 total, 102 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]

lk:test> insert into testorders4 values(11,varchar'BAD',double'100.15',date'2019-05-20');
INSERT: 1 row

Query 20211018_062151_00097_4rik7, FINISHED, 5 nodes
Splits: 102 total, 102 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]

lk:test> insert into testorders4 values(10,varchar'SUCCESS',double'125.15',date'2019-05-17');
INSERT: 1 row

Query 20211018_062159_00098_4rik7, FINISHED, 5 nodes
Splits: 102 total, 102 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]

lk:test> insert into testorders4 values(11,varchar'BAD',double'100.15',date'2019-05-20');
INSERT: 1 row

Query 20211018_062201_00099_4rik7, FINISHED, 5 nodes
Splits: 102 total, 102 done (100.00%)
0:01 [0 rows, 0B] [0 rows/s, 0B/s]

lk:test> insert into testorders4 values(11,varchar'BAD',double'100.15',date'2019-05-20');
INSERT: 1 row

Query 20211018_062205_00100_4rik7, FINISHED, 5 nodes
Splits: 102 total, 102 done (100.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]

lk:test> SELECT * from testorders4;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       11 | BAD         |     100.15 | 2019-05-20
       11 | BAD         |     100.15 | 2019-05-20
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
       11 | BAD         |     100.15 | 2019-05-20
       11 | BAD         |     100.15 | 2019-05-20
(6 rows)

Query 20211018_062214_00101_4rik7, FINISHED, 4 nodes
Splits: 22 total, 22 done (100.00%)
0:00 [6 rows, 6.93KB] [16 rows/s, 19.1KB/s]
lk:test> select * from testorders4 where orderkey=10;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
(2 rows)

Query 20211018_063122_00007_8zqxi, FINISHED, 4 nodes
Splits: 22 total, 22 done (100.00%)
0:01 [2 rows, 2.35KB] [2 rows/s, 3.28KB/s]

lk:test> select * from testorders4 where orderkey=10;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
(2 rows)

Query 20211018_063131_00008_8zqxi, FINISHED, 4 nodes
Splits: 22 total, 22 done (100.00%)
0:00 [2 rows, 2.35KB] [7 rows/s, 8.73KB/s]

lk:test> select * from testorders4 where orderkey=10;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
(2 rows)

Query 20211018_063136_00009_8zqxi, FINISHED, 2 nodes
Splits: 18 total, 18 done (100.00%)
0:00 [2 rows, 2.35KB] [7 rows/s, 8.51KB/s]

lk:test> select * from testorders4 where orderkey=10;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
(2 rows)

Query 20211018_063138_00010_8zqxi, FINISHED, 3 nodes
Splits: 18 total, 18 done (100.00%)
0:00 [2 rows, 2.35KB] [10 rows/s, 11.8KB/s]
lk:test> set session heuristicindex_filter_enabled=false;
SET SESSION
lk:test> select * from testorders4 where orderkey=10;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
(2 rows)

Query 20211018_063202_00016_8zqxi, FINISHED, 4 nodes
Splits: 22 total, 22 done (100.00%)
0:00 [2 rows, 2.35KB] [5 rows/s, 6.49KB/s]

lk:test> select * from testorders4 where orderkey=10;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
(2 rows)

Query 20211018_063204_00017_8zqxi, FINISHED, 4 nodes
Splits: 22 total, 22 done (100.00%)
0:00 [2 rows, 2.35KB] [12 rows/s, 14.5KB/s]

lk:test> select * from testorders4 where orderkey=10;
 orderkey | orderstatus | totalprice | orderdate
----------+-------------+------------+------------
       10 | SUCCESS     |     125.15 | 2019-05-17
       10 | SUCCESS     |     125.15 | 2019-05-17
(2 rows)

Query 20211018_063205_00018_8zqxi, FINISHED, 4 nodes
Splits: 22 total, 22 done (100.00%)
0:00 [2 rows, 2.35KB] [13 rows/s, 15.6KB/s]

Test Result

bTree index reduce splits number.

Conclusion

Regression Passed

openLooKeng / hetu-core

内容风险标识