Skip to content

[refine](column) support const sources in column insertion - #67145

Open
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:support-const-column-insert
Open

[refine](column) support const sources in column insertion#67145
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:support-const-column-insert

Conversation

@Mryange

@Mryange Mryange commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

IColumn::insert_range_from() and insert_indices_from() required callers to pass the same concrete column type as the destination. A top-level ColumnConst source therefore had to be materialized or handled at each call site. Missing that handling could make paths such as MutableBlock::add_rows() cast a ColumnConst to the destination's concrete type and abort.

Root cause: constant-source handling was outside the common IColumn insertion contract. This change makes the public insertion methods non-virtual entry points that recognize a top-level ColumnConst source and append its single nested value through insert_many_from(). Concrete column implementations move behind insert_range_from_impl() and insert_indices_from_impl(), preserving their existing behavior for ordinary sources and const destinations. The duplicate handling in FullSorter is removed, while the SegmentIterator workaround from #67042 is intentionally retained.

Unit tests cover range and index insertion for numeric, string, nullable, and array constants, as well as both MutableBlock::add_rows() overloads.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: N/A

Problem Summary: MutableBlock and sort paths could receive top-level ColumnConst sources, but the virtual range and index insertion interfaces required concrete non-const columns. Callers therefore had to detect and expand constant columns manually, and some paths could fail with a bad cast. This change applies a non-virtual interface around the insertion methods, routes ColumnConst sources through insert_many_from(), and moves concrete implementations behind insert_range_from_impl() and insert_indices_from_impl().

### Release note

None

### Check List (For Author)

- Test: BE build with sh build.sh --be -j48; ASAN BE UT with ColumnConstTest.* and BlockTest.MutableBlockAddRowsSupportsConstNullableSource (19 tests passed)

- Behavior changed: No

- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange

Mryange commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 16881 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e08b7c0c6ca782dfd6d4d88e8c33f87a762d7d8e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17612	3136	3106	3106
q2	2110	269	228	228
q3	10211	885	519	519
q4	4670	245	201	201
q5	7688	568	386	386
q6	143	118	93	93
q7	541	505	403	403
q8	9241	985	885	885
q9	3501	2400	2390	2390
q10	6495	835	694	694
q11	394	191	181	181
q12	617	253	193	193
q13	18137	1532	1173	1173
q14	150	147	135	135
q15	q16	440	394	363	363
q17	1440	921	726	726
q18	3061	2250	2226	2226
q19	1290	956	823	823
q20	397	272	198	198
q21	5664	1725	1929	1725
q22	332	269	233	233
Total cold run time: 94134 ms
Total hot run time: 16881 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3538	3441	3429	3429
q2	518	381	366	366
q3	2283	2358	2266	2266
q4	1203	1175	896	896
q5	2155	2105	2085	2085
q6	168	124	85	85
q7	1071	930	845	845
q8	1611	1423	1428	1423
q9	3147	3133	3110	3110
q10	1834	1775	1604	1604
q11	370	273	248	248
q12	446	438	358	358
q13	1493	1546	1166	1166
q14	174	180	153	153
q15	q16	403	399	354	354
q17	3570	3269	3155	3155
q18	4774	4434	4724	4434
q19	944	910	872	872
q20	1001	955	833	833
q21	3838	3081	3218	3081
q22	393	357	319	319
Total cold run time: 34934 ms
Total hot run time: 31082 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 81831 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e08b7c0c6ca782dfd6d4d88e8c33f87a762d7d8e, data reload: false

query5	4280	411	323	323
query6	400	149	128	128
query7	4944	434	234	234
query8	308	119	114	114
query9	8664	2908	2894	2894
query10	390	219	185	185
query11	5402	1020	922	922
query12	121	70	69	69
query13	1184	440	329	329
query14	6006	2175	2072	2072
query14_1	1970	1958	1962	1958
query15	172	120	108	108
query16	938	384	353	353
query17	819	459	358	358
query18	2349	330	239	239
query19	166	140	113	113
query20	70	69	70	69
query21	203	104	90	90
query22	5488	5269	5252	5252
query23	6572	6282	5887	5887
query23_1	6053	6102	6106	6102
query24	7357	1085	783	783
query24_1	759	791	790	790
query25	446	320	268	268
query26	1246	253	134	134
query27	2752	415	261	261
query28	4690	1503	1494	1494
query29	950	446	347	347
query30	256	158	134	134
query31	832	404	332	332
query32	133	79	77	77
query33	477	222	183	183
query34	1049	825	497	497
query35	403	381	343	343
query36	565	596	528	528
query37	122	82	72	72
query38	1000	857	827	827
query39	497	477	491	477
query39_1	441	494	469	469
query40	214	93	78	78
query41	58	55	58	55
query42	75	71	73	71
query43	248	245	213	213
query44	1030	551	558	551
query45	111	109	99	99
query46	808	830	526	526
query47	751	734	712	712
query48	310	303	232	232
query49	546	255	224	224
query50	762	252	184	184
query51	8202	8340	8228	8228
query52	68	66	57	57
query53	194	211	151	151
query54	217	181	249	181
query55	79	60	55	55
query56	197	158	186	158
query57	690	662	651	651
query58	213	158	158	158
query59	1227	1232	1127	1127
query60	250	183	191	183
query61	119	138	118	118
query62	380	201	178	178
query63	170	138	145	138
query64	2799	665	603	603
query65	1628	1581	1549	1549
query66	1852	277	208	208
query67	9761	9516	9366	9366
query68	2760	1261	748	748
query69	367	240	196	196
query70	664	633	604	604
query71	246	175	165	165
query72	2355	1682	1490	1490
query73	661	563	334	334
query74	1565	1231	1131	1131
query75	1152	1100	957	957
query76	2295	742	515	515
query77	254	263	217	217
query78	3892	3733	3206	3206
query79	2121	808	562	562
query80	1406	332	284	284
query81	506	157	132	132
query82	642	117	96	96
query83	289	214	195	195
query84	297	113	90	90
query85	805	359	290	290
query86	430	181	174	174
query87	1006	978	900	900
query88	2779	2100	2096	2096
query89	279	198	173	173
query90	1993	125	129	125
query91	137	120	99	99
query92	79	67	68	67
query93	1415	1101	723	723
query94	679	248	218	218
query95	555	311	219	219
query96	809	596	276	276
query97	1007	1055	1014	1014
query98	143	141	136	136
query99	439	343	315	315
Total cold run time: 176832 ms
Total hot run time: 81831 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 14.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e08b7c0c6ca782dfd6d4d88e8c33f87a762d7d8e, data reload: false

query1	0.00	0.00	0.01
query2	0.07	0.03	0.03
query3	0.25	0.11	0.11
query4	1.60	0.10	0.10
query5	0.18	0.17	0.17
query6	1.25	0.72	0.70
query7	0.03	0.00	0.01
query8	0.05	0.03	0.02
query9	0.29	0.21	0.22
query10	0.34	0.35	0.33
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.31	0.31	0.29
query14	0.47	0.45	0.44
query15	0.37	0.35	0.34
query16	0.23	0.23	0.23
query17	0.66	0.72	0.66
query18	0.18	0.17	0.16
query19	1.13	1.19	1.19
query20	0.01	0.01	0.01
query21	15.43	0.15	0.12
query22	5.08	0.04	0.05
query23	16.18	0.25	0.10
query24	3.03	0.32	0.25
query25	0.11	0.04	0.04
query26	0.79	0.16	0.12
query27	0.04	0.03	0.02
query28	3.68	0.60	0.27
query29	12.48	3.12	2.56
query30	0.25	0.11	0.13
query31	2.75	0.37	0.17
query32	3.54	0.32	0.23
query33	1.34	1.53	1.41
query34	15.36	2.19	1.77
query35	1.74	1.69	1.69
query36	0.46	0.29	0.29
query37	0.06	0.04	0.04
query38	0.05	0.03	0.03
query39	0.03	0.03	0.02
query40	0.12	0.08	0.08
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 90.41 s
Total hot run time: 14.64 s

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 93.33% (14/15) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 65.29% (29525/45223)
Line Coverage 50.05% (317914/635196)
Region Coverage 45.90% (261074/568814)
Branch Coverage 47.08% (120447/255810)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants