Equality on an ARRAY-typed field means CONTAINS

[Bukhtawar]

Description

Registers EQUAL/NOTEQUAL overloads for [ARRAY<T>, scalar] argument pairs in PPLFuncImpTable: tags = 'x' on an ARRAY-typed column now resolves to Calcite's ARRAY_CONTAINS(tags, 'x') (element equality), and != to NOT(ARRAY_CONTAINS(...)).

Why. This gives PPL the same semantics a Lucene term query has on a multi-valued field: a document matches when any value equals the term. Columnar storage engines surface multi-valued fields as ARRAY-typed columns — e.g. the OpenSearch composite engine's Parquet format, where a keyword field mapped multi_value: true is stored as a LIST<element> column (opensearch-project/OpenSearch#22685). Previously the analyzer rejected the comparison outright:

EQUAL function expects {[IP,IP],[COMPARABLE_TYPE,COMPARABLE_TYPE]}, but got [ARRAY,STRING]

Design notes.

• The overloads are registered before the scalar comparison overloads so an [ARRAY, scalar] pair resolves to contains; all scalar comparisons are untouched.
• The new ARRAY_ELEMENT_COMPARABLE type checker matches only when arg0 is ARRAY, arg1 is not an ARRAY (array-to-array equality stays unsupported rather than silently meaning overlap), and the element type is comparable with the value under the same rules scalar comparisons use — PPLComparableTypeChecker.isComparable, widened from private to package-private rather than duplicated.
• ARRAY_CONTAINS is element equality, not regex — mvfind (unanchored regex) is not a substitute for term equality.
• Engine-executability: DataFusion executes this natively as array_has; the Calcite Enumerable path uses Calcite's own ARRAY_CONTAINS implementation.

Verification. Unit test asserts resolve(EQUAL, ARRAY<VARCHAR> ref, 'alpha') produces ARRAY_CONTAINS($0, 'alpha'). Verified end-to-end against a live OpenSearch composite-engine cluster (Parquet LIST column → PPL where tags = 'alpha' → DataFusion array_has): contains-match, exact element equality (no substring match), and != as NOT(contains) with three-valued-logic null exclusion. The consuming integration test lives in the OpenSearch PR above (MultiValueFieldIT.testEqualsOnMultiValueColumnMeansContains) and is muted there until this change ships in the published snapshot.

Related Issues

Companion to opensearch-project/OpenSearch#22685 (multi-value keyword fields in the composite engine).

Check List

• New functionality includes testing.
• New functionality has been documented.
• Commits are signed per the DCO using --signoff or -s.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ Recommended focus areas for review Debug Output The test contains System.out.println statements on lines 29-30 and 34. These debug statements should be removed before merging to production code. They will clutter test output and serve no purpose in a committed test. System.out.println("array type = " + arrayRef.getType().getSqlTypeName()); System.out.println("literal type = " + literal.getType().getSqlTypeName()); RexNode resolved = PPLFuncImpTable.INSTANCE.resolve(builder, BuiltinFunctionName.EQUAL, arrayRef, literal); System.out.println("resolved = " + resolved);

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Handle commutative equality for arrays The ARRAY_ELEMENT_COMPARABLE checker only validates [ARRAY, scalar] but doesn't handle the reversed case [scalar, ARRAY]. Equality is commutative, so value = array_field should also resolve to ARRAY_CONTAINS. Consider adding a symmetric overload or making the checker bidirectional. core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java [985-990] register( EQUAL, (FunctionImp2) (builder, array, value) -> builder.makeCall(SqlLibraryOperators.ARRAY_CONTAINS, array, value), ARRAY_ELEMENT_COMPARABLE); +register( + EQUAL, + (FunctionImp2) + (builder, value, array) -> + builder.makeCall(SqlLibraryOperators.ARRAY_CONTAINS, array, value), + new PPLTypeChecker() { + @Override + public boolean checkOperandTypes(List<RelDataType> types) { + if (types.size() != 2) return false; + RelDataType valueType = types.get(0); + RelDataType arrayType = types.get(1); + if (arrayType.getSqlTypeName() != SqlTypeName.ARRAY + || valueType.getSqlTypeName() == SqlTypeName.ARRAY) return false; + RelDataType elementType = arrayType.getComponentType(); + return elementType != null + && PPLTypeChecker.PPLComparableTypeChecker.isComparable(elementType, valueType); + } + @Override + public String getAllowedSignatures() { return "[ELEMENT_TYPE,ARRAY]"; } + @Override + public List<List<RelDataType>> getParameterTypes() { + RelDataType anyType = TYPE_FACTORY.createSqlType(SqlTypeName.ANY); + return List.of(List.of(anyType, anyType)); + } + }); Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies that equality is commutative and the current implementation only handles [ARRAY, scalar] but not [scalar, ARRAY]. This is a significant functional gap that could cause queries with reversed operands to fail or behave unexpectedly.	Medium
	Support reversed operands for inequality Similar to EQUAL, NOTEQUAL should also support the reversed operand order [scalar, ARRAY] since inequality is also commutative. Add a symmetric overload to ensure value != array_field works correctly. core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java [991-998] register( NOTEQUAL, (FunctionImp2) (builder, array, value) -> builder.makeCall( SqlStdOperatorTable.NOT, builder.makeCall(SqlLibraryOperators.ARRAY_CONTAINS, array, value)), ARRAY_ELEMENT_COMPARABLE); +register( + NOTEQUAL, + (FunctionImp2) + (builder, value, array) -> + builder.makeCall( + SqlStdOperatorTable.NOT, + builder.makeCall(SqlLibraryOperators.ARRAY_CONTAINS, array, value)), + new PPLTypeChecker() { + @Override + public boolean checkOperandTypes(List<RelDataType> types) { + if (types.size() != 2) return false; + RelDataType valueType = types.get(0); + RelDataType arrayType = types.get(1); + if (arrayType.getSqlTypeName() != SqlTypeName.ARRAY + || valueType.getSqlTypeName() == SqlTypeName.ARRAY) return false; + RelDataType elementType = arrayType.getComponentType(); + return elementType != null + && PPLTypeChecker.PPLComparableTypeChecker.isComparable(elementType, valueType); + } + @Override + public String getAllowedSignatures() { return "[ELEMENT_TYPE,ARRAY]"; } + @Override + public List<List<RelDataType>> getParameterTypes() { + RelDataType anyType = TYPE_FACTORY.createSqlType(SqlTypeName.ANY); + return List.of(List.of(anyType, anyType)); + } + }); Suggestion importance[1-10]: 8 __ Why: Similar to the equality case, this identifies that NOTEQUAL should also support reversed operands since inequality is commutative. The missing overload could cause value != array_field queries to fail, making this a critical functional issue.	Medium