[SPARK-58730][SQL] Support the ANSI SQL/JSON JSON_ARRAY constructor function

[ganeshashree]

What changes were proposed in this pull request?

Implement the ANSI SQL:2016 JSON_ARRAY constructor function (feature T811), which builds a JSON array from a list of argument values and returns it as JSON text.

Syntax (a dedicated primaryExpression production, mirroring JSON_TABLE, because the
argument list carries non-expression syntax):

JSON_ARRAY ( [ value [ FORMAT JSON ] [, ...] ]
             [ { NULL | ABSENT } ON NULL ]
             [ RETURNING data_type ] )

• New JsonArray Catalyst expression that serializes each element through the same
  Jackson writer as to_json, so numbers, decimals, datetimes, and nested
  structs/arrays/maps render consistently.
• (NULL | ABSENT) ON NULL: ABSENT ON NULL (the standard default) drops NULL
  elements; NULL ON NULL keeps them as JSON null.
• RETURNING: defaults to STRING and is restricted to string types (VARIANT deferred);
  CHAR/VARCHAR are normalized to STRING.
• Per-element FORMAT JSON: splices already-JSON string text verbatim instead of quoting
  it (validated for well-formedness at eval). A nested JSON constructor carries it
  implicitly. The splice/quote decision is frozen at parse time so a later optimizer
  rewrite (e.g. CollapseProject inlining a JSON_ARRAY alias) cannot flip it.
• JSON_ARRAY and ABSENT added as non-reserved keywords (matching JSON_TABLE/JSON_VALUE).

Example:

SELECT JSON_ARRAY(1, 'x', true);              -- [1,"x",true]
SELECT JSON_ARRAY(1, NULL, 3);                -- [1,3]        (ABSENT ON NULL default)
SELECT JSON_ARRAY(1, NULL, 3 NULL ON NULL);   -- [1,null,3]
SELECT JSON_ARRAY(JSON_ARRAY(1, 2), 3);       -- [[1,2],3]    (nested: implicit FORMAT JSON)
SELECT JSON_ARRAY('[1,2]');                   -- ["[1,2]"]    (plain string: quoted)
SELECT JSON_ARRAY('[1,2]' FORMAT JSON);       -- [[1,2]]      (explicit FORMAT JSON: spliced)

Why are the changes needed?

Spark implements none of the ANSI SQL:2016 constructor JSON functions, forcing users migrating from Oracle, SQL Server, MySQL, Trino, or BigQuery to rewrite JSON_ARRAY(a, b, c) as to_json(array(a, b, c)), which is verbose and does not carry the standard's ON NULL / RETURNING / FORMAT JSON semantics.

Does this PR introduce any user-facing change?

Yes. It adds the new JSON_ARRAY SQL constructor function and the new non-reserved keywords JSON_ARRAY and ABSENT. There is no change to existing queries (both keywords remain usable as identifiers, as they are non-reserved).

How was this patch tested?

• New JsonArraySuite (49 cases): scalar/mixed/nested values, both ON NULL modes, RETURNING (collated + CHAR/VARCHAR normalization), implicit/explicit FORMAT JSON (splicing, per-row validation, error truncation), plan-stability of the splice decision, foldable/throwable/collation behavior, and analysis-time type rejection.
• Golden-file coverage in json-functions.sql (SQLQueryTestSuite).
• SQLKeywordSuite and regenerated keywords*.sql.out for the new keywords; checkError coverage for the new error conditions.
• New reference doc page sql-ref-syntax-qry-select-json-array.md.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 4.8)

[uros-b]

Thank you @ganeshashree! cc @cloud-fan