Use Calcite's validation system for type checking & coercion#4892

Open

yuancu wants to merge 117 commits intoopensearch-project:mainfrom

yuancu:issues/4636

Commits on Dec 16, 2025

WIP: try introducing sql validator without disabling the legacy type checking
yuancu
committed
Override deriveType in validator to allow type checking on UDTs
yuancu
committed
Add special handling for datetime coercion
yuancu
committed
Override coerceOperandType and castTo methods to apply casting to udf logics
yuancu
committed
Fix interval semantic mismatch between sql and ppl
yuancu
committed
Fix quarter interval bug in calcite (counte-react)
yuancu
committed
Comment out function overloadings to make basic functions work
yuancu
committed
Remove unused validator operators
yuancu
committed
Update sargFromJson method in ExtendedRelJson
yuancu
committed
Prepend following rules for datetime comparisons: (date, time) -> timestamp; (time, timestamp) -> timestamp (1240/1599)
yuancu
committed
Directly delegate type checking to sql type checkers (1292/1599 | 1252/1872)
yuancu
committed
Enable IP comparison (1332/1559 | 1409/1915)
yuancu
committed
Fix expected type checking for agg functions (1287/1599 | 1423/1914)
yuancu
committed
Define operand types for json functions
yuancu
committed
Define operand types and return type inferences for array functions (1356/1599 | 1476/1915)
yuancu
committed
Add hive sql library operators for array_slice function, which is used in mvindex's implementation (1580/2015)
yuancu
committed
Define overrides for atan function to allow conditional sql call rewriting (1579/2015)
yuancu
committed
Update operand types for percentile approx to allow with addtional type hint
yuancu
committed
Correct span function type routing (allow any)
yuancu
committed
Define operand type for DISTINCT_COUNT_APPROX
yuancu
committed
Unconditionally rewrite COUNT() to COUNT(*) in sql level to allow type inference (1701/2015)
yuancu
committed
Define operand types for EnhancedCoalesce, ScalarMin & Max; remove SqlLibrary.SPARK from lookup table to reduce confusion (1717/2018)
yuancu
committed
Correct type checkers for mvappend; fix regexp lookup; correct isempty and isblank (1738/2018)
yuancu
committed
Initiate a new RelToSqlConverter every time as it is stateful (1769/2018)
yuancu
committed
Override add for string concat and number addition (1773/2018)
yuancu
committed
Switch sql dialect to opensearch spark sql dialect (1772/2018)
yuancu
committed
Align null order in collation (1783/2018)
yuancu
committed
Skip validations for bin-on-timestamps (1799/2027)
yuancu
committed
Do not remove sort in subqueries when converting sql to rel (1820/2027)
yuancu
committed
Trim unused fields after when converting sql back to rel (1857/2027)
yuancu
committed
Pass on join type of logical correlate to lateral join (1858/2027)
yuancu
committed
Support semi and anti join in the converted SQL (1860/2027)
yuancu
committed
Disable insertion of json_type operator (1864/2027)
yuancu
committed
Rewrite IN / NOT IN with tuple inputs to row(...tuple) to conform to correct SQL syntax (1867/2027)
yuancu
committed
Update exception type of testSpanByImplicitTimestamp
yuancu
committed
Update PplTypeCoercionRule to allow CAST(IP as STRING)
yuancu
committed
Fix float literal by inserting compulsory cast
yuancu
committed
Extend spark dialect to support cast null to IP (1880/2028)
yuancu
committed
Support sql-udt conversion of composite types (1881/2028)
yuancu
committed
Add geoip of string override because the udt type is erased during validation (1884/2028)
yuancu
committed
Skip validation when grouping by two cases (1885/2028)
yuancu
committed
Embed collations into window for streamstats (1910/2054, before rebase: 1887/2028)
yuancu
committed
Skip validation when logical values is used to create empty rows (1912/2054)
yuancu
committed
Remove EnhancedCoalesceFunction in favor of built-in Coalesce function (1913/2054)
yuancu
committed
Fix ITs that are affected by allowing implicit coercion from number to string (1915/2055)
yuancu
committed
Return original logical plan when get error message 'Aggregate expressions cannot be nested' (1916/2055)
yuancu
committed
Reorganize class structures and refactor for readability
yuancu
committed

Commits on Dec 19, 2025

Commits on Dec 22, 2025

Commits on Dec 26, 2025

Merge remote-tracking branch 'origin/main' into issues/4636
yuancu
committed

Commits on Dec 29, 2025

Commits on Jan 9, 2026

Merge remote-tracking branch 'origin/main' into issues/4636
yuancu
committed

Commits on Jan 12, 2026

Update plans, UTs, doctests as is after merging with main
yuancu
committed

Commits on Jan 14, 2026

Merge OpenSearchSqlToRelConverter in CalciteToolsHelper into the same subclass for validation
yuancu
committed

Commits on Jan 19, 2026

Commits on Jan 22, 2026

Address comments
yuancu
committed

Commits on Jan 23, 2026

Add @NonNull annotation to getOperandMetadata() and provide default implementation
yuancu
committed

Commits on Jan 27, 2026

Create a new sql validator each time
yuancu
committed

Commits on Feb 4, 2026

Merge remote-tracking branch 'origin/main' into issues/4636
yuancu
committed

Commits on Apr 3, 2026

Commits on Apr 4, 2026

WIP: Fixing integration tests
yuancu
committed

Commits on Apr 5, 2026

Fix post-merge test failures after merging origin/main
yuancu
committed