added isnotnull condition support

[ThyTran1402]

Description

Adds support for field IS [NOT] NULL as a predicate syntax in PPL, as a synonym for the existing isnull(field) / isnotnull(field) functions.
Previously, users coming from a SQL background would naturally write where field is not null in PPL, only to receive a parse error with no guidance.

Related Issues

Resolves #5262

Check List

• New functionality includes testing.
• New functionality has been documented.
• New functionality has javadoc added.
• New functionality has a user manual doc added.
• New PPL command checklist all confirmed.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff or -s.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Failed to generate code suggestions for PR

[github-actions]

Failed to generate code suggestions for PR

[ThyTran1402]

Do you know why it did not pass the tests 😢 ? @Swiddis

[Swiddis]

Unit tests are failing due to spotless violations, it says how to fix them here: https://github.com/opensearch-project/sql/actions/runs/23671443599/job/68965646433?pr=5278#step:6:9358

Doctests are failing because the cluster is returning incorrect results: https://github.com/opensearch-project/sql/actions/runs/23671443599/job/68965646435?pr=5278#step:6:661

Are you running these locally before pushing? Integ tests are slow locally, but at least ./gradlew spotlessApply spotlessCheck test doctest should get you most of the way there. You can always check the test action to see exactly what each CI check is doing. I also recommend setting up pre-commit, which will connect with our pre-commit steps to auto-fix your formatting on every commit.

If you want to work on this interactively, I'm available on the opensearch slack

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

📝 TODO sections ppl/src/test/java/org/opensearch/sql/ppl/parser/AstExpressionBuilderTest.java [320]: search operator should not include functionCall, need to change antlr.

🔀 No multiple PR themes

⚡ Recommended focus areas for review Keyword Conflict Adding NULL as a new lexer token may conflict with existing uses of null as an identifier or field name in PPL queries. While NULL is added to keywordsCanBeId, there may be edge cases where field names or identifiers named null break in unexpected ways. This should be thoroughly tested with queries that use null as a field name or part of a field path. NULL: 'NULL'; Grammar Ambiguity The isNullPredicate rule uses expression IS nullNotnull, where nullNotnull is NOT? NULL. This could introduce ambiguity or precedence issues when NOT appears in complex boolean expressions. For example, a IS NOT NULL AND b IS NULL might be parsed unexpectedly if NOT is consumed by nullNotnull rather than the outer logical expression. Careful testing of compound expressions is warranted. | expression IS nullNotnull # isNullPredicate ; nullNotnull : NOT? NULL Arrays.asList Usage Arrays.asList(visit(ctx.expression())) creates a fixed-size list. While this works for a single argument, it is inconsistent with how other similar function calls are built in this file. Consider using Collections.singletonList(...) for clarity, or verify that the Function constructor handles this correctly. Arrays.asList(visit(ctx.expression())));

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Verify all expected null-matching rows The test testIsNullPredicate only verifies a single row ("Virginia"), but there may be multiple rows with a null age in TEST_INDEX_BANK_WITH_NULL_VALUES. If the dataset contains more than one record with a null age, the test will fail or give a false positive. Consider verifying the total number of rows returned as well, or ensure the expected rows match all null-age records in the test dataset. integ-test/src/test/java/org/opensearch/sql/ppl/WhereCommandIT.java [203] -verifyDataRows(result, rows("Virginia")); +verifyDataRows(result, rows("Virginia"), rows("Hattie"), rows("Nanette")); Suggestion importance[1-10]: 4 __ Why: The suggestion raises a valid concern about whether the test dataset contains multiple null-age records that should be verified. However, the improved code assumes specific additional rows ("Hattie", "Nanette") without confirmed knowledge of the test dataset, making it potentially incorrect.	Low
	Simplify single-element list construction The condition ctx.nullNotnull().NOT() == null checks for the absence of NOT, which correctly maps to IS NULL. However, the logic should be verified: when NOT is absent, it should return IS_NULL, and when NOT is present, it should return IS_NOT_NULL. The current logic appears correct, but using Arrays.asList for a single-element list is unnecessary — consider using Collections.singletonList or List.of for clarity and minor efficiency. ppl/src/main/java/org/opensearch/sql/ppl/parser/AstExpressionBuilder.java [252-259] return new Function( ctx.nullNotnull().NOT() == null ? IS_NULL.getName().getFunctionName() : IS_NOT_NULL.getName().getFunctionName(), - Arrays.asList(visit(ctx.expression()))); + Collections.singletonList(visit(ctx.expression()))); Suggestion importance[1-10]: 3 __ Why: Using Collections.singletonList or List.of instead of Arrays.asList for a single element is a minor style improvement with negligible impact. The existing code is functionally correct.	Low

[ThyTran1402]

Hi @Swiddis

Thank you for your guidance and suggestions on working on this issue. I found that it's helpful to run the test locally as you suggested, to pass it locally first, and it passed the tests this time =))

Can you please review it when you get a chance?