fix: auto-add missing placeholders to LLM judge prompts

[Chibionos]

Summary

This PR fixes the validation error that occurs when {{ActualOutput}} or {{ExpectedOutput}} placeholders are missing from LLM judge evaluator prompts. Instead of raising an error, we now automatically append the missing placeholders in clearly marked sections at the end of the prompt.

Problem

Users were encountering this error:

ValueError: Failed to create evaluator from file 'evals/evaluators/evaluator-xxx.json': 
1 validation error for LegacyLlmAsAJudgeEvaluator
prompt
  Value error, Prompt must contain both {ActualOutput} and {ExpectedOutput} placeholders

Solution

Modified the validation logic in both evaluator implementations to be more user-friendly:

Behavior

• ✅ Both placeholders present: No changes made to the prompt
• ✅ One placeholder missing: Automatically append only the missing placeholder in a new section
• ✅ Both placeholders missing: Automatically append both placeholders in separate sections

Example

If a prompt is missing {{ExpectedOutput}}, it will be transformed from:

Evaluate this output: {{ActualOutput}}

To:

Evaluate this output: {{ActualOutput}}

## Expected Output
{{ExpectedOutput}}

Changes

• Modified src/uipath/eval/evaluators/legacy_llm_as_judge_evaluator.py
  • Changed validate_prompt_placeholders from raising ValueError to auto-fixing prompt
• Modified src/uipath/eval/evaluators/llm_as_judge_evaluator.py
  • Changed validate_prompt_placeholders from raising UiPathEvaluationError to auto-fixing prompt

Benefits

• ✅ Better UX: No more confusing validation errors
• ✅ Backward compatible: Prompts with both placeholders work exactly as before
• ✅ Fail-safe: Ensures evaluators always have access to required data
• ✅ Clear sections: Missing placeholders are added with descriptive headers

Testing

The fix can be tested with evaluator configs that have:

1. Both placeholders present (should work as before)
2. Only {{ActualOutput}} present ({{ExpectedOutput}} should be auto-added)
3. Only {{ExpectedOutput}} present ({{ActualOutput}} should be auto-added)
4. Neither placeholder present (both should be auto-added)

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 noreply@anthropic.com

[Chibionos]

Enhancement: XML Tags for Better Parsing

Added XML-style opening and closing tags around the auto-added placeholders to help the LLM clearly distinguish between outputs, especially when dealing with large JSON objects.

Updated Format

When missing placeholders are auto-added, they now include tags:

## Actual Output
<ActualOutput>
{{ActualOutput}}
</ActualOutput>

## Expected Output
<ExpectedOutput>
{{ExpectedOutput}}
</ExpectedOutput>

Benefits

• ✅ Clear boundaries: LLM can easily identify where each output starts and ends
• ✅ JSON-friendly: Works well with complex nested JSON structures
• ✅ Parseable: XML-style tags are familiar to LLMs and easy to parse
• ✅ Consistent: Same format whether one or both placeholders are missing

Example with Large JSON

Before tags, it would be unclear where one output ends:

## Expected Output
{{ExpectedOutput}}
{"complex": {"nested": {"json": "here"}}, "more": "data"}

With tags, boundaries are crystal clear:

## Expected Output
<ExpectedOutput>
{"complex": {"nested": {"json": "here"}}, "more": "data"}
</ExpectedOutput>

This makes it much easier for the LLM to correctly parse and compare the actual vs expected outputs.