Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 58 additions & 1 deletion docs/template-markers.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,64 @@ This distinction allows resources (like templates) to be available as pipeline r

## {{ agent_name }}

Should be replaced with the human-readable name from the front matter (e.g., "Daily Code Review"). This is used for display purposes like stage names.
Should be replaced with the human-readable name from the front matter
(e.g., `Daily Code Review`). The value is substituted **as-is**, with
no quoting or escaping — front-matter `name` values are free-form and
have not been validated against YAML scalar rules.

> ⚠️ This marker is only safe inside a position that is **not parsed as
> YAML** (currently only `src/data/threat-analysis.md`, which is a
> markdown body). YAML positions inside the generated pipelines use
> [`{{ pipeline_name }}`](#-pipeline_name-) (top-level `name:` line)
> or [`{{ agent_display_name }}`](#-agent_display_name-)
> (`displayName:` positions). Both emit a fully-quoted-and-escaped
> double-quoted YAML scalar, so colons, embedded `"`, and other
> plain-scalar-unsafe characters in the agent name cannot break parsing.

## {{ agent_display_name }}

Should be replaced with the front-matter agent name, emitted as a
**YAML double-quoted scalar** with proper escaping for `\`, `"`,
`\n`, `\r`, `\t`, and other ASCII control characters. Used for
`displayName:` positions inside the generated YAML where the templates
previously hand-wrapped `{{ agent_name }}` in double quotes (which
silently corrupted any agent name containing an embedded `"`).

For an agent named `My "special": agent`, this expands to:

```yaml
displayName: "My \"special\": agent"
```

Used in `src/data/1es-base.yml` (1ES stage display name) and
`src/data/stage-base.yml` (stage-target stage display name). The marker
deliberately does **not** include the `-$(BuildID)` suffix that
[`{{ pipeline_name }}`](#-pipeline_name-) carries — stage labels are
static and don't need per-run uniqueness.

## {{ pipeline_name }}

Should be replaced with the front-matter agent name plus the
`-$(BuildID)` suffix, always emitted as a **YAML double-quoted scalar**
with the same escaping rules as `{{ agent_display_name }}`. Used only
for the top-level pipeline `name:` line, which in Azure DevOps is the
build-number format string. The `-$(BuildID)` suffix is the
[varying token ADO requires](https://learn.microsoft.com/azure/devops/pipelines/process/run-number)
to give each run a unique display name in the runs view; without it,
every run shows the same name.

For an agent named `Daily safe-output smoke: noop`, this expands to:

```yaml
name: "Daily safe-output smoke: noop-$(BuildID)"
```

`$(BuildID)` is an ADO macro and is expanded at queue time after YAML
parsing; `$` has no special meaning inside a YAML double-quoted scalar
so the macro passes through untouched.

Used in `src/data/base.yml` and `src/data/1es-base.yml` only. The
job- and stage-level templates don't emit a top-level pipeline name.

## {{ engine_install_steps }}

Expand Down
120 changes: 120 additions & 0 deletions src/compile/common.rs
Original file line number Diff line number Diff line change
Expand Up @@ -996,6 +996,43 @@ pub fn sanitize_filename(name: &str) -> String {
.join("-")
}

/// Emit `s` as a YAML double-quoted scalar (always quoted, never plain).
///
/// We always quote because the value is substituted into YAML positions
/// where colons and other plain-scalar-unsafe characters are common in
/// agent names (e.g. `"Daily safe-output smoke: noop"`). A bare scalar
/// like `name: Daily safe-output smoke: noop-$(BuildID)` is invalid YAML
/// because the second colon is interpreted as a mapping indicator.
///
/// `$(...)` ADO macros pass through untouched — `$` has no special meaning
/// inside a YAML double-quoted scalar and ADO expands the macro at queue
/// time after YAML parsing.
///
/// `reject_pipeline_injection` already strips newlines and template /
/// pipeline-command sequences from front-matter `name` values, so the
/// escape table only has to cover `\` and `"`. Tabs and ASCII control
/// characters are escaped too as a belt-and-braces measure.
pub fn yaml_double_quoted(s: &str) -> String {
let mut out = String::with_capacity(s.len() + 2);
out.push('"');
for ch in s.chars() {
match ch {
'\\' => out.push_str("\\\\"),
'"' => out.push_str("\\\""),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
'\u{0085}' => out.push_str("\\x85"),
'\u{2028}' => out.push_str("\\u2028"),
'\u{2029}' => out.push_str("\\u2029"),
c if (c as u32) < 0x20 => out.push_str(&format!("\\x{:02x}", c as u32)),
c => out.push(c),
}
}
out.push('"');
out
}

/// Default self-hosted pool for 1ES templates.
pub const DEFAULT_ONEES_POOL: &str = "AZS-1ES-L-MMS-ubuntu-22.04";
/// Default Microsoft-hosted VM image for non-1ES templates.
Expand Down Expand Up @@ -2856,6 +2893,17 @@ pub async fn compile_shared(
let checkout_steps = generate_checkout_steps(&front_matter.checkout);
let checkout_self = generate_checkout_self();
let agent_name = sanitize_filename(&front_matter.name);
// Top-level pipeline `name:` value (the ADO build-number format).
// Always quoted so colons / embedded `"` in the agent name can't
// break parsing. Includes `-$(BuildID)` because ADO needs a varying
// token in the build-number format — without one, every run shows
// the same name in the runs view.
let pipeline_name =
yaml_double_quoted(&format!("{}-$(BuildID)", front_matter.name));
// Stage / job `displayName:` value. Always quoted (same escaping
// rationale as `pipeline_name`) but with NO BuildID suffix — stage
// labels are static and shouldn't carry per-run uniqueness suffixes.
let agent_display_name = yaml_double_quoted(&front_matter.name);

// 3. Run extension validations
for ext in extensions {
Expand Down Expand Up @@ -3069,6 +3117,8 @@ pub async fn compile_shared(
("{{ checkout_repositories }}", &checkout_steps),
("{{ agent }}", &agent_name),
("{{ agent_name }}", &front_matter.name),
("{{ agent_display_name }}", &agent_display_name),
("{{ pipeline_name }}", &pipeline_name),
("{{ agent_description }}", &front_matter.description),
("{{ engine_run }}", &engine_run),
("{{ engine_run_detection }}", &engine_run_detection),
Expand Down Expand Up @@ -3996,6 +4046,76 @@ mod tests {
assert_eq!(sanitize_filename("test_case"), "test-case");
}

// ─── yaml_double_quoted ──────────────────────────────────────────────────

#[test]
fn test_yaml_double_quoted_plain_string() {
assert_eq!(yaml_double_quoted("hello"), r#""hello""#);
}

#[test]
fn test_yaml_double_quoted_string_with_colon_is_safe() {
// The bug this helper exists to fix: an agent name like
// "Daily safe-output smoke: noop" must not be emitted bare in the
// top-level pipeline `name:` line, where the second colon would
// be parsed as a YAML mapping indicator.
assert_eq!(
yaml_double_quoted("Daily safe-output smoke: noop-$(BuildID)"),
r#""Daily safe-output smoke: noop-$(BuildID)""#
);
}

#[test]
fn test_yaml_double_quoted_escapes_backslash() {
assert_eq!(yaml_double_quoted(r"a\b"), r#""a\\b""#);
}

#[test]
fn test_yaml_double_quoted_escapes_double_quote() {
assert_eq!(yaml_double_quoted(r#"say "hi""#), r#""say \"hi\"""#);
}

#[test]
fn test_yaml_double_quoted_escapes_whitespace_controls() {
assert_eq!(yaml_double_quoted("a\nb"), r#""a\nb""#);
assert_eq!(yaml_double_quoted("a\rb"), r#""a\rb""#);
assert_eq!(yaml_double_quoted("a\tb"), r#""a\tb""#);
}

#[test]
fn test_yaml_double_quoted_escapes_yaml_line_separators() {
assert_eq!(yaml_double_quoted("a\u{0085}b"), r#""a\x85b""#);
assert_eq!(yaml_double_quoted("a\u{2028}b"), r#""a\u2028b""#);
assert_eq!(yaml_double_quoted("a\u{2029}b"), r#""a\u2029b""#);
}

#[test]
fn test_yaml_double_quoted_escapes_other_control_chars() {
// Bell (0x07) is a low ASCII control char — should escape as \x07.
assert_eq!(yaml_double_quoted("a\u{0007}b"), r#""a\x07b""#);
}

#[test]
fn test_yaml_double_quoted_passes_through_ado_macros() {
// $(BuildID), $(Build.SourcesDirectory) etc. have no special meaning
// inside a YAML double-quoted scalar; ADO expands them at queue time
// after YAML parsing.
assert_eq!(
yaml_double_quoted("$(Build.BuildId)/$(System.JobId)"),
r#""$(Build.BuildId)/$(System.JobId)""#
);
}

#[test]
fn test_yaml_double_quoted_passes_through_unicode() {
// Non-ASCII characters pass through as-is — YAML 1.2 supports UTF-8
// in double-quoted scalars natively.
assert_eq!(
yaml_double_quoted("résumé — 你好"),
r#""résumé — 你好""#
);
}

// ─── generate_pr_trigger ─────────────────────────────────────────────────

#[test]
Expand Down
4 changes: 2 additions & 2 deletions src/data/1es-base.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This template extends the 1ES Unofficial Pipeline Template with Copilot CLI,
# AWF network isolation, and MCP Gateway — matching the standalone pipeline model.

name: {{ agent_name }}-$(BuildID)
name: {{ pipeline_name }}
{{ parameters }}
{{ schedule }}
{{ pr_trigger }}
Expand Down Expand Up @@ -34,7 +34,7 @@ extends:
runPrerequisitesOnImage: false # Pool image has 1ES prerequisites preinstalled
stages:
- stage: AgentStage
displayName: "{{ agent_name }}"
displayName: {{ agent_display_name }}
jobs:
{{ setup_job }}

Expand Down
2 changes: 1 addition & 1 deletion src/data/base.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

name: {{ agent_name }}-$(BuildID)
name: {{ pipeline_name }}
{{ parameters }}
resources:
repositories:
Expand Down
2 changes: 1 addition & 1 deletion src/data/stage-base.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

stages:
- stage: {{ stage_prefix }}
displayName: "{{ agent_name }}"
displayName: {{ agent_display_name }}
jobs:
{{ setup_job }}
- job: {{ stage_prefix }}_Agent
Expand Down
42 changes: 41 additions & 1 deletion tests/compiler_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ fn assert_required_markers(content: &str) {
"{{ checkout_repositories }}",
"{{ allowed_domains }}",
"{{ source_path }}",
"{{ agent_name }}",
"{{ pipeline_name }}",
"{{ engine_run }}",
"{{ compiler_version }}",
"{{ integrity_check }}",
Expand Down Expand Up @@ -3528,6 +3528,46 @@ fn test_1es_compiled_output_is_valid_yaml() {
);
}

/// Names with embedded `"` and `:` must survive YAML escaping in both
/// the top-level `name:` line and any `displayName:` positions.
///
/// Regression: until `{{ pipeline_name }}` was introduced both positions
/// used a bare `{{ agent_name }}` substitution which broke if the
/// front-matter name contained colons (`name: a: b` parsed as a YAML
/// mapping) or embedded double quotes (`displayName: "a "b" c"` parsed
/// as broken scalars). Now both positions go through `yaml_double_quoted`
/// via a single `{{ pipeline_name }}` marker.
#[test]
fn test_compiled_yaml_survives_tricky_agent_name_standalone() {
let compiled = compile_fixture("tricky-name-agent.md");
assert_valid_yaml(&compiled, "tricky-name-agent.md");

// The top-level pipeline name must contain the escaped form of the
// embedded `"` (rendered as `\"`) AND retain the colon.
assert!(
compiled.contains(r#"name: "My \"special\": agent with quotes-$(BuildID)""#),
"standalone output should contain escaped pipeline name; got:\n{compiled}"
);
}

#[test]
fn test_compiled_yaml_survives_tricky_agent_name_1es() {
let compiled = compile_fixture("tricky-name-1es-agent.md");
assert_valid_yaml(&compiled, "tricky-name-1es-agent.md");

// Top-level pipeline name carries the `-$(BuildID)` suffix because
// the ADO build-number format needs a varying token; the stage
// displayName does NOT carry the suffix (stage labels are static).
assert!(
compiled.contains(r#"name: "My \"special\": agent with quotes (1ES)-$(BuildID)""#),
"1ES output should contain escaped pipeline name; got:\n{compiled}"
);
assert!(
compiled.contains(r#"displayName: "My \"special\": agent with quotes (1ES)""#),
"1ES output should contain escaped stage displayName; got:\n{compiled}"
);
}

/// Test that the minimal standalone fixture produces valid YAML with correct structure
#[test]
fn test_standalone_minimal_compiled_output_is_valid_yaml() {
Expand Down
15 changes: 15 additions & 0 deletions tests/fixtures/tricky-name-1es-agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
name: 'My "special": agent with quotes (1ES)'
description: "Fixture covering YAML escaping for embedded \" and : in name (1ES target)"
target: 1es
permissions:
read: my-read-arm-connection
---

## Tricky-Name Agent (1ES)

Fixture used by `tests/compiler_tests.rs` to verify that agent names
containing embedded double quotes and colons survive YAML escaping for
the 1ES target, where the stage's `displayName:` is generated via
`{{ agent_display_name }}` and the top-level pipeline `name:` is
generated via `{{ pipeline_name }}`.
14 changes: 14 additions & 0 deletions tests/fixtures/tricky-name-agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
name: 'My "special": agent with quotes'
description: "Fixture covering YAML escaping for embedded \" and : in name"
target: standalone
permissions:
read: my-read-arm-connection
---

## Tricky-Name Agent

Fixture used by `tests/compiler_tests.rs` to verify that agent names
containing embedded double quotes and colons survive YAML escaping in
both the top-level `name:` line (via `{{ pipeline_name }}`) and
any `displayName:` positions (via `{{ agent_display_name }}`).
16 changes: 8 additions & 8 deletions tests/safe-outputs/add-build-tag.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading