Skip to content

Add crash diagnostics to device test pipeline step #11298

@jonathanpeppers

Description

@jonathanpeppers

Context

PR #11224 migrates device tests from RunTestApp + NUnit XML + RenameTestCases to dotnet test + TRX. The old CreateErrorResultsFile in RenameTestCases.cs currently produces a "Possible Crash / {config}" synthetic test failure when no test results XML is found, but the error message provides no analysis of why the crash happened — just a raw logcat dump.

In build 14010260, the Mono.Android.NET_Tests-NoAot test failed with "Possible Crash / Release" because the emulator's system_server died mid-test (lowmemorykiller triggered, all system services died). Diagnosing this required manually downloading the logcat artifact and scanning thousands of lines for crash indicators.

Proposal

After dotnet test completes (or crashes), add a pipeline step in apk-instrumentation.yaml that:

  1. Only runs when the test step failed
  2. Dumps adb logcat -d and scans for known crash patterns
  3. Emits ##[warning]Crash indicator: <description> messages so they appear directly in the AZDO build summary

Crash patterns to detect

Pattern Message
lowmemorykiller or lmkd Low memory killer triggered
Multiple ServiceManager: service '...' died System services died (system_server likely crashed)
FATAL EXCEPTION FATAL EXCEPTION in test process
signal 11 (SIGSEGV) or signal 6 (SIGABRT) Native crash signal detected
tombstoned: received crash request Tombstone generated (native crash)
statsd.*system server restart system_server restarted

Benefits

  • Crash diagnostics appear inline in the AZDO build log — no need to download artifacts
  • Works regardless of test framework (NUnit, MTP, etc.)
  • Shell-based, no C# changes required
  • When the process is killed externally (e.g., emulator crash), on-device code can't report anything — the pipeline is the only reliable place to analyze what happened

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageIssues that need to be assigned.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions