Skip to content

Fix ProviderDbContextTests pipeline flake; reorganize and quality-pass test suite#520

Merged
NikTilton merged 8 commits into
mainfrom
jschick/fix-providerdb-disposed-test
May 12, 2026
Merged

Fix ProviderDbContextTests pipeline flake; reorganize and quality-pass test suite#520
NikTilton merged 8 commits into
mainfrom
jschick/fix-providerdb-disposed-test

Conversation

@jschick04
Copy link
Copy Markdown
Collaborator

Summary

Fixes the intermittent ObjectDisposedException in the Eventing test project on the ADO pipeline (ProviderDbContextTests.PerformUpgradeIfNeeded_WithV2SchemaAndNullParameters_Throws), and — per follow-up scope — performs a full audit and reorganization of the test suite using industry best-practice categorization (Unit vs Integration), plus a quality pass to drop weak / duplicated tests, parameterize mirror tests, and add coverage where it was missing.

Pipeline fix

The original failure was a SQLite handle-disposal race caused by Microsoft.Data.Sqlite reusing pooled handles across test runs that touched the same on-disk DB path. Fix: move SQLite-substrate tests out of the unit project into a dedicated Integration project with serial xUnit execution, and centralize cleanup (ClearAllPools + retry loop) in a shared SqliteTestDb helper in tests/Shared/EventLogExpert.Eventing.TestUtils.

Test reorganization (Unit ↔ Integration)

Categorization rule applied uniformly:

  • Unit = in-process, deterministic, no real I/O / SQLite / OS APIs / network / time / shared mutable state. Mocked collaborators OK; bUnit component tests with mocked services qualify.
  • Integration = crosses a real boundary (file system, real SQLite, real OS APIs / Win32 P/Invoke, real time).

Net result: tests are rebalanced — total count unchanged, but each test now lives in the right project.

Project Before After
Components.Tests 144 144
Eventing.Tests 477 288
UI.Tests 1039 915
EventDbTool.IntegrationTests (renamed from .Tests) 19 46
Eventing.IntegrationTests (new) 334 + 2 skip
UI.IntegrationTests (new) 124
Total 1679 + 2 skip 1851 + 2 skip

Quality pass

  • 2A: Dropped low-value smoke / duplicate tests; refactored Dispose idempotency tests into a clear contract; parameterized RenderXml table tests.
  • 2B: Parameterized invalid-log-name and cache-kind mirror tests.
  • 2C: Dropped 6 weak resolver constructor smoke tests.
  • 2D-1: Added 27 new tests for EventDbTool covering RegexHelper, MtaProviderSource failure paths, CreateDatabaseCommand failure paths + batch-boundary at 101 providers + skip-all, ShowCommand, and Program.BuildServiceProvider (singleton + verbose flag → log level contract). Visibility: CreateDatabaseCommand.CreateDatabase and ShowCommand.ShowProviderInfo switched private→internal (matches the existing DiffDatabaseCommand precedent).
  • Reviewer-panel follow-up: A 4-agent reviewer panel (3 code-review + rubber-duck) flagged 3 tests as misclassified — they call real Win32 APIs (LoadLibraryExW / FormatMessageW) directly or transitively. Moved EventMessageProviderTests, NativeErrorResolverTests, and LocalProviderEventResolverTests to the Integration project.

Deferred (intentional, captured in plan)

  • Razor component coverage for SettingsModal, FilterRow, SubFilterRow, FilterRowChrome, FilterGroupModal, FilterCacheModal — paused at user request, can resume in a follow-up PR (Fluxor + bUnit pattern is established via DebugLogModalTests).
  • MtaProviderSource happy-path test — needs a real .evtx fixture with sibling LocaleMetaData/*.MTA; the no-MTA failure path IS covered, so the ""no local fallback"" guarantee is asserted for the most likely regression.

Verification

All 6 test projects pass locally on Release (Windows): 1,851 passed + 2 skipped, 0 failed.

Commits

  1. dda5fd3 Phase 1 — ADO flake fix (Eventing.IntegrationTests + shared TestUtils + serial xUnit)
  2. 85e7409 Phase 2 — Unit↔Integration split (UI.IntegrationTests + EventDbTool.IntegrationTests)
  3. f65ed05 Phase 2A — drop weak smoke / duplicate tests, refactor Dispose contract, parameterize RenderXml
  4. 98ca7c3 Phase 2B — parameterize invalid-log-name and cache-kind mirror tests
  5. 4cf9ac6 Phase 2C — drop 6 weak resolver constructor smoke tests
  6. a4267cc Phase 2D-1 — EventDbTool coverage (27 tests)
  7. c202a78 Reviewer-panel fix — move OS-API-dependent provider and resolver tests to Integration

Copilot AI review requested due to automatic review settings May 12, 2026 03:25
@jschick04 jschick04 requested a review from a team as a code owner May 12, 2026 03:25
@jschick04 jschick04 marked this pull request as draft May 12, 2026 03:25
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes an intermittent SQLite-related test flake by moving SQLite/OS-bound tests into dedicated Integration test projects configured for serial execution, and centralizes SQLite cleanup via a shared SqliteTestDb helper. It also reorganizes and refactors the test suite (Unit vs Integration), removes low-value tests, and adds new integration coverage for the EventDbTool commands/helpers.

Changes:

  • Introduced shared SQLite cleanup helper (SqliteTestDb) and updated tests to use it instead of ad-hoc pool-clearing/GC/delete logic.
  • Added/updated Integration test projects (UI/Eventing/EventDbTool) with xunit.runner.json enforcing serial execution; adjusted namespaces and project references accordingly.
  • Parameterized/refactored tests (e.g., cache-kind mirroring, invalid log name scenarios) and added new EventDbTool integration tests for RegexHelper, CreateDatabaseCommand, ShowCommand, Program.BuildServiceProvider, and MTA failure paths.

Reviewed changes

Copilot reviewed 56 out of 57 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/Unit/EventLogExpert.UI.Tests/TestUtils/Constants/Constants.DebugLog.cs Removes debug-log constants no longer needed in unit tests.
tests/Unit/EventLogExpert.UI.Tests/TestUtils/Constants/Constants.Database.cs Removes DB constants from unit tests (moved to integration test utils).
tests/Unit/EventLogExpert.Eventing.Tests/Resolvers/EventResolverCacheTests.cs Parameterizes cache tests across Description/Value caches to reduce duplication.
tests/Unit/EventLogExpert.Eventing.Tests/Resolvers/EventResolverBaseTests.cs Updates TestUtils namespace import to shared TestUtils project.
tests/Unit/EventLogExpert.Eventing.Tests/Providers/ProviderDetailsTests.cs Updates TestUtils namespace import to shared TestUtils project.
tests/Unit/EventLogExpert.Eventing.Tests/ProviderDatabase/ProviderDetailsMergerTests.cs Updates TestUtils namespace import to shared TestUtils project.
tests/Unit/EventLogExpert.Eventing.Tests/ProviderDatabase/CompressedJsonValueConverterTests.cs Removes low-value constructor smoke test.
tests/Unit/EventLogExpert.Eventing.Tests/EventLogExpert.Eventing.Tests.csproj Adds reference to shared Eventing TestUtils project.
tests/Unit/EventLogExpert.Components.Tests/Database/SettingsUpgradeProgressBannerTests.cs Tightens Dispose idempotency assertion.
tests/Shared/EventLogExpert.Eventing.TestUtils/SqliteTestDb.cs Adds centralized SQLite pool clearing + retry delete helpers.
tests/Shared/EventLogExpert.Eventing.TestUtils/EventUtils.cs Moves EventUtils into shared TestUtils namespace.
tests/Shared/EventLogExpert.Eventing.TestUtils/EventLogExpert.Eventing.TestUtils.csproj Converts prior test utility project into a shared library; adjusts properties/references.
tests/Shared/EventLogExpert.Eventing.TestUtils/Constants/Constants.Resolver.cs Updates namespace to shared TestUtils.
tests/Shared/EventLogExpert.Eventing.TestUtils/Constants/Constants.Provider.cs Updates namespace to shared TestUtils (includes common provider/log constants).
tests/Shared/EventLogExpert.Eventing.TestUtils/CompressionTestUtils.cs Updates namespace to shared TestUtils.
tests/Integration/EventLogExpert.UI.IntegrationTests/xunit.runner.json Disables xUnit parallelization for integration suite.
tests/Integration/EventLogExpert.UI.IntegrationTests/TestUtils/DatabaseSeedUtils.cs Moves UI DB seeding utils into integration test namespace.
tests/Integration/EventLogExpert.UI.IntegrationTests/TestUtils/Constants/Constants.DebugLog.cs Adds integration-only debug-log constants.
tests/Integration/EventLogExpert.UI.IntegrationTests/TestUtils/Constants/Constants.Database.cs Adds integration-only DB constants (replacing deleted unit version).
tests/Integration/EventLogExpert.UI.IntegrationTests/GlobalUsings.cs Adds global xUnit using for the new project.
tests/Integration/EventLogExpert.UI.IntegrationTests/EventLogExpert.UI.IntegrationTests.csproj New UI integration test project referencing UI/Eventing/shared TestUtils.
tests/Integration/EventLogExpert.UI.IntegrationTests/DebugLog/DebugLogServiceTests.cs Updates namespace and constants import for integration project split.
tests/Integration/EventLogExpert.UI.IntegrationTests/Database/DatabaseServiceTests.cs Updates namespace and TestUtils imports for integration project split.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/xunit.runner.json Disables xUnit parallelization for integration suite.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/TestUtils/Constants/Constants.Provider.cs Removes redundant constants file (now sourced from shared TestUtils).
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Resolvers/VersatileEventResolverTests.cs Updates namespace/imports; replaces ad-hoc SQLite cleanup with SqliteTestDb.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Resolvers/LocalProviderEventResolverTests.cs Moves to integration namespace and shared TestUtils imports.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Resolvers/EventProviderDatabaseEventResolverTests.cs Moves to integration namespace; centralizes DB cleanup via SqliteTestDb.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Readers/EventLogWatcherTests.cs Removes redundant constructor smoke tests; updates constants import.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Readers/EventLogSessionTests.cs Updates constants import to shared TestUtils.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Readers/EventLogReaderTests.cs Parameterizes invalid-log-name tests; updates constants import to shared TestUtils.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Readers/EventLogInformationTests.cs Updates constants import to shared TestUtils.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Providers/RegistryProviderTests.cs Updates constants import to shared TestUtils.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Providers/ProviderMetadataTests.cs Updates constants import to shared TestUtils.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Providers/EventMessageProviderTests.cs Moves OS-API-dependent tests to integration namespace and trims low-value cases.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Providers/EventMessageProviderIntegrationTests.cs Updates constants import to shared TestUtils.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/ProviderDatabase/ProviderDbContextTests.cs Moves to integration namespace; centralizes DB cleanup via SqliteTestDb.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/Interop/NativeErrorResolverTests.cs Moves Win32-dependent tests into integration namespace.
tests/Integration/EventLogExpert.Eventing.IntegrationTests/EventLogExpert.Eventing.IntegrationTests.csproj Adds shared TestUtils reference + ensures xunit.runner.json copied to output.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/xunit.runner.json Disables xUnit parallelization for integration suite.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/UpgradeDatabaseCommandTests.cs Updates namespace/TestUtils imports for integration project rename/split.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/TestUtils/DatabaseTestUtils.cs Reuses shared SqliteTestDb for cleanup; updates namespace.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/TestUtils/Constants/Constants.Database.cs Updates namespace for integration project split.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/ShowCommandTests.cs Adds integration tests for ShowCommand behavior (errors/warnings/output).
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/RegexHelperTests.cs Adds integration tests for RegexHelper behaviors (null/empty/timeout/invalid).
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/ProviderSourceTests.cs Updates namespace/TestUtils imports for integration split.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/ProgramTests.cs Adds DI contract tests for BuildServiceProvider (singleton + log level).
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/MtaProviderSourceTests.cs Adds failure-path tests for MTA discovery and logging contracts.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/MergeDatabaseCommandTests.cs Updates namespace/TestUtils imports for integration split.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/GlobalUsings.cs Adds global xUnit using for the new project.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/EventLogExpert.EventDbTool.IntegrationTests.csproj New EventDbTool integration test project; ensures xunit.runner.json copied to output.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/DiffDatabaseCommandTests.cs Updates namespace/TestUtils imports for integration split.
tests/Integration/EventLogExpert.EventDbTool.IntegrationTests/CreateDatabaseCommandTests.cs Adds integration tests covering validation, “no empty db” contract, batching, skip-set behavior.
src/EventLogExpert.EventDbTool/ShowCommand.cs Makes ShowProviderInfo internal for test access.
src/EventLogExpert.EventDbTool/Properties/AssemblyInfo.cs Updates InternalsVisibleTo to the new integration test assembly name.
src/EventLogExpert.EventDbTool/CreateDatabaseCommand.cs Makes CreateDatabase internal for test access.
EventLogExpert.slnx Updates solution structure: adds integration test projects + shared TestUtils; removes old unit EventDbTool tests project.
Comments suppressed due to low confidence (1)

tests/Shared/EventLogExpert.Eventing.TestUtils/EventLogExpert.Eventing.TestUtils.csproj:6

  • <AllowUnsafeBlocks>true</AllowUnsafeBlocks> is enabled for this shared TestUtils library, but there is no unsafe code in the project. Consider removing this property to keep the compilation surface area minimal (and only enable unsafe where it's required).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/Unit/EventLogExpert.EventDbTool.Tests/RegexHelperTests.cs Outdated
NikTilton
NikTilton previously approved these changes May 12, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 61 out of 61 changed files in this pull request and generated 1 comment.

Comment thread tests/Unit/EventLogExpert.Eventing.Tests/Resolvers/EventResolverCacheTests.cs Outdated
@jschick04 jschick04 force-pushed the jschick/fix-providerdb-disposed-test branch from 3d8d8bb to f19e1b3 Compare May 12, 2026 15:21
@jschick04 jschick04 requested a review from Copilot May 12, 2026 15:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 66 out of 66 changed files in this pull request and generated 1 comment.

@jschick04 jschick04 marked this pull request as ready for review May 12, 2026 15:32
@jschick04 jschick04 requested a review from NikTilton May 12, 2026 15:32
@NikTilton NikTilton merged commit 45b193f into main May 12, 2026
10 checks passed
@NikTilton NikTilton deleted the jschick/fix-providerdb-disposed-test branch May 12, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants