Skip to content

Test Project feature#2162

Open
spetersenms wants to merge 17 commits intomicrosoft:mainfrom
spetersenms:TestProject
Open

Test Project feature#2162
spetersenms wants to merge 17 commits intomicrosoft:mainfrom
spetersenms:TestProject

Conversation

@spetersenms
Copy link
Contributor

@spetersenms spetersenms commented Mar 9, 2026

❔What, Why & How

This new feature brings the concept of test projects to AL-Go for GitHub. The purpose of this is to allow the user to split building and testing into separate projects. This can be especially useful if some tests are unstable and might require a re-run to succeed. In such scenarios, it's nice not to have to wait for compilation again.

A test project is a special type of project that depends on one or more regular projects. It will download and install artifacts from those dependent projects and run all test apps it finds. It allows any customization or settings used in regular projects, such as build modes or conditional settings. The only limitation is test projects are not allowed to build apps.

Related to issue: #

✅ Checklist

  • Add tests (E2E, unit tests)
  • Update RELEASENOTES.md
  • Update documentation (e.g. for new settings or scenarios)
  • Add telemetry

@spetersenms spetersenms marked this pull request as ready for review March 9, 2026 15:42
@spetersenms spetersenms requested a review from a team as a code owner March 9, 2026 15:42
Copilot AI review requested due to automatic review settings March 9, 2026 15:42
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces the concept of test projects to AL-Go for GitHub — a new project type that separates test execution from compilation. A test project does not build any apps; instead, it depends on one or more regular projects, installs the apps they produce (including test apps), and runs tests against them. This enables re-running tests without a full rebuild.

Changes:

  • New testProject setting for project-level .AL-Go/settings.json, with schema validation, default value, and documentation
  • Dependency resolution and build order logic to treat test projects as dependents of their upstream projects
  • Pipeline changes to correctly handle running a test project with no local app/test folders

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Actions/AL-Go-Helper.ps1 Adds testProject handling in AnalyzeRepo (validates no buildable code, sets test runner flags) and AnalyzeProjectDependencies (injects build-order dependencies via synthetic markers)
Actions/RunPipeline/RunPipeline.ps1 Skips the "repository is empty" early exit for test projects
Actions/.Modules/ReadSettings.psm1 Adds testProject with an empty-array default to the settings defaults
Actions/.Modules/settings.schema.json Adds testProject as an array-of-strings property to the JSON schema
Scenarios/settings.md Documents the new testProject setting
RELEASENOTES.md Describes the test projects feature and its usage
Tests/DetermineProjectsToBuild.Test.ps1 Adds six new test scenarios covering dependency resolution, validation errors, transitive deps, and multi-upstream cases

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

"templateBranch" = ""
"appDependencyProbingPaths" = @()
"useProjectDependencies" = $false
"testProject" = @()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is an array, it feels like this should be plural no? So testProjects

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe? The thought behind it is that the current project is a test project (singular) if this setting is set. testProjects might not be a good name since that would indicate it depends on several test projects? Maybe testProjectDependencies instead?

Comment on lines +732 to +734
$settings.installTestRunner = $true
$settings.installTestFramework = $true
$settings.installTestLibraries = $true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not just take these from the repos settings file? You may not need all these 🤷‍♂️

RELEASENOTES.md Outdated

- Resolve the dependency so the test project always builds after its target project(s).
- Install the Test Runner, Test Framework, and Test Libraries into the container.
- Run all tests from the installed test apps (`runTestsInAllInstalledTestApps`).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Run all tests from the installed test apps (`runTestsInAllInstalledTestApps`).
- Run all tests from the installed test apps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants