Combine Packit jobs running Contest into fewer jobs#14392
Open
comps wants to merge 3 commits intoComplianceAsCode:masterfrom
Open
Combine Packit jobs running Contest into fewer jobs#14392comps wants to merge 3 commits intoComplianceAsCode:masterfrom
comps wants to merge 3 commits intoComplianceAsCode:masterfrom
Conversation
This was likely a leftover from Beakerlib-era Fedora "downstream" testing - when we stopped doing it, we moved the only remaining valid test here. However since we run cTests via Github Actions in upstream, this extra test is likely unnecessary and complicates our Packit testing setup. Signed-off-by: Jiri Jaburek <comps@nomail.dom>
Keeping plans/tests separate is not necessary, and the use case is isolated enough that it makes sense to keep all pieces of it together. Signed-off-by: Jiri Jaburek <comps@nomail.dom>
The new layout has much fewer jobs: - centos-stream-8-x86_64:contest-oscap - centos-stream-8-x86_64:contest-ansible - centos-stream-9-x86_64:contest-oscap - centos-stream-9-x86_64:contest-ansible - centos-stream-10-x86_64:contest-oscap - centos-stream-10-x86_64:contest-ansible while keeping at least some separation for re-running. Within each job, all tests still execute in parallel, as parallel tmt plans, so there shouldn't be any extra performance hit or added delay. The coalescing will however vastly reduce the amount of Testing Farm "requests", hopefully reducing load on TF a lot, in addition to reducing load on Github runners. --- I opted for defining the plans on the Contest side instead of in tests/tmt/ because support for plan importing (as we did before) is limited and ie. doesn't allow filtering by tags. Having the plans in Contest allows us to automatically filter out profiles which are subsets of others, tests that always fail by design, etc., etc. We also don't need to worry about which tests/profiles are on which CentOS Stream, since Contest has "adjust" rules for that already, and an empty plan is automatically SKIPPED by Testing Farm. Signed-off-by: Jiri Jaburek <comps@nomail.dom>
b8ca92f to
67d48fc
Compare
Collaborator
Author
|
Since this still seems to be running the old |
Collaborator
Author
|
Actually, it works here too - just the checks preview right above comment box is broken (shows old workflows). If you go to Checks of this PR, under Packit-as-a-Service, you'll see the new jobs and they all passed, ie. https://github.com/ComplianceAsCode/content/pull/14392/checks?check_run_id=63549779897 |
Mab879
approved these changes
Feb 13, 2026
Member
Mab879
left a comment
There was a problem hiding this comment.
We will need to adjust the required checks and rebase all opens PRs for this to take effect. I will leave this open so we can coordinate when we want this to happen.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
The new layout has much fewer jobs:
while keeping at least some separation for re-running.
Within each job, all tests still execute in parallel, as parallel tmt plans, so there shouldn't be any extra performance hit or added delay. The only disadvantage is that
/packit retest-failedor manual re-trigger will run all plans within that job (ie. all CentOS Stream 8 Ansible testing).The coalescing will however vastly reduce the amount of Testing Farm "requests", hopefully reducing load on TF a lot, in addition to reducing load on Github runners.
I opted for defining the plans on the Contest side instead of in
tests/tmt/because support for plan importing (as we did before) is limited and ie. doesn't allow filtering by tags.Having the plans in Contest allows us to automatically filter out profiles which are subsets of others, tests that always fail by design (
/static-checks/diff), etc., etc.We also don't need to worry about which tests/profiles are on which CentOS Stream, since Contest has "adjust" rules for that already, and an empty plan is automatically
SKIPPEDby Testing Farm.Rationale:
The current approach causes HEAVY load on Testing Farm's public ranch, one PR creation or push spinning up ~100 TF jobs (called "requests"), and Testing Farm public ranch can handle only ~130 running at any time.
Combining multiple parallel TMT plans into one TF "request" helps reduce that greatly, even if the underlying amount of cloud-reserved machines is the same.
In addition, this PR reduces our use of Github runners a lot, meaning more PRs can likely run CI at the same time.
The disadvantage is that there's a higher risk that any one
oscapexecution ends inoscapfreezing, and us needing to restart the whole Packit testing job. So there is a balance to be struck:The current approach tries to strike some balance (see the job list above).
Review Hints:
I have tested this in my fork, but this PR CI should test it again just fine. No rebasing of other PRs or impact on other PRs is expected, old Packit config should continue working well (for now, until maybe something in Contest changes a year down the road).
NOTE: There are extra commits in the PR cleaning up some things, see their descriptions for rationale.