Replace GTest with custom framework, add performance test filtering and coverage#744
Open
Replace GTest with custom framework, add performance test filtering and coverage#744
Conversation
Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
- Modified test/mp_unit/mp_unit_tests.hpp to use ../framework.hpp instead of gtest/gtest.h - Enhanced test/framework.hpp with GTest-compatible APIs: - Added Environment base class for global test setup/teardown - Added TestInfo and UnitTest classes for test metadata access - Added GTEST_SKIP macro support via SkipHelper class - Added namespace alias 'testing' for compatibility - Added InitGoogleTest and AddGlobalTestEnvironment helper functions - Updated test/framework.cc with implementations for new classes - All mp_unit test files now use framework.hpp through mp_unit_tests.hpp - Formatting applied via lint.sh Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
- Remove duplicate static getMPIRank() and getMPISize() functions - Add full namespace qualification to GTEST_SKIP macro Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
- Move test framework from test/perf/ to test/ for shared use - Add GTest-compatible macros (TEST, TEST_F, EXPECT_*, ASSERT_*, etc.) - Remove GTest dependency from CMakeLists.txt - Add test_framework library for unit and mp_unit tests - Add code coverage support with lcov (MSCCLPP_ENABLE_COVERAGE option) - Update perf tests to use shared framework Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
Support --gtest_filter command line argument for test filtering, compatible with Azure pipeline configurations. Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
- Move PerfTestResult struct definition outside vector declaration - Move getCurrentTimestamp to anonymous namespace - Add documentation for GTEST_SKIP macro explaining RAII pattern Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Remove GTest and implement custom testing framework
Replace GTest with custom framework and add code coverage
Feb 11, 2026
The recent removal of GTest and introduction of custom test framework requires MPI dependency which is not needed for CodeQL analysis. Disable test building in CodeQL workflows to fix the build failures. CodeQL only needs to analyze the core library code, not the tests. Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
Copilot
AI
changed the title
Replace GTest with custom framework and add code coverage
Fix CodeQL workflows by disabling test builds
Feb 11, 2026
Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
- Make MPI REQUIRED for test builds (clearer error messages) - Add project include directories to test_framework library - Fix core_tests.cc to use custom framework correctly - Fix mp_unit_tests.hpp to use mscclpp::test namespace - Add FAIL() macro with streaming support for test messages - Building tests now works in Docker environment with GPU bypass Tests can now be built using: docker run --rm -v $(pwd):/workspace -w /workspace \ ghcr.io/microsoft/mscclpp/mscclpp:base-dev-cuda12.4 bash -c \ "mkdir build && cd build && cmake -DMSCCLPP_BYPASS_GPU_CHECK=ON \ -DMSCCLPP_USE_CUDA=ON .. && make -j" Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
- Move helper classes inside namespace for proper access - Remove duplicate class definitions outside namespace - Tests can now build in Docker with CUDA toolkit installed - Remaining issues: ErrorCode and TransportFlags need operator<< for EXPECT_EQ Successfully building with Docker: docker run --rm -v $(pwd):/workspace -w /workspace \ ghcr.io/microsoft/mscclpp/mscclpp:base-dev-cuda12.4 bash -c \ "mkdir build && cd build && cmake -DMSCCLPP_BYPASS_GPU_CHECK=ON \ -DMSCCLPP_USE_CUDA=ON .. && make -j4" Note: Some unit tests (errors_tests.cc, core_tests.cc) need operator<< defined for ErrorCode and TransportFlags to compile with custom framework. Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
Copilot
AI
changed the title
Fix CodeQL workflows by disabling test builds
Fix test builds with GPU bypass in Docker environment
Feb 11, 2026
- Remove build_test/ directory containing CMake cache and build files - Update .gitignore to exclude build_*/ pattern to prevent future accidents These CMake artifacts (CMakeCache.txt, CMakeFiles/, generated headers) were accidentally committed and should never be in version control. Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
Copilot
AI
changed the title
Fix test builds with GPU bypass in Docker environment
Remove CMake build artifacts from repository
Feb 11, 2026
- Add nlohmann::ordered_json metrics field to TestResult struct - Add nlohmann/json.hpp include to test/framework.hpp - Link test_framework with nlohmann_json::nlohmann_json - Replace PerfTestResult with TestResult in test/perf/framework.cc - Move perf utility functions to utils namespace for consistency - Remove duplicate PerfTestResult struct definition This consolidates the two similar structs into one, reducing code duplication while maintaining all necessary fields for both unit tests (passed/failure_message) and performance tests (metrics). Verified build succeeds with Docker: docker run --rm -v $(pwd):/workspace -w /workspace \ ghcr.io/microsoft/mscclpp/mscclpp:base-dev-cuda12.4 bash -c \ "cd /workspace/build && make -j4 fifo_test" Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
Co-authored-by: chhwang <8018170+chhwang@users.noreply.github.com>
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
|
/azp run mscclpp-ut |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
|
/azp run mscclpp-ut |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
|
/azp run mscclpp-ut |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
|
/azp run mscclpp-ut |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
|
/azp run mscclpp-ut |
|
Azure Pipelines failed to run 1 pipeline(s). |
Contributor
|
/azp run mscclpp-ut |
|
Azure Pipelines failed to run 1 pipeline(s). |
Contributor
|
/azp run mscclpp-ut |
|
Azure Pipelines failed to run 1 pipeline(s). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test/framework.*) that covers only what the tests actually use — a unifiedTEST()macro with SFINAE-based fixture auto-detection,EXPECT_*/ASSERT_*assertions, environments, and setup/teardown.--exclude-perf-testsflag and substring-based negative filteringMSCCLPP_ENABLE_COVERAGECMake option with gcov/lcov; CI uploads to Codecovtest/perf/into main test targets