feat: Cleanup Qt-based tray implementation

[Kishi85]

Description

This reworks the Qt-based tray with an adapted version of QtTrayMenu.h/cpp from upstream as suggested in LizardByte/Sunshine#4907.

This version restores libnotify for notifications due to potential problems caused by DBus interface limitations when running with elevated privileges (e.g. as used by Sunshine). The new libnotify integration also tries to separate the used codepaths as best as possible to keep the implementation clean and easily understandable (with a fallback to basic notifications using QtTrayMenu::showMessage).

There is still an issue with left clicking the tray icon on Wayland (also present in the current AI based implementation) which caused by https://qt-project.atlassian.net/browse/QTBUG-139921 that can unfortunately not be worked around.

This version is platform-neutral for QtTrayMenu.h/cpp code with Linux platform-specific parts (and libnotify integration) restricted to tray_linux.cpp. Qt-provided features are used as much as possible to keep this platform-independent.

Screenshot

Issues Fixed or Closed

Roadmap Issues

Type of Change

• feat: New feature (non-breaking change which adds functionality)
• fix: Bug fix (non-breaking change which fixes an issue)
• docs: Documentation only changes
• style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
• refactor: Code change that neither fixes a bug nor adds a feature
• perf: Code change that improves performance
• test: Adding missing tests or correcting existing tests
• build: Changes that affect the build system or external dependencies
• ci: Changes to CI configuration files and scripts
• chore: Other changes that don't modify src or test files
• revert: Reverts a previous commit
• BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

• Code follows the style guidelines of this project
• Code has been self-reviewed
• Code has been commented, particularly in hard-to-understand areas
• Code docstring/documentation-blocks for new or existing methods/components have been added or updated
• Unit tests have been added or updated for any new or modified functionality

AI Usage

• None: No AI tools were used in creating this PR
• Light: AI provided minor assistance (formatting, simple suggestions)
• Moderate: AI helped with code generation or debugging specific parts
• Heavy: AI generated most or all of the code changes

[codecov]

Codecov Report

❌ Patch coverage is 61.26984% with 122 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.38%. Comparing base (dc4be3a) to head (5f5e43d).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/QtTrayMenu.cpp	54.00%	59 Missing and 33 partials ⚠️
src/tray_linux.cpp	73.91%	11 Missing and 19 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #121      +/-   ##
==========================================
+ Coverage   63.75%   64.38%   +0.63%     
==========================================
  Files           4        5       +1     
  Lines         858      643     -215     
  Branches      313      209     -104     
==========================================
- Hits          547      414     -133     
+ Misses        205      153      -52     
+ Partials      106       76      -30     

Flag	Coverage Δ
Linux-qt5	55.16% <61.26%> (-1.96%)	⬇️
Linux-qt6	55.16% <61.26%> (-1.96%)	⬇️
Windows	70.67% <ø> (ø)
macOS	56.63% <ø> (-2.66%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/tray_linux.cpp	65.51% <73.91%> (+6.05%)	⬆️
src/QtTrayMenu.cpp	54.00% <54.00%> (ø)

... and 1 file with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc4be3a...5f5e43d. Read the comment docs.

[Kishi85]

I've fixed most of the issues I've found so far (added all the necessary pointer valid checks/etc. and moved the exit() order around so everything gets cleaned up in reverse order (clearing references as necessary on the go). But no matter what I do the CI pipeline segfaults at this one specific test:

[ RUN      ] TrayTest.TestCheckboxStates
/home/runner/work/_temp/d5b937f4-9556-4df2-8f27-571e2cb10ca8.sh: line 1:  8581 Segmentation fault      (core dumped) ./test_tray --gtest_color=yes --gtest_output=xml:test_results.xml
Error: Process completed with exit code 139.

I can run all the tests fine locally after I've added support for spectacle screenshots due to using KDE, tests running both with clang and gcc compiled project. It just wont complete when run on the CI-container with ubuntu-latest. I'm a bit out of ideas here and it looks like I have to test this on a native Ubuntu-latest install at some point to continue this further.

[ReenigneArcher]

@Kishi85 have a look at #123

[Kishi85]

@Kishi85 have a look at #123

I'll pick that test change if you dont mind as it IMHO makes more sense logically

[Kishi85]

Also the change you did for showing the menu on wayland in #123 does not work. Same issue as with show() due to that bug in Qt:
qt.qpa.wayland: Failed to create grabbing popup. Ensure popup QWidgetWindow(0x55c3cc28b8c0, name="QMenuClassWindow") has a transientParent set and that parent window has received input.

Looks like the only way to get left-click to work on Linux/Wayland is for Qt to fix the bug, every other workaround either fails silently or yields above message unfortunately.

[Kishi85]

One main deficit of the QSystemTray::showMessage notification is that it does not hide immediately when you click it even if it triggers the callback. Neither can you programmatically close the message via the trayIcon object hence the screenshots showing multiple balloons due to tests firing in rapid succession.

After looking around for a bit the best thing (in terms of interoperability and functionality) seems to be using libnotify just for the notifications (as the XDG desktop notification path is not really viable due to the process privilege constraints).

Good thing here is that now that most of the Qt Tray Menu stuff is now isolated, it should be easy to do something similar for notifications and have a clean implementation.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[ReenigneArcher]

Unless you want me to wait, I think I'm good with merging this as is. I don't think anyone is going to test it out on Sunshine until it's merged into the pre-releases.

[Kishi85]

Unless you want me to wait, I think I'm good with merging this as is. I don't think anyone is going to test it out on Sunshine until it's merged into the pre-releases.

Go for it. If there's anything to add I'll do a follow up PR.

[Kishi85]

@ReenigneArcher
5f5e43d seems to cause issues now (just realized this while trying a game on my TV).
I'll put up a PR to revert that specific commit in a second as without that change everything is running smoothly here.

[ReenigneArcher]

@Kishi85 so with these changes, do you think it could pave the way to use QT on Windows and macOS as well?

[Kishi85]

@Kishi85 so with these changes, do you think it could pave the way to use QT on Windows and macOS as well?

The QtTrayMenu class should work on any QT supported platform. The only thing that likely needs attention per platform is notifications due to already noted restrictions of QSystemTrayIcon::showMessage.