On Linux (Fedora 26 with GNOME in particular) there is a problem with VLC that the UI is broken because of Qt bugs. This is not related to VLC though. Running VLC on XWayland instead would be a good short-time solution.
However, VLC doesn't honor the default platform backend settings of the host Qt library. The same goes for the QT_QPA_PLATFORM variable.
Making it possible to select the platform backend would be a good short-term workaround for the issues we currently experience.
I can write a short patch if this is accepted as a valid issue.
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Child items
0
No child items are currently assigned. Use child items to break down this issue into smaller parts.
Linked items
0
Link issues together to show that they're related.
Learn more.
Well... if you can write a patch that works, that would be great. But I doubt that's even possible without breaking something else or changing the interface of the Qt library.
However, VLC doesn't honor the default platform backend settings of the host Qt library.
How do we get this information, from our code?
The same goes for the QT_QPA_PLATFORM variable.
I don't see why we need to do anything, tbh. It should be managed by Qt5, automatically...
In /modules/gui/qt/qt.cpp:414 there is the check that just goes along the lines of "if it's possible to connect to wayland, just use wayland for the UI". I'm now on your IRC channel, I'm going to wait for Victorien.
It went unnoticed because on my workstation, requesting Gnome under Wayland automatically fallback to Gnome Under Xorg, whereas I should have a wayland display (using intel card).
The current workaround for Gnome under Wayland for vlc users is to use:
WAYLAND_DISPLAY='' vlc
or
cvlc
Why does vlc force a platform at all? There are a number of out-of-tree platform plugins, doing this means that all of them won't work with vlc without patching, in addition to the problem of not respecting user settings.
My suggestion would be that the "-platform" hardcoding in qt.cpp is simply removed. Then the default plugin used will be selected, or it can be overridden if the user sets QT_QPA_PLATFORM, as all of this is already done in Qt itself. Would such a change be accepted?
If no platforms work, VLC needs to fall back to another interface than Qt. But Qt exits the whole process. To work around that Qt deficiency, VLC checks if the X11 display is available, and only then starts Qt, forcing the X11 back-end. The proper solution requires changing the Qt API, which is not something that this bug tracker can take care of.
Alternatively, we need to try to instantiate a QApplication in a separate process first, to see if it exits or works. But this is really slow because the really slow QApplication constructor then runs twice instead of just once. Unfortunately, that seems to be the only viable option with Qt5.
There are a number of out-of-tree platform plugins, doing this means that all of them won't work with vlc without patching, in addition to the problem of not respecting user settings.
Well yes... but then again... so far, nobody has proposed a viable alternative.
My suggestion would be that the "-platform" hardcoding in qt.cpp is simply removed.
As pointed out several times, that alone would cause VLC to crash at start-up in plain text mode, which is not acceptable.
Then the default plugin used will be selected, or it can be overridden if the user sets QT_QPA_PLATFORM, as all of this is already done in Qt itself. Would such a change be accepted?