I'm looking to figure out if it's possible to automate linux gui apps, either on wayland or x.org.
I think that triggering mouse or keyboard inputs should be possible, but I'm not sure how to figure out where the button or text box I want to click on is located on the screen. In Windows we have the api which allows you to trigger events based on those xml selectors. Does anyone know if there is any similar way to find gui elements in Linux? I'm also not sure if wayland and x should offer these apis, or the desktop evironment, such as Gnome.
Thank you.
I'm new to Windows development, and am looking for assistance on where to get started for a particular project.
In short, I want to create a windowed application that allows a user to send keyboard and mouse inputs to another application, by interacting with various UI controls via touch. Essentially a custom on-screen keyboard/touchpad that can be used for sending keyboard-shortcuts to other applications.
There are two applications in Windows 10 that perform exactly the way I would want my new app to - the On-Screen Keyboard and Touchpad:
https://support.microsoft.com/en-us/help/4337906/windows-10-open-the-on-screen-touchpad
https://support.microsoft.com/en-us/help/10762/windows-use-on-screen-keyboard
At the most basic level, I want to define my own interface (or allow the end user to define their own), and use the same code that the onscreen keyboard/touchpad are using for handling touch events and injecting inputs into the system.
I'm uncertain at what level I would need to start to get the functionality I need - UWP? WPF? C++?
If anyone has any insight into how the on-screen utilities were built, I think that would give me an excellent head start.
I've developed an app based on NodeJS with Electron for a web agency and try to extend it to the touch bar functionality of the MacBook Pro.
Developers launch the app once, which then automatically starts various processes in the background, such as compiling SASS files, optimizing images, etc.
Typically, these developers work with programs such as SublimeText or Atom. However, it often happens that the app expects user input during programming or inserts hints to optimize the code. I would like to display these things in the Touch Bar so that the user does not have to switch between the apps during programming. Unfortunately, the touch bar view of my app is hidden when switching to SublimeText or another app and replaced by the current app in the foreground.
Is there a way to display an app's specific touch bar while the app is running (even hidden) permanently without replacing it? I wonder if that is at least natively possible?
I couldn't find any information about this on the Electron documentary or somewhere else…
Thank you in advance!
We're using Red Hat Linux 6.4, and our application is built using Qt. The application has multiple windows and we support a layout system where our users can save the application layout and restore it later. The application is cross-platform, and on Windows, everything is fine. On Linux, we're having problems restoring windows when a window spans multiple monitors. Our configuration uses a single virtual X display spanning all monitors, and the users can manually position and size windows across the monitors as desired.
What we've found is that the window manager is enforcing a policy on windows that are programmatically set and forcing them not to span across divide between two monitors. When we attempt to restore a saved layout containing a window that spanned monitors, the window manager reduces its size and repositions at as it sees fit. Basically, as long as the user makes the change by dragging and resizing the window, the window manager respects it, but an application that programmatically sets it gets overridden. I'm sure someone somewhere thought this was a reasonable restriction, but our customers disagree.
A developer here has spent days searching and experimented trying to find a way to work around this behavior programmatically, or better yet, tell the window manager to stop doing that. We're using the GNOME desktop and Qt 4.8.x.
Any ideas?
Thank you,
Doug McGrath
I want to know how to create a simple desktop environment, such as KDE, Gnome and LXDE, for Linux, or specifically, Ubuntu/Kubuntu, but I do not know how to begin and how complex it is.
I want to create a simple, efficient and lightweight desktop and to reproduce some good features from KDE, Gnome and Windows XP. I do not want to reinvent the wheel, I want to design the graphical interface and main dialogs.
I could use C++ with Qt, or maybe with X11 calls. A simple desktop like TWM would be the first step, then I could add taskbar, start menu, new features and play with new ideas.
Thanks.
Edit
I have installed icewm and I think it is what I need to begin. On the login screen, I can choose if I use KDE (desktop environment) or icewm (window manager). I do not understand what a window manager actually is.
I have downloaded icewm source code and I am confused. There are some concepts that I have to understand, such as, what happens after the login screen when the desktop is loaded, how a window works, etc.
Edit 2
I think I need a good tutorial. I have to understand how a Window Manager or Desktop Environment work. There are some concepts that I have to know.
This is no simple feat but by no means impossible.
Other people have done it, there are plenty of DE or WM out there so there is no reason that you, with patient, skill and lots coffee couldn't do something great.
Learn the difference between DE and WM.
Test different DE and WM, maybe one does exactly what you want it to do?
Make your own custom DE
Here is a guide to get you started creating your own custom DE
After doing all this you should be better equipped to figure out what you actually want to do.
Fork a project and get started.
These are all open source projects, so I suggest you grab one of the simpler window managers and rummage around in its source code. I should warn you, though: this kind of project is not for the faint-hearted, and the likelihood of your little toy project becoming a mainstream desktop is low, so don't go into this imagining it's going to be anything more than a learning exercise (and if it does become something more, you'll be pleasantly surprised).
Strategically, your best bet is to fork an existing WM, rather than trying to build a brand new one from scratch.
One final point: Qt is a windowing framework for C++ that sits on top of a window manager. It isn't a window manager, per se, like Gnome/TWM/etc. are. And, as #ptomato rightly points out, Gnome is actually a desktop environment that itself sits on top of a window manager, which is selectable, but defaults to Mutter.
I would use X calls and learn how to boot to a CLI (command line interface). Booting to a CLI allows you to switch between window managers / desktop environments. Then I would design a window manager. And finally finish off with a desktop environment.
It starts with frame buffers. These are how 80's graphics worked.
Then you have the X window system. This however doesn't show you multiple windows at once. Only one at a time.
A window manager is the program that controls where your windows go. This allows for multiple X windows to take up one screen.
A desktop environment then builds on that to add things like a task bar, start menu and desktop icons.