How do I make my Rust program communicate to systemd's WatchDog? - rust

I've followed this tutorial:
https://www.linode.com/docs/guides/start-service-at-boot/
and set everything up.
Since my problem is exactly this:
https://unix.stackexchange.com/questions/298027/restart-systemd-service-when-output-is-no-longer-generated
I've also followed the answers here.
The problem with the solution: after 7 hours of runtime, the program stuck. No output generated, systemd didn't restart it.
in my_service.service are both suggested mandatory entries
everything else is correct
WatchdogSec=30
Restart=on-watchdog
Can I manually make my Rust program communicate with systemd? And if yes - how? I'd love to make it notify systemd at particular lines periodically.
The docs couldn't be more minimal:
https://docs.rs/systemd/latest/systemd/daemon/constant.STATE_WATCHDOG.html
I don't understand a single thing..
Another example crate libsystemd, there's 3 search results for WatchDog and this one is the most relevant I guess.
https://docs.rs/libsystemd/latest/libsystemd/daemon/enum.NotifyState.html#variant.Watchdog
IDK how I am able to accomplish anything here. Do I just paste this line anywhere in the program that I want?
libsystemd::daemon::NotifyState
how will it know the PID?
In any case: each of those packages has multiple methods, and my program hangs after anywhere from 1-24 hours, trial and error might take weeks.
How can I make my Rust program communicate to the systemd from inside threads, or if impossible - just manually set it up and ensure the WatchDog receives the signal as I want it? What's the line of code for sending a single notification from Rust program to systemd?
As mentioned above the go-to logic is simple: if no print output, restart the program.

You have to send the message somehow. By looking for functions that use the NotifyState I found both systemd::daemon::notify and libsystemd::daemon::notify either of which will do.
For example:
use systemd::daemon::{notify, STATE_WATCHDOG};
notify(false, [(STATE_WATCHDOG, "1")].iter()).unrwap();

Related

PyO3 - prevent user submitted code from looping and blocking server thread

I'm writing a game in Rust where each player can submit some python scripts to the server in order to automate various tasks in the game. I plan on using pyo3 to run the python from rust.
However, I can see an issue arising if a player submits a script like this:
def on_event(e):
while True:
pass
Now when the server calls the function (using something like PyAny::call1()) the thread will hang as it reaches the infinite loop.
My first thought was to have pyo3 execute the python one statement at a time, therefore being able to exit if the script been running for over a certain threshold, but I don't think pyo3 supports this.
My next idea was to give each player their own thread to run their own scripts on, that way if one of their scripts got stuck it only affected their gameplay. However, I still have the issue of not being able to kill a thread when it gets stuck in an infinite loop - if a lot of players submitted scripts that just looped, lots of threads would start using a lot of CPU time.
All I need is way to execute python scripts in a way such that if one of them does loop, it does not affect the server's performance at all.
Thanks :)
One solution is to restrict the time that you give each user script to run.
You can do it via PyThreadState_SetAsyncExc, see here for some code. It uses C calls of the interpreter, which you probably can access in Rust (with PyO3 FFI magic).
Another way would be to do it on the OS level: if you spawn a process for the user script, and then kill it when it runs for too long. This might be more secure if you limit what a process can access (with some OS calls), but requires some boilerplate to communicate between the host.

Respond Y dynamically to a shell program

We have a startup script for an application (Owned and developed by different team but deployments are managed by us), which will prompt Y/N to confirm starting post deployment. But the number of times it will prompt will vary, depends on changes in the release.
So the number of times it will prompt would vary from 1 to N (Might be even 100 or more than that).
We have automated the deployment and startup using Jenkins shell script jobs. But startup prompts number is hardcoded to 20 which might be more at sometime.
Could anyone please advise how number of prompts can be handled dynamically. We need to pass Y whenever there is pattern in the output "Do you really want to start".
Checked few options like expect, read. But not able to come up with a solution.
Thanks in advance!
In general, the best way to handle this is by (a) using a standard process management system, such as your distro's preferred init system; or, if that's not possible, (b) to adjust the script to run noninteractively (e.g., with a --yes or --noninteractive option).
Barring that, assuming your script reads from standard input and not the TTY, you can use the standard program yes and pipe it into the command you're running, like so:
$ yes | ./deploy
yes prints y (or its argument) over and over until it's killed, usually by SIGPIPE.
If your process is reading from /dev/tty instead of standard input, and you really can't convince the other team to come to their senses and add an appropriate option, you'll need to use expect for this.

Simple Qt threading mechanism with progress?

I want to look for files with given extensions recursively from a given root directory and to display the number of files currently found in my GUI.
Since this kind of processing may be long, the GUI may be blocked.
I could just wait for the end of the processing and get the file count, but I am learning Qt (PyQt), so I see this as a training.
So I have read Qt doc:
When to Use Alternatives to Threads, and I don't think it's for me.
Then I read:
Choosing an Appropriate Approach, and I think my solution is the first one:
Run a new linear function within another thread, optionally with
progress updates during the run
But in this case you have 3 choices:
Qt provides different solutions:
Place the function in a reimplementation of QThread::run() and start the QThread. Emit signals to update progress. OR
Place the function in a reimplementation of QRunnable::run() and add the QRunnable to a QThreadPool. Write to a thread-safe variable
to update progress. OR
Run the function using QtConcurrent::run(). Write to a thread-safe variable to update progress.
Could you tell me how to choose the best one?
I have read some "solutions" but I'd like to understand why you should use one methodology instead of another one.
And also since I am looking for files, I may have a directory in which many files would match the search criteria. So it would mean lots of interruptions. Is there something special to keep in mind regarding this?
Thank you!
From what I know (hopefully more can chime in).
QThread offers support with signal interaction. For example, you'd be able to stop your concurrent function with a signal. Not sure how you'd do that with the other options, if at all.
Things to keep in mind: widgets all have to live in the main thread, but can communicate with other other threads via signals & slots.
Another quick thread on the topic w/ some decent bullet-points.
https://qt-project.org/forums/viewthread/50165/
Best of luck on your project, and welcome to Qt!

What's stopping my command from terminating?

I'm writing a command line tool for installing Windows services using Node JS. After running a bunch of async operations, my tool should print a success message then quit. Sometimes however, it prints its success message and doesn't quit.
Is there a way to view what is queued on Node's internal event loop, so I can see what is preventing my tool from quitting?
The most typical culprit for me in CLI apps is event listeners that are keeping the process alive. I obviously can't say if that's relevant to you without seeing your code, though.
To answer your more general question, I don't believe there are any direct ways to view all outstanding tasks in the event loop (at least not from JS-land). You can, however, get pretty close with process._getActiveHandles() and process._getActiveRequests().
I really recommend you look up the documentation for them, though. Because you won't find any. They're undocumented. And they start with underscores. Use at your own peril. :)
try to use some tools to clarify the workflow - for example, the https://github.com/caolan/async#waterfall or https://github.com/caolan/async#eachseriesarr-iterator-callback
so, you don't lose the callback called and can catch any erros thrown while executing commands.
I think you also need to provide some code samples that leads to this errors.

As a user, I want a status bar (or similar) to notify me that a job is working when using a Wx.Python gui app

Can someone recommend a straight forward way of adding some type of graphical notification (status bar, spinning clocks, etc...) to my wx.Python gui application? Currently, it searches logs on a server for unique strings, and often times takes upwards to 3-4 minutes to complete. However, it would be convenient to have some type of display letting a user know that the status of the job towards finishing/completion. If I added a feature like this, I'm not sure, but I'm afraid I may have to look at using threads ... and I'm a complete newbie to Python? Any help and direction will be appreciated.
Yes, you'd need to use threads or queues or something similar. Fortunately, there are some excellent examples here: http://wiki.wxpython.org/LongRunningTasks and this tutorial I wrote is pretty straight-forward too: http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/
Using threads isn't that hard. Basically you put the long running part in the thread and every so often, you send a status update to your GUI using a thread-safe method, like wx.PostEvent or wx.CallAfter. Then you can update your statusbar or a progress bar or whatever you're using.

Resources