I have a process that needs to be extensible by loading shared libraries. Is there a way to run the shared library code in a sandbox environment (other than an external process) so that if it segfaults it doesn't crash the process and has limitations on how much memory it can allocate, the cpu cycles it can use, etc.
I don't think there is a clean way to do it. You could try:
Catching segfaults and recovering from them (tricky, architecture specific, but doable)
Replacing calls to malloc/calloc for the library with an instrumented version that would count the allocated space (how to replace default malloc by code)
Alternatively use malloc hooks (http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html)
CPU cycles are accounted for the whole process, so I don't think there is any way you could get the info for just the library. Only viable option - manuallty measure ticks for every library call that your code makes.
In essence - this would be fun to try, but I recommend you go with the separate process approach and use RPC, quotas, ulimits etc.
No. If the shared library segfaults your process will segfault (the process is executing the library code). If you run it as an external process and use an RPC mechanism then you'd be okay as for crashing, but your program would need to detect when the service wasn't available (and something would need to restart it). Tools like chroot can sandbox processes, but not the individual libraries that an executable links to.
Related
I have a node process that runs tasks. When I run an intensive task, in top it can show more than 100% CPU usage (around 110%). From some research that I was doing, I figured that nodejs was single-threaded meaning it would only be running on one CPU per process.
Is it possible that the workload could take up the whole CPU so it moves some of the load to another CPU? Was unable to find a clear answer on this.
Other than specifically coding with WorkerThreads (which it doesn't sound like you are using), nodejs runs your Javascript code in only a single thread (e.g. the interpreter itself just uses one thread to run Javascript opcodes).
But, nodejs does have other threads that are used in the implementation of library functions such as file system operations and crypto operations and for the garbage collector. And, some 3rd party libraries may use native threads in their own implementation. So, it is definitely possible for nodejs to use more than just one core. It really depends upon what the code/task is doing and what library functions are being called.
Is it possible that the workload could take up the whole CPU so it moves some of the load to another CPU?
It does not move the running of your Javascript to another CPU. But as I said above, some library functions that use native code may use additional threads.
Is it possible to profile only a shared library without looking the main program ?
For example, I developed a plugin and I would like to profile but with no need to profile the whole application. I just want to see the bottleneck of my plugin. (Of course, I would like to profile it while the main application is running and has loaded my plugin...)
I'm working on linux and I'm used to callgrind, but for the curiosity, I'm also interested by the possibilities on all systems, so I let the question general.
I'm interested in this because the main program is quite slow, and don't want to add the overhead of profiling on since I'm not interested by the main program performance here...
In Linux perf statistical profiling tool has very low overhead (1-2%), so you can profile entire application with perf record ./your_application and then analyze generated profile perf.data with perf report command. You may filter perf report output to some shared libraries or search function names of your plugin. Read more at http://www.brendangregg.com/perf.html
Callgrind is not just a profiler, it is binary recompiler used to implement exact profiler with instrumentation approach and it has 10-20 times overhead for any code, even when profiling tool is not enabled.
Your plugin only runs during certain times, right? Like when the user requests certain activities? People use this method with any IDE, manually pausing during that time. The pause will land in the plugin according to how much time it uses. Before you pause there is no performance impact because the app runs full speed, while in the pause, it is stopped, which you don't care because you're diagnosing your plugin.
Hoi.
I am working on an experiment allowing users to use 1% of my CPU. That's like your own Webserver; but a big dynamic remote execution framework (dont ask about that), and I dont want users to use API functions like create files, no sockets, no threads, no console output, nothing.
Update1: People will be sending me binaries, so interrupt 0x80 is possible. Therefore... Kernel?
I need to limit a process so it cannot do anything but use a single pipe. Through that pipe the process will use my own wrapped and controlled API.
Is that even possible? I thought like a Linux kernel module.
The issues with limiting RAM and CPU are not primary here, for that there's something on google.
Thanks in advance!
The ptrace facility will allow your program to observe and control the operation of another process. Using the PTRACE_SYSCALL flag, you can stop the child process before every syscall, and make a decision about whether you want to allow that system call to proceed.
You might want to look at what Google is doing with their Native Client technology and the seccomp sandbox. The Native Client (NaCl) stuff is intended to let x86 binaries supplied by a web site run inside a user's local browser. The problem of malicious binaries is similar to what you face, so most of the technology/research probably applies directly.
I want to provide a way to upload plugins (assemblies) to a site by users for a scripting propose. Through mono.cecil I can analyse those assemblies and limit access only to a predefined list of functions, but I also need to limit memory usage, execution time and kill the thread if it goes to overdraft this resources.
I think I can monitor the memory usage by the profiler api, but as I know there are no tools to abort thread with guarantee. Is there any way to abort thread with guarantee? Maybe I should run code using embedding mono and control the execution of thread in native part of an application, is it possible?
You could use Thread.Abort() as long as you don't allow the plugin code to ResetAbort().
Thread level control was not practical IMHO (anyone did that in the past). Typically you should consider process level control of memory usage or application domain level.
My VPS account has been occasionally running out of memory. It's using Apache on Linux. Support says it's a slow memory leak and has enabled MaxRequestsPerChild to deal with it.
I have a few questions about this. When a child process dies, will it cause my scripts to lose session data? Does anyone have advice on how I can track down this memory leak?
Thanks
No, when a child process dies you will not lose any data unless it was in the middle of a request at the time (which should not happen if it exits due to MaxRequestsPerChild).
You should try to reproduce the memory leak using an identical software stack on your test system. You can use tools such as Valgrind to try to detect it.
You can also try a debug build of your web server and its modules, which will enable you to detect what's going on.
It's difficult to reproduce the behaviour of production systems in non-production ones. If you have auto-test coverage of your web application, you could try using your full auto-test suite, but in practice this is unlikely to cover every code path therefore may miss the leaky one.
When a child process dies, will it cause my scripts to lose session data?
Without knowing what scripting language and session handler you are using (and the actual code) it rather hard to say.
In most cases, using scripting languages in modules or via [fast] cgi, then its very unlikely that the session data would actually be lost - although if the process dies in the middle of processing a request it may not get the chance to write the updated session back to whatever is storing the session. And in the very unlikely event it dies during the writeback, it may corrupt the session data. These are quite exceptional circumstances.
OTOH if your application logic is implemented via a daemon (e.g. a Java container) then its quite probable that memory leaks could accumulate (although these would be reported against a different process).
Note that if the problem is alleviated by setting MaxRequestsPerChild then it implies that the problem is occurring in an Apache module.
The production releases of Apache itself, in my experience, is very stable without memory leaks. However I've not used all the modules. Not sure if ExtendedStatus gives a breakdwon of memory usage by module - might be worth checking.
I've previously seen problems with the memory management of modules loaded by the PHP module not respecting PHP's memory limits - these did clear down at the end of the request though.
C.