I am running a ghostscript command from the shell to convert a postscript file to JPG, like so:
gs -dBATCH - dSAFER -dNOPAUSE -sDEVICE=jpeg -sOutputFile=out.jpg source.ps
Most of the time this works fine, but occasionally a bad file will cause it to hang.
As I am not an expect in GhostScript, I can't say whether there are any built-in failsafe mechanisms that could prevent it from failing, or at least make it fail in a more graceful manner (right now I have to kill the process)
Thanks
On those bad files, I would suggest trying them with either -dNOTRANSPARENCY and/or -dNOINTERPOLATION. Disabling transparency, if it makes a difference, will likely cause the output to be incorrect, but it would give you a hint as to whether you've found a bug, or a slow file. Transparency blending and image interpolation are both areas that can easily consume a lot of CPU time and memory.
You might try leaving it running overnight, again in an attempt to establish whether this is a bug or not.
Also, if you're not already doing so, you could consider upgrading to the latest release (9.05), we've fixed a number of problems, and improved performance somewhat in the last few releases.
Finally, if you have an example you can share, report it with the example at Ghostscript Bugzilla
Parenthetically, using a Postscript RIP in a traditional "server" configuration generally relies on a Postscript infinite loop - the "server loop" is usually implemented in Postscript.
Chris
PostScript is a completely general programming language. So PostScript programs, like programs in any other full programming language, can get stuck in endless loops as well as go wrong in all the other usual ways. The Halting Theorem proves that, in general, it is impossible to predict whether a given program will get stuck in a loop or not purely from some automatic analysis of it (other than actually running it).
The only way you can guard against hangs is impose some kind of arbitrary time limit on the execution of a PostScript program, and kill the Ghostscript process when that time is exceeded.
Related
I'm working on solving non-linear optimization problems. Currently I'm evaluating different algorithms to find out which one fits my problem best. I'm using MATLAB 2020b on Ubuntu 20.04 LTS.
I currently got IPOPT with the HSL solvers up and running. My problem consists of a few hundred variables (~500 at the moment). Switching to MA97 didn't show any performance improvements. Probably my problem is too small? Nevertheless, I'd like to check if the parallelism of MA97 compared to e.g. MA27 is working properly, hence, if I compiled everything correctly.
Is there any sample problem where I can verify if MA97 is working multi-threaded but MA27 not?
Several approaches suggested:
Try to debug from Matlab into the native code and see what IPOPT is calling into. This approach is tricky because Matlab itself uses OpenMP.
Use proc filesystem, if there are subdirectories under /proc/self/tasks, the process is multi-threaded. This approach has the same issues as above (Matlab backend will likely be using multi-threading).
Use environmental variables to limit the number of OpenMP threads (OMP_THREAD_LIMIT) and check for performance changes. Will need to measure this difference specifically around the call to IPOPT, as again, Matlab will be using OpenMP for its own functionality.
Matlab has a built-in profiler:
% start profiling
profile on
% your code ...
% launch profile viewer
profile viewer
Also, the IPOPT logs may be helpful. If the solver is multithreaded, there should be a difference between elapsed real-time and CPU time. This scales with parallelism, i.e
CPU time ∝ threads count * elapsed real-time
This is a rough approximation which is only valid up to the point you become resource-constrained on the number of threads.
I hope you already solved your problem. But I want to reply to help others. If you pass option linear_solver ma97 IPOPT should use HSL MA97 solver. I dont know how it can be done from MATLAB but if you add working directory "ipopt.opt" file IPOPT will read this file and apply specified options.
File content: (no equality sign)
linear_solver ma97
I would like to use gnuplot 5.x on my employer's hardware platforms (Dell workstations & laptop running Windows 7 and 10 respectively). Our system administration has identified "buffer overflow vulnerabilities" associated with this software.
A simple search for "gnuplot buffer overflow" on Google yields some information related to this, e.g.:
https://sourceforge.net/p/gnuplot/bugs/2093/
https://sourceforge.net/p/gnuplot/bugs/1413/
https://research.loginsoft.com/bugs/buffer-overflow-vulnerability-in-ps_options-gnuplot-5-2-5/
I am not a C++ savvy software programmer nor a cyber-security specialist. What I see in some of these posts is comments like:
Link #1: "an attacker might use this flaw to overflow important data to hijack the control flow".
Link #3: "This allows an attacker to cause Denial of Service (Segmentation fault and Memory Corruption) or possibly have unspecified other impacts when a victim opens a specially crafted file".
And I am thinking to myself: "Really?". Are these assessments credible, and if so, how can such a long-established and widely-used tool have such a serious flaw?
As you can imagine, I am having trouble getting the installation and usage of gnuplot approved by my employer. Any evidence-based information you can provide to shed light on this matter would be very much appreciated.
Thank you very much.
Maziar.
All programs have bugs. You point to two specific bugs that were identified in 2018, reported on the project bug tracker, and fixed in the first subsequent release of the program. In both cases the bugs were of the sort "you could crash this program if you give it certain improperly formed commands". Those particular bugs no longer exist in current versions of the program, since they were reported and fixed. But other bugs of the sort probably exist in gnuplot and in every single program running on your computer. That's life in an imperfect world.
In other words, you or your employer's security policy must decide whether it is really a problem that "this program might crash if you feed it garbage".
A more realistic concern is that gnuplot is essentially a scripting language. It can read and write any file that the user has permission to access. If you run a gnuplot script that came from a malicious source, it might overwrite your files or issue harmful commands to your system. This is the same concern as if you downloaded and executed a *.com or *.exe file or a python script any other series of commands provided by a malicious source. It's not really an indictment of the program, only of lax security practices by the user.
Is there any good reason not to run a brief unknown (30 line) assembly script inline in a usermode c program for dynamic analysis directly on my laptop?
There's only one system call to time, and at this point I can tell that it's a function that takes a c string and it's length, and performs some sort of encryption on it in a loop, which only iterates through the string as long as the length argument tells it.
I know that the script is (supposed to be) from a piece of malicious code, but for the life of me I can't think of any way it could possibly pwn my computer barring some sort of hardware bug (which seems unlikely given that the loop is ~ 7 instructions long and the strangest instruction in the whole script is a shr).
I know it sounds bad running an unknown piece of assembly code directly on the metal, but given my analysis up to this point I can't think of any way it could bite me or escape.
Yes, you can but I won't recommend it.
The problem is not how dangerous is the code this time (assuming you really understand all of the code and you can predict the outcome of any system call), the problem is that it's a slippery slope and it's not worth it considering what's at stake.
I've done quite a few malware analysis and rarely happened that a piece of code caught me off guard but it happened.
Luckily I was working on a virtual machine within an isolated network: I just restored the last snapshot and stepped through the code more carefully.
If you do this analysis on your real machine you may take the habit and one day this will bite you back.
Working with VMs, albeit not as comfortable as using your OS native GUI, is the way to go.
What could go wrong with running a 7 lines assembly snippet?
I don't know, it really depends on the code but a few things to be careful about:
Exceptions. An instruction may intentionally fault to pass the control to an exception handler. This is why it very important that you totally understand the code: both the instruction and the data.
System calls exploits. A specially crafted input to a system call may trigger a 0-day or an unpatched vulnerability in your system. This is why is important that you can predict the outcome of every system call.
Anti debugger techniques. There are a lot of way a piece of code could escape a debugger (I'm thinking Windows debugging here), it's hard to remember them all, be suspicious of everything.
I've just named a few, it's catastrophically possible that an hardware bug could lead to privileged code execution but if that's really a possibility then nothing but a spare sacrificable machine will do.
Finally, if you are going to run the malware (because I assume the work of extracting the code and its context is too much of a burden) up to a breakpoint on your machine, think of what's at stake.
If you place the break point on the wrong spot, if the malware takes another path or if the debugger has a glitchy GUI, you may loose your data or the confidentiality of your machine.
I'n my opinion is not worth it.
I had to make this premise for generality sake but we all sin something, don't we?
I've never run a piece of malware on my machine but I've stepped through some with a virtual machine directly connected on the company network.
It was a controlled move, nothing happened, the competent personnel was advised and it was an happy ending.
This may very well be your case: it can just be a decryption algorithm and nothing more.
However, only you have the final responsibility to judge if it is acceptable to run the piece of code or not.
As I remarked above, in general it is not a good idea and it presupposes that you really understand the code (something that is hard to do and be honest about).
If you think these prerequisites are all satisfied then go ahead and do it.
Before that I would:
Create an unprivileged user and deny it access to my data and common folders (ideally deny it everything but what's is necessary to make the program work).
Backup the the critical data, if any.
Optionally
Make a restore point.
Take an hash of the system folders, a list of installed services and the value of the usual startup registry keys (Sysinternals have a tool to enum them all).
After the analysis, you can check that nothing important system-wide has changed.
It may be helpful to subst a folder and put the malware there so that a dummy path traversal stops in that folder.
Isn't there better solution?
I like using VMs for their snapshotting capabilities, though you may stumble into an anti-VM check (but they are really dumb checks, so it's easy to skip them).
For a 7-line assembly I'd simply rewrite it as a JS function and run it directly in a browser console.
You can simply transform each register in a variable and transcript the code, you don't need to understand it globally but only locally (i.e. each instruction).
JS is handy if you don't have to work with 64-bit quantities because you have an interpreter in front of you right now :)
Alternatively I use any programming language I have at hand (One time even assembly it self, it seems paradoxical but due to a nasty trick I had to convert a 64-bit piece of code to a 32-bit one and patch the malware with it).
You can use Unicorn to easily emulate a CPU (if the architecture is supported) and play with your shellcode without any risk.
I'd like to write a x11 terminal emulator, but I don't know how I should spawn and communicate with the shell, is there any basic (pseudo- or C) code for that? like what sort of PTY to create, how to bind the shell to it, what signals I have to catch or send, etc. don't really feel like sorting through the whole xterm sources.
EDIT: oh and I want to implement a way of communicating with any applications in it, how shall I do the feature discovery? some hidden ansi sequence in the "clients", hoping it's not colliding with other terminal emulators? some environment variable, hoping it's not colliding with the "clients" or removed by the shell?
YAT (yet another terminal) https://github.com/jorgen/yat is suitable for embedding in Qt Quick programs. Contributions for improvement are welcome. (Disclaimer: a friend started that project, and I work on it sometimes.) It takes a mostly correct approach (e.g. it uses a Linux pseudo-terminal properly, something I didn't know about before my friend was explaining that), and has a lot of features; however the parser is written from scratch and is not feature-complete or bug-free yet.
Unfortunately most terminal implementations so far have been starting from scratch, or with a one-off monolithic fork (from rxvt for example), which is a lot of work and results in all of them being incomplete. So I think a better alternative would be to use a reusable logic-only library called libvterm: http://www.leonerd.org.uk/code/libvterm/ or to base your terminal on one which already uses that. That way if you find bugs and fix them, you'll improve the whole ecosystem.
https://github.com/timmoorhouse/imgui-terminal is interesting, and works (at least somewhat) but is a prime candidate to be rewritten with libvterm, IMO. If you are into immediate-mode rendering in OpenGL, it might be a good choice anyway.
http://41j.com/hterm/ does use libvterm, and adds a few features which libvterm doesn't have, for inline graphics rendering (ReGIS and PNG). But the code is not elegant enough or portable enough, IMO, and the graphics rendering "floats" over the text rather than being truly inline. It still might be an adequate starting point for some use cases. In my fork https://github.com/ec1oud/hackterm I got it to build with mostly modern system libraries, however it still depends on an outdated version of SDL, which is included.
OK, if anyone also need this, and is using lua, I found the http://www.tset.de/lpty library works fine. still testing ansi escapes and stuff, but should work.
Is there a programming language which can be programmed entirely in interactive mode, without needing to write files which are interpreted or compiled. Think maybe something like IRB for Ruby, but a system which is designed to let you write the whole program from the command line.
I assume you are looking for something similar to how BASIC used to work (boot up to a BASIC prompt and start coding).
IPython allows you to do this quite intuitively. Unix shells such as Bash use the same concept, but you cannot re-use and save your work nearly as intuitively as with IPython. Python is also a far better general-purpose language.
Edit: I was going to type up some examples and provide some links, but the IPython interactive tutorial seems to do this a lot better than I could. Good starting points for what you are looking for are the sections on source code handling tips and lightweight version control. Note this tutorial doesn't spell out how to do everything you are looking for precisely, but it does provide a jumping off point to understand the interactive features on the IPython shell.
Also take a look at the IPython "magic" reference, as it provides a lot of utilities that do things specific to what you want to do, and allows you to easily define your own. This is very "meta", but the example that shows how to create an IPython magic function is probably the most concise example of a "complete application" built in IPython.
Smalltalk can be programmed entirely interactively, but I wouldn't call the smalltalk prompt a "command line". Most lisp environments are like this as well. Also postscript (as in printers) if memory serves.
Are you saying that you want to write a program while never seeing more code than what fits in the scrollback buffer of your command window?
There's always lisp, the original alternative to Smalltalk with this characteristic.
The only way to avoid writing any files is to move completely to a running interactive environment. When you program this way (that is, interactively such as in IRB or F# interactive), how do you distribute your programs? When you exit IRB or F# interactive console, you lose all code you interactively wrote.
Smalltalk (see modern implementation such as Squeak) solves this and I'm not aware of any other environment where you could fully avoid files. The solution is that you distribute an image of running environment (which includes your interactively created program). In Smalltalk, these are called images.
Any unix shell conforms to your question. This goes from bash, sh, csh, ksh to tclsh for TCL or wish for TK GUI writing.
As already mentioned, Python has a few good interactive shells, I would recommend bpython for starters instead of ipython, the advantage of bpython here is the support for autocompletion and help dialogs to help you know what arguments the function accepts or what it does (if it has docstrings).
Screenshots: http://bpython-interpreter.org/screenshots/
This is really a question about implementations, not languages, but
Smalltalk (try out the Squeak version) keeps all your work in an "interactive workspace", but it is graphical and not oriented toward the command line.
APL, which was first deployed on IBM 360 and 370 systems, was entirely interactive, using a command line on a modified IBM Selectric typewriter! Your APL functions were kept in a "workspace" which did not at all resemble an ordinary file.
Many, many language implementations come with pure command-line interactive interpreters, like say Standard ML of New Jersey, but because they don't offer any sort of persistent namespace (i.e., when you exit the program, all your work is lost), I don't think they should really count.
Interestingly, the prime movers behind Smalltalk and APL (Kay and Iverson respectively) both won Turing Awards. (Iverson got his Turing award after being denied tenure at Harvard.)
TCL can be programmed entirely interactivly, and you can cetainly define new tcl procs (or redefine existing ones) without saving to a file.
Of course if you are developing and entire application at some point you do want to save to a file, else you lose everything. Using TCLs introspective abilities its relatively easy to dump some or all of the current interpreter state into a tcl file (I've written a proc to make this easier before, however mostly I would just develop in the file in the first place, and have a function in the application to resources itself if its source changes).
Not sure about that, but this system is impressively interactive: http://rigsomelight.com/2014/05/01/interactive-programming-flappy-bird-clojurescript.html
Most variations of Lisp make it easy to save your interactive work product as program files, since code is just data.
Charles Simonyi's Intentional Programming concept might be part way there, too, but it's not like you can go and buy that yet. The Intentional Workbench project may be worth exploring.
Many Forths can be used like this.
Someone already mentioned Forth but I would like to elaborate a bit on the history of Forth. Traditionally, Forth is a programming language which is it's own operating system. The traditional Forth saves the program directly onto disk sectors without using a "real" filesystem. It could afford to do that because it didn't ran directly on the CPU without an operating system so it didn't need to play nice.
Indeed, some implementations have Forth as not only the operating system but also the CPU (a lot of more modern stack based CPUs are in fact designed as Forth machines).
In the original implementation of Forth, code is always compiled each time a line is entered and saved on disk. This is feasible because Forth is very easy to compile. You just start the interpreter, play around with Forth defining functions as necessary then simply quit the interpreter. The next time you start the interpreter again all your previous functions are still there. Of course, not all modern implementations of Forth works this way.
Clojure
It's a functional Lisp on the JVM. You can connect to a REPL server called nREPL, and from there you can start writing code in a text file and loading it up interactively as you go.
Clojure gives you something akin to interactive unit testing.
I think Clojure is more interactive then other Lisps because of it's strong emphasis of the functional paradigm. It's easier to hot-swap functions when they are pure.
The best way to try it out is here: http://web.clojurerepl.com/
ELM
ELM is probably the most interactive you can get that I know of. It's a very pure functional language with syntax close to Haskell. What makes it special is that it's designed around a reactive model that allows hot-swapping(modifying running code(functions or values)) of code. The reactive bit makes it that whenever you change one thing, everything is re-evaluated.
Now ELM is compiled to HTML-CSS-JavaScript. So you won't be able to use it for everything.
ELM gives you something akin to interactive integration testing.
The best way to try it out is here: http://elm-lang.org/try