Writing a Linux Terminal emulator - linux

I'd like to write a x11 terminal emulator, but I don't know how I should spawn and communicate with the shell, is there any basic (pseudo- or C) code for that? like what sort of PTY to create, how to bind the shell to it, what signals I have to catch or send, etc. don't really feel like sorting through the whole xterm sources.
EDIT: oh and I want to implement a way of communicating with any applications in it, how shall I do the feature discovery? some hidden ansi sequence in the "clients", hoping it's not colliding with other terminal emulators? some environment variable, hoping it's not colliding with the "clients" or removed by the shell?

YAT (yet another terminal) https://github.com/jorgen/yat is suitable for embedding in Qt Quick programs. Contributions for improvement are welcome. (Disclaimer: a friend started that project, and I work on it sometimes.) It takes a mostly correct approach (e.g. it uses a Linux pseudo-terminal properly, something I didn't know about before my friend was explaining that), and has a lot of features; however the parser is written from scratch and is not feature-complete or bug-free yet.
Unfortunately most terminal implementations so far have been starting from scratch, or with a one-off monolithic fork (from rxvt for example), which is a lot of work and results in all of them being incomplete. So I think a better alternative would be to use a reusable logic-only library called libvterm: http://www.leonerd.org.uk/code/libvterm/ or to base your terminal on one which already uses that. That way if you find bugs and fix them, you'll improve the whole ecosystem.
https://github.com/timmoorhouse/imgui-terminal is interesting, and works (at least somewhat) but is a prime candidate to be rewritten with libvterm, IMO. If you are into immediate-mode rendering in OpenGL, it might be a good choice anyway.
http://41j.com/hterm/ does use libvterm, and adds a few features which libvterm doesn't have, for inline graphics rendering (ReGIS and PNG). But the code is not elegant enough or portable enough, IMO, and the graphics rendering "floats" over the text rather than being truly inline. It still might be an adequate starting point for some use cases. In my fork https://github.com/ec1oud/hackterm I got it to build with mostly modern system libraries, however it still depends on an outdated version of SDL, which is included.

OK, if anyone also need this, and is using lua, I found the http://www.tset.de/lpty library works fine. still testing ansi escapes and stuff, but should work.

Related

Is there a way to replace the current process with a new one in a cross-platform way in Rust?

Similar to the linux execve(3) syscall, I want to replace the current process with a new one in Rust, in a way that works on both Unix and Windows systems. I'm fine with using crates if necessary, although I would like to stay away from unsafe.
So far, the only thing I've found in the standard library is std::os::unix::process::CommandExt::exec, but this only works on Unix. Looking for crates, I found the exec crate, but it doesn't appear to support Windows. However, this open PR mentions that it's possible to use the wexecvp syscall on Windows to achieve the same functionality, although it doesn't work on the Windows Runtime and only works on Win32. (That's good enough for me, though.) I'm at a dead-end on how to do this without breaking out libc::wexecvp and unsafe.
Is there a way to replace the current process with a new one in a cross-platform way in Rust?
This is a question that is independent of the programming language used, because it's about the process models of operating systems.
The answer is simply No.
The Windows process model doesn't work this way -- it's not a *nix.
Some things are fundamentally nonportable. So, you'll need to figure out what your functional requirement is (e.g. stdio handle inheritance, permissions, environment, etc.), and how to implement that on the other platform(s).
I can't get more specific because you'd need to provide more details on what "replace the current process by another one" means for your app You can always launch another program and quit, which naively sounds like the same thing :-).

VisualOcaml wanted to write a deminer (rather than using Ocaml's graphics library)

In Ocaml, is there easier ways to write a graphics-based toy programs like deminer (like the one that comes with Windows 95)? I find the only way is to start by scratch using Ocaml's graphics library. There must be better ways around?
There are bindings to the SDL library, that provides more features than Graphics.
There are actually several of them, and I'm not exactly sure which is best:
SdlCaml is a part of the [GLcaml] project
the OcamlSDL library
I think SdlCaml is more bare metal (probably partly automatically generated), and OCamlSDL is an older (but still occasionally updated) library with a larger user base.
Note however that Graphics is simple to use for a start, and you can still move to something more sophisticated later. If you run into speed-of-rendering issues, you have to use double buffering, as explained in the manual.

What language is easiest to develop command line/simple GUI for Linux?

I need to develop a large set of tools to be run from the server command line (i.e. not client-server architecture). The systems does not have to be high-performance; I just want something that is easy to develop with.
Which technologies are out there I can use to build simple GUI to be run from the command line? I need only menus where I can select a line/check-box/enter free text in a dialog.
Edit: forgot to add, access to Mysql (i.e. drivers available) is essential.
Shell, with dialog, the old stand-by - http://www.linuxjournal.com/article/2807
EDIT- If it's MySQL-related, take a look at PERL-Tk and DBI.
python + ncurses would be a good combo here.
i like using perl's re.pl from the Devel::REPL library for quickie cli interfaces. read on a bit for my rationale before downvoting!
in this type of app it sounds like you will be doing query-type operations. these naturally lend themselves to a "repl" style interraction. re.pl gives you all of the goodies, namely command editing and history. all you need to write are the functions that users will call. the nice thing is that users who know perl will realize they can use any installed module to extend the functionality of your system on their own. i my case, i used re.pl to create a mysqlclient-like tool to access and display data that was being compressed in a way that the standard mysqlclient couldn't deal with.
i cite perl because it's DBI is the best database abstraction and it is what i have used....but the rationale can be extended to other tools. python's repl or any other would provide the same benefit.
You could use Mono for Linux and write your program in C# .NET, then make it work for Linux, since Mono allows so.
As far as graphic command line interfaces go, one of the best frameworks is ncurses. It abstracts away most of the ugliness associated with graphic command line applications.
I have to say, use Python, because I like it.
But text-based interfaces are pretty much not worth it, because they seem like a good idea until you look at the details:
There isn't really a standard keyboard navigation model for text-UIs; they all use their own scheme
How is unicode supported? (Hint: this is nontrivial)
What about different keyboard layouts? What key does someone press if their keyboard doesn't have, say, a "home", "end", or "Escape" ?
ncurses does not provide a widget set, only low-level operations. The answers to the above questions aren't easy.
It really shows that nobody has put much thought into keyboard-and-text-driven terminal-based UIs recently, or these would all have been solved.
Web interfaces have them solved, in fact, you can use a text-mode web browser if you like.
Modern devices like i(phone|pad)s and even cheap mobile phones have a web browser which is good enough.
It is easy to write a web application which uses a very simple style (few images, little Javascrfipt) and have it work without much effort on a variety of devices.
So I would say go with dmckee's comment "go with what you know".
By building your own terminal-based interface, you are going to box yourself into a corner in the long term.

Interactive programming language?

Is there a programming language which can be programmed entirely in interactive mode, without needing to write files which are interpreted or compiled. Think maybe something like IRB for Ruby, but a system which is designed to let you write the whole program from the command line.
I assume you are looking for something similar to how BASIC used to work (boot up to a BASIC prompt and start coding).
IPython allows you to do this quite intuitively. Unix shells such as Bash use the same concept, but you cannot re-use and save your work nearly as intuitively as with IPython. Python is also a far better general-purpose language.
Edit: I was going to type up some examples and provide some links, but the IPython interactive tutorial seems to do this a lot better than I could. Good starting points for what you are looking for are the sections on source code handling tips and lightweight version control. Note this tutorial doesn't spell out how to do everything you are looking for precisely, but it does provide a jumping off point to understand the interactive features on the IPython shell.
Also take a look at the IPython "magic" reference, as it provides a lot of utilities that do things specific to what you want to do, and allows you to easily define your own. This is very "meta", but the example that shows how to create an IPython magic function is probably the most concise example of a "complete application" built in IPython.
Smalltalk can be programmed entirely interactively, but I wouldn't call the smalltalk prompt a "command line". Most lisp environments are like this as well. Also postscript (as in printers) if memory serves.
Are you saying that you want to write a program while never seeing more code than what fits in the scrollback buffer of your command window?
There's always lisp, the original alternative to Smalltalk with this characteristic.
The only way to avoid writing any files is to move completely to a running interactive environment. When you program this way (that is, interactively such as in IRB or F# interactive), how do you distribute your programs? When you exit IRB or F# interactive console, you lose all code you interactively wrote.
Smalltalk (see modern implementation such as Squeak) solves this and I'm not aware of any other environment where you could fully avoid files. The solution is that you distribute an image of running environment (which includes your interactively created program). In Smalltalk, these are called images.
Any unix shell conforms to your question. This goes from bash, sh, csh, ksh to tclsh for TCL or wish for TK GUI writing.
As already mentioned, Python has a few good interactive shells, I would recommend bpython for starters instead of ipython, the advantage of bpython here is the support for autocompletion and help dialogs to help you know what arguments the function accepts or what it does (if it has docstrings).
Screenshots: http://bpython-interpreter.org/screenshots/
This is really a question about implementations, not languages, but
Smalltalk (try out the Squeak version) keeps all your work in an "interactive workspace", but it is graphical and not oriented toward the command line.
APL, which was first deployed on IBM 360 and 370 systems, was entirely interactive, using a command line on a modified IBM Selectric typewriter! Your APL functions were kept in a "workspace" which did not at all resemble an ordinary file.
Many, many language implementations come with pure command-line interactive interpreters, like say Standard ML of New Jersey, but because they don't offer any sort of persistent namespace (i.e., when you exit the program, all your work is lost), I don't think they should really count.
Interestingly, the prime movers behind Smalltalk and APL (Kay and Iverson respectively) both won Turing Awards. (Iverson got his Turing award after being denied tenure at Harvard.)
TCL can be programmed entirely interactivly, and you can cetainly define new tcl procs (or redefine existing ones) without saving to a file.
Of course if you are developing and entire application at some point you do want to save to a file, else you lose everything. Using TCLs introspective abilities its relatively easy to dump some or all of the current interpreter state into a tcl file (I've written a proc to make this easier before, however mostly I would just develop in the file in the first place, and have a function in the application to resources itself if its source changes).
Not sure about that, but this system is impressively interactive: http://rigsomelight.com/2014/05/01/interactive-programming-flappy-bird-clojurescript.html
Most variations of Lisp make it easy to save your interactive work product as program files, since code is just data.
Charles Simonyi's Intentional Programming concept might be part way there, too, but it's not like you can go and buy that yet. The Intentional Workbench project may be worth exploring.
Many Forths can be used like this.
Someone already mentioned Forth but I would like to elaborate a bit on the history of Forth. Traditionally, Forth is a programming language which is it's own operating system. The traditional Forth saves the program directly onto disk sectors without using a "real" filesystem. It could afford to do that because it didn't ran directly on the CPU without an operating system so it didn't need to play nice.
Indeed, some implementations have Forth as not only the operating system but also the CPU (a lot of more modern stack based CPUs are in fact designed as Forth machines).
In the original implementation of Forth, code is always compiled each time a line is entered and saved on disk. This is feasible because Forth is very easy to compile. You just start the interpreter, play around with Forth defining functions as necessary then simply quit the interpreter. The next time you start the interpreter again all your previous functions are still there. Of course, not all modern implementations of Forth works this way.
Clojure
It's a functional Lisp on the JVM. You can connect to a REPL server called nREPL, and from there you can start writing code in a text file and loading it up interactively as you go.
Clojure gives you something akin to interactive unit testing.
I think Clojure is more interactive then other Lisps because of it's strong emphasis of the functional paradigm. It's easier to hot-swap functions when they are pure.
The best way to try it out is here: http://web.clojurerepl.com/
ELM
ELM is probably the most interactive you can get that I know of. It's a very pure functional language with syntax close to Haskell. What makes it special is that it's designed around a reactive model that allows hot-swapping(modifying running code(functions or values)) of code. The reactive bit makes it that whenever you change one thing, everything is re-evaluated.
Now ELM is compiled to HTML-CSS-JavaScript. So you won't be able to use it for everything.
ELM gives you something akin to interactive integration testing.
The best way to try it out is here: http://elm-lang.org/try

The right language for OpenGL UI prototyping. Ditching Python

So, I got this idea that I'd try to prototype an experimental user interface using OpenGL and some physics. I know little about either of the topics, but am pretty experienced with programming languages such as C++, Java and C#. After some initial research, I decided on using Python (with Eclipse/PyDev) and Qt, both new to me, and now have four different topics to learn more or less simultaneously.
I've gotten quite far with both OpenGL and Python, but while Python and its ecosystem initially seemed perfect for the task, I've now discovered some serious drawbacks. Bad API documentation and lacking code completion (due to dynamic typing), having to import every module I use in every other module gets tedious when having one class per module, having to select the correct module to run the program, and having to wait 30 seconds for the program to start and obscure the IDE before being notified of many obvious typos and other mistakes. It gets really annoying really fast. Quite frankly, i don't get what all the fuzz is about. Lambda functions, list comprehensions etc. are nice and all, but there's certainly more important things.
So, unless anyone can resolve at least some of these annoyances, Python is out. C++ is out as well, for obvious reasons, and C# is out, mainly for lack of portability. This leaves Java and JOGL as an attractive option, but I'm also curious about Ruby and Groovy. I'd like your opinion on these and others though, to keep my from making the same mistake again.
The requirements are:
Keeping the hell out of my way.
Good code completion. Complete method signatures, including data types and parameter names.
Good OpenGL support.
Qt support is preferable.
Object Oriented
Suitable for RAD, prototyping
Cross-platform
Preferably Open-Source, but at least free.
It seems you aren't mainly having a problem with Python itself, but instead with the IDE.
"Bad API documentation"
To what API? Python itself, Qt or some other library you are using?
"lacking code completion (due to dynamic typing)"
As long as you are not doing anything magic, I find that PyDev is pretty darn good at figuring these things out. If it gets lost, you can always typehint by doing:
assert isinstance(myObj, MyClass)
Then, PyDev will provide you with code completion even if myObj comes from a dynamic context.
"having to import every module I use in every other module gets tedious when having one class per module"
Install PyDev Extensions, it has auto-import on the fly. Or collect all your imports in a separate module and do:
from mymodulewithallimports import *
"having to select the correct module to run the program"
In Eclipse, you can set up a default startup file, or just check "use last run configuration". Then you never have to select it again.
"before being notified of many obvious typos and other mistakes"
Install PyDev Extensions, it has more advanced syntax checking and will happily notify you about unused imports/variables, uninitialized variables etc.
Looking just at your list I'd recommend C++; especially because Code Completion is so important to you.
About Python: Although I have few experience with OpenGL programming with Python (used C++ for that), the Python community offers a number of interesting modules for OpenGL development: pyopengl, pyglew, pygpu; just to name a few.
BTW, your import issue can be resolved easily by importing the modules in the __init__.py files of the directory the modules are contained in and then just importing the "parent" module. This is not recommended but nonetheless possible.
I don't understand why nobody has heard of the D programing language?
THIS IS THE PERFECT SOLUTION!!!!
The only real alternative if you desire all those things is to use Java, but honestly you're being a bit picky about features. Is code completion really that important a trait? Everything else you've listed is traditionally very well regarded with Python, so I don't see the issue.
The text editor (not even an IDE) which I use lets you import API function definitions. Code completion is not a language feature, especially with OpenGL. Just type gl[Ctrl+I] and you'd get the options.
I tried using Java3D and java once. I realized Java3D is a typical Java API... lots of objects to do simple things, and because it's Java, that translates to a lot of code. I then moved to Jython in Eclipse to which cleaned up the code, leaving me with only the complexity of Java3D.
So in the end, I went in the opposite direction. One advantage this has over pure python is I can use Java with all of Eclipse's benefits like autocomplete and move it over to python when parts get unwieldy in Java.
It seems like Pydev can offer code completion for you in Eclipse.
I started off doing OpenGL programming with GL4Java, which got migrated to JOGL and you should definately give it (JOGL) a try. Java offers most of the features you require (plus Eclipse gives you the code completion) and especially for JOGL there are a lot of tutorials out there to get you started.
Consider Boo -- it has many of Python's advantages while adopting features from elsewhere as well, and its compile-time type inference (when variables are neither explicitly given a specific type or explicitly duck typed) allows the kind of autocompletion support you're asking about.
The Tao.OpenGL library exposes OpenGL to .NET apps (such as those Boo compiles), with explicit support for Mono.
(Personally, I'm mostly a Python developer when not doing C or Java, but couldn't care less about autocompletion... but hey, it's your question; also, the one-class-per-module convention seems like a ridiculous amount of pain you're putting yourself through needlessly).

Resources