How does tmux, vim, emacs, etc transcend the UI limitations of *nix terminals? - linux

When I'm writing a program for use on the command line, I notice that there's some limitations. For instance, I can't draw a 1-pixel-thick horizontal or vertical line like tmux does when it separates panes in a window. I can only move the cursor down, not up like VI seemingly does. I can't refresh information on the top of the page if the cursor is at the bottom.
So, when programs like tmux and vi do this, I have to wonder if they are:
drawing the screen from top to bottom every update (which I think is highly unlikely because otherwise I could scroll up in my terminal and see each redraw)
using some library that enables graphics in the terminal, like SDL, which I also think is unlikely.
using some standard syscall I don't know about
or finally
taking advantage of some feature of Linux/Unix of which I'm completely unaware.
So, how do these programs generate such a rich UI in such a seemingly limited shell? So long as the answer gives me just enough fodder to go on a Google rampage, I'll be happy.
I'm also assuming that these programs use some common method to do these things, but if that's wrong let me know.

A typical terminal emulator has a lot more features than are immediately apparent.
Essentially a program just needs to output short sequences of bytes that represent various commands such as move cursor (up|down|left|right), change color, scroll region, erase region, etc.
These commands typically begin with the escape character (the same character that is generated when you press the esc key while typing in a terminal) followed by various other characters, depending on which action is desired.
A good starting point for understanding how it works would be the Wikipedia Article about ANSI escape codes

You can do it by hand by putting the terminal into raw mode and writing directly to the terminal using low-level operations but the standard way to do it is to use the ncurses library.

Related

Get text around input caret on Linux

Motivation: I'm trying to write scripts which send keystrokes to the currently focused window. Right now I use xdotool, which lets me send raw keystrokes. However, I want the exact keystrokes to be a function of the current text around the input caret in the focused window.
Problem: Is there a generic way of reading the state of the text input caret -- both its current position as well as the text around it? Intuitively, I want the content of the current "text box" as well as the location of the cursor within that text box. Perhaps this is not possible in the general case, but is there a way of doing it which would work for emacs and firefox? I'm running Ubuntu Linux
Further motivation: due to a bad case of RSI I control my computer by voice rather than typing. This works by setting up voice-activated scripts that are triggered by saying different phrases. When dictating English prose, it would be helpful to automatically capitalize words at the beginning of sentences. This automatic capitalization can be accomplished by reading the characters immediately before the input caret, checking if they contain a period, and if so, capitalizing the start of the next phrase that I dictate by voice.
Thanks so much! If anybody can help me here, it would greatly increase my day-to-day accessibility.
Since there is no standard widget toolkit for X11, but only a buch of independently developed arbitrary toolkits, there is no generic way to implement this.
As far as X11 and tools operating on its level (like xdotool) is concerned, there's only windows of either the InputOutput variety (i.e. visible windows, that receive events and one can draw to) or the just Input which are invisible and only receive events. There are no further refined "widgets" so to speak. You get a pixel grid, which you can draw to.
Accessibility interfaces are the burden of the toolkits (or if you don't use a toolkit – then you're a badass – you, the developer), to implement: https://www.freedesktop.org/wiki/Accessibility/
The absolute generic way would be to take a screenshot of the currently focused window, employ a computer vision / machine learning based solution to identify the caret, then OCR the line of text around it. And to be honest, IMHO doing it that way would probably be a lot more reliable than hoping for the accessibility interfaces to be properly implemented.

How to partly clean command promt

I'm creating a small game in python where two players choose 1 of 3 characters and fight each other by turns. So when I launch game in cmd, there are some info that I want to refresh on cmd every turn so I use "import os" and "os.system('cls')". This cleans whole window. The thing is I want some information to stay on screen, like how much one's character done damage on last rounds etc. Is it possible at all?
Or maybe its possible to do that when I open my program, two cmd screens opens, both store different information and communicate with each other?
ANSI escape codes are your friend. Among other things, it allows you to move the cursor around the screen. For example
print('\e[2A\e[3C')
will move the cursor two spaces up and three to the right. This wikipedia article gives a good synopsis, see the section on CSI sequences. These codes work on most Unix/Linux OSs and newer versions of Windows 10.

vim - Cursor randomly jumps on Linux

This only occurs when I am using vim on Linux (it's Kali Linux to be precise, though I haven't tested it on other distributions). I am using a standard German keyboard layout.
Sometimes when I type in vim (it often happens when I exit insert mode or when I use :w, maybe only on one of these since I often do one after the other), the cursor randomly jumps elsewhere, usually about 100 lines upwards (I don't have an exact number). At the same time, the next number in the line my cursor was in is decremented.
I suspect that this happens because I hit some sequence of keys to quickly, since this, on my Linux distribution, can cause some special characters to be inserted due to one of the keys modifying the other. For example, if I type "yt" quickly with this keyboard, it becomes "yŧ" (with a second bar on the t)
This by itself is somewhat annoying to me so if someone knows a way to turn that off on Linux while still retaining the basic keyboard layout, this would solve my problem, but telling me the exact command I accidentally executed so I can avoid/remove it will also help.
As far as I can remember, this problem only occurred when I was editing .texfiles, but that is also what I have been using vim the most for recently, so I wouldn't assume that it only happens there.
Still, I can post my list of plugins and my .vimrc if necessary. Just in case it has something to do with only LaTeX files, the only vim plugin I have for that is vimtex.
The command you are looking for is mapped to control + x by default. It decrements the next number on the given line.

Shell formatting language

On linux, console applications have the ability to format their output. They can set font color, set background color and can place signs everywehre on the console. Using that it is, for example, possible to implement a tetris game right into the console.
I´m wondering how one can do that. I think they use a output markup language or something else. Can anyone tell me where I can learn more about this?
Thanks very much!
Most console applications involving a lot of motion or color are built using the ncurses library. Some very common examples would be irssi (IRC client), mc (Midnight Commander, the console file browser), mutt (POP3/IMAP mail client)
It seems like you are already aware of the escape codes used to modify console colors. A good list of console color escape sequences (for Bash) can be found here.
You obviously need to get a hold of those every-popular Unix video games, rogue, srogue, larn, hack, and/or nethack. They have a long and venerable history.
Notably, these all use the standard curses — or more recently, ncurses — library. Here’s a screen shot.
Since they have no joystick, motion is with vi commands. They are hands-down the very best way to hone your vi motion skills ever invented: no more two-finger typing for you! You stop thinking about motion; it just becomes a part of your fingers’ muscle memory. You really have to play them to get a feel for the awesome “Zen” state you can get into playing them:
After enough practice, it feels as though your fingers themselves remember how to play the piece. You don’t even watch them. They've a job to do, and once they’ve learned it, can go about that job remarkably free of direct supervision. The key to clearing the mind of the outside world, so that the program becomes the dominant reality, is what a musician would call “finger memory”. (You might have heard athletes or dancers refer to it as muscle memory, but when we’re talking about using the computer, it really is the fingers that count.)
[...] Of course, that's not really what’s going on; it only seems to be. Your fingers don’t really remember. But a part of your brain that controls them does, even though “you” don’t realize it. What’s happened is that you've so successfully assimilated the moves needed that conscious direction is no longer required. The little lighthouse keeper behind your forehead can worry about other things, assured that your fingers will do the job you’ve trained them to do. Your eyes are on the screen, the program in your head, and your head is in the program. Your fingers become an unnoticed extension of your will. [...]
[...] There’s no question that, for certain tasks, the keyboard is clearly the optimally efficient input device. Consider the game of rogue or one of its more recent incarnations. You wouldn’t want to use anything but a keyboard there. The command set is just too rich. Trying to play the game with a mouse‐and‐menu interface instead of a keyboard one would slow you down by at least two orders of magnitude.
The rogue family of video games are also notable for showing how to write a video game for a regular terminal like a vt100 or an xterm, which I believe is what you are looking for. I’d probably use a more modern language than C these days, but all the same principles still apply. Both Perl and Python have good interfaces to these standard libraries.
It's not so much a markup language as a series of escape sequences that trigger the terminal viewer to format in a certain way.
You can send ANSI escape sequences before your output to indicate that the following output should be a certain color, weight, background. You can also send sequences that jump the cursor to specific locations to continue writing output.
If you are going to do a full blown app you should consider using some library such as ncurses which makes these manageable.

Inefficiency in Vim

I consider myself somewhat familiar with Vim,
hate the arrow keys (let alone the mouse),
regularly look up tips and plugins to get the most out of this tool,
use it daily to manage my cloud servers, etc.
However, I always find myself doing the same mistakes probably inherited from the GUI-world:
too often switching to visual mode to see what piece of code I'm about to manipulate,
undoing changes to retrieve lost statements because I forget to utilize registers (and pasting code on temporary lines just to grab it after other edits),
relying on Ctrl-C & Ctrl-V when working with operating system's clipboard,
keep pressing j button to browse through lengthy files to find certain functions.
Probably my Hungarian keyboard layout prevents me from being faster as most of the special characters (/, [, etc.) are only available as a key combination (with Shift or Alt Gr).
Given this specific situation, what pieces of advice could you give me? Have you faced similar bad habits when you were a Vim-novice? I'd like to see my productivity skyrocket (who wouldn't?). Thanks in advance.
I've found a simple, effective strategy. Choose one action, one task or one set of keys that you think is unnecessarily slow. Figure out a better way of doing this using the vim manual or googling or a plugin etc. Force yourself to use this every time. Rinse, and repeat. The path to efficiency is one-by-one elimination of the slow parts.
I'd also recommend just reading the vim manual from time to time - even if you don't remember everything, knowing something's out there is very helpful.
This probably applies well beyond vim, but
something that worked for me was finding a specific feature that I knew would
be more efficient and concentrate on using that for a week or two.
Just one feature at a time, and possibly using it excessively.
After a couple of weeks it becomes automatic and you can move on to the
next thing.
I learn programming tricks the same way. eg. I'll have a month of using lambda expressions for everything, then a month of mapping and filtering.
(not on production code though)
Probably my Hungarian keyboard layout prevents me from being faster as most of the special > characters (/, [, etc.) are only available as a key combination (with Shift or Alt Gr).
I'm sitting in front of german keyboards all day long and know this problem very well. Some keyboard layouts are simply not very suited for programming / using vim. I think its safe to assume that most programming languages and keyboard shortcuts were designed with the us-layout in mind.
My advice: reset your keyboard layout to us-english and practive touch-typing on that layout (typing without looking at the keys). It won't matter that the keyboard labels are wrong and you'll be much more comfortable using vim hotkeys.
The only problem that still remains for me is to produce language specific characters (german umlauts such as ä,ö,ü) wich i assume will also be a problem for hungarian. For that I use a combination of vim-digraphs, linux window manager digraph-key and windows layout-switching hotkeys.
just keep using it. The more you use it, the better you become at it. VIM isn't too bad. The main thing is you just have to remember that it isn't always in edit mode.

Resources