Ive read numerous posts about tty. They all start with the historical reasons for the tty name. Please leave this out and just describe the tty system as it exists today. Then they talk about how a tty is a file and that stdin, stdout and sterr of a process started in a terminal are all mapped to this file.
How are three files mapped to a single file?
Some say that tty allows line editing before enter is hit and does other line discipline stuff. There is a blog post which says that each tty has its own stdin and stdout . The blog post by Linus Akesson which i'm still grappling with explains that there is in fact a tty driver in the kernel and a tty device file. then there is the controlling terminal, sessions, terminal emulators, raw and cooked modes, pty and what not.
To better understand what tty is, can someone explain to me a what happens in this simple situation:
A terminal is opened and it runs the default shell. From the shell a process is run and it asks for input.
What happens when the call scanf is made?
How does the terminal know that scanf is called?
The editing buffer in the terminal which we see afterwards(the line where we enter text) - where does it come from? Does this buffer exist in the tty device file and is being outputted like an stdout file is printed?
Which process is controlling this buffer? The tty driver?
What happens when we press enter? Does the tty driver 'submit' the line to the stdin part of the tty device?
How does the process know that input has been submitted.
And the output part: When the same process outputs something, does it write to the tty device? But isnt the tty already outputting the current editing buffer line?
IF theres a better way to decribe what a tty does without answering the above questions then please do so If i missed some crucial part please fill in as you think necessary.
This turned out really long, so brace yourself...
TTY Device
You are looking at a TTY as a screen divided to e.g. 80x24 tiles.
But a TTY is a console: it contains an input device (generally connected to a keyboard) and an output device (generally connected to a screen).
TTY Abstraction
While the TTY is connected to (physical or simulated) devices, Unix processes don't see devices, they see an abstraction. This abstraction consists of an input stream, an output stream, and a control interface.
The control interface can turn on/off some "fancy" features like sending the input it receives not only to the process using the terminal, but also to its own output stream (that feature is called "echo" and can be controlled with stty echo), but the fact that you are seeing something on the terminal output doesn't mean it's connected to any form of stdout.
This abstraction is implemented in the kernel by the TTY driver and the line discipline. You can think of the line discipline as the Strategy design pattern.
This abstraction has to be available to userspace, and the way Unix drivers export anything to userspace is by creating special files such as /dev/tty.
A TTY file allows you to read() the input stream, write() to the output stream, and turn features on/off via ioctl()
TTY Files
Usually, every time you launch a new terminal, a new TTY file is created by the driver.
Any process can open a tty file, regardless of whether that file will be the process' stdin, stdout, stderr, all of them, or none of them.
You can see for yourself: Open a terminal and type tty. Let's say it printed /dev/pts/3.
Now open another terminal and run:
exec 10>/dev/pts/3 # open /dev/pts/3 as file descriptor 10
echo hello >&10 # write "hello" through file descriptor 10
This will cause echo to write hello to file descriptor 10, which is the first terminal. Accordingly, the first terminal will print hello.
Standard Streams
Unix implements the 3 standard streams, stdin, stdout, and stderr. These receive no special treatment whatsoever from the terminal driver or line discipline, and most of their implementation is in the shell.
When you start your terminal emulator, it opens a tty file, say /dev/pts/3. It then creates a new process (fork()), opens /dev/pts/3 as file descriptors 0 (stdin), 1 (stdout), and 2 (stderr), and then executes the shell.
Which means that when the shell starts, it has the terminal file, with its streams and control interfaces. When the shell writes to either stdout or stderr, both writes go to the TTY's output stream.
When the shell executes another process, the process inherits /dev/pts/3 as its file descriptors 0, 1, 2, unless the shell does redirection, or the executed programs changes these file descriptors.
Specific Answers
Now we're ready to answer your questions:
What happens when the call scanf is made?
scanf() calls read(STDIN), which calls the TTY driver's implementation of read().
In cooked mode, this will block until the input stream has buffered a full line. In raw mode, it will block until at least one character was read.
Then the TTY input buffer is copied to scanf's buffer.
How does the terminal know that scanf is called?
It doesn't. If you type something into the terminal while the program is running and not awaiting your input, it will be buffered in the terminal's input stream.
Then, whenever scanf is called, if at all, it will read that buffer. If scanf isn't called, then the program ends, control returns to the shell. and the shell reads that buffer. You can see it by running sleep 30, and while it's running, type another command and press enter. The shell will execute it after sleep is done:
bash-4.3$ sleep 30
echo hello
bash-4.3$ echo hello
hello
bash-4.3$
The editing buffer in the terminal which we see afterwards(the line where we enter text) - where does it come from? Does this buffer exist in the tty device file and is being outputted like an stdout file is printed?
The buffer exists in the kernel and is attached to the TTY file.
If the terminal's echo feature is on, the line discipline will send the input stream not only to the input buffer, but also to the output stream.
If the terminal is in cooked (default) mode, the line discipline will give special treatment characters like backspace (yes, backspace is a character, ASCII 8). In the case of backspace, it will remove the last character from the input buffer, and send to the output stream a control sequence to delete the last character from the screen.
Note that the buffer is managed separately from what you see on the screen.
Which process is controlling this buffer? The tty driver?
The buffer is in the kernel, and not controlled by a process, but rather by the line discipline, which is controlled by the TTY driver.
What happens when we press enter? Does the tty driver 'submit' the line to the stdin part of the tty device?
When you press "enter" a linefeed character (\n) is added to the buffer, and, if any process is waiting on terminal input, the input buffer is copied to the process' buffer and the process becomes unblocked and continues to run.
The more interesting question is what happens when you press something that is NOT "enter". In raw mode, it doesn't matter if it's "enter" or not, because \n doesn't get any special treatment. In cooked mode, however, the input buffer is not copied and the reading process is not notified.
How does the process know that input has been submitted.
The process calls e.g. scanf() which read(STDIN), which will block the process until input is available. When input is available, the TTY driver will unblock the blocked process (i.e. wake it up).
Note that this is not special to TTY files or to STDIN, it applies to all files, that's just how read() works.
Also note that scanf() doesn't know whether or not STDIN is a TTY file.
When the same process outputs something, does it write to the tty device?
when you call something like printf(), it calls write(STDOUT), which calls the TTY driver's implementation of write(), which writes to the TTY's output stream.
Again, note that printf() doesn't know whether or not STDOUT is a TTY file.
But isnt the tty already outputting the current editing buffer line?
In Unix, a file (any file, not just TTY files) can be opened and written to by multiple writers, and no synchronization among them is guaranteed.
As you can see with the echo hello >&10 example above, the process running in the terminal is not the only thing which can write to the TTY's output stream, but even an unrelated process can write to the TTY's output stream.
And when echo is enabled, the line discipline can also write to the TTY's output stream.
All these writes will be interleaved, the driver won't try to synchronize them or make sense of them.
I have a small issue with my code, running in Python 3. I'm trying to fool Raspbian, in order to make it believe a tty is an external device.
However, I can't read a single word I wrote previously with os.write(slave, text.encode()), using something like os.read(slave, 512).
I open the tty as follow master, slave = os.openpty()
I think I'm missing a parameter or something, but I can't find out what.
I tried accessing the tty in another terminal, with a cat <, with a subprocess, but the program still block when it has to read.
Please explain what is the problem.
Regards.
I think your mistake here is that you are trying to read the slave. If you read the master instead you should get your output.
Quote from: http://www.rkoucha.fr/tech_corner/pty_pdip.html
A pseudo-terminal is a pair of character mode devices also called pty. One is master and the other is slave and they are connected with a bidirectional channel. Any data written on the slave side is forwarded to the output of the master side. Conversely, any data written on the master side is forwarded to the output of the slave side as depicted in figure 2.
RPI
I'm now studying what is the mechanism behind Ctrl+Alt+F1~F7.
I found some infomation talk about framebuffer, VT, and tty. I know framebuffer is an anstraction to graphic card manipulation and tty(and pts) is something a program interact with. I also find some people talking about VT(virtual terminal), but I don't know what is the relationship between them.
After searching on the web, I made a figure.
I think a VT is a set of monitor and keyboard. If I want to use many tty in one set(one VT), I have to switch between ttys because I only have one monitor to show them. This is VT Swich. But what is VT actually be? A driver, a module, or something else?
I guess if I have two set of monitors and keyboards connect to my computer, I can group them to form two VTs. And I can show my tty1 in monitor1 and control it use keyboard1, and tty2 with monitor2 and keyboard2. I can do VT Switch in both VT, too. Is that true?
And I think I can view framebuffer(fbcon + fbdev) as a graphic card driver, but it is design for console, not for general purpose(proprietary drivers).
Also, there are some special files under /dev
/dev/tty* and /dev/pts/* are for tty
echo can send input to the tty, and cat can recieve output from the tty.
/dev/vcs* are for screen of VT
cat can get the text in the screen, but use it under X get all zero(not empty).
/dev/fb* are for framebuffer
cat can get the whole pixel in the screen, but use it under X get all zero(not empty). I heard that X server doesn't use framebuffer, so what does it use?
Please tell me where I am wrong.
Correct tty and pty are terminals. but there is small difference. pty is pseudo terminals. open a terminal type tty will return pts terminal. tty are generally physical if you have used embedded linux you can redirect linux console to different tty which are uart terminals. if you use a 9 pin uart cable. then you can use ttyX.
vcs are differant screens were used pre X-Server erra. or now in server where X server( or even Wayland Server) is not installed. in run level 3 and 5. to try use CTRL + ALT + F3 to access /dev/vc3. you can echo and cat simmilarly.
frame buffer is part of Graphic subsystem its not supposed to be used with cat. It is matrix. its abstraction over any screen connected. to be used by graphical drivers and applications.
I'm reading The TTY demystified, trying to get some understanding about tty, pty.
After reading the first half. I'm not getting the complete picture of how the whole thing is working when I'm type some commands into an xterm or ssh.
Below is the picture visualizing my understanding, I don't know if it's correct or not, please help verify.
when I type a command on the keyboard
keyboard ->tty->xterm->pty(master)->pty(slave)->user processes
(the purple line)
when a user process generates some output
user processes->pty(slave)->pty(master)->xterm->tty->display
(the blue line)
The diagram seems confused with regard to the box for "tty":
a pseudo-terminal has devices that might be named "tty" and "pty" (in the pre-Unix98 configuration). The /dev-entry for tty corresponds to the pseudo-terminal master.
xterm reads keystrokes as X events via the X server from whatever the keyboard device happens to be.
I need to test a serial port application on Linux, however, my test machine only has one serial port.
Is there a way to add a virtual serial port to Linux and test my application by emulating a device through a shell or script?
Note: I cannot remap the port, it hard coded on ttys2 and I need to test the application as it is written.
Complementing the #slonik's answer.
You can test socat to create Virtual Serial Port doing the following procedure (tested on Ubuntu 12.04):
Open a terminal (let's call it Terminal 0) and execute it:
socat -d -d pty,raw,echo=0 pty,raw,echo=0
The code above returns:
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/2
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/3
2013/11/01 13:47:27 socat[2506] N starting data transfer loop with FDs [3,3] and [5,5]
Open another terminal and write (Terminal 1):
cat < /dev/pts/2
this command's port name can be changed according to the pc. it's depends on the previous output.
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/**2**
2013/11/01 13:47:27 socat[2506] N PTY is /dev/pts/**3**
2013/11/01 13:47:27 socat[2506] N starting data transfer loop with FDs
you should use the number available on highlighted area.
Open another terminal and write (Terminal 2):
echo "Test" > /dev/pts/3
Now back to Terminal 1 and you'll see the string "Test".
You can use a pty ("pseudo-teletype", where a serial port is a "real teletype") for this. From one end, open /dev/ptyp5, and then attach your program to /dev/ttyp5; ttyp5 will act just like a serial port, but will send/receive everything it does via /dev/ptyp5.
If you really need it to talk to a file called /dev/ttys2, then simply move your old /dev/ttys2 out of the way and make a symlink from ptyp5 to ttys2.
Of course you can use some number other than ptyp5. Perhaps pick one with a high number to avoid duplicates, since all your login terminals will also be using ptys.
Wikipedia has more about ptys: http://en.wikipedia.org/wiki/Pseudo_terminal
Use socat for this:
For example:
socat PTY,link=/dev/ttyS10 PTY,link=/dev/ttyS11
There is also tty0tty http://sourceforge.net/projects/tty0tty/ which is a real null modem emulator for linux.
It is a simple kernel module - a small source file. I don't know why it only got thumbs down on sourceforge, but it works well for me. The best thing about it is that is also emulates the hardware pins (RTC/CTS DSR/DTR). It even implements TIOCMGET/TIOCMSET and TIOCMIWAIT iotcl commands!
On a recent kernel you may get compilation errors. This is easy to fix. Just insert a few lines at the top of the module/tty0tty.c source (after the includes):
#ifndef init_MUTEX
#define init_MUTEX(x) sema_init((x),1)
#endif
When the module is loaded, it creates 4 pairs of serial ports. The devices are /dev/tnt0 to /dev/tnt7 where tnt0 is connected to tnt1, tnt2 is connected to tnt3, etc.
You may need to fix the file permissions to be able to use the devices.
edit:
I guess I was a little quick with my enthusiasm. While the driver looks promising, it seems unstable. I don't know for sure but I think it crashed a machine in the office I was working on from home. I can't check until I'm back in the office on monday.
The second thing is that TIOCMIWAIT does not work. The code seems to be copied from some "tiny tty" example code. The handling of TIOCMIWAIT seems in place, but it never wakes up because the corresponding call to wake_up_interruptible() is missing.
edit:
The crash in the office really was the driver's fault. There was an initialization missing, and the completely untested TIOCMIWAIT code caused a crash of the machine.
I spent yesterday and today rewriting the driver. There were a lot of issues, but now it works well for me. There's still code missing for hardware flow control managed by the driver, but I don't need it because I'll be managing the pins myself using TIOCMGET/TIOCMSET/TIOCMIWAIT from user mode code.
If anyone is interested in my version of the code, send me a message and I'll send it to you.
You may want to look at Tibbo VSPDL for creating a linux virtual serial port using a Kernel driver -- it seems pretty new, and is available for download right now (beta version). Not sure about the license at this point, or whether they want to make it available commercially only in the future.
There are other commercial alternatives, such as http://www.ttyredirector.com/.
In Open Source, Remserial (GPL) may also do what you want, using Unix PTY's. It transmits the serial data in "raw form" to a network socket; STTY-like setup of terminal parameters must be done when creating the port, changing them later like described in RFC 2217 does not seem to be supported. You should be able to run two remserial instances to create a virtual nullmodem like com0com, except that you'll need to set up port speed etc in advance.
Socat (also GPL) is like an extended variant of Remserial with many many more options, including a "PTY" method for redirecting the PTY to something else, which can be another instance of Socat. For Unit tets, socat is likely nicer than remserial because you can directly cat files into the PTY. See the PTY example on the manpage. A patch exists under "contrib" to provide RFC2217 support for negotiating serial line settings.
Using the links posted in the previous answers, I coded a little example in C++ using a Virtual Serial Port. I pushed the code into GitHub: https://github.com/cymait/virtual-serial-port-example .
The code is pretty self explanatory. First, you create the master process by running ./main master and it will print to stderr the device is using. After that, you invoke ./main slave device, where device is the device printed in the first command.
And that's it. You have a bidirectional link between the two process.
Using this example you can test you the application by sending all kind of data, and see if it works correctly.
Also, you can always symlink the device, so you don't need to re-compile the application you are testing.
Would you be able to use a USB->RS232 adapter? I have a few, and they just use the FTDI driver. Then, you should be able to rename /dev/ttyUSB0 (or whatever gets created) as /dev/ttyS2 .
I can think of three options:
Implement RFC 2217
RFC 2217 covers a com port to TCP/IP standard that allows a client on one system to emulate a serial port to the local programs, while transparently sending and receiving data and control signals to a server on another system which actually has the serial port. Here's a high-level overview.
What you would do is find or implement a client com port driver that would implement the client side of the system on your PC - appearing to be a real serial port but in reality shuttling everything to a server. You might be able to get this driver for free from Digi, Lantronix, etc in support of their real standalone serial port servers.
You would then implement the server side of the connection locally in another program - allowing the client to connect and issuing the data and control commands as needed.
It's probably non trivial, but the RFC is out there, and you might be able to find an open source project that implements one or both sides of the connection.
Modify the linux serial port driver
Alternately, the serial port driver source for Linux is readily available. Take that, gut the hardware control pieces, and have that one driver run two /dev/ttySx ports, as a simple loopback. Then connect your real program to the ttyS2 and your simulator to the other ttySx.
Use two USB<-->Serial cables in a loopback
But the easiest thing to do right now? Spend $40 on two serial port USB devices, wire them together (null modem) and actually have two real serial ports - one for the program you're testing, one for your simulator.
-Adam
$ socat -d -d pty,link=/tmp/vserial1,raw,echo=0 pty,link=/tmp/vserial2,raw,echo=0
Will generate symlinks of /tmp/vserial1 and /tmp/vserial2 for generated virtual serial ports in /dev/pts/*
Resource
Combining all other amazingly useful answers, I found the below command to be VERY useful for testing on different types of Linux distros where there's no guarantee you're going to get the same /dev/pts/#'s every time and/or you need to test multiple psuedo serial devices and connections at once.
parallel 'i="{1}"; socat -d -d pty,raw,echo=0,link=$HOME/pty{1} pty,raw,echo=0,link=$HOME/pty$(($i+1))' ::: $(seq 0 2 3;)
Breaking this down:
parallel runs the same command for each argument supplied to it.
So for example if we run it with the --dryrun flag it gives us:
i="0"; socat -d -d pty,raw,echo=0,link=$HOME/pty0 pty,raw,echo=0,link=$HOME/pty$(($i+1))
i="2"; socat -d -d pty,raw,echo=0,link=$HOME/pty2 pty,raw,echo=0,link=$HOME/pty$(($i+1))
This is due to the $(seq x y z;) at the end, where
x = start #, y = increment by, and z = end # (or # of devices you need to spawn)
parallel 'i="{1}"; echo "make psuedo_devices {1} $(($i+1))"' ::: $(seq 0 2 3;)
Outputs:
make psuedo_devices 0 1
make psuedo_devices 2 3
Gathering all this together the final above command symlinks the proper psuedo devices together regardless of whats in /dev/pts/ to whatever directory supplied to socat via the link flag.
pstree -c -a $PROC_ID gives:
perl /usr/bin/parallel i="{1}"; socat -d -d pty,raw,echo=0,link=$HOME/pty{1} pty,raw,echo=0,link=$HOME/pty$(($i+1)) ::: 0 2
├─bash -c i="0"; socat -d -d pty,raw,echo=0,link=$HOME/pty0 pty,raw,echo=0,link=$HOME/pty$(($i+1))
│ └─socat -d -d pty,raw,echo=0,link=/home/user/pty0 pty,raw,echo=0,link=/home/user/pty1
└─bash -c i="2"; socat -d -d pty,raw,echo=0,link=$HOME/pty2 pty,raw,echo=0,link=$HOME/pty$(($i+1))
└─socat -d -d pty,raw,echo=0,link=/home/user/pty2 pty,raw,echo=0,link=/home/user/pty3
ls -l $HOME/pty* yield:
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty0 -> /dev/pts/4
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty1 -> /dev/pts/6
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty2 -> /dev/pts/7
lrwxrwxrwx 1 user user 10 Sep 7 11:46 /home/user/pty3 -> /dev/pts/8
This was all because I was trying running tests against a platform where I needed to generated a lot of mach-serial connections and to test their input/output via containerization (Docker). Hopefully someone finds it useful.