Question:
If I have a pty or tty master/slave pair, what can I not do with it if I only have the slave node's file descriptor? Or, put another way: what can I only do if I have the master node's file descriptor?
My Current Understanding:
I grok the "typical" relationship of a terminal/console/SSH having the master end for interfacing with a human, and one or more program (e.g. a shell and its children processes) being on the slave end. And I (loosely) grok the more unusual(/archaic?) usecases like using a TTY for other kinds of data links, like PPP. This question is not a "I don't get this TTY business" question. I'm asking about the ("low-level"?) "API" stuff: e.g. is there any termios/ioctl manipulations or other programmatic changes to the TTY pair that cannot be accomplished if you don't have access to the master FD?
I guess the obvious ones are:
I can only read/write from the master end if I have the master end's FD.
grantpt/unlockpt/ptsname can only be used on the master end's FD.
Anything else?
I've been on/off reading some man pages and experimenting on my Linux machines: the basic stuff one would want to do with a pty (e.g. stty columns 78, etc) seems to work on "either end". But I would suspect there's stuff only a process holding a file descriptor of the master end can do (especially because the master-slave name dichotomy suggests some unilateral control/dominion). And of course since I'm only testing on Linux, there's possible behavior differences between various versions/configurations of Linux and vs. the Unixes, so I don't want to assume that what I'm seeing is portable.
Motivation
(In case someone wants to know why I want to know)
General knowledge/curiosity.
I'm not in love with the current selection of command line tools for working with ptys. Without getting into the details, I've looked at reptyr, ptyget, expect/empty, screen/tmux(/neercs? the one with reptyr-like feature), dtach/abduco, and none of them hit my sweetspot of minimalist versatility. I'm trying to become more informed so I can better evaluate existing solutions and/or better design my own tool(s) to scratch my particular itch.
Thanks to StackOverflow's related-questions suggestions and other online searching since asking this, I've found a (partial?) answer:
Enabling or disabling packet mode on a PTY in Linux can only be done if you have the master FD [See TIOCPKT at this manpage]
Getting the Session ID associated with a TTY in Linux can only be done if you have the master FD (unclear if this is expected/intended behavior) [See TIOCGSID at this manpage]
Re-sizing the TTY is only portable from the master FD in practice (a terminal emulator might resize the TTY when it's resized, but an application with just the slave FD has no real certainty that the master size resizes accordingly, or that the terminal driver will even accept a resize from the slave end). [Source]
There is a trick for telling if the slave end of a TTY is opened, that you can't do if you don't have the master FD. [Source]
I'll try to keep coming back to edit this as I learn more.
Related
By now I have now spent at least 10 hours trying to get my head around the famous blog post by Linus Akesson, and Im still struggling. So let me ask my doubts about tty/ptty as a series of short questions.
1) Is the tty/ptty in user space or kernel space?
2) What is tty/ptty's connection to devices or drivers or some numbering or something?
3) The tty seems to be linked to something called the controlling terminal of a process, What is the relation and is every process related to a terminal?
4) On the whole I still dont understand where the heck this terminal concept fits in. A process wants to read something from the stdio, cant it simply do it from the required device file. What exactly is the problem that the tty intends to take care of?
5) I read somewhere that there are attempts to move the tty from the userspace to the kernel space. Is the tty simply a historical residue than a strong design feature.??
A clarification (which might answer some of your questions):
I think you meant pty (and not ptty) which is pseudo-tty/pseudo-terminal.
A tty (/dev/ttyx) - stands for teletype - is the original terminals (used a line printer for output and a keyboard for input!). A terminal is basically just a user interface device that uses text for input and output.
A pty (/dev/pty/n) is a pseudo-terminal - it's a software implementation that appears to the attached program like a terminal, but instead of communicating directly with a "real" terminal, it transfers the input and output to another program. It's the end point of telnet/SSH or even the GNOME terminal.
For example, when you ssh into a remote machine and run ls, the ls output is sent to a pseudo-terminal, the other side of which is attached to the SSH daemon.
EDIT:
As far as I know, the tty and so pty, are usermode. BUT they represent terminal-driver. What I mean is: the device file /dev/tty1 is the first virtual console. Most code lives in drivers/char, in the files tty_io.c and n_tty.c and vt.c (kernel source). In contrast to character devices in order to open those files tty_open routine is called, and trust me, it's way messier than opening a character device...
Tty/pty stands for terminal drivers mentioned above but they stands for serial ports (the"numbering" you said). I know very little about it so I don't want to say incorrect data... but you can search the net about it (or someone else can continue from here)
EDIT2:
You have changed the question so now it seems like I spoke out of context...
Anyway, tty has many different roles even nowday. Terminal driver is the way user-kernel can "communicate". There are some techniques such as terminal drivers, character device etc.
If you still have a question please comment and don't change the whole post....
I'm looking for a way to determine the local input (mouse/keyboard) idle time,
remotely through SSH, without having root access or access to the currently logged on user Xauthority.
I know the following solution which works if you have access to the X server:
detecting keyboard, mouse activity in linux
But is it possible without having to connect to the X server?
Is there another way? E.g. indirectly via CPU or memory usage of certain processes?
Any ideas welcome.
Using w and /dev solutions will only get you so far, since it could be that the user is around, but has not typed anything in the shell - for example, he/she could be playing some game. A better approach would be to poll /proc/interrupts. The local interrupts for the mouse and keyboard are often under "i8042" (though in some rare cases it might be different). You might want to try: "grep i8042 /proc/interrupts". This will yield IRQ 1 (keyboard) and IRQ 12 (usually, the mouse). You can get the values, store them, and then poll occasionally (no callback, alas), to get the counts. If the numbers changed, interrupts occurred - meaning keyboard (IRQ 1) or mouse (IRQ 12) were touched/pressed etc. Key presses generally generate two interrupts (key down, key up). Mouse movement is more erratic.
This has several advantages:
1) If the user so much as touches the mouse, or presses a key - you know
2) You can do so programmatically (i.e. fopen() /proc/interrupts , or (alternatively) /proc/stat, and get the "intr" line) and fread() the relevant lines
3) You don't even need to be root for this.
As an unprivileged, distinct user, I'd try to do this by monitoring for new processes spawned with the target user ID. If you see a new process, chances are it means the user caused it to be spawned, unless its parent is cron (or one of a handful of similar supervisors, you'll have to try it on your system and filter out the few false alarms you get at first).
This won't give you a very precise idle time, but perhaps a maximum bound. If you want something more precise, you probably ought to have privileges to do it.
The commands w and ls -l /dev/pts are the best ones I can think of. Between them, you can get a pretty good view of idleness on the linux console, ssh, and xterm sessions.
If the user is just sitting on a local thing playing with firefox or something, this won't catch it, but I'm not sure you can get that without access to X or being root. Looking for cpu usage isn't reliable because a lot of apps use a little cpu even when idle, or virtually zero when used, and watching /proc/x_server_pid/* doesn't help either, since X can still have activity even when the user is idle.
Looking for new processes like the other answer said is the best I can think of if /dev/pts doesn't help and you can't actually look at the input.
I would like to be able to get a list of all of the file descriptors (now considering this question to pertain to actual files) that a process ever opened during the runtime of the process. The problem with polling /proc/(PID)/fd/ is that you only get a snapshot in time of what is currently open. Is there a way to force linux to keep this information around long enough to log it for the entire run of the process?
First, notice that a file descriptor which is open-ed then close-d by the application is recycled by the kernel (a future open could give the same file descriptor). See open(2) and close(2) and read Advanced Linux Programming.
Then, consider using strace(1); you'll be able to log all the syscalls (or perhaps just open, socket, close, accept, ... that is the syscalls changing the file descriptor table). Of course strace is using the ptrace(2) syscall (which you probably don't want to bother using directly).
The simplest way would be to run strace -o /tmp/mytrace.tr yourprog argments... and to look, e.g. with some pager like less, into the quite big /tmp/mytrace.tr file.
As Gearoid Murphy commented you could restrict the output of strace using e.g. -e trace=file.
BTW, to debug Makefile-s this is the wrong approach. Learn more about remake.
My task at the moment is to port a driver for some 16550-compatible chip from QNX to Linux. The chip provides several UARTs, each one seen as a standard 16550 serial port, albeit with some extensions.
Now, in QNX, the whole device driver is packed into a standalone executable, that acts both as a driver and as an initial configurator for the provided UARTs (baud rates, loopback modes etc.) That is just natural in QNX because there device drivers run in user space and are little more than standard executables.
On Linux, OTOH, the driver is now implemented as a kernel module, loadable at will. More, that module is provided by the producer, so I would not want to modify or patch it too much.
For me, the remaining task is to provide some mechanism for setting up those UARTs' parameters. They are seen as /dev/ttyPREFIXX devices. I intend to do that through a standard C-programmed executable calling standard termios (ie tcsetattr() or ioctls) on the serial ports of interest.
Which leads me to the question: is my approach right? And, if yes, then how to achieve a persistent configuration? As I perceive the fact (from this example: http://www.easysw.com/~mike/serial/serial.html), the termios functions act on OPEN devices. In short: they open a device, they set up the parameters, they read or write, then close the port. After closing the port, is the configuration (baud rate etc.) lost? I hope it is not, because it is stored, already, into the hardware.
Can somebodey confirm to me that the configuration is persistent? And, if not, how to achieve that persistence, for the future applications that would open again that port and will expect it with some pre-established parameters? If not, should I modify the module kernel to accept some parameters and, then, do the configuration at the load time?
The approach that I intend for now is to write that C executable that opens the ports, sets up their configuration, then close the ports. I hope the latter applications will see the ports with the desired configuration.
Thank you.
You might want to have a look at stty and setserial. The venerable Serial-HOWTO (wow, when was the last time I actually recommended a HOWTO to anyone?) is probably also a good starting point.
Well, I found the answer here: http://www.gnu.org/software/libc/manual/html_node/Mode-Functions.html#Mode-Functions
Quote: "Although tcgetattr and tcsetattr specify the terminal device with a file descriptor, the attributes are those of the terminal device itself and not of the file descriptor. This means that the effects of changing terminal attributes are persistent; if another process opens the terminal file later on, it will see the changed attributes even though it doesn't have anything to do with the open file descriptor you originally specified in changing the attributes."
This clears the issue.
Is it possible to 'hibernate' a process in linux?
Just like 'hibernate' in laptop, I would to write all the memory used by a process to disk, free up the RAM. And then later on, I can 'resume the process', i.e, reading all the data from memory and put it back to RAM and I can continue with my process?
I used to maintain CryoPID, which is a program that does exactly what you are talking about. It writes the contents of a program's address space, VDSO, file descriptor references and states to a file that can later be reconstructed. CryoPID started when there were no usable hooks in Linux itself and worked entirely from userspace (actually, it still does work, depending on your distro / kernel / security settings).
Problems were (indeed) sockets, pending RT signals, numerous X11 issues, the glibc caching getpid() implementation amongst many others. Randomization (especially VDSO) turned out to be insurmountable for the few of us working on it after Bernard walked away from it. However, it was fun and became the topic of several masters thesis.
If you are just contemplating a program that can save its running state and re-start directly into that state, its far .. far .. easier to just save that information from within the program itself, perhaps when servicing a signal.
I'd like to put a status update here, as of 2014.
The accepted answer suggests CryoPID as a tool to perform Checkpoint/Restore, but I found the project to be unmantained and impossible to compile with recent kernels.
Now, I found two actively mantained projects providing the application checkpointing feature.
The first, the one I suggest 'cause I have better luck running it, is CRIU
that performs checkpoint/restore mainly in userspace, and requires the kernel option CONFIG_CHECKPOINT_RESTORE enabled to work.
Checkpoint/Restore In Userspace, or CRIU (pronounced kree-oo, IPA: /krɪʊ/, Russian: криу), is a software tool for Linux operating system. Using this tool, you can freeze a running application (or part of it) and checkpoint it to a hard drive as a collection of files. You can then use the files to restore and run the application from the point it was frozen at. The distinctive feature of the CRIU project is that it is mainly implemented in user space.
The latter is DMTCP; quoting from their main page:
DMTCP (Distributed MultiThreaded Checkpointing) is a tool to transparently checkpoint the state of multiple simultaneous applications, including multi-threaded and distributed applications. It operates directly on the user binary executable, without any Linux kernel modules or other kernel modifications.
There is also a nice Wikipedia page on the argument: Application_checkpointing
The answers mentioning ctrl-z are really talking about stopping the process with a signal, in this case SIGTSTP. You can issue a stop signal with kill:
kill -STOP <pid>
That will suspend execution of the process. It won't immediately free the memory used by it, but as memory is required for other processes the memory used by the stopped process will be gradually swapped out.
When you want to wake it up again, use
kill -CONT <pid>
The more complicated solutions, like CryoPID, are really only needed if you want the stopped process to be able to survive a system shutdown/restart - it doesn't sound like you need that.
Linux Kernel has now partially implemented the checkpoint/restart futures:https://ckpt.wiki.kernel.org/, the status is here.
Some useful information are in the lwn(linux weekly net):
http://lwn.net/Articles/375855/ http://lwn.net/Articles/412749/ ......
So the answer is "YES"
The issue is restoring the streams - files and sockets - that the program has open.
When your whole OS hibernates, the local files and such can obviously be restored. Network connections don't, but then the code that accesses the internet is typically more error checking and such and survives the error conditions (or ought to).
If you did per-program hibernation (without application support), how would you handle open files? What if another process accesses those files in the interim? etc?
Maintaining state when the program is not loaded is going to be difficult.
Simply suspending the threads and letting it get swapped to disk would have much the same effect?
Or run the program in a virtual machine and let the VM handle suspension.
Short answer is "yes, but not always reliably". Check out CryoPID:
http://cryopid.berlios.de/
Open files will indeed be the most common problem. CryoPID states explicitly:
Open files and offsets are restored.
Temporary files that have been
unlinked and are not accessible on the
filesystem are always saved in the
image. Other files that do not exist
on resume are not yet restored.
Support for saving file contents for
such situations is planned.
The same issues will also affect TCP connections, though CryoPID supports tcpcp for connection resuming.
I extended Cryopid producing a package called Cryopid2 available from SourceForge. This can
migrate a process as well as hibernating it (along with any open files and sockets - data
in sockets/pipes is sucked into the process on hibernation and spat back into these when
process is restarted).
The reason I have not been active with this project is I am not a kernel developer - both
this (and/or the original cryopid) need to get someone on board who can get them running
with the lastest kernels (e.g. Linux 3.x).
The Cryopid method does work - and is probably the best solution to general purpose process
hibernation/migration in Linux I have come across.
The short answer is "yes." You might start by looking at this for some ideas: ELF executable reconstruction from a core image (http://vx.netlux.org/lib/vsc03.html)
As others have noted, it's difficult for the OS to provide this functionality, because the application needs to have some error checking builtin to handle broken streams.
However, on a side note, some programming languages and tools that use virtual machines explicitly support this functionality, such as the Self programming language.
This is sort of the ultimate goal of clustered operating system. Mathew Dillon puts a lot of effort to implement something like this in his Dragonfly BSD project.
adding another workaround: you can use virtualbox. run your applications in a regular virtual machine and simply "save the machine state" whenever you want.
I know this is not an answer, but I thought it could be useful when there are no real options.
if for any reason you don't like virtualbox, vmware and Qemu are as good.
Ctrl-Z increases the chances the process's pages will be swapped, but it doesn't free the process's resources completely. The problem with freeing a process's resources completely is that things like file handles, sockets are kernel resources the process gets to use, but doesn't know how to persist on its own. So Ctrl-Z is as good as it gets.
There was some research on checkpoint/restore for Linux back in 2.2 and 2.4 days, but it never made it past prototype. It is possible (with the caveats described in the other answers) for certain values of possible - I you can write a kernel module to do it, it is possible. But for the common value of possible (can I do it from the shell on a commercial Linux distribution), it is not yet possible.
There's ctrl+z in linux, but i'm not sure it offers the features you specified. I suspect you asked this question since it doesn't