41.2.1 How We Made It This Far (Back?)

Originally, UNIX ran on a small machine that talked only to teletypes, or ttys for short. The UNIX kernel had to collect up input lines, allowing minor corrections - erasing the previous character and killing (erasing wholly) the input line - and translating a few "special" characters for controlling programs. Teletypes were printers, incapable of erasing, so the erase and kill characters were just ordinary printing characters, namely # and @. The original special characters were CTRL-d (for end-of-file), DEL (to interrupt), and CTRL-\ (to quit). The kernel also mapped input RETURN codes to the newline character, so that users could push the big RETURN key, on teletypes that had those.

These teletypes had some peculiarities. In particular, they used a moving print head (or carriage), and this print head took a noticeable amount of time to return from the right margin to the left. If sent continuous printing text, a teletype could smear characters all over the paper during a carriage return. [1] The UNIX kernel therefore had to allow for a delay after a carriage return. At the same time, the kernel did "output processing" by changing newlines to the teletype's carriage return and linefeed codes, if necessary. [2] A few teletypes allowed only uppercase characters, and UNIX grew support for these as well. UNIX did get away without something common to other operating systems, however: UNIX systems assumed that all teletypes were "full duplex" and used "remote echo." This meant, in essence, that both the teletype and the UNIX system could send to each other at the same time; and the teletype would not print what you typed until told to do so by the UNIX host. [3]

[1] This is an exaggeration. Printing during a carriage return was occasionally used as a diagnostic for checking the motor speed. The character printed during the return was supposed to appear exactly halfway along the line.

[2] Some teletypes really processed a newline as a "new line," i.e., a carriage return and linefeed, but most left this up to the host computer.

[3] Full duplex/remote echo and half duplex/local echo tended to go together. In particular, a half duplex system - which was not the same as a simplex system - had to have local echo to avoid being annoying to use. Fortunately, this is irrelevant today. The concept of "duplex" has fallen by the wayside, and everything is full duplex, or at least simulates it internally.

UNIX also had to provide a way for special applications, such as UUCP (1.33), to get input characters without any processing. This was the so-called raw mode. The kernel service was all-or-nothing: in raw mode, every input and output character was left alone, and passed directly - and immediately - to the application. In "cooked" mode, the kernel did input and output translations and delays.

Along with the ability to set raw or cooked mode, the kernel allowed changing each of the special characters (5.9) and allowed control of some of the simpler aspects of the serial port interface, such as parity and baud rate. [4] The baud rate, perhaps better called the bit rate, of the original teletype was 110 bits per second (bps), or 11 characters per second. (The machines really did print exactly 11 times each second, with one possibility being quietly to print nothing.) Early computer modems ran at 110 and 300 baud, and there were a standard set of serial port speeds: 50, 75, 110, 134.5, 150, 200, 300, 600, 1200, 1800, 2400, 4800, and even 9600 bps, which was considered terribly fast. UNIX systems used serial cards with two additional "external control" rates labeled A and B; these became exta and extb. Some UNIX systems still support exactly (and only) these rates, and tie exta to 19200 bps and extb to 38400 bps.

[4] Parity is used for error checking. Parity is simply the number of "1" bits. If you have the value 1001001, and even parity, the parity bit should be 1, because 1001001 has three 1 bits - an odd number - and adding another 1 makes this even. If the parity bit fails to match, at least one bit is wrong. It could, of course, be the parity bit itself. Moreover, with a tty port, there may not be anything you can do to fix the error - most UNIX kernels just drop the bad input character - but the check is available.

Eventually, teletype printers began to be displaced. First there came so-called glass ttys-CRT displays that tried to act just like a teletype - and then smarter terminals, ones that could (gasp) move a cursor around the screen, and edit the display in place. These used special control and escape codes to do the editing. They also provided the opportunity to write full-screen editors. UNIX had to evolve to adapt to these new constraints. Unfortunately, by this time there were two main branches of UNIX. One would eventually become 4BSD, or Berkeley UNIX; the other was to become System V.

The goals for both systems were similar, and thus both wound up with comparable approaches. Berkeley UNIX, however, attempted both to retain backwards compatibility and to provide a nice user interface, while the original System V system discarded compatibility in favor of efficiency and a "complete" interface - one that allowed doing everything a serial port could do.

Berkeley UNIX thus acquired three terminal modes. It retained the original raw and cooked modes, and added a new one called cbreak. [5] In cbreak mode, some input processing was done, but most characters were sent on to the application as they arrived. Since the kernel was not collecting lines, the erase and line-kill characters were unneeded; these were sent on unchanged. Most of the process control characters - interrupt, quit, and a new stop or suspend code - were still interpreted. To allow users to type these codes, a new "literal next" or "quote" character was introduced. Berkeley UNIX also added more output processing, including a special translation option for certain Hazeltine Corporation displays and features such as proper tab handling, output flush, and word erase.

[5] This cbreak mode has sometimes been referred to as "half-baked."

The System V base, on the other hand, dropped the idea of raw mode entirely. Instead, this system provided an individual control for each option. The icanon option, for instance, controlled whether input lines were to be collected or "canonicalized." The isig option controlled signals: when off, the interrupt (DEL, or in modern systems, CTRL-c) and quit characters were just ordinary characters. The inpchk option controlled input parity checking, and so forth. Similarly, output processing had individual flags: ocrnl for carriage return-newline control, opost for output processing in general. By turning everything off individually, an application could get the same effect as the old system's raw mode. The kernel also allowed control over the number of data bits in each serial frame, the number of stop bits, and so forth. [6] Thus, while Berkeley UNIX had nice line editing, it was incapable of attaching to five-bit Baudot systems. System V lacked the user interface features, but could talk to almost anything.

[6] Different systems use anything from five to nine bits in a serial-port "byte." Most people, however, do not need to care about all this. Most systems just use eight bits, either as seven data bits and a parity check, or as eight data bits without parity. Thus, most people can ignore these options, and stick with either "seven bits, even parity" or "eight bits, no parity."

Since then, the world has become simpler in one way - those old printing teletypes are gone nearly everywhere, for instance - but more complicated in another. These days, many computers use bitmapped displays rather than individual remote terminals. UNIX systems support networking, and use windowing systems such as the X Window System (1.31). These in turn bring a myriad of options, window managers, look-and-feel, and so on. But they all have one thing in common: to run old applications, each window or network login must provide a virtual terminal interface. UNIX systems generally do this with pseudo teletypes or ptys (41.8). Each pty exists to emulate a display terminal, which in turn is mainly pretending to be a teletype printer. (Sometimes one has to wonder where the progress lies.)

A POSIX standardization committee has settled on a standard interface, both at the UNIX kernel level and for the stty command. Most UNIX systems, including Berkeley UNIX, have moved to embrace this standard. While it leaves a few loose ends - mainly for reasons involving backwards compatibility for System V-it allows systems both the flexibility of the System V interface and the features of the Berkeley approach. This means that while windows and networks may be emulating ancient teletypes, at least they are all doing it in the same way.

41.2.2 Handling Most Characters

With all that as background, let's take a look at what happens to an input character, from the time you type it until an application can react. The details may vary - often wildly - depending on your system and whether you are using a window, a terminal, a network, or some combination of all three, but the overall idea is the same. For simplicity, we will assume you have an ordinary terminal. We will call this "the terminal" and the kernel's idea of it "the tty."

Suppose you type the letter x. The terminal sends the ASCII code (51.3) for a lowercase X (120) to the UNIX kernel's tty. The kernel then looks at the tty state. Assume for the sake of discussion that the tty is in cooked or icanon mode, and that none of the special characters has been set to x. Then the letter x is placed in an input buffer and echoed back to the terminal, causing an x to be displayed on your screen. But if you really wanted to type a c, you would now type your erase character (5.9) (usually CTRL-h, BACKSPACE, or DELETE, which may or may not all be the same or all different, depending on your particular terminal or keyboard). The code for this character will also be sent to the tty; this time it will match your erase character. The kernel will then remove the last character from the input buffer. Since this - a lowercase X - is an ordinary printing character, the kernel will send a single backspace, or the sequence "backspace space backspace," to the terminal. This will generally back the cursor up over the character and then erase it from the screen. (On a POSIX system, you get the latter by setting echoe mode.) Finally, when you type RETURN or ENTER, your terminal sends an ASCII code 13. Since icrnl is set, the kernel changes this to 10 (newline), which it then echoes to the terminal. Since onlcr is set, this sends both a code 13 (carriage return) and a 10 (linefeed) to the terminal. The kernel sees that 10 is a newline, wraps up the collected buffer, and passes it on to whatever application is currently reading from the tty.

If you turn off icanon (or turn on cbreak), the kernel takes any partially collected buffer and passes those characters to the application, then passes on each ordinary input character as it comes in. The kernel still echoes input back to the terminal. If you turn off the echo flag in the tty, the kernel will stop doing echoing. This is how a full-screen editor like vi works: it turns off icanon, turns off echo, and turns off some, but not all, of the special characters. The vi program can then do its own echoing, so that when you type i to go into insert mode, no i appears on your terminal.

One of several difficult areas involves turning icanon back on. In particular, there may be some characters you typed at the terminal while icanon was off. These reached the tty, which packaged them up and sent them off to an application. The application may not have read them yet, but as far as the tty is concerned, they are gone. Thus, you may not be able to recover them for your current input line. Older Berkeley UNIX systems are able to handle this case, but System V systems that use the STREAMS interface are not. As long as your system is fast enough, though, you will never notice, because applications will always turn icanon on before you can type anything at the terminal.

41.2.3 What About TABs?

Tabs are another difficult issue. The history here predates computing; typewriter tabs are sometimes used as the "right" model. Nonetheless, different terminals behave differently, and different people make different assumptions about how tabs should work. The ASCII code for TAB, code 9, is intended to move the cursor right to the next tabstop. But where is that? Moreover, once the cursor has gone there, how does the kernel move it back if you decided to erase the tab?

Many UNIX kernels can be told to expand tabs. When they do this, they set the tabstops at every eight characters. This is where they think tabstops belong. That is, if you print a newline, two ordinary letters, and a tab, the tab will turn into six spaces. If a tty is in icanon/cooked mode, and is expanding tabs, it can "unexpand" them to backspace over the tab. Berkeley kernels will do this, and it works fairly well. They can get it wrong, however, under certain conditions. For instance, if you set the tty to pass tabs unmodified, and if the terminal itself puts tabstops at every ten characters - this would be the proper setting for dealing with a DEC-10, for instance - the kernel tty code will put out fewer backspaces than needed.

Even if the terminal sets its tabstops at eight, the kernel's tty code and the terminal can get different ideas of the current cursor column. Most Berkeley kernels count control codes as "ordinary" output characters, for instance, even though those characters are likely to have no effect on the cursor, or might even move it to an arbitrary position. To help prevent input control characters from goofing up backspacing, Berkeley kernels can echo them as two-character sequences. For instance, CTRL-g will normally echo as ^G. Erasing such a control character works properly: the tty code puts out two backspaces, two spaces, and two more backspaces. Erasing more characters, possibly including a TAB, then still works. This "control echo" can be switched on and off individually as well.

In addition to carriage return delays, which exist to allow time for the teletype's print carriage to move left, some UNIX systems also support tab delays, for more or less the same reason. Like return-delays, these are pretty much outmoded and useless. The POSIX standard leaves room for both kinds of delay, but does not mandate either one. You may see them in stty output, as cr2, cr3, tab1, and the like, but your system's default is probably "no delay," and few people are likely to change this deliberately.

Article 41.4 has some higher-level information about TABs.

41.2.4 Flow Control (We Hope)

Finally, flow control - avoiding lost input and output characters - is perhaps the dirtiest swamp of all. Most of the terminals built in the 1980s support, and at higher speeds require, something called \h'-1p'XON/XOFF flow control. Here, when the terminal falls behind in printing characters, it shouts "stop!" by sending an XOFF character - ASCII code 19, or CTRL-s - to the UNIX system. If the UNIX machine does not stop soon enough, some text will be lost. When the terminal is ready for more, it sends a "go" character - an XON, ASCII code 17, or CTRL-q. These were never intended as a general flow control mechanism - on some of the original teletypes, they turned the paper tape punch off and on - but they have that meaning now. Unfortunately, most terminals also allow users to type CTRL-s and CTRL-q, but they provide no way to distinguish between the terminal yelling "stop" and the user pushing CTRL-s. The result is a constant battle between people who want to use CTRL-s and computer systems that want to take it for themselves.

Other systems, notably HP-based systems, use something called ENQ/ACK flow control. Here the terminal and the host system must agree up-front on a minimum buffer size. Then either system is allowed to send that many characters to the other, after which it must stop and wait for a "go-ahead" signal. Each system requests such a signal by sending an "enquire": ASCII code 5, or CTRL-e. When the listening system encounters the ENQ, and is ready for more, it sends an acknowledgement: ASCII code 6, or CTRL-f. This system is superior to the XON/XOFF system in one way, as it never has problems with a busy system failing to stop immediately on command, but it still does not prevent users from typing CTRL-e and CTRL-f. Moreover, it is not implemented on most UNIX systems.

A third method of flow control, and the most reliable where it is available, is the so-called out of band approach. "Out of band" simply means that users cannot accidentally simulate it by typing control characters. Out of band control can be done in software, using something similar to HP's ENQ/ACK and some encoding tricks, but in practice, most UNIX machines that support any kind of out of band flow control use something called either "hardware flow control" or "RTS/CTS flow control." (This can be implemented with no special hardware at all on many systems, so the latter name is better.)

With RTS/CTS flow control, two existing serial-cable wires, RTS and CTS, are "taken over." (RTS and CTS-which stand for Request to Send and Clear to Send respectively - were originally intended for use with half duplex modems. Since half duplex modems are today merely museum pieces, this is a sensible approach, but it does violate the RS232 standard.) RTS at the terminal is cross-connected to CTS at the host computer, and vice versa. The terminal and the computer both assert RTS whenever they are ready to receive data, and wait for CTS before sending. Unfortunately, not enough systems implement this, and of those that do, many get it wrong. [7] Thus, while RTS/CTS flow control offers the possibility of working perfectly, you cannot count on it. Still, it is worth looking for an rts/cts option in your UNIX's stty.

[7] For instance, on Sun workstations, RTS/CTS is supported in hardware, but the particular Zilog chip that does this also uses the DCD (Data Carrier Detect) line to control the receiver. Thus, if you set stty crtscts, you cannot tell a modem to dial out, because DCD is off. It is possible to work around this, but only with control over both the hardware and the UNIX kernel.

41.2.5 Then What?

If you think this is complicated, just hope you never have to deal with synchronous transmission, RS422, DIN connectors, lightning strike protection, and many of the other hardware and electrical aspects that surround computer communications. Getting two arbitrary computers to talk to each other can be excessively difficult. Here again, standards come to the rescue. If everything you have is proper RS232- modulo (52.9), perhaps, RTS/CTS flow control - and POSIX, things should usually go smoothly.