Diagramming an Electronic Sentence: How the Linux Shell Runs Commands
You’ve sat down at your computer and opened your command terminal. You enter a few simple commands and the data dances. The computer bends to your will and renders the results you need to organize data for work or launch the game you just finished writing. Given this easy access to computing power, one could be forgiven for being oblivious or indifferent to the processes that happen out of sight of the user. Furthermore, one could be forgiven for not realizing that this heavy lifting is done by a program like any other, with its own code base, contributors, bug reporting, etc. This type of program is know as a Command Line Interface. For UNIX-like systems , including Linux, this program is the Shell.
There are many different iterations of the shell: sh, ksh, zsh, etc. The Bourne Again Shell, or bash, is the most commonly used shell currently. No matter the version or distribution of Linux, though, what the shell does is the same: the shell accepts, parses, and validates user commands, translates them into syscalls to interact with the kernel which drives the work of computation at the machine level. That is compactly stated, and there is much to unpack there. To uncoil this, let’s use an analogy from grade school: diagramming a sentence.
It’s important to remember as we build onto this metaphor that Linux programming syntax is imperative (“Computer, do this; Computer, do that other thing”). That is to say, the subject of the sentence is the computer, so we don’t need to diagram the subject, just the verb, objects, indirect objects, and adverbs of the input language.
Breaking Down the Sentence
Well we all remember diagramming sentences, tediously labeling verbs, objects, and adjectives. Now imagine if that instead of growing into fluency, a student instead developed superhuman speed at diagramming sentences and that was it’s only way for them to interact with the world. That is the mind of the shell. Fortunately, tedium is computing’s wheelhouse.
On passing a command to the shell, the computer takes breaks down the entire passed line into its constituent components. For instance, when the command ls -l is entered, the first step is to copy the command into a buffer using the getline() command. In our analogy this would be like writing a sentence on a chalkboard so that it could be diagrammed in front of the class. The pointer which getline() assigns the vale to is then split into an array of constituent parts. In this instance, the array would look like {ls, -l}. It is important for our metaphor that we recognize the general syntax of Linux commands: Verb | Adverbs | Object. It knows to look for the verb (command) in the first, or [0], position in the array above. Now that the command has been stored and split up into its constituent components, the shell begins unpack and act on them.
Expansion and Alias
Before the computer can begin to execute components of the array, it must search for Aliases and Expansion and uncoil their contents. Aliases are a kind of variable that stores an action, like assigning a hotkey to a keyboard. Calling alias lk='ls -la' creates an alias such that when lk is called it returns as if ls -la had been called. Aliases are stored in the the .bashrc file that is located at the home directory.
Expansion tokens ($,*) indicate that indicate a expansion is required to return the full result. The token $ indicates that what follows is a variable, not a literal. By convention shell variables are all uppercase. For example, PS1 is the prompt string 1 variable. Calling echo $PS1 will display the Prompt String 1 variable which stored alongside the aliases in the .bashrc file. Some other common examples of variables are $HOME or $PATH.
The special character * acts as a string wildcard. A common use of the wildcard token is gcc *.c -o myfile. In that example, it will find every file that ends with .c and compile it together. As a side note, it is a testament to C’s enduring influence that * as wildcard is a convention found in many places in computer science, SQL queries being an easy example.
In our analogy, this could be thought of as listing out the full text of an acronym rather than simply leaving in its abbreviated form; or like translating something out of shortened slang (i.e., Jake for Jacob). Once the process has uncoiled all the implied content of aliases and expansion, it can move on to how to handle the verb.
Checking for a Built-in
It’s important to remember at this point that everything in Linux is a file. Now that the process has checked for Alias and Expansion, it begins searching for the binary file that contains the set of instructions to execute the command, i.e., the verb. Think of it like looking up a very precise definition in a dictionary. The first dictionary it checks is the built-in list, the equivalent of a desk reference. Built-ins are exactly that. They are a short list of commands that are “built in” to Linux. Common examples would be cd, echo, and the ls we passed in in search of c files.
Finding the PATH
If the command is not a built-in, one must look in a larger dictionary to find its definition. That is where the PATH comes in. The PATH is a variable that stores directory locations that the computer searches in order to find a broader, user-defined set of functions. If after searching the PATH no file with that command name is found, an error will inform the user. Once the definition (i.e. the discrete set of instructions to execute the command) has been found either as a builtin or in the various PATH directories, the computer can finally set about executing them.
Fork and Execute
At this point, we’ve learned to diagram, parse, uncoil, and look up where commands are. Now to it’s time to fork and execute. Fork creates a copy, or child, of the current process. Linux does this in case some error happens while processing the command. The parent copy of the shell uses wait() to do exactly that until the child is done with calling an execute function. There are a variety of these functions but for our purposes we’ll use execve(). excecve() executes the code in the child copy of the shell. If no error is returned, control is passed back to the parent, who has been wait()-ing. This sort of run-in-clone approach is essential to maintaining data integrity. Upon execute, the shell displays again the prompt (a customizable environmental variable known as PS1) and another command may be entered.
Metaphor in Example
Let’s put our metaphor to work and implement. Let’s imagine that your grammar teacher copies a sentence to be diagrammed to the board. The instructions are that you must write down on scratch paper a version of the diagram with answer, then come to board and copy the answer for all to see. You being the star student, the teacher has added a special challenge. You are also to include a short definition of the verb in your work . And GO!
In the metaphor, the teacher is the user, passing the command to the board, not unlike the getline() call we mentioned earlier. Splitting the words and labeling them is as as we saw above. You, following the instructions, make a copy of the sentence on you scratch paper a kind of analogue forked clone like the cloned pid we saw in fork(). You look at your quick reference and don’t see the word so you search for it in a more compendious dictionary as you would searching for built-ins, then searching the PATH. You have the definition (the file path to the executable file) and the diagram and head to the wait()-ing board. You return all the requested information and throw your scratch paper in the recycling as you walk back to your desk. And the teacher moves on to the next sentence and student.
Conclusion
There you have it. An analogized understanding of how the shell executes commands. Parse, unwind implications of alias and expansion (slang, acronyms), find the exact definition of the verb either from builtin or from somewhere on the PATH, then run the command in a cloned version of the shell to prevent errors. I hope this explanation has been helpful.
