Unix Guide

Written by Chris Gregg and Nick Troccoli

There are walkthrough videos for many of the commands listed below. You can view all the walkthrough videos here.

The Filesystem

When logged in to myth, or any other machine, we can navigate the filesystem, - go in and out of folders ("directories"), make files, etc. We can think of the filesystem as a tree, with the root of the tree labeled "/" (forward slash). Files and directories (also called "folders") make up the nodes of the tree, and directories can contain more files and directories.

For example, you may have a directory on your computer called "CS107", which has some files and directories in that directory. It turns out there's a fun tree command that can show you a list of the files and directories in a tree format (names that end with / are directories, names that end with * are executable - files):

myth$ tree cs107 -F
cs107
├── 
│   ├── main*
│   ├── main.c
│   ├── main.cpp~
│   └── readme.txt
├── assign1/
│   ├── file1.txt
│   ├── file2.txt
│   ├── folder1/
│   │   └── myProgram.c
│   └── folder2/
└── assign2/
    ├── docs/
    └── readme.txt

6 directories, 8 files

There are two special files in every directory: . (the current directory) and .. (the parent directory). . is useful to refer to files in the current directory, like running a program by saying ./myprogram. .. is useful for navigating up the filesystem, like going up a level by saying cd ...

If you want to refer to a file or folder, you can specify its location in two ways:

  • an absolute path: how to get there from the root of the filesystem (/), or
  • a relative path: how to get there from where you currently are

For example, lets say we want to open file1.txt above in vim. Regardless of where we are in the filesystem, we can always refer to it like this: vim /cs107/assign1/file1.txt. However, it may be easier to refer to it from where we currently are. For instance, let's say we're in the cs107 folder. We can also say vim assign1/file1.txt. This is using a relative path; we are specifying how to get there from where we currently are. Note that there is no leading /, because we aren't starting from the filesystem root; we start from our current location. Another example: let's say we're in the /cs107/assign1/folder1 folder. We could say vim ../file1.txt.

When you log into the myth computers, you are automatically put in your home directory (~), your personal file space on myth. This is not /, as it turns out - it is a path like /afs/ir/users/t/r/troccoli. But you can refer to it via the shorthand ~.

Autocomplete

Your terminal remembers unix commands you enter, and can autocomplete commands for you. For example, if you press the up arrow key, the terminal will auto-fill your most-recently-typed command. You can continue pressing the up arrow key to go through commands you executed in order of decreasing recency. You can use the down arrow key to go through commands in order of increasing recency. If you want to execute that command, just press ENTER.

Second, if you type an exclamation point (also known as bang) followed by a character or a string and then enter, you will run the last command that starts with that character or string. E.g.,

myth$ ls -a
.  ..  hello  hello.c  hello.c~  innerFolder  readme.txt  samples
myth$ cd ..
myth$ !l
ls -a
.   assign0  assign2  .do_not_look.txt  file2.txt
..  assign1  assign3  file1.txt     .this_is_hidden.txt
myth$

Typing !l runs ls -a again, because the previous command that started with l was ls -a.

Third, if you press Ctl-r, you can then start typing characters and the line will start being populated with the last command that starts with the characters you have typed. This is easier to see in the video linked at the top of the page; also try it yourself!

Fourth, hit the tab key while you are typing a command, and the shell will automatically finish the command for you. Or, if it is ambiguous, it will provide options (you might have to type tab again). For example, if you want to type the history command, you can type his-tab and the rest of the command will be filled in:

myth$ his (then hit tab, at which point the entire history command will show)
myth$ history
...

If, on the other hand, you typed hi and then tab twice, you would see this:

myth$ hi (then the tab key twice)
hipercdecode hipsopgm history
myth$ hi

The shell is telling you that there are three commands that start with hi, and you have to continue with the letter you know is next. Then, if you type tab again, you can finish the completion.

Tab completion is extra-handy when you want to type file names or directories. Instead of typing a long file name, you can simply start typing a name and then type tab to have the name autocomplete. If only part of the name completes, that means there are multiple options, and typing tab again will give you the options that the shell is considering.

Commands To Navigate The Filesystem

Run the pwd command and it will output the absolute path to your current location ("working directory"): 
myth$ pwd
/afs/.ir/users/c/g/cgregg
myth$ cd 107
myth$ pwd
/afs/.ir/users/c/g/cgregg/107

Run the cd command and specify the location you would like to navigate to, as an absolute or relative path: 
myth$ ls
file2.c  Makefile  myfile.txt  MyFolder MyFolder2
myth$ cd MyFolder
myth$ ls
item1.txt  item2.txt
myth$ cd ../MyFolder2
myth$ ls
contents.c
Enter cd with no argument to go back to your home directory (~).

Enter ls with no arguments to view the contents of the current directory: 
myth$ ls
assign0  assign1  assign2  assign3  file1.txt  file2.txt
myth$
Optionally specify an absolute or relative path to view the contents of that directory instead: 
myth$ ls assign0
triangle.c Makefile custom_tests
There are many "flags" you can specify (arguments beginning with a dash) to change the behavior of ls. You can view them all in the manual pages: man ls. One common one is -l, which lists files with extra information: 
myth$ ls -l
total 12
drwx------ 2 cgregg operator 2048 Sep  6 11:42 assign0
drwx------ 4 cgregg operator 2048 Sep  6 08:42 assign1
drwx------ 3 cgregg operator 2048 Sep  6 08:43 assign2
drwx------ 2 cgregg operator 2048 Sep  6 09:44 assign3
-rw------- 1 cgregg operator   18 Sep  6 11:50 file1.txt
-rw------- 1 cgregg operator   46 Sep  6 11:50 file2.txt
myth$
This prints out whether something is a directory (starts with 'd' or not), its permissions, file size in bytes, last modified date, and the filename. The permissions are not that important to know right now, but the "rwx" simply means that you have the ability to (r)ead, (w)rite, and (e)xecute a file or the files in a directory. For sizes here, file1.txt is 18 bytes long, for instance. Another common flag is -a which lists all files, including "hidden" files, which begin with a period. You can also use flags together: 
myth$ ls -al
total 16
drwx------ 6 cgregg operator 2048 Sep  6 12:02 .
drwx------ 7 cgregg operator 2048 Sep  6 08:46 ..
drwx------ 2 cgregg operator 2048 Sep  6 11:42 assign0
drwx------ 4 cgregg operator 2048 Sep  6 08:42 assign1
drwx------ 3 cgregg operator 2048 Sep  6 08:43 assign2
drwx------ 2 cgregg operator 2048 Sep  6 09:44 assign3
-rw------- 1 cgregg operator    4 Sep  6 12:02 .do_not_look.txt
-rw------- 1 cgregg operator   18 Sep  6 11:54 file1.txt
-rw------- 1 cgregg operator   46 Sep  6 11:55 file2.txt

man displays the manual pages for a given command. E.g. man ls. To use a manual page, you can use the arrow keys to scroll up and down the page, and you type q to quit. You can also perform a search inside a man page by typing a forward slash (/) and then a search term, followed by the enter key. If you are not sure what manual page you want, but you do have a keyword, you can search the manual system to find the page you want, by using the command man -k keyword. 
myth$ man -k printf
Printf (3o)          - Formatted output functions.
asprintf (3)         - print to allocated string
...
printf (1)           - format and print data
printf (3)           - formatted output conversion
...
The numbers in parentheses is the manual "section", and a command may show up in different sections. For example, in the above listing, there are two printf commands, one in section 1, and one in section 3. Unix commands are in section 1, and C library functions are in section 3. To find a command in a particular section, put the section number first: 
myth$ man 1 printf # show the unix printf command manual page
myth$ man 3 printf # show the C library function manual page

Commands To Create and Delete Files and Directories

You can use cp and specify the source you are copying, and the destination where you are copying to, in that order (cp source dest). For example, let's say we want to copy a hello.c file from the assign0 directory into the assign1 directory: 
myth$ cp assign0/hello.c assign1/hello.c
The destination can be a path with or without a filename. If you leave the filename off, the file will have the same name as the original. 
# do the same thing
myth$ cp assign0/hello.c assign1/hello.c
myth$ cp assign0/hello.c assign1/
Use the -r ("recursive") flag to copy an entire directory. For example, let's copy the entire assign0 directory into the existing directory called assign1: 
myth$ cp -r assign0 assign1
On myth, your profile has been set up to warn you if a file already exists. This is not the case on many other Linux systems, so be careful! On other systems, copying a file with cp replaces a file with the same name in that location without asking you or telling you that it has removed the original file (permanently).

mv works like cp, but it moves instead of copies. You can also use it to rename something by moving it to the same location, but specifying a different name: 
myth$ mv originalFileName newFileName
You can move entire directories without any flags: 
myth$ mv directoryOldName directoryNewName
You will be asked before overwriting files with mv, but this is not normally the case on other systems, so be careful!

Navigate to where you would like to create the folder and use the mkdir command, specifying the name of the folder you would like to create: 
myth$ mkdir MyNewFolder
myth$ cd MyNewFolder
If a folder with that name already exists at that location, `mkdir` prints an error message.

Use the rmdir command to delete a directory that is empty: 
myth$ rmdir assign5
Use the rm command to delete files (this is permanent!). On myth you will be prompted to remove a file, but on most linux systems this is not the default behavior, so be careful! 
myth$ rm hello.c
rm: remove regular file 'hello.c'? y
myth$
To remove an entire directory that is not empty, use rm with the -r ("recursive") flag. This will prompt to confirm deleting each file: 
myth$ rm -r assign7
rm: descend into directory ‘assign7’? y
rm: remove regular file ‘assign7/readme.txt’? y
rm: remove regular file ‘assign7/hello.c’? y
...
If you want to bypass the manual prompts (be careful!), use the -f ("force") flag as well. Be very careful with the -rf flags!. The files are permanently deleted without prompting you for confirmation, even if you mistype. 
myth$ rm -rf assign7
myth$

Commands To Print, Search and Compare Files

The cat command ("concatenation") prints a file to the terminal. 
myth$ cat hello.c
#include<stdio.h>
#include<stdlib.h>

int main() {
    printf("Hello, World!\n");
    return 0;
}
myth$
You can also use cat to print multiple files one after the other, e.g., cat file1 file2 file3.

grep lets you search specified files for a specified pattern: grep [pattern] [file(s) to search]. For example, if we were looking for all the lines where the binky function is called in program.c, we could use search the source file as follows: 
grep "binky()" program.c
The pattern you search for can be a "regular expression", which means you do not have to specify an exact string, but rather some general pattern, that you are looking for. For example, . matches any character, e.g, 'a..' matches 'abc' and also 'adf'. * matches zero or more repeats of char to its left, e.g., 'ab*'' matches 'abbbbb' and also 'a'. ^ matches the beginning of the line. $ matches the end of the line. For example, let's pretend binky takes some arguments. We can instead search: 
grep "binky(.*)" program.c
Certain punctuation characters such as * and $ have special meaning to the terminal and may get transformed before passing these arguments along to the program. It is best to get into the habit of enclosing the pattern argument in single-quotes when invoking grep to ensure the pattern is received as intended. Also check out the manual page for grep for more customization options!

The find command lets you search for files with a given name. You specify the directory you want to search, and then the name to search for, after -name. For example, let's say we want to search the assign1 folder for all files named hello.c: 
myth$ find assign1 -name "hello.c"
assign1/hello.c
myth$
The name to search for is usually in quotes. This uses a different search syntax than grep, so you can search for patterns like this to search for all .c files, for example: 
myth$ find . -name "*.c"
./assign0/hello.c
./assign1/folder1/myProgram.c
./assign1/hello.c
./assign3/loop.c
myth$
Check out the manual pages for more customization options!

You can specify the names of 2 files to the diff command and it will output all differences: 
myth$ diff hello.c hello2.c
5c5
<     printf("Hello, World!\n");
---
>     printf("Howdy, World!\n");
myth$
This output tells us that line 5 in both files (5c5) is different. Specifically, hello2.c does not have the first line, but has the second line. You can read it as "what changes do I need to make to go from hello.c to hello2.c?". The answer here is we would remove the line printing "Hello"... and replace it with the line printing "Howdy"....Here's another example: 
myth$ diff hello.c hello3.c
3a4
> // a comment
5c6
<     printf("Hello, World!\n");
---
>     printf("HellO, World!\n");
myth$
In this case, line 3 from hello.c was not present and was "added" at line 4 in file 2. Then, we have the changed line, which happens at line 5 in hello.c and at line 6 in hello3.c. If we re-run the command with the opposite order of the files, we get: 
4d3
< // a comment
6c5
<     printf("HellO, World!\n");
---
>     printf("Hello, World!\n");
Diff always notes the differences with respect to the first file, so in this case, the comment line 4 was present in the first file but not the second file, so it was deleted (and would have been present in the second file at line 3). Note that the order of lines in the files do matter. Incidentally, this command works by solving the "longest common subsequence" problem that you might have seen in CS106B!

Manipulating Input and Output

Use the > operator to save the output of a command to a certain file: 
myth$ ./hello > outputFile.txt
myth$ cat outputFile.txt
Hello, World!
myth$
If you want to append the output to a file that already exists, use >>: 
myth$ ./hello >> outputFile.txt
myth$ cat outputFile.txt
Some text that was here already
Hello, World!
myth$

Use the < operator to send the contents of a file as input to a command. This works for programs that read user input while they are running. 
myth$ ./addTwoNumbers
This program will add two doubles!
Please enter a double: 40.5
Please enter a double: 1.5
The sum is: 42.000000
myth$ cat twoNumbers.txt
40.5
1.5
myth$ ./addTwoNumbers < twoNumbers.txt
This program will add two doubles!
Please enter a double: Please enter a double: The sum is: 42.000000
Note that the input isn't actually shown on the screen (which may screw up your formatting). In CS107, we generally avoid reading input from a user, and if we do need to read input, we explicity use command-line parameters for our programs.

The original unix methodology for utility programs was "make small programs that only do one thing, and do them well." For example, the cat command simply takes one or more filenames and prints them one after another. The "killer" idea for unix was that, once you have small programs that each do one thing, you can then chain the output of one program into the input of another program, called piping. The result of this is the same as writing the output of the first command to some temporary file, and then reading the input into the second command. The "pipe" | operator lets us do this. For this example, note that the wc -l command outputs the number of lines in the input. If we wanted to calculate the number of times we call binky in program.c, we could do the following: 
grep "binky()" program.c | wc -l
This says "run the grep command, and send the output of this command as input to the wc command." You can pipe multiple times, as well. For instance, if we want to find the number of diff lines that contain #include for two files, we could do: 
diff file1.c file2.c | grep "#include" | wc -l