Writing shell scripts

Here is where the fun begins

With the thousands of commands available for the command line user, how can you remember them all? The answer is, you don't. The real power of the computer is its ability to do the work for you. To get it to do that, we use the power of the shell to automate things. We write scripts.

Scripts are collections of commands that are stored in a file. The shell can read this file and act on the commands as if they were typed at the keyboard. In addition to the things you have learned so far, the shell also provides a variety of useful programming features to make your scripts truly powerful.

What are scripts good for? A wide range of tasks can be automated. Here are some of the things I automate with scripts:

  • A script gathers up all the files (over 2200) in this site on my computer and transmits them to my web server.

  • The SuperMan pages are built entirely by a script.

  • Every Friday night, all my computers copy their files to a "backup server" on my network. This is performed by a script.

  • A script automatically gets the current updates from my Linux vendor and maintains a repository of vital updates. It sends me an email message with a report of tasks that need to be done.

As you can see, scripts unlock the power of your Linux machine. So let's have some fun!

Contents

  1. Writing your first script and getting it to work
    1. Writing a script
    2. Setting permissions
    3. Putting it in your path
  2. Editing the scripts you already have
    1. Commands, commands, everywhere
    2. Aliases
    3. Shell functions
    4. type
    5. .bashrc
  3. Here Scripts
    1. Writing an HTML file with a script
  4. Substitutions - Part 1
    1. Variables
    2. How to create a variable
    3. Where does the variable's name come from?
    4. How does this increase our laziness?
    5. Environment Variables
  5. Substitutions - Part 2
    1. --help and other tricks
    2. Assigning a command's result to a variable
    3. Constants
  6. Quoting
    1. Single and double quotes
    2. Quoting a single character
    3. Other backslash tricks
  7. Shell Functions
    1. Keep your scripts working
  8. Some Real Work
    1. show_uptime
    2. drive_space
    3. home_space
    4. system_info
  9. Flow Control - Part 1
    1. if
    2. What is a "condition"?
    3. Exit status
    4. test
    5. exit
    6. Testing for root
  10. Stay Out of Trouble
    1. Empty variables
    2. Missing quotes
    3. Isolating problems
    4. Watching your script run
  11. Keyboard Input and Arithmetic
    1. read
    2. Arithmetic
  12. Flow Control - Part 2
    1. More branching
    2. Loops
    3. Building a menu
  13. Positional Parameters
    1. Detecting command line arguments
    2. Command line options
    3. Getting an option's argument
    4. Integrating the command line processor into the script
    5. Adding interactive mode
  14. Flow Control - Part 3
  15. Errors and Signals and Traps (Oh My!) - Part 1
    1. Exit status
    2. Checking the exit status
    3. An error exit function
    4. AND and OR lists
    5. Improving the error exit function
  16. Errors and Signals and Traps (Oh My!) - Part 2
    1. Cleaning up after yourself
    2. trap
    3. Signal 9 From Outer Space
    4. A clean_up function
    5. Creating safe temporary files

Learning the shell

Why bother?

Why do you need to learn the command line anyway? Well, let me tell you a story. Not long ago we had a problem where I used to work. There was a shared drive on one of our file servers that kept getting full. I won't mention that this legacy operating system did not support user quotas; that's another story. But the server kept getting full and stopping people from working. One of the software engineers in our company spent the better part of a day writing a C++ program that would look through the directories of all the users and add up the space they were using and make a listing of the results. Since I was forced to use the legacy OS while I was on the job, I installed a version of the bash shell that works on it. When I heard about the problem, I realized I could do all the work this engineer had done with this single line:

du -s * | sort -nr > $HOME/space_report.txt

Graphical user interfaces (GUIs) are helpful for many tasks, but they are not good for all tasks. I have long felt that most computers today do not use electricity. They instead seem to be powered by the "pumping" motion of the mouse! Computers were supposed to free us from manual labor, but how many times have you performed some task you felt sure the computer should be able to do? You ended up doing the work by tediously working the mouse. Pointing and clicking, pointing and clicking.

I once heard an author remark that when you are a child you use a computer by looking at the pictures. When you grow up, you learn to read and write. Welcome to Computer Literacy 101. Now let's get to work.

Contents

  1. What is "the shell"?
    1. What's an xterm, gnome-terminal, konsole, etc.?
    2. Starting a Terminal
    3. Testing the Keyboard
    4. Using the Mouse
  2. Navigation
    1. File System Organization
    2. pwd
    3. cd
  3. Looking Around
    1. ls
    2. less
    3. file
  4. A Guided Tour
    1. /
    2. /boot
    3. /etc
    4. /bin, /usr/bin
    5. /sbin, /usr/sbin
    6. /usr
    7. /usr/local
    8. /var
    9. /lib
    10. /home
    11. /root
    12. /tmp
    13. /dev
    14. /proc
    15. /mnt
  5. Manipulating Files
    1. Wildcards
    2. cp
    3. mv
    4. rm
    5. mkdir
  6. I/O Redirection
    1. Standard Output
    2. Standard Input
    3. Pipes
    4. Filters
  7. Permissions
    1. File permissions
    2. chmod
    3. Directory permissions
    4. Becoming the superuser for a short while
    5. Changing file ownership
    6. Changing group ownership
  8. Job Control
    1. A practical example
    2. Putting a program in the background
    3. Listing your processes
    4. Killing a process
    5. A little more about kill
Source

Linux : Searching with "find"

The find command is one of the darkest and least understood areas of Linux, but it is also one of the most powerful. The biggest problem with find is that it has more options than most people can remember -- it truly is capable of doing most things you could want.

This article is excerpted from the newly published book Red Hat Fedora 5 Unleashed . Excerpt provided by Sams Publishing.

Admittedly, the find command does not help itself by using X-style parameters. The Unix standard is -c, -s, and so on, whereas the GNU standard is --dosomething, --mooby, and so forth. X-style parameters merge the two by having words preceded by only one dash. The most basic usage is this:

find -name "*.txt"

That query searches the current directory and all subdirectories for files that end in .txt. The previous search finds files ending in .txt but not .TXT, .Txt, or other case variations. To search without case sensitivity, use -iname instead of -name. You can optionally specify where the search should start before the -name parameter, like this:

find /home -name "*.txt"

Another useful test is -size, which lets you specify how big the files should be to match. You can specify your size in kilobytes and optionally also use + or - to specify greater than or less than. For example:

 find /home -name "*.txt" -size 100k 
 find /home -name "*.txt" -size +100k 
 find /home -name "*.txt" -size -100k 

The first brings up files of exactly 100KB, the second only files greater than 100KB, and the last only files less than 100KB.

The -user option enables you to specify the user that owns the files you are looking for. So, to search for all files in /home that end with .txt, are under 100KB, and are owned by user paul, you would use this:

find /home -name "*.txt" -size -100k -user paul

You can flip any of the conditions by specifying -not before them. For example, you can add a -not before -user paul to find matching files owned by everyone but paul:

find /home -name "*.txt" -size -100k -not -user paul

You can add as many -not parameters as you need, even using -not -not to cancel each other out! (Yes, that is pointless.) Keep in mind, though, that -not -size -100k is essentially equivalent to -size +100k, with the exception that the former will match files of exactly 100KB whereas the latter will not.

You can use -perm to specify which permissions a file should have for it to be matched. This is tricky, so read carefully. The permissions are specified in the same way as with the chmod command: u for user, g for group, o for others, r for read, w for write, and x for execute. However, before you give the permissions, you need to specify a plus, a minus, or a blank space. If you specify neither a plus nor a minus, the files must exactly match the mode you give. If you specify -, the files must match all the modes you specify. If you specify +, the files must match any the modes you specify. Confused yet?

The confusion can be cleared up with some examples. This next command finds all files that have permission o=r (readable for other users). Notice that if you remove the -name parameter, it is equivalent to * because all filenames are matched.

find /home -perm -o=r

Any files that have o=r set are returned from that query. Those files also might have u=rw and other permissions, but as long as they have o=r, they will match. This next query matches all files that have o=rw set:

find /home -perm -o=rw

However, that query does not match files that are o=r or o=w. To be matched, a file must be readable and writeable by other users. If you want to match readable or writeable (or both), you need to use +, like this:

find /home -perm +o=rw

Similarly, this next query matches files that are only readable by user, group, and others:

find /home -perm -ugo=r

Whereas this query matches files as long as they are readable by the user, or by the group, or by others, or by any combination of the three:

find /home -perm +ugo=r

If you use neither + nor -, you are specifying the exact permissions to search for. For example, the next query searches for files that are readable by user, group, and others but not writeable or executable by anyone:

find /home -perm ugo=r

You can be as specific as you need to be with the permissions. For example, this query finds all files that are readable for the user, group, and others and writeable by the user:

find /home -perm ugo=r,u=w

To find files that are not readable by others, use the -not condition, like this:

find /home -not -perm +o=r

Now, on to the most advanced aspect of the find command: the -exec parameter. This enables you to execute an external program each time a match is made, passing in the name of the matched file wherever you want it. This has very specific syntax: Your command and its parameters should follow immediately after -exec, terminated by \;. You can insert the filename match at any point using {} (an opening and a closing brace side by side).

So, you can match all text files on the entire system (that is, searching recursively from / rather than from /home as in our previous examples) over 10KB, owned by paul, that are not readable by other users, and then use chmod to enable reading, like this:

find / -name "*.txt" -size +10k -user paul -not -perm +o=r -exec chmod o+r {} \;

When you type your own -exec parameters, be sure to include a space before \;. Otherwise, you might see an error such as missing argument to ´-exec'.

Do you see now why some people think the find command is scary? Many people learn just enough about find to be able to use it in a very basic way, but hopefully you will see how much it can do if you give it a chance.

Linux Partition HOWTO

1. Introduction
1.1. What is a partition?
1.2. Other Partitioning Software:
1.3. Related HOWTOs
1.4. Additional information on your system:
2. Devices
2.1. Device names
2.2. Device numbers
3. Partition Types
3.1. Partition Types
3.2. Foreign Partition Types
3.3. Primary Partitions
3.4. Logical Partitions
3.5. Swap Partitions
4. Partitioning requirements
4.1. What Partitions do I need?
4.2. Discussion:
4.3. File Systems
4.4. Swap Partitions
5. Partitioning with fdisk
5.1. fdisk usage
5.2. Four primary partitions
5.3. Mixed primary and logical partitions
5.4. Submitted Examples
6. Labels
6.1. Volume Labels
6.2. Device Labels
7. Formatting an ext2/3 partition
.1. Simple Invocation
.2. Reserved blocks
8. Recovering a Deleted Partition Table
9. Setting Up Swap Space
9.1. Swap Files
9.2. Swap Files
9.3. Multiple Swap Areas
10. Appendix
10.1. Formating Partitions
10.2. Activating Swap Space
10.3. Mounting Partitions
10.4. Some facts about file systems and fragmentation