--------------------
Shell Metacharacters

>     `prog >file` direct standard output to file, (over-writes file)
>>    `prog >>file` append standard output to file
<     `prog <file` take standard input from file
|     `p1 | p2` connect standard output of p1 to standard input of p2
<<str  here document: standard input follows, up to next str on a line by 
       itself, one way to represent a multiline string
*      match any string of zero or more characters in filenames (a glob
       expansion, not a regex)
?      match any single character in filenames
[ccc]  match any single character from ccc in filenames; ranges
       like 0-9 or a-z are legal
;      command terminator: `p1;p2` does p1, then p2
&      like ; but doesn't wait for p1 to finish
`...`  (grave/backtick)      run command(s) in ...; output replaces `...`
(...)  run command(s) in ... in a sub-shell
{...}  run command(s) in ... in current shell (rarely used)
$1, $2 etc.    $0...$9 replaced by positional arguments to shell file
$var   value of shell variable var
${var} value of var; avoids confusion when concatenated with text
\       \c take character c literally, \newline discarded
'...'   take ... string literally
"..."   take ... string literally after $, `...` and \ interpreted



--
$#     Provides the total number of arguments passed to the shell script or function.
$*, $@ Represents all the command-line arguments at once.
"$*"   Represents all the command-line arguments as a single string.
"$@"   Represents all the command-line arguments as separate, individual strings.


------------------------------
POSIX built-in shell variables (preceded by $, e.g. "$#")

#     Number of arguments given to current process.
@     Command-line arguments to current process. Inside double quotes, expands
      to individual arguments.
*     Command-line arguments to current process. Inside double quotes, expands
      to a single argument.
-     (hyphen) Options given to shell on invocation.
?     Exit status of previous command.
$     Process ID of shell process.
PPID  Process ID of parent process.
!     Process ID of last background command. Use this to save process ID numbers
      for later use with the wait command.
0     (zero) The name of the shell program, in most cases, including full
      invocation path.
PWD   Current working directory, containing no components of type symbolic link,
      no components that are dot, and no components that are dot-dot when the
      shell is initialized.
ENV   Used only by interactive shells upon invocation; the value of $ENV is
      parameter-expanded. The result should be a full pathname for a file to
      be read and executed at startup. This is an XSI requirement.
HOME  Home (login) directory.
IFS   Internal field separator; i.e., the list of characters that act as word
      separators. Normally set to space, tab, and newline.
LANG  Default name of current locale; overridden by the other LC_* variables.
LC_ALL     Name of current locale; overrides LANG and the other LC_* variables.
LC_COLLATE   Name of current locale for character collation (sorting) purposes.
LC_CTYPE     Name of current locale for character class determination during
             pattern matching.
LC_MESSAGES  Name of current language for output messages.
LINENO    Line number in script or function of the line that just ran.
NLSPATH    The location of message catalogs for messages in the language given
           by $LC_MESSAGES (XSI).
PATH Search path for commands.
PS1  Primary command prompt string. Default is "$ ".
PS2  Prompt string for line continuations. Default is "> ".
PS4  Prompt string for execution tracing with set -x. Default is "+ ".

More POSIX built-ins can be found here (but seriously, don't trust all of them):
http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html


------------------------------------
Finding Programs in your enviornment

which <utility>         Returns the path to the utility if it exists, 
                        heritage from C shell.
command -v <utility>    Returns the path to the utility, -v flag keeps it
                        silent, -V is default behavior.  Very portable.
whereis <utility>       Looks up executable, man page, and source- not based
                        on the current environment.

----------------------------------------------
Evaluation and Substitution of Shell Variables

$var      Value of var; nothing if var undefined.  ENV variables are, by
          convention, typically typed all caps.
${var}    Same as $var, useful if alphanumerics follow variable name.
${var-thing}  Value of var if defined, otherwise thing, $var remains unchanged.
${var:-word}  If var exists and isn't null, return it's value; otherwise,
              return word.
              Purpose: To return a default value if the variable is undefined.
              Example: ${count:-0} evaluates to 0 if count is undefined.
${var:=word}  If var exists and isn't null, return it's value; otherwise set it
              to word and return it's value.
              Purpose: To set a variable to a default value if it is undefined.
              Example: ${count:=0} sets count to 0 if it is undefined.
${var:?message}  If var exists and isn't null, return it's value; otherwise,
                 print 'var: message', and abort the current command of script.
                 Omitting message produces the default message parameter null or
                 not set.  Note, however, that interactive shells do not have to
                 abort.  (Behavior and exit code varies across shells.)
                 Purpose: To catch errors that result from variables being
                 undefined.
                 Example: ${count:?"undefined!"} prints 'count: undefined!' and
                 exits if count is undefined.
${var:+word}  If var exists and isn't null, return word; otherwise, return null.
              Purpose: To test for the existence of a variable.
              Example: ${count:+1} returns 1 (which could mean "true") if count
              is defined.

-----------------------------------------
POSIX dirname(1) and basename(1) Examples

path           dirname        basename
"/usr/lib"     "/usr"         "lib"
"/usr/"        "/"            "usr"
"usr"          "."            "usr"
"/"            "/"            "/"
"."            "."            "."
".."           "."            ".."

Command                Results
dirname /              /
dirname //             / or //
dirname /a/b/           /a
dirname //a//b//       //a
dirname                Unspecified
dirname a              . ($? = 0)
dirname ""             . ($? = 0)
dirname /a             /
dirname /a/b           /a
dirname a/b            a


-----------------------------------
Pattern-matching Varialbe Operators

${variable#pattern}    If the pattern matches the beginning of the variable’s
                       value, delete the shortest part that matches and return
                       the rest.
                       Example: ${path#/*/}  Result: tolstoy/mem/long.file.name
${variable##pattern}   If the pattern matches the beginning of the variable’s
                       value, delete the longest part that matches and return
                       the rest.
                       Example: ${path##/*/}  Result: long.file.name
${variable%pattern}    If the pattern matches the end of the variable’s value,
                       delete the shortest part that matches and return the
                       rest.
                       Example: ${path%.*}  Result: /home/tolstoy/mem/long.file

${variable%%pattern}   If the pattern matches the end of the variable’s value,
                       delete the longest part that matches and return the rest.
                       Example: ${path%%.*}  Result: /home/tolstoy/mem/long


-------------------------------
Subshell and code block summary

Construct   Delimiters  Separate Process  Recognized where
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Subshell    ( )         Yes               Anywhere on the line
Code block  { }         No                After newline, semicolon, or keyword


----------------
Test Expressions
for use with test(1), and [ , as builtin and utility.
(By the way, did you know that '/bin/[' exists as a binary?)

string         string is not null
-b file        file is a block device file.
-b file        file is a block device file.
-c file        file is a character device file.
-d file        file is a directory.
-e file        file exists.
-f file        file is a regular file.
-g file        file has its setgid bit set.
-h file        file is a symbolic link.
-L file        file is a symbolic link. (Same as –h.)
-n string      string is non-null.
-p file        file is a named pipe (FIFO file).
-r file        file is readable.
-S file        file is a socket.
-s file        file is not empty.
-tn            File descriptor n points to a terminal.
-u file        file has its setuid bit set.
-w file        file is writable.
-x file        file is executable, or file is a directory that can be searched.
-z string      string is null.
s1=s2          Strings s1 and s2 are the same.
s1!=s2         Strings s1 and s2 are not the same.
n1 -eq n2      Integers n1 and n2 are equal.
n1 -ne n2      Integers n1 and n2 are not equal.
n1 -lt n2      n1 is less than n2.
n1 -gt n2      n1 is greater than n2.
n1 -le n2      n1 is less than or equal to n2.
n1 -ge n2      n1 is greater than or equal to n2.

-a             Logical and operator
n1 -gt n2 -a n1 -gt n3
-o             Logical or operator
n1 -gt n2 -o n1 -gt n3

---------------------
echo Escape Sequences

\a      Alert character, usually the ASCII BEL character.
\b      Backspace.
\c      Suppress the final newline in the output. Furthermore, any characters
        left in the argument, and any following arguments, are ignored (not
        printed).
\f      Formfeed.
\n      Newline.
\r      Carriage return.
\t      Horizontal tab.
\v      Vertical tab.
\\      A literal backslash character.
\0ddd   Character represented as a 1- to 3-digit octal value.


-----------------------
printf Escape Sequences

\a      Alert character, usually the ASCII BEL character.
\b      Backspace.
\c      Suppress any final newline in the output.a Furthermore, any characters
        left in the argument, any follow- ing arguments, and any characters left
        in the format string are ignored (not printed).
\f      Formfeed.
\n      Newline.
\r      Carriage return.
\t      Horizontal tab.
\v      Vertical tab.
\\      A literal backslash character.
\ddd    Character represented as a 1- to 3-digit octal value. Valid only in the
        format string. \0ddd Character represented as a 1- to 3-digit octal
        value.

--------------------------------
POSIX BRE and ERE metacharacters

\      BRE/ERE      Usually, turn off the special meaning of the following
                    character. Occasionally, enable a special meaning for the
                    following character, such as for \(...\) and \{...\}.
.      BRE/ERE      Match any single character except NUL. Individual programs
                    may also disallow match- ing newline.
*      BRE/ERE      Match any number (or none) of the single character that
                    immediately precedes it. For EREs, the preceding character
                    can instead be a regular expression. For example, since .
                    (dot) means any character, .* means “match any number of any
                    character.” For BREs, * is not special if it’s the first
                    character of a regular expression.
^      BRE/ERE      Match the following regular expression at the beginning of
                    the line or string. BRE: spe- cial only at the beginning of
                    a regular expression. ERE: special everywhere.
$      BRE/ERE      Match the preceding regular expression at the end of the
                    line or string. BRE: special only at the end of a regular
                    expression. ERE: special everywhere.
[...]  BRE/ERE      Termed a bracket expression, this matches any one of the 
                    enclosed characters. A hyphen (-) indicates a range of
                    consecutive characters. (Caution: ranges are locale-
                    sensitive, and thus not portable.) A circumflex (^) as the
                    first character in the brackets reverses the sense: it
                    matches any one character not in the list. A hyphen or close
                    bracket (]) as the first character is treated as a member of
                    the list. All other metacharac- ters are treated as members
                    of the list (i.e., literally). Bracket expressions may
                    contain collating symbols, equivalence classes, and
                character classes (described shortly).
\{n,m\} BRE     Termed an interval expression, this matches a range of
                occurrences of the single character that immediately
                precedes it. \{n\} matches exactly n occurrences, \{n,\ }
                matches at least n occurrences, and \{n,m\} matches any
                number of occurrences between n and m. n and m must be
                between 0 and RE_DUP_MAX (minimum value: 255), inclusive.
\( \)   BRE     Save the pattern enclosed between \( and \) in a special
                holding space. Up to nine subpatterns can be saved on a
                single pattern. The text matched by the subpat- terns can be
                reused later in the same pattern, by the escape sequences 
                \1 to \9. For example, \(ab\).*\1 matches two occurrences of
                ab, with any number of characters in between.
\n      BRE     Replay the nth subpattern enclosed in \( and \) into the
                pattern at this point. n is a number from 1 to 9, with 1
                starting on the left.
{n,m}   ERE     Just like the BRE \{n,m\} earlier, but without the
                backslashes in front of the braces.
+       ERE     Match one or more instances of the preceding regular expression.
?       ERE     Match zero or one instances of the preceding regular expression.
|       ERE     Match the regular expression specified before or after.
()      ERE     Apply a match to the enclosed group of regular expressions.

-------------------------------------------
Simple regular expression matching examples

scully    The six letters scully, anywhere on a line
^scully   The six letters scully, at the beginning of a line
scully$   The six letters scully, at the end of a line
^scully$  A line containing exactly the six letters scully, and nothing else
[Ss]cully Either the six letters Scully, or the six letters scully, anywhere
          on a line
scu.ly    three letters scu, any character, and the two letters ly, anywhere
          on a line
scu.*ly   The three letters scu, any sequence of zero or more characters, and
          the two letters ly, anywhere on a line (e.g., scuyly, scully,
          scuWHOlly, and so on)


-----------------------
POSIX character classes

[:alnum:]      Alphanumeric characters
[:alpha:]      Alphabetic characters
[:blank:]      Space and tab characters
[:cntrl:]      Control characters
[:digit:]      Numeric characters
[:graph:]      Nonspace characters
[:lower:]      Lowercase characters
[:print:]      Printable characters
[:punct:]      Punctuation characters
[:space:]      Whitespace characters
[:upper:]      Uppercase characters
[:xdigit:]     Uppercase characters


------------------------
printf Format Specifiers

%b    The corresponding argument is treated as a string containing escape
%c    ASCII character. Print the first character of the corresponding argument.
%d, %i    Decimal integer.
%e    Floating-point format ([-]d.precisione[+-]dd).
%E    Floating-point format ([-]d.precisionE[+-]dd).
%f    Floating-point format ([-]ddd.precision).
%g    %e or %f conversion, whichever is shorter, with trailing zeros removed.
%G    %E or %f conversion, whichever is shorter, with trailing zeros removed.
%o    Unsigned octal value.
%s    String.
%u    Unsigned decimal value.
%x    Unsigned hexadecimal number. Use a–f for 10 to 15.
%X    Unsigned hexadecimal number. Use A–F for 10 to 15.
%%    Literal %.


--------------------
Meaning of precision

%d, %i, %o, %u, %x, %X
         The minimum number of digits to print. When the value has fewer digits,
         it is padded with leading zeros. The default precision is 1.
%e, %E   The minimum number of digits to print. When the value has fewer digits,
         it is padded with zeros after the decimal point. The default precision
         is 6. A precision of 0 inhibits printing of the decimal point.
%f       The number of digits to the right of the decimal point.
%g, %G   The maximum number of significant digits.
%s       The maximum number of characters to print.


----------------
Flags for printf

–      Left-justify the formatted value within the field.
+      Always prefix numeric values with a sign, even if the value is positive.
#      Use an alternate form: %o has a preceding 0; %x and %X are prefixed with
       0x and 0X, respectively; %e, %E, and %f always have a decimal point in
       the result; and %g and %G do not have trailing zeros removed.
0      Pad output with zeros, not spaces. This happens only when the field width
       is wider than the converted result. In the C language, this flag applies
       to all output formats, even nonnumeric ones. For the printf command, it
       applies only to the numeric formats.

---------------
Basic wildcards

?       Any single character
*       Any string of characters
[set]   Any character in set
[!set]  Any character not in set


---------------------------------
Using the set construct wildcards

[abc]          a, b, or c
[.,;]          Period, comma, or semicolon
[-_]           Dash or underscore
[a-c]          a, b, or c
[a-z]          Any lowercase letter
[!0-9]         ny nondigit
[0-9!]         ny digit, or an exclamation mark
[a-zA-Z]       Any lower- or uppercase letter
[a-zA-Z0-9_-]  Any letter, any digit, underscore, or dash


---------
expr operators

e1|e2    If e1 is nonzero or non-null, its value is used. Otherwise, if e2 is
         nonzero or non-null, its value is used. Otherwise, the final value is
         zero.
e1&e2    If e1 and e2 are non-zero or non-null, the return value is that of e1.
         Otherwise, the final value is zero.

e1=e2    Equal.
e1!=e2   Not equal.
e1<e2    Less than.
e1<=e2   Less than or equal to.
e1>e2    Greater than.
e1>=e2   Greater than or equal to.
         These operators cause expr to print 1 if the indicated comparison is
         true, 0 otherwise. If both oper- ands are integers, the comparison is
         numeric; otherwise, it’s a string comparison.

e1+e2    The sum of e1 and e2.
e1-e2    The difference of e1 and e2.
e1*e2    The product of e1 and e2.
e1/e2    The integer division of e1 by e2 (truncates).
e1%e2    The remainder of the integer division of e1 by e2 (truncates).
e1:e2    Match of e1 to BRE e2; see the expr(1) manpage for details.
( expression )  The value of expression; used for grouping, as in most
                programming languages.
integer    A number consisting only of digits, although an optional leading 
           minus sign is allowed. Sadly, unary plus is not supported.
string    A string value that cannot be mistaken for a number or an operator.


--------------------
arithmetic operators

++ --
        Left to Right  Increment and decrement, prefix and postfix
+-!~
        Right to Left  Unary plus and minus; logical and bitwise negation
* /%
        Left to Right  Multiplication, division, and remainder
+-
        Left to Right  Addition and subtraction
<< >>
        Left to Right  Bit-shift left and right
< <= > >=
        Left to Right  Comparisons
= = !=
        Left to Right  Equal and not equal
&
        Left to Right  Bitwise AND
^
        Left to Right  Bitwise Exclusive OR
|
        Left to Right  Bitwise OR
&&
        Left to Right  Logical AND (short-circuit)
||
        Left to Right  Logical OR (short-circuit)
?:
        Right to Left  Conditional expression
= += -= *= /= %= &= ^= <<= >>= |=
        Right to Left  Assignment operators

--------------------
Sort key field types

b      Ignore leading whitespace.
d      Dictionary order.
f      Fold letters implicitly to a common lettercase.
g      Compare as general floating-point numbers.  GNU version only.
i      Ignore nonprintable characters.
n      Compare as (integer) numbers.
r      Reverse the sort order.


---------------------
Shell options for set
(be warned, set is the interface for misfit features in all shells)

-a   allexport      POSIX         Export all subsequently defined variables.
-b   notify         POSIX         Print job completion messages right away, instead of waiting for next prompt. Intended for interactive use.
-B   braceexpand    bash          Enable brace expansion. On by default. See “Brace Expansion” [14.3.4] for more information.
-C   noclobber      POSIX         Don’t allow > redirection to existing files. The >| oper- ator overrides the setting of this option. Intended for interactive use.
-e   errexit        POSIX         Exit the shell when a command exits with nonzero status.
-f   noglob         POSIX         Disable wildcard expansion.
-m   monitor        POSIX         Enable job control (on by default). Intended for inter- active use.
-n   noexec         POSIX         Read commands and check for syntax errors, but don’t execute them. Interactive shells are allowed to ignore this option.
-u   nounset        POSIX         Treat undefined variables as errors, not as null.
-v   verbose        POSIX         Print commands (verbatim) before running them.
-x   xtrace         POSIX         Print commands (after expansions) before running them.
     ignoreeof      POSIX         Disallow Ctrl-D to exit the shell.
     nolog          POSIX         Disable command history for function definitions.
     vi             POSIX         Use vi-style command-line editing. Intended for interactive use.

-A   ksh88,ksh93  Array assignment. set +A does not clear the array. Refer to Korn shell “Indexed Arrays”for more information.
-H   trackall (ksh)             Same as POSIX -f noglob, Disable wildcard expansion.
-s   ksh88, ksh93    Sort the positional parameters.


Common shell options which, for portability, should be avoided:

-h   hashall (bash) POSIX  Locate and remember the location of commands called from function bodies when the function is defined, instead of when the function is executed (XSI).
-k   histexpand  bash            Enable !-style history expansion. On by default.
-P   physical    bash            Use the physical directory structure for commands that change directory.
     history     bash            Enable command history. On by default.
     posix       bash            Enable full POSIX compliance.
     markdirs    ksh88, ksh93    Append a / to directories when doing wildcard expansion.
     pipefail    ksh93           Make pipeline exit status be that of the last command that fails, or zero if all OK. ksh93n or newer.
     viraw       ksh88, ksh93    Use vi-style command-line editing. Intended for interactive use. This mode can be slightly more CPU- intensive than set -o vi.
-p   privileged  bash, ksh88,    ksh93    Attempt to function in a more secure mode. The details differ among the shells; see your shell’s docu- mentation.
-t   bash,       ksh88, ksh93    Read and execute one command and then exit. This is obsolete; it is for compatibility with the Bourne shell and should not be used.
     bgnice      ksh88, ksh93    Automatically lower the priority of all commands run in the background (with &).
     emacs       bash, ksh88,    ksh93    Use emacs-style command-line editing. Intended for interactive use.
     gmacs       ksh88, ksh93    Use GNU emacs-style command-line editing. Intended for interactive use.



---------------------------------------------------------
POSIX shell built-in commands, a real-world portable list
(If utility exists, built-in will be run unless utility is explicitly called)

: (colon)      Do nothing (just do expansions of arguments).
. (dot)        Read file and execute it's contents in current shell,
               (to 'source' a file in bash).
alias          Set up shorthand for command or command line (interactive use).
bg             Put a job in background (interactive use).
break          Exit from surrounding for, while, or until loop.
cd             Change working directory
command        Locate built-in and external commands; find a built-in command instead of an identically named function.
continue       Skip to next iteration of for, while, or until loop.
eval           Process arguements as a command line.
exec           Replace shell with given program or change I/O for shell.
exit           Exit from shell.
export         Create environment variables.
false          Do nothing, unsuccessfully.
fc             Work with command history (interactive use).
fg             Put background job in foreground (interactive use).
getopts        Process command-line options, (also less portable getopt).
jobs           List
kill           Send signals.
newgrp         Start new shell with new group ID (obsolete).
pwd            Print working directory.
read           Read a line from standard input.
readonly       Make variables read-only (unassignable).
return         Return from surrounding function.
set            Set options or positional parameters.
shift          Shift command-line arguements.
times          Print accumulated user and system CPU times for the shell and it's children.
trap           Set up signal-catching routine.
true           Do nothing, successfully.
umask          Set/show file permission mask.
unalias        Remove alias definitions (interactive use).
unset          Remove definitions of variables or functions.
wait           Wait for background job(s) to finish.