===== 1. Shell Something Out =====

==== Printing in the terminal ====

  * We cannot echo exclamation mark (!) in double quotes. But these will work:<code>echo Hello world !
echo 'Hello world !'
echo "Hello world \!"</code>
  * We can use **printf** instead of **echo**
  * No new line by default with printf, so we have to add it when needed:<code>printf  "%-5s %-10s %-4s\n" No Name  Mark 
printf  "%-5s %-10s %-4.2f\n" 1 Sarath 80.3456 
printf  "%-5s %-10s %-4.2f\n" 2 James 90.9989 
printf  "%-5s %-10s %-4.2f\n" 3 Jeff 77.564</code>
  * In '%-5s' above, '-' is used to specify the left alignment, otherwise the text is right aligned.
  * echo can be used with escape sequences when the flag -e is provided (with double-quoted strings):<code>echo -e "1\t2\t3"</code>

  * We can change the output color with escaped codes: for instance:  reset = 0, black = 30, red = 31, green = 32, yellow = 33, blue = 34, magenta = 35, cyan = 36, and white = 37:<code>echo -e "\e[1;31m This is red text \e[0m"</code>
  * For colored background use:  reset = 0, black = 40, red = 41, green = 42, yellow = 43, blue = 44, magenta = 45, cyan = 46, and white=47:<code>echo -e "\e[1;42m Green Background \e[0m"</code>


==== Playing with variables and environment variables ====

  * Get process ID with pgrep, for instance:<code>pgrep gedit</code>
  * List the environment variables of a process with:<code>cat /proc/$PID/environ</code>

<note>Previous commands doesn't seem to work in cygwin.</note>
  
  * We can use **tr** to substitute characters:<code>echo "Coucou manu." | tr 'o' '8'</code>

  * To find length of a string:<code>length=${#var}</code>
  * To Identifying the current shell:<code>echo $SHELL</code>
  * To check for super user:<code>If [ $UID -ne 0 ]; then
    echo Non root user. Please run as root.
else
    echo Root user
fi</code>
  * To modify the Bash prompt string:<code>PS1="PROMPT>"</code>
  * For the PS1 var there are also certain special characters that expand to system parameters. For example, \u expands to username, \h expands to hostname, and \w expands to the current working directory.

==== Function to prepend to environment variables ====

  * We can add in .bashrc the method:<code>prepend() { [ -d "$2" ] && eval $1=\"$2':'\$$1\" && export $1; }</code>
  * Or, to fix the issue of empty variable when predending:<code>prepend() { [ -d "$2" ] && eval $1=\"$2\$\{$1:+':'\$$1\}\" && export $1 ; }</code>
  * Note: **${parameter:+expression}** will expand to **expression** if **parameter** is set and not null.
  * And then call:<code>prepend PATH /opt/myapp/bin
prepend LD_LIBRARY_PATH /opt/myapp/lib</code>

==== Math with the shell ====

  * We mainly use **let**, **( ( ) )**, **[]**, **expr** and **bc**.
  * Using **let** for basic operations:<code>no1=4;
no2=5;
let result=no1+no2
echo $result

# incrementation:
let no1++  

# decrementation:
let no1--  

# shorthands:
let no+=6
let no-=6
</code>

  * Using [] operator as alternate method:<code>result=$[ no1 + no2 ]

# using the $ prefix inside [] is legal:
result=$[ $no1 + 5 ]
</code>

  * Using ( () ) or expr:<code>result=$(( no1 + 50 ))
result=`expr 3 + 4`
result=$(expr $no1 + 5)</code>

  * For floating point operations we can use bc:<code>echo "4 * 0.56" | bc

no=54; 
result=`echo "$no * 1.5" | bc`
echo $result

# passing additional parameters to bs with prefixes to the operation
# and semicolon delimiters:

# Decimal places scale with bc:
# Note that the default scale is 0 and bc is thus outputting integers.
echo "scale=2;3/8" | bc

# Base conversion with bc:
no=100
echo "obase=2;$no" | bc

no=1100100
echo "obase=10;ibase=2;$no" | bc

# Calculating square roots and pows:
echo "scale=3;sqrt(10)" | bc
echo "10^10" | bc
</code>

==== Playing with file descriptors and redirection ====

  * Reserved file descriptors are:
    * 0: stdin
    * 1: stdout
    * 2: stderr

  * Saving to a file is done with:<code>echo "This is a sample text 1" > temp.txt</code>
  * To append to a file we use:<code>echo "This is a sample text 1" >> temp.txt</code>
  * To view file content:<code>cat temp.txt</code>
  * To redirect stderr exclusively to a file and stdout to another file:<code>cmd 2>stderr.txt 1>stdout.txt</code>
  * To redirect stderr and stdout to a single file by converting stderr to stdout:<code>cmd 2>&1 >output.txt</code>
  * We can also redirect to /dev/null to discard an output stream:<code>cmd 2>/dev/null</code>

  * We can use the tee command to make a copy of the output stream instead of redirecting:<code># Prototype: command | tee FILE1 FILE2

cat a* | tee out.txt | cat -n

# To append to a file we use the flag -a:
cat a* | tee -a out.txt | cat -n
</code>

  * We can use stdin as a command argument. It can be done by using - as the  filename argument for the command:<code>cmd1 | cmd2 | cmd -</code>

  * Redirection from a file to a command:<code>cmd < file</code>
  * Redirection froma text block within script:<code>cat<<EOF>log.txt
LOG FILE HEADER
This is a test log file
Function: System statistics
EOF</code>
  
  * To create custom file descriptors:<code>exec 3<input.txt # open for reading with descriptor number 3

echo this is a test line > input.txt
exec 3<input.txt

# Use the descriptor:
cat<&3

# To use the descriptor again we must reassign it with exec.

# For write descriptor with truncate mode:
exec 4>output.txt
echo newline >&4
cat output.txt

# For write descriptor with append mode:
exec 5>>output.txt
echo newline >&5
cat output.txt
</code>

==== Arrays and associative arrays ====

  * For associative arrays we need bash version 4 or higher.
  * Array definitions:<code># Values are stored in consecutive locations starting
# from index 0:
array_var=(1 2 3 4 5 6)

# Definition with set of index-value pairs:
array_var[0]="test1"
array_var[1]="test2"
array_var[2]="test3"
array_var[3]="test4"</code>

  * To print array content:<code># Print content at given index:
echo ${array_var[0]}

index=5
echo ${array_var[$index]}

# Print all the values:
echo ${array_var[*]}

# or:
echo ${array_var[@]}
</code>

  * To get array legnth:<code>echo ${#array_var[*]}</code>
  * Using associative arrays:<code>declare -A ass_array

# Then we add elements with:
ass_array=([index1]=val1 [index2]=val2)

# Or:
ass_array[index1]=val1
ass_array'index2]=val2

# For instance:
declare -A fruits_value
fruits_value=([apple]='100dollars' [orange]='150 dollars')
echo "Apple costs ${fruits_value[apple]}"

# To list array indexes:
echo ${!array_var[*]}

# Or:
echo ${!array_var[@]}
</code>

==== Visiting aliases ====

  * Create an alias with:<code>alias new_command='command sequence'
# For instance:
alias install='sudo apt-get install'</code>

  * To keep permanent aliases, we have to add them in .bashrc:<code>echo 'alias cmd="command seq"' >> ~/.bashrc</code>

  * To Remove alias:<code>unalias example
# or:
alias example=</code>

  * We can escape aliases with exacuting a command with:<code>\command</code>

==== Grabbing information about the terminal ====

  * Using **tput** and **stty**:<code># Get num columns and rows:
tput cols
tput lines

# get current terminal name:
tput longname # not working on cygwin

# Move cursor to position 10,10:
tput cup 10 10

# set background color n=0 to 7:
tput setb n

# set foreground color n=0 to 7:
tput setf n

# show bold text:
tput bold

# start and end underlining:
tput smul
tput rmul

# delete from cursor to end of line:
tput ed
</code>

  * To hide a password we use stty:<code>echo -e "Enter password: "
stty -echo
read password
stty echo
echo
echo Password read.</code>

==== Getting and setting dates and delays ====

  * To Read date:<code>date

# to print the epoch (num secs since 1970-01-01 00:00:00 UTC):
date +%s

# find out the epoch of a given date:
date --date "Thu Nov 18 08:07:21 IST 2010" +%s

# Retrieve the week day of a date:
date --date "Jan 20 2001" +%A

# Check time taken by a command:
start=$(date +%s)
commands;
statements;
end=$(date +%s)
difference=$(( end - start))
echo Time taken to execute commands is $difference seconds.
</code>

  * We can delay execution in script with the **sleep** command:<code>echo -n Count:
tput sc # store cursor position
count=0;
while true;
do
    if [ $count -lt 40 ];
    then
        let count++;
        sleep 1; # sleep a given number of seconds
        tput rc # restore cursor position.
        tput ed # clear until end of line.
        echo -n $count;
    else exit 0;
    fi
done</code>

==== Debugging the script ====

  * Debug a complete script file:<code>bash -x scripts.sh</code>
  * Debug only a portion of the script:<code>for i in {1..6};
do
    set -x
    echo $i
    set +x
done
echo "Script executed"</code>

  * We can provide our own formatting for instance with a _DEBUG environment variable:<code>function DEBUG()
{
    [ "$_DEBUG" == "on" ] && $@ || :  # in bash ":" tells the script to do nothing.
}

for i in {1..10}
do
    DEBUG echo $i
done

# Then we would run:
_DEBUG=on ./script.sh
</code>

==== Functions and arguments ====

  * Functions can be defined with:<code>function fname()
{
    statements;
}

# Or alternately:

fname()
{
    statements;
}</code>

  * In a function "$@" expands as "$1" "$2" "$3" and so on
  * In a function "$*" expands as "$1c$2c$3", where c is the first character of IFS.
  * Functions in bash support recursion:<code>F() 
{ 
  echo $1; F hello; sleep 1; 
}</code>

  * Functions can be exported:<code>export -f fname</code>
  * Reading the return value (status) of a command:<code>cmd; 
echo $?;

# For instance:
CMD="command" #Substitute with command for which you need to test the 
exit status
$CMD
if [ $? -eq 0 ];
then
    echo "$CMD executed successfully"
else
    echo "$CMD terminated unsuccessfully"
fi</code>

==== Reading the output of a sequence of commands in a variable ====

  * We can read the output of a sequence with a **subshell method**:<code>cmd_output=$(COMMANDS)

# For instance:
cmd_output=$(ls | cat -n)
echo $cmd_output</code>

  * Or with a **back quotes**:<code>cmd_output=`COMMANDS`

# For instance:
cmd_output=`ls | cat -n`
echo $cmd_output</code>

  *  We can spwan a separate process with ( ):<code>pwd; # current working directory
(cd /bin; ls);
pwd; # Still the same current working directory</code>
 
  * Subshell should be quoted to preserve the newline character:<code>out="$(cat tex.txt)"
echo $out</code>

==== Reading n characters without pressing the return key ====

  * Read n chars:<code>read -n number_of_chars variable_name

# For instance:
read -n 2 var
echo $var


# Read a password:
read -s var

# Display a message before reading:
read -p "Enter input:" var

# Read after timeout:
read -t timeout var  # timeout is a number of seconds.

# Read until delimiter:
read -d delim_char var

# For instance:
read -d ":" var
</code>

==== Running a command until it succeeds ====

  * We can define the repeat method as:<code>repeat()
{
  while true
  do
    $@ && return
  done
}</code>

  * We can optimize by adding a delay and avoiding the spawn of the "true" binary with:<code>repeat() { while :; do $@ && return; sleep 30; done }</code>

==== Field separators and iterators ====

  * Update IFS to read data:<code>data="name,sex,rollno,location"
# To read each of the item in a variable, we can use IFS.
oldIFS=$IFS
IFS=, now,
for item in $data;
do
    echo Item: $item
done
IFS=$oldIFS
</code>

  * Receive some indexed values on a line with IFS:<code>line="root:x:0:0:root:/root:/bin/bash" 
oldIFS=$IFS;
IFS=":"
count=0
for item in $line;
do
     [ $count -eq 0 ]  && user=$item;
     [ $count -eq 6 ]  && shell=$item;
    let count++
done;
IFS=$oldIFS
echo $user\'s shell is $shell;</code>
  
  * We can generate sequences with:<code>echo {1..50}
echo {a..z}
echo {A..Z}
echo {a..h}

# for instance:
for i in {a..z}; 
do 
  echo $i;
done;</code>

  *  For loop can also take the format of for in C:<code>for((i=0;i<10;i++))
{
    commands; # Use $i
}</code>

  * While loop format:<code>while condition
do
    commands;
done</code>

  * until format:<code>x=0;
until [ $x -eq 9 ]; # [ $x -eq 9 ] is the condition
do
    let x++; echo $x;
done</code>

==== Comparisons and tests ====

  * IF syntax:<code>if condition;
then
    commands;
fi</code>
  * else if syntax:<code>if condition; 
then
    commands;
else if condition; then
    commands;
else
    commands;
fi</code>

  * To make if conditions shorter we can use logical operators:<code>[ condition ] && action; # action executes if the condition is true
[ condition ] || action; # action executes if the condition is false</code>

  * Performing mathematical conditions is done with:<code>[ $var -eq 0 ]  # It returns true when $var equal to 0.
[ $var -ne 0 ] # It returns true when $var is not equal to 0</code>

  * Available mathematical operators for conditions are:
    * -gt : greater than
    * -lt : less than
    * -ge : greater or equal
    * -le : less than or equal

  * Combining multiple test conditions is done with:<code>[ $var1 -ne 0 -a $var2 -gt 2 ]  # using AND -a
[ $var1 -ne 0 -o var2 -gt 2 ] # OR -o</code>

  * Filesystem related tests:<code>[ -f $var ] # This returns true if the given variable holds a regular file path or filename

[ -x $var ] # This returns true if the given variable holds a file path or filename that is executable

[ -d $var ] # This returns true if the given variable holds a directory path or directory name

[ -e $var ] # This returns true if the given variable holds an existing file

[ -c $var ] # This returns true if the given variable holds the path of a character device file

[ -b $var ] # This returns true if the given variable holds the path of a block device file

[ -w $var ] # This returns true if the given variable holds the path of a file that is writable

[ -r $var ] # This returns true if the given variable holds the path of a file that is readable

[ -L $var ] # This returns true if the given variable holds the path of a symlink</code>

  * String comparaisons:<code>[[ $str1 = $str2 ]] # This returns true when str1 equals str2
# that is, the text contents of str1 and str2 are the same.

[[ $str1 == $str2 ]] # It is an alternative method for string equality check

# We can check whether two strings are not the same as follows:

[[ $str1 != $str2 ]] # This returns true when str1 and str2 mismatch 

# We can find out the alphabetically smaller or larger string as follows:

[[ $str1 > $str2 ]] # This returns true when str1 is alphabetically greater than str2

[[ $str1 < $str2 ]] # This returns true when str1 is alphabetically lesser than str2

[[ -z $str1 ]] # This returns true if str1 holds an empty string

[[ -n $str1 ]] # This returns true if str1 holds a nonempty string</code>

  * It is sometimes easier to combine multiple conditions using logical operators:<code>if [[ -n $str1 ]] && [[ -z $str2 ]] ;
then
    commands;
fi</code>

  * We can use the test command isntead of the [] operator:<code>if  [ $var -eq 0 ]; then echo "True"; fi
# can be written as:
if  test $var -eq 0 ; then echo "True"; fi</code>