------------------------------------------------------------------------------- Output variables and attributes declare -p variable For functions use -f ------------------------------------------------------------------------------- Bash Variable substitution ${var:-word} # use var otherwise use 'word' ${var:+word} # if var is non-null use 'word' else nothing ${var:=word} # assign var with 'word' if unset, then return var # (remove ':' to apply on unset or empty) ${var+word} # if var has been set use 'word' else nothing # # EG: use $count in ping with -c option if set # ping -q ${count:+-c "$count"} "$HOSTNAME" # # This is the best 'bourne shell' way to test if var # or array/hash element has been set (even is empty) # EG: [[ ${var+has_been_set} ]] ${var@Q} # quote the variable properly for use in bash # var="a\"b'c" echo "$var" '-->' "${var@Q}" # # => a"b'c --> 'a"b'\''c' # Note: quoting the ${var@Q} makes no difference ${#var} # variable string length in characters (not bytes) ${var:offset} # var from offset chars to end (starts at 0) ${var:offset:len} # only return len chars from offset (neg removes N chars) # negatives (mean offset from end) # # var=$( echo {a..z} | tr -d ' ' ) # echo ${var:1:1} # get second char 'b' # echo ${var:5:3} # get 3 chars 'fgh' # echo ${var:0:-3} # all but last 3 chars # echo ${var: -10:5} # space needed for neg 'qrstu' # echo ${var:(-10):5} # parenthsis works too # echo ${var:(-3)} # last 3 chars 'xyz' ${var#word} # remove prefix 'word' start of string (may be glob!) ${var##word} # use the longest matching prefix glob (* can include /) # EX: basename: echo "user = ${HOME##*/}" # prefix grep test... # This: if echo "$answer" | grep -iq "^y"; then # to: if [ "$answer" != "${answer#[Yy]}" ]; then ${var%word} # remove suffix 'word' from end of string (may be glob!) # EX: dirpath: echo "${BASH_SOURCE%/*}" # ${var%%word} # use the longest matching suffix glob # example echo "Login IP:" ${SSH_CLIENT%% *} ${var/sub/repl} # substr replace first occurance of 'sub' with 'repl' # Example: substitute $HOME with '~' in directory path # echo ${PWD/${HOME}/\~} ${var//sub/repl} # substr replace all occurances of 'sub' with 'repl' # o=${1//%/%25} # do percent first # o=${o//&/%26} # o=${o////%2F} # --- urlencode $1 (minimal) # o=${o//=/%3D} # o=${o//\?/%3F} # o=${o//+/%2B} # handle + # o=${o// /+} # handle space # printf "%s" $o" ${var/#sub/repl} # substr replace 'sub' at start with 'repl' ${var/%sub/repl} # substr replace 'sub' at end with 'repl' # the '/repl' is optional if you just want to remove sub # see the start of "file.txt" for examples # ${file//-/_} # substitute all '-' for '_' ${var^} # output var with first char uppercased ${var,,} # output var with everything lowercased # ^ uppercase , lowercase ~ swap upper-lower ${!reference} # contents of the variable that $reference names # See "Indirect Variables" below ${!prefix*} # list variables starting with prefix ------------------------------------------------------------------------------- Newlines in Variables... Note that var=`cmd` will save the newline characters (quotes not needed) But to use them you must quote the variable! m=$( mount | awk '{print $1}' ) echo $m # => / /usr /home echo "$m" # => / # /usr # /home echo ${m@Q} # bash quote output # => $'/\n/usr\n/home' NOTE: the final newline of a command is alwasy automatically removed! In otherwords outside quotes, newlines in the input are treated purely as white space between arguments and thus ignored. Inside quotes newlines are retained as newlines. ------------------------------------------------------------------------------- test or '[' vs '[[' '[' is legacy but works with older shells, and thus provides more backwards compatible scripting. However as it was based as a command, and is executed like a command. As such its arguments MUST be quoted. While [[ is a bash builtin expression, so aruments do not need quoting. and can directly test if the variable is actually defined (see next). Because of this the two tests execute differently. For example b='/*' [ $b ] && echo true || echo false # ERRORS too many arguments [ "$b" ] && echo true || echo false # quoting makes it work [[ $b ]] && echo true || echo false # expression has no problems [[ -n "$b" ]] && echo not_empty [[ -z "$b" ]] && echo zero_length [[ -v b ]] && echo true || echo false # variable has been set [[ ! -v b ]] && echo true || echo false # variable has NOT been set Note the last is true for any, non-empty, value of b NOTE: Bash ((..)) and $((..)) is for mathematics, not string testing! Expr fails badly in many situations Avoid it. Use BASH '[[..]]' or '((..))' as appropriate instead. For details of 'expr' problems see https://antofthy.gitlab.io/apps/expr.txt ------------------------------------------------------------------------------- Variable Testing Rules of thumb when using '[...]' * Always quote all variables being tested (unless in a builtin) * Only use the boolean type test (below) when you have complete control of all posible settings. * Prepend X (or other alphanumberic) when comparing unknown strings. * Don't use ! if you can avoid it (too many changes between shells) WARNING: The "test" command and hence the bash '[...]' is run as a program! This means variables are substituted before the program runs, to test the results. It has no idea about actual variables, unlike bash '[[...]]' syntax. Boolean Variable tests (variable contents must be fully controlled)... var="true" # var is true var="" # var is false [ "$var" ] && echo 'var is true' [ -n "$var" ] && echo 'var is true' [ -z "$var" ] && echo 'var is false (empty)' [ "${var+set}" ] && echo 'var has been set (may be empty)' Giving it a more verbose setting works better for -x tracing... FLAG="FLAG_IS_TRUE" if [ -n "$FLAG" ]; then ... Test of variables containing ANY string (see PROBLEM CASES below) To do this prefix the strings with something like 'X' option=-xyzzy [ "X$option" != X ] && echo option is defined (not null) [ "X$option" = 'X-f' ] && echo "option is '-f'" [ "X$a" = "X$b" ] && echo "$a equals $b" PROBLEM CASES... These cases cause the "[...]" tests to fail badly! These fail in old bourne shell, but all but first work in BASH! [ $var ] # but var="a = b" (return false instead of true) [ "$var" ] # but var=-a (actually any option starting with '-') [ "$a" = "$b" ] # but a='(' and b=')' This is why you must use the string test above (with 'X' prefixes). NOTE test ! ... differs from UNIX to UNIX Under BASH, it is the NOT of the next 3 arguments that follow (non-sensible). Under Solaris SH, it seems to handle precedence correctly. EG: test "" -a ! "" is false as you would expect under BASH BUT test ! "" -a "" is true for BASH and false for Solaris SH ----------- BASH: Newer shells use [[..]] that overcomes all these problems. The quoting and contents does not matter. As [[..]] is designed as a bash builtin that knows about variables. var="true" # var is true var="" # var is false unset var # var is undefined [[ -v var ]] && echo "TRUE" || echo "FALSE" echo ${#var} # length of var 0 = empty/unset/false [[ -v var ]] # var is set, regardless of value (NOTE no '$') [[ $var ]] # var is not empty [[ -n $var ]] # ditto [[ -z $var ]] # var is unset or empty [ "$a" = "$b" ] # variables are the same (see below) NOTE: if '$' is shown, it is required! EG: only the -v test does not use '$' Unfortunatally '-v' option added Bash 4.2 so is not in RH6 WARNING: for bash [[..]] expressions '=' and '==' treat right side as glob! That is... [[ aaa = a?a ]] # is true [[ "aaa" = "a?a" ]] # is flase This is the case even if assigned to variables! a=aaa b='a?a' [[ $a = $b ]] # is true ------------------------------------------------------------------------------- Is a variable defined or set (empty or otherwise) For simple variables in BASH you can use [[ -v var ]] It knows the difference between unset and empty. It can be done using '[', but it is not easy Variable substition test (works with older shells)... # Using of a substition test.... if [ "${var+defined]}" ]; then echo "Variable is defined" fi Does an array have elements (BASH or ZSH)... NOTE: array element 0 is also the non-array value of variable if [ ${#var[@]} -eq 0 ]; then echo "Variable undefined, or Array/Hash is Empty" elif [ "X$var" = "X" ]; then echo "Variable is a NULL string" else echo "Variable has non-null value of \"$var\"" fi # The first test cannot be used for testing a specific array element, # which may be sparse arrays (undefined elements between defined elements). Array Elements (including Sparse Arrays)... array[5]="element" [[ -v array ]] && echo "array is set" || echo "array not set" [[ -v array[3] ]] && echo "array[3] is set" || echo "array[3] not set" [[ -v array[5] ]] && echo "array[5] is set" || echo "array[5] not set" # => array not set # => array[3] not set # => array[5] is set # same results as above... [ "${array+set}" ] && echo "array is set" || echo "array not set" [ "${array[3]+set}" ] && echo "array[3] is set" || echo "array[3] not set" [ "${array[5]+set}" ] && echo "array[5] is set" || echo "array[5] not set" array[5]="" # and repeat tests -- still works unset array[5] # and repeat -- and that element is now 'not set' Associative Array or Hash unset hash; declare -A hash hash[set]="" [[ -v hash[not] ]] && echo "hash[not] is set" || echo "hash[not] not set" [[ -v hash[set] ]] && echo "hash[set] is set" || echo "hash[set] not set" [ "${hash[not]+set}" ] && echo "hash[not] is set" || echo "hash[not] not set" [ "${hash[set]+set}" ] && echo "hash[set] is set" || echo "hash[set] not set" unset hash[set] # and repeat -- and it is now 'not set' ------------------------------------------------------------------------------- Bash string matching - Globs vs RegExpr Globbing can be used with either '=' or '==' in [[...]] or in variable expandsions such as ${var%...} Do not use quotes to allow the glob expandsion) Quotes are also not needed for variables. name=abcd [[ $name == a* ]] && echo true || echo false # => true WARNING: The '*' in the globs match all characters, including '/' That is it acts more like '**' for paths path=/usr/local/bin/command [[ $shell == /*/command ]] && echo true || echo false # => true Extended globs Enable with: shopt +s extglob *(pattern) zero or more of parenthesis +(pattern) one or more ?(pattern) zero or one @(pattern) only one NOTE: Glob *(pattern) is equevlent to (regex)* While Glob @(pattern) => (regex) --- Regular expression ( '=~' ) NOTE: The RE must NOT be quoted! Any part that is quoted in the RE must be an exact match to the string. if [[ "start-abc-ijk-xyz-end" =~ -(.*)-(.*)-(.*)- ]]; then echo "${BASH_REMATCH[@]@Q}" # results, quoted fi # => '-abc-ijk-xyz-' 'abc' 'ijk' 'xyz' BASH_REMATCH is an special built-in array 0 = the string the expression matched (equivelent to sed: \&) n = positional sub-expression (equivelent to sed: \1 \2 .. \N ) The best way to solve quoting issues is to put the pattern in a variable This has no problem with the built-in '[[' exprssion. string="start-123-456-789-end" pattern="-(.*)-(.*)-(.*)-" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => '-123-456-789-' '123' '456' '789' The positional sub-expressions are number by parentesis, so repeating the sub-expressions will only return the last match. :-( string="start-123-456-789-end" pattern="(-[^-]*)+" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => '-123-456-789-end' '-end' There is no 'before' or 'after' match provision unless you add it yourself. Note, minimal matching is not provided, each expresion will gobble as much of the string as it can, so you need to craft your RE very carefully. string="to-be-or-not-to-be" pattern="^([^-]*)-(.*)-(.*)" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => 'to-be-or-not-to-be' 'to' 'be-or-not-to' 'be' ----- Wierdness... * can match nothing! # One pattern... string="start-123-456-789-end" pattern="(-[^-]+)" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => '-123' '-123' # repeated pattern pattern="(-[^-]+)(-[^-]+)(-[^-]+)" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => '-123-456-789' '-123' '-456' '-789' # Using '+' works (BUT only last match is output - as previously shown) pattern="(-[^-]+)+" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => '-123-456-789-end' '-end' # Using a count - works (last count match returned) pattern="(-[^-]+){3}" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => '-123-456-789' '-789' # But '*' will always find a zero match at the end! string="start-123-456-789-end" pattern="(-[^-]+)*" [[ $string =~ $pattern ]] && echo "${BASH_REMATCH[@]@Q}" # => '' '' That is (...)* always matched no elements, despite it being supposedly greedy. Basically as it will match 'zero' matches as well! Simularly using counts... # one or more... Works fine... pattern="(-[^-]+)+" # -> "-123-456-789-end" "-end" pattern="(-[^-]+){1,}" # -> "-123-456-789-end" "-end" # zero or more ('*' equivelent)... returns empty... pattern="(-[^-]+)*" # -> "" "" pattern="(-[^-]+){0,}" # -> "" "" # same for '?' or '{0,1}... it returns a empty match pattern="(-[^-]+)?" # -> "" "" pattern="(-[^-]+){0,1}" # -> "" "" ------------------------------------------------------------------------------- Indirect Variables and Name References https://mywiki.wooledge.org/BashFAQ/006 Indirect uses a variable to hold a variables name # Fast Summery.... a=b $a=store_in_b echo $b eval "echo \$$a" echo ${!a} # Bourne shell (using "eval" - DANGER DANGER) b=123 a=b eval "echo \$$a" eval "echo \${$a}" set_ABC_var=987 set=ABC eval "echo \${set_${set}_var}" # Bash (from version 2) (dangerious, non-intutitive) # indirect set via read # NOTE: a newline is appended by "<<<" and removed by "read" command a=b read $a <<< "string" echo "b = $b" # => b = string # indirect set via typeset (obsolete) a=b typeset $a=typeset echo "b = $b" # => b = typeset # indirect set via declare (replaced typeset) a=b declare $a=declare echo "b = $b" # => b = declare # indirect access b=access echo "a -> $a = ${!a}" # => a -> b = access # NOTE: do not confuse "${!var}" this with "${!var*}" # which will LIST all variables starting with 'var' (not a reference) BASH "declare -n" nameref 'name referance' variable attribute EG: Define variable to lookup anoher variables value # declared 'a' to point to 'b' b=assign_b declare -n a=b # see the attribute for 'a' -- bash knows it is a referance declare -p a # => declare -n a="b" declare -p b # => declare -- b="assign_b" # NOW $a and $b both point to same data echo "a = $a b = $b" # => a = assign_b b = assign_b a=assign_a; echo "a = $a b = $b" # => a = assign_a b = assign_a b=assign_b; echo "a = $a b = $b" # => a = assign_b b = assign_b # unsetting does not remove the attribute unset b; echo "a = $a b = $b" # => a = b = b=assign_b; echo "a = $a b = $b" # => a = assign_b b = assign_b unset a; echo "a = $a b = $b" # => a = b = a=assign_a; echo "a = $a b = $b" # => a = assign_a b = assign_a b=assign_b; echo "a = $a b = $b" # => a = assign_b b = assign_b # NOTE the attribute is ONLY on variable 'a' not on 'b' # you can remove and re-add reference attibute declare +n a; echo "a = $a b = $b" # => a = b b = assign_b declare -n a; echo "a = $a b = $b" # => a = assign_b b = assign_b # Name referances to arrays works... a[5]=whatever declare -n ref="a" echo "ref[5] = ${ref[5]}" # => ref[5] = whatever # reference to an element of an array! declare -n ref="a[5]" echo "ref = ${ref}" # => ref = whatever # but it is not an array itself, and '0' does not return the value! echo "ref[0] = ${ref[0]}" # => ref[0] = echo "ref[5] = ${ref[5]}" # => ref[5] = declare +n ref # remove name referance ------------------------------------------------------------------------------- Bash Arrays, ARRAY Also see http://mywiki.wooledge.org/BashFAQ/005 NOTES: * Indexes start at zero * The zeroth element is the same as the variable of the same name. * Elements do not need to exist (sparse array) but are "" if referenced, * Referencing an undefined element, does NOT define the element. * bash treats array subscripts as an arithmetic expression EG equivelent to a $(( )), so variables can be used without '$' prefix Pre-declaring an array (not actually needed) declare -a xyz[50] Basic (compact or filled arrays) letters=( a b c ) # just define in sequence declare -p letters # how to re-create the array completely # => declare -a letters=([0]="a" [1]="b" [2]="c") letters+=( d e z ) # add elements to end of array # Without parenthesis, this is string append letters[5]=f # add/change a element echo "${letters[@]}" # list all elements printf "'%s' " "${letters[@]}"; echo echo ${#letters[@]} # number of elements echo "${letters[4]}" # referencing an element echo "${letters[-2]}" # negative references from end echo "${letters[-1]}" # list last element echo ${letters[${#letters[@]}-1]} # list last element (non-sparse arrays) unset 'letters[${#letters[@]}-1]' # JUNK last element (quotes are needed!) letters+=f # PUSH element onto end of array letters[${#letters[@]}]=f # explicit indexed push on end i=2 # unset index 2, and compact array letters=( "${letters[@]::$i}" "${letters[@]:$((i+1))}" ) i=2; elem='c' # reinsert element at index 2 letters=( "${letters[@]::$i}" "$elem" "${letters[@]:$i}" ) keys="${!letters[*]}" # list the keys as space seperated list echo $keys # the keys used keys=("${!letters[@]}") # keys as an array echo ${#keys[@]} # number of keys (as apposed to array size) Copy Array (WARNING: this compresses a sparse array) new_array=( "${old_array[@]}" ) # removes any 'undefined' elements The zeroth element is the same as the variable name without the index. variable[0]=zero echo $variable zero variable=different echo ${variable[0]} different echo ${#variable} # length of zeroth element (0=unset or null) 8 Array slices (get parts on an array)... echo "${letters[@]:3:2}" # list elements 3 and 4 (c and d) echo "${letters[@]: -1}" # list last element (NOTE THE SPACE) echo "${letters[@]:(-1)}" # list last element alternative # see Positional Parameters can also use sparse syntax args_array=( "$@" ) # save porgram arguments into an array Array as a Stack / Queue / List... NOTE: using last element at top produces 'subscript' errors on empty stack. But using first element as top of stack produces empty strings, but no error. unset 'stack' # initialise or reset (no number, until used) stack=( $(seq 2 6) ) stack+=( last ) # PUSH - Append values onto end stack=( first "${stack[@]}" ) # UNSHIFT - Prepend onto start of stack printf "'%s' " "${stack[@]}";echo # PRINT stack echo ${#stack[@]} # COUNT number of elements on stack echo "$stack" # SHIFT out first element - empty string if empty stack=("${stack[@]:1}") # and remove first element - no error if empty echo "${stack[-1]}" # POP last element ('bad array subscript' if empty) unset 'stack[-1]' # and remove last element ( " ) Stack Alternative (stores in top to bottom sequence) http://www.tldp.org/LDP/abs/html/arrays.html#STACKEX This provides functions with limit protections Loop though array (filled array, not sparse) # Just print the array... printf '%s\n' "${letters[@]}" # alternative # Or clean up 'declare -p' output declare -p letters | sed 's/.*(\(.*\)" *)/" \1/; s/" \[[0-9]*\]="/, /g; s/^, //' # => a, b, c, d, e, f # or use multiple lines (remains shell parsible)... declare -p letters | sed 's/declare -. //; s/ *\[/\n [/g; s/ *)$/\n)/' # => letters=( # [0]="a" # ... # ) # loop through elements (ignore index) for i in "${letters[@]}"; do echo "'$i'" done # with indexes (if not sparse) for(( i=0; i < ${#letters[@]}; i++ )); do echo "$i : '${letters[i]}'" done In Reverse (using indexes) for (( i=${#letters[@]}-1 ; i>=0 ; i-- ));do echo "${letters[i]}" done Loop through to find index of a element (works for sparse and hash too) value='c' for i in "${!letters[@]}"; do [[ "${letters[$i]}" = "${value}" ]] && break done echo $i Passing arrays to functions, as either as single array, or by nameref # Enumerate any type of array or hash print_array() { local name array i name="$1" declare -n array="$name" # let us reference array elements for i in "${!array[@]}"; do # loop though indices printf '%s[%s] = "%s"\n' "$name" "$i" "${array[$i]}" done } print_array letters # the above is simular to (but outputs a single line!) declare -p letters # or attempt to make multi-line declare -p letters | sed 's/ *\[/\n [/g;s/ *)$/\n)/' Note: when hash is used in ((..)) you need to backslash the $ of the index. Or remove the $ completely due to the mechanics of ((..)) https://github.com/koalaman/shellcheck/wiki/SC2149 index=42 array[$index]=100 echo "${array[$index]}" # 100 echo "$(( array[\$index]-31 ))" # 69 echo "$(( array[index]-31 ))" # 69 echo "$(( array[$index]-31 ))" # -31 -- fails! Read file into an array # Words words=( $(<"file") ) # you do not need to worry about quotes # or [#] in the file. Due to quote order read -d'\n' -a words 7 6 5 4 3 2 1 Function, using nameref's reverse() { # reverse array_var reverse_var # second is the output array local i r_in r_out declare -n r_in="$1" r_out="$2" # reference array arguments r_out=() # clear the destination array for i in "${r_in[@]}"; do r_out=("$i" "${r_out[@]}") done declare +n r_in r_out } array=(a b c d e f) reverse array rev echo "${rev[@]}" # => f e d c b a shopt 'extdebug' trick (compresses sparse arrays) If shopt 'extdebug' then array "${BASH_ARGV[@]}" containes the reverse of a functions parameters! array=(1 2 3 4 5 6 7) rev() { rev==("${BASH_ARGV[@]}"); } shopt -s extdebug rev "${array[@]}" shopt -u extdebug echo "${rev[@]}" # => 7 6 5 4 3 2 1 External 'tac' (no new lines in values) array=(1 2 3 4 5 6 10 11 12) read -d'\n' -a array < <(printf '%s\n' "${array[@]}"|tac) echo "${array[@]}" # => 12 11 10 6 5 4 3 2 1 Double eval (ugly, but one line) array=(a b c x y z) eval eval "'rev=('" "'\"\${array[-'{1..${#array[@]}}']}\"'" "')'" echo "${rev[@]}" # => z y x c b a --- Sort an array (shell built-ins only) #!/bin/bash a=( "$@" ) # Bubble Sort Array for (( i=0; i<(${#a[@]}-1); i++ )); do for (( j=i+1; j<(${#a[@]}); j++ )); do if [[ ${a[j]} < ${a[i]} ]]; then t=${a[j]}; a[j]=${a[i]}; a[i]=$t; fi done done # Output the array echo "${a[@]}" =======8<-------- ------------------------------------------------------------------------------- Sparse Array Handling sparse[4]="y" # create sparse array of values sparse[7]="n" sparse=( [4]="y" [7]="n" ) # alternative create method (equivelent) sparse[13]='' # add a null value in sparse array sparse[22]=last # add another parameter printf '"%s" ' "${sparse[@]}"; echo # list elements wth quotes # => "y" "n" "" "last" declare -p sparse # view everything # => declare -a sparse=([4]="y" [7]="n" [13]="" [22]="last") echo ${#sparse[@]} # number of defined elements # => 4 # number of elements, not index of last element echo "${!sparse[*]}" # what indexes are defined # => 4 7 13 22 echo ${sparse[4]} # get a specific value # => y echo "${sparse[-1]}" # output the last element ('last') # => last unset sparse[-1] # unset last element echo "${sparse[-1]}" # => 'q' sparse[22]=last # readd last element echo "${sparse[@]:14:1}" # lists the next element on or after 14 # => last # This is actually element 22! printf '"%s" ' "${sparse[@]:5:3}"; echo # => "q" "" "last" # list 3 items from index 5 on Testing is elements defined or empty Note undefined is often treated as an empty string, so be careful! sparse=( [4]=y [7]=q [13]='' [22]=last ) indexes=(${!sparse[*]}) # array of indexes # what is being tested... echo "${indexes[*]}" # the indexes used by the sparse array # => 4 7 13 22 echo ${indexes[-1]} # lookup index of last argument in sparse # => 22 # compare number of elements with index of last element if [ ${#sparse[@]} -ne ${indexes[-1]} ]; then echo array is sparse else echo array has no gaps fi # => array is sparse Loop through Sparse Array sparse=( [4]=y [7]=q [13]='' [22]=last ) for i in "${sparse[@]}"; do # loop through elements (ignore index) echo "'$i'" done # => 'y' # 'q' # '' # 'last' for i in ${!sparse[*]}; do # list using indexes to elements echo "$i : '${sparse[i]}'" done # => 4 : 'y' # 7 : 'q' # 13 : '' # 22 : 'last' # Enumerate any type of array or hash print_array() { local name array i name="$1" declare -n array="$name" # let us reference array elements for i in "${!array[@]}"; do # loop though indices printf '%s[%s] = "%s"\n' "$name" "$i" "${array[$i]}" done } print_array sparse # the above is simular to (but outputs a single line!) declare -p sparse # or make multi-line declare -p sparse | sed 's/ *\[/\n [/g;s/ *)$/\n)/' Copy Sparse or associative Arrays exactly sparse=( [4]=y [7]=q [13]='' [22]=last ) # copy by looping through elements for i in "${!sparse[@]}"; do new[$i]=${sparse[$i]} done declare -p new # => declare -a new=([4]="y" [7]="q" [13]="" [22]="last") # OR copy by renaming the declare output! temp=$(declare -p sparse); eval "${temp/sparse=(/copy=(}" declare -p copy # => declare -a copy=([4]="y" [7]="q" [13]="" [22]="last") Compress Sparse Array sparse=( [4]=y [7]=q [13]='' [22]=last ) compress=( "${sparse[@]}" ) # compress array indexes - no longer sparse echo ${#sparse[@]} # Number of defined elements # => 4 echo ${#compress[@]} # => 4 echo ${!sparse[*]} # what indexes are in use # => 4 7 13 22 echo ${!compress[*]} # => 0 1 2 3 echo ${compress[3]} # what is the last (number-1) element # => last echo ${compress[-1]} # alternative (bash v4.2) # => last --- Specific Element Handling sparse=( [4]=y [7]=q [13]='' [22]=last ) echo ${#sparse[22]} # Length of a specific element # => 4 test_element() { if [[ -v sparse[$1] ]]; then if [[ ${sparse[$1]} ]]; then echo "defined, string length ${#sparse[$1]}" else echo "defined, empty string" fi else echo "not defined" fi } test_element 12 # => not defined test_element 13 # => defined, empty string test_element 22 # => defined, string length 4 ------------------------------------------------------------------------------- Bash Associative Array (Hash) Example WARNING: They MUST be declared! # From Bash v4 on you can use associative arrays.... typeset -A homedir # Declare associative array homedir=( # Compound assignment [jim]=/home/jim [silvia]=/home/silvia [alex]=/home/alex ) # assignment homedir[ormaaj]=/home/ormaaj name=anthony homedir[$name]=/home/$name typeset -A hash index=banana hash[$index]=86 # Updating hash in a ((..)) echo "${hash[$index]}" # => 86 echo "$(( hash[index]-17 ))" # => -17 -- wrong (no key 'index') echo "$(( hash[$index]-17 ))" # => 69 -- correct (substitutes key) echo "$(( ${hash[$index]}-17 ))" # => 69 -- correct (substitutes value) echo "$(( hash[\$index]-17 ))" # => 69 -- CORRECT (no substitute) # Replacing - with -= and only these work, others FAIL... hash[$index]=86; (( hash[$index]-=17 )); echo "${hash[$index]}" hash[$index]=86; (( hash[\$index]-=17 )); echo "${hash[$index]}" # CORRECT => 69 # Does key exist not_key=tomato [[ -v hash[\$not_key] ]] && echo true || echo false # => false [[ -v hash[\$index] ]] && echo true || echo false # => true [[ -v hash[$index] ]] && echo true || echo false # => true (substitute) Note: when hash is used in ((..)) you need to backslash the $ of the index. Or remove the $ completely due to the mechanics of ((..)) https://github.com/koalaman/shellcheck/wiki/SC2149 This however is only needed if modifing the variable in the hash! EG: incrementing below... Loop through Hash Array # outputs as a single line! declare -p homedir # => declare -a homedir=([alex]="/home/alex" ... [silvia]="/home/silvia" ) # cleaned up declare -p homedir | sed 's/^declare -a //; s/=//; s/ )/)/; s/[]["'\'']//g;' # attempt to make multi-line (unsorted) declare -p homedir | sed 's/declare -A //; s/ *\[/\n [/g; s/ *)$/\n)/' # => homedir=( [alex]="/home/alex" [jim]="/home/jim" [silvia]="/home/silvia" ) # Enumerate any type of array or hash (unsorted) print_array() { local name array i name="$1" declare -n array="$name" # reference the given array name for i in "${!array[@]}"; do # loop though indices printf '%s[%s] = "%s"\n' "$name" "$i" "${array[$i]}" done } print_array homedir # counting statistics declare -A counts=([abc]=3 [xyz]=1 [ljk]=5) host=abc (( counts[$host]++ )) # NOTE SC2149 above declare -p counts | sed 's/^declare -A //; s/=//; s/ )/)/; s/[]["'\'']//g;' # => count=(xyz=1 ljk=5 abc=4) # variable order, NOT sorted Sorted Loop through Array declare -A counts=([abc]=3 [xyz]=1 [ljk]=5) # Sort the output! (you can NOT set variables in a pipelined loop) for i in "${!counts[@]}"; do printf ' %-10s (count %d)\n' "$i" "${counts[$i]}" done | sort # Sorted array of indexes mapfile -t sorted < <(printf '%s\n' "${!counts[@]}" | sort ) for i in "${sorted[@]}"; do printf ' %-10s (count %d)\n' "$i" "${counts[$i]}" done # Using read on sorted indexes while read -r i; do printf ' %-10s (count %d)\n' "$i" "${counts[$i]}" done < <( printf '%s\n' "${!counts[@]}" | sort ) # sort by value? # AT moment only adding value to output and sorting that. ------------------------------------------------------------------------------- LIST functions (plain Bourne Shell) Also usable for PATH-like environment variables NOTE the use of sub-shells (...) to scope the IFS setting list="a:b:c:d" # list variable element="e" # element of the list sep=":" # list_separator (not white space) Append to list (even if empty!) list="${list:+$list$sep}$element" Split up list function split_list () { ( IFS="$sep" set -- $list for f in "$@"; do echo -n "${f:-.} " done; echo ) } Loop over list for item in `( IFS="$sep"; echo $list; )`; do echo item=$item done Count of elements count=`IFS="$sep"; set - $list; echo $#` get I'th element (shell must understand shift argument) element=`IFS="$sep"; set - $list; shift $I; echo $1` get first element element=`IFS="$sep"; set - $list; echo $1` delete first element #list=`echo "$list" | sed "/$sep/"\!"d; /:/s/^[^$sep]*$sep//;"` list=`echo "$list" | cut -d"$sep" -f2-` delete specific element over the whole list -- not right??? list=`echo "$list" | sed "s|${element}${sep}||g; s|${sep}${element}\$||;"` Of course BASH arrays supersedes all this. -------------------------------------------------------------------------------