-------------------------------------------------------------------------------
Grep and grep related things...

Also see  https://antofthy.gitlab.io/info/misc/regex.txt

-------------------------------------------------------------------------------

   cgrep       color grep (gnu grep has this built in)
   pcregrep    pcre-tools utilities grep (multiline, or whole file grep)

-------------------------------------------------------------------------------
Enable gnu-grep highlight with background color (for spaces)

  export GREP_COLORS='ne:mt=01;31;44'
  export GREP_OPTS='--color'
  alias grep command grep '$GREP_OPTS'

  echo $'abc def xyz' | grep 'def'
  abc def xyz     <- def is highlighted

To show the whole file with all the highlighted matches, add a 'empty match'
Grep will NOT hightlight a zero length match

  echo $'abc\n  def  \nxyz' | grep 'def\|$' -
  abc
    def           <-def is highlighted
  xyz

To add highlights in a pipeline, you need some extra flags
(You may also want to add '--line-buffered' if output is intermittent)

  echo $'abc\n  def  \nxyz' |
    grep --color=always 'def\|$' - |
      less -R

Multiple highlights (not simple -- see grep alternatives below)...

  cat log |
    GREP_COLORS='mt=01;31' grep --color=always 'fail' |
    GREP_COLORS='mt=01;32' grep --color=always 'success' |
      less -R

-------------------------------------------------------------------------------
Highlight trailing whitespace

  Show all lines, highlighting trailign space
    GREP_COLORS='mt=44' grep --color '[[:blank:]]*$'

  Show just the lines with trailing space highlighted
    GREP_COLORS='mt=44' grep --color '[[:blank:]]\+$'

  I recommend redefining grep highlight to include background color
    export GREP_COLORS='ne:mt=31;44'

Cat can also do this, by adding '$'

    cat -E  or cat --show-ends

-------------------------------------------------------------------------------
Prefix Testing

Example anything starting with 'y' case insensitive

  if grep -iq "^y" <<< "$answer"; then

shell-builtin solution

  if [[ "$answer" != "${answer#[Yy]}" ]]; then

-------------------------------------------------------------------------------
Get the full quote string with words...

  Get the first (-m1) URL for a ".tar.gz" file

    url-$( curl -skL "$url" | grep -om1 '[^"]*tar\.gz[^"]*'  )


-------------------------------------------------------------------------------
Get value following a key....

Zero-width positive look behind (grep perl-RE)
EG the word after 'beta'

  echo "alpha beta gamma delta" | grep -oP 'beta \K\S+'
  # => gamma

Another method of zero-width positive look behind

  echo "alpha beta gamma delta" | grep -oP '(?<=beta )\S+'
  # => gamma

or more exact and less senstive...

  echo "alpha beta gamma delta" | grep -oP '\bbeta\s+\K\S+'
  # => gamma


Shell built-in only method...
  string='abc,10.11.13.14,def,1.2.3.4,geh,6.7.54.23'
  dev=def         # find the value after this!
  s=",$string,"   # make a copy with extra pre/post strings
  case $s in
    *,"$dev",*) ip=${s#*",$dev,"}; ip=${ip%%,*} ;;
    *)          ip='' ;;   # not found case
  esac
  printf '%s\n' "$ip"


Practical Examples...

  # get the mount points (after the '%' character)...
  df | grep -oP '% \K.*'

  # get GID of the group "users"  (non-awk solution)
  grep -oP '^users:x:\K\d+' /etc/group

  # look for quoted assignment (note start of line ^ in RE)
  # EG  PASSWORD="xyzzy123"
  grep -oP '^PASSWORD="\K[^"]+' config_file

  # Just get the first match (key could match multiple times)
  grep -m1 -oP '^sslclientcert\s*=\s*\K.*' /etc/yum.repos.d/redhat.repo

-------------------------------------------------------------------------------
Matching words

Moved to info/misc/regex.txt

-------------------------------------------------------------------------------
Multi-line Matching

  Two lines with keywords, any number of lines between...
  EG: Between START and END

      grep -Pz '\bSTART\b(.|\n)*\bEND\b' file

    or using perl '/s' (single line) option to make '.' match newlines

      grep -Pz '(?s)\bSTART\b.*\bEND\b' file

    or non-greedy (minimal number of lines) matching

      grep -Pz '(?s)\bSTART\b.*?\bEND\b' file

    OR non-gnu grep  (no shortest length)

      < input_file   tr '\n' '\a' | grep -o 'START.*END' | tr '\a' '\n'


Grep only the first N lines of multiple files

  gawk 'FNR>10 {nextfile} /pattern/ { print FILENAME ; nextfile }' filenames


-------------------------------------------------------------------------------
Grep for two or more words in same paragraph (multi-line)

paragraph mode of perl is better for dealing with paragraphs!

  perl -000ne 'print "-- $ARGV --\n$_"
              if /\bone\b/i && /\btwo\b/i && /\bthree\b/i;'

-------------------------------------------------------------------------------
Grep Alternatives

sed/awk/perl   -- of course!

ack

  Recursive Grep -- by default
  Automatically Skips Subversion, Git and VCS repository directories
  Skips 'images' and other binarys (which can be bad)

  Examples
    ack --color --passthru --pager="${PAGER:-less -R}" pattern files

    ack --color --passthru pattern files | less -R

    export ACK_PAGER_COLOR="${PAGER:-less -R}"
    ack --passthru pattern files

ag
  "The Silver Searcher"
  https://github.com/ggreer/the_silver_searcher
  dnf install the_silver_searcher


-----
Color Highlight Grep Alternatives...

colout
  colors patterns in files as specified in pipelines
  http://nojhan.github.io/colout/

  Example: color the network error code...
    tail -f /var/log/nginx/access.log | \
      colout ' 5\d\d ' red | \
      colout ' 4\d\d ' yellow | \
      colout ' 3\d\d ' cyan | \
      colout ' 2\d\d ' green

    ls -l ~ | colout "^(d*)-*(rwx)(rwx)(r-x)" blue,red,yellow,green

    echo "Progress [########################] 100%" | colout "#" Rainbow

    # color the percentages from 0% blue to 100% red
    df | colout --scale 0,100 "\[(.*)%\]" scale

highlight
  small shell script version of colout,
  mapping color names to sed patterns
  https://github.com/kepkin/dev-shell-essentials

  example...
    command_here | highlight green "input" | highlight red "output"

-------------------------------------------------------------------------------