-------------------------------------------------------------------------------
Creating and using Template files.

Templating is used in the configuration of software as it is being installed.
That is one 'source config' is used to define all the places in multiple files
where that value needs to be used.  Such as in scripts, configuration files,
for data such as Path, Programs to call, data files, web addresses, etc.

It is also (and more intuitively thought about) for the preperation and mailing
of pre-formatted mail to large numbers of people.

Typically files/strings will have special placeholders, which are specially
labeled strings not typically found in the file, that are replaced with
pre-defined values for that label.  These values are pre-defined from a common
higherlevel configuration source (source code templating), or perhaps is
a large array of such values from a database, (mail templating).

Examles if placeholders that may be used.
   ${label} or $label
   <% $label %>
   ===label===
   {{label}}
   VAR_label

The substition needs to be done regardless of quoting or other syntax of the
file, as such ideally the placeholder should be unique and not part of the
normal syntax of the file, whatever that may be (program, data, etc).

IT also needs ways to add the 'placeholder' without the substitution.

However there is no ideal solution that is common to all UNIX systems, that
works in all situations.

Here we look at various techniques and methods, I have seen.

===============================================================================
envsubst

This is a GUI tool, and as such is available on almost all UNIX machines.

This looked for  $VARIABLE  and ${VARIABLE} strings in files and replaces them
with the contents of the environment variables of the same name.  If no such
environment variable is found, it is left as is.

Example...

Input...
=======8<--------CUT HERE----------
<VirtualHost *:${PORT}>
    ServerName ${SERVER_NAME}
    ServerAlias ${SERVER_ALIAS}
    DocumentRoot "${DOCUMENT_ROOT}"
</VirtualHost>
=======8<--------CUT HERE----------

Command....
=======8<--------CUT HERE----------
export PORT="443"
export SERVER_NAME="example.com"
export SERVER_ALIAS="www.example.com"
export DOCUMENT_ROOT="/var/www/html/"
envsubst < apache.tmpl > my_apache_site.conf
=======8<--------CUT HERE----------

Output...
=======8<--------CUT HERE----------
<VirtualHost *:443>
    ServerName example.com
    ServerAlias www.example.com
    DocumentRoot "/var/www/html/"
</VirtualHost>
=======8<--------CUT HERE----------

Notes, and posible Problems...
  * There are a lot of environment variables that are already set
    that may also be substituted, but probably should NEVER be substituted!
    For example $PATH is common in shell scripts, as it is a envvar!
  * The ${..} is used for situations like  "${noun}ify"  (Bash Syntax)

-------------------------------------------------------------------------------
"sed" templating...

This uses 's|||' commands to avoid conflict with '/'

  =======8<--------CUT HERE----------
  sed -e 's|===BROWSERDIR===|'"$BROWSERDIR"'|' \
      -e 's|===LIBRARY_RC===|'"$LIBRARY_RC"'|' \
      -e '/===COLOR_SETTINGS_DIVIDER===/d'   \
      < file.tmpl  >file
  =======8<--------CUT HERE----------

NOTE: this method works well in Makefiles!

BUT while the above can handle '/' it will not handle quotes, or other
special 'sed' controls in the substituted values.

A simular technique can be used in perl, with better handling.

-------------------------------------------------------------------------------
Bash (Pure) Templating

  =======8<--------CUT HERE----------
  #!/bin/bash
  while read -r line ; do
    while [[ "$line" =~ (.*)(\$\{[a-zA-Z_][a-zA-Z_0-9]*\})(.*) ]] ; do
      PRE=${BASH_REMATCH[1]}
      POST="${BASH_REMATCH[4]}${line:$end_offset:${#line}}"
      LABEL="{BASH_REMATCH[3]}"
      eval 'VALUE="$'$VARNAME'"'
      line="$PRE$VALUE$POST"
      end_offset=${#PRE}
    done
    echo -n "${line:0:-1}"
  done
  =======8<--------CUT HERE----------

-------------------------------------------------------------------------------
Perl templating

This uses environment variables to replace {{label}} placeholders.

perl -pe 's/\{\{(\w+)\}\}/(defined $ENV{$1}?$ENV{$1}:"missing var $1")/eg' \
     < infile > outfile

OR using a perl hash table of substitutons...

=======8<--------CUT HERE----------
#!/usr/bin/env perl
my %replace = ( 'dbName' => 'testdb', 'somethingElse' => 'fooBar' );
undef $/;
my $buf = <STDIN>;
$buf =~ s/\$\{$_\}/$replace{$_}/g for keys %replace;
print $buf;
=======8<--------CUT HERE----------

-------------------------------------------------------------------------------
Python Templating Class

This is in many ways simular to "envsubst" above, but using a pythion
dictionary instead of environment variables.

It replaces   ${name} or $name with the value of 'name' if present in
the given dictionary.

=======8<--------CUT HERE----------
# A Python program to demonstrate the
# working of the string template
from string import Template

# List Student stores the name and marks of three students
Student = [('Ram',90), ('Ankit',78), ('Bob',92)]

# We are creating a basic structure to print the name and
# marks of the students.
t = Template('Hi $name, you have got $marks marks')

for i in Student:
     print (t.substitute(name = i[0], marks = i[1]))

=======8<--------CUT HERE----------

-------------------------------------------------------------------------------
Bash-TPL     https://github.com/TekWizely/bash-tpl
  (approx 1100 line single script)

  Small BASH script to do templating, that also tries to ensure correct
  formating (when you add extra indenting for a templated section.

  Default the placeholders are:  <% $NAME %>
  But this can be changed!

  Values come from environment, OR from numbered arguments...
     bash-tpl test.tpl TekWizely
  with "test.tpl"
     Hello <% $1 %>

  Also includes other directives (like 'INCLUDE')
  Ability to run a small scripts to work out the substition.
     As a placeholder, line, or even a block within the file.
     Hello <%% echo $NAME | tr '[:lower:]' '[:upper:]' %>
  Template comments.
  or a whole line (startung with '%')
  And many other features.

-------------------------------------------------------------------------------
Bash "templater" - https://github.com/vicentebolea/bash-templater

Bash Equivelent (reading from a variable list)...

Basically reads variables into script and a array of the labels used.
Then goes though template file line by line replacing keywords in a loop

  templater var_file <file.tmpl > file

  with var_file of the form
    NAME=John Doe
    EMAIL=john.doe\@mail.server

  example template file...
    @@@NAME@@@ <@@@EMAIL@@@>

=======8<--------CUT HERE----------
source "$1"       # read variables into bash!
mapfile -t keywords < <(sed -n 's/^\([A-Za-z0-9_]\+\)=.*$/\1/p' "$1")
for read line; do
  for keyword in "${keywords[@]}"; do
    value="$(echo "${!keyword}" | sed 's/\//\\\//g')"
    line=`sed "s@\@\@\@${keyword}\@\@\@@${!keyword}@g" <<<"$line"`
  done
  echo "$line"
done < /dev/stdin
=======8<--------CUT HERE----------

WARNING: it reads variables into the bash script directly!
As such you can NOT use specific varibales like 'keyword' or 'line' or some
bash specific variables, or it will have unexpected results.

-------------------------------------------------------------------------------
Bash "renderest" - https://github.com/relaxdiego/renderest

Looks up the variable names from the template file, and
uses"{{label}}" placeholders in the input template.
This generates "sed" expressions from provided environment variables, giving
a warning if a variable is not found.

In summery...

=======8<--------CUT HERE----------
# Get all variable names used in the template
varnames=$(grep -oE '\{\{([A-Za-z0-9_]+)\}\}' $1 |
            sed -rn 's/.*\{\{([A-Za-z0-9_]+)\}\}.*/\1/p' | sort | uniq)
# sed expressions
expressions=""
for varname in $varnames; do
  # checks for existance of varname
  [ -z ${!varname} ] && echo "ERROR: $varname is empty/not-found" >&2
  value="$(echo "${!varname}" | sed 's/\//\\\//g')"
  expressions+="-e 's/\{\{$varname\}\}/${value}/g' "
done
eval sed "$expressions" "$1"
=======8<--------CUT HERE----------

NOTE: The 'eval' probbaly should be removed!

-------------------------------------------------------------------------------
Apache Server Side Includes.  (SHTML)

Files ending in ".shtml" can be processed by apache to do variable
substituitions and file inclusions, before the resulting HTML is set to calling
web clients.  Any <!-- ... --> expressions embedded in the SHTML, is replaced.
These can include common variable and file substitutions, which is what I used
it for.

I used SHTML a lot in my web pages, and my web site relied on it.  I did not
what to re-work that templating system when I switched over to a static page
web server provider.

Specifically I used "Templated Includes", as shown in...
   https://antofthy.gitlab.io/info/www/apache_ssi.txt

It was simple and let me desing a 'html' object that I can add to the resulting
HTML files hundreds of times.

Unfortunatally it is only used by the Apache Web Server!

My solution was to have a local only Apache webserver which I then call
to convert ".shtml" pages to ".html" pages before uploading them to the
static web server provider (github).

This also allowed me to generate very fancy directory index pages, where
needed, before uploading to my web provider.  Without having to change the way
I did things.

I would LOVE it if a offline SHTML to HTML converter was developed.

-------------------------------------------------------------------------------
Other Templating programs

mo     - mustache templates in bash

esh    - embedded shell (written in sh)

cookie - https://github.com/bbugyi200/cookie
         not a lot of documentation

shtpl  - https://github.com/mlorenzo-stratio/shtpl
         Bash uses jinja templating syntax (lots of if/loop macro controls)

sigil  - https://github.com/gliderlabs/sigil
         simple template and macros,
         label values can be set from command line
         Placeholds can be  ${label} (-p option) or {{ $label }}
         or even user specified delimiters
         written in "Go" and based on "Go" templating package

-------------------------------------------------------------------------------