hilite

#!/usr/bin/env perl -w
#
# hilite: Colorize multiple regex matches in a stream.
# 2006-11-03: Written by Steven J. DeRose.
#
use strict;
use Getopt::Long;

use sjdUtils;
use alogging;
use ColorManager;

our %metadata = (
    'title'        => "hilite.pl",
    'description'  => "Colorize multiple regex matches in a stream.",
    'rightsHolder' => "Steven J. DeRose",
    'creator'      => "http://viaf.org/viaf/50334488",
    'type'         => "http://purl.org/dc/dcmitype/Software",
    'language'     => "Perl 5.18",
    'created'      => "2006-11-03",
    'modified'     => "2022-12-16",
    'publisher'    => "http://github.com/sderose",
    'license'      => "https://creativecommons.org/licenses/by-sa/3.0/"
);
our $VERSION_DATE = $metadata{'modified'};

=pod

=head1 Usage

Prints the file(s), or STDIN if no files are specified (or with --text,
text from the command line),
colorizing matches to any regex(es) specified,
using the normal ANSI terminal escapes for color (or by inserting HTML for colors).

For example, to highlights all matches to the `-e` regex, from an input file,
in bold red on a black background.

    hilite -c red/black/bold -e "foo\w*" myFile.txt

B<Note>: Set I<-c> before I<-e>. I<-c> applies to I<following> expressions.

To hilight all of "hello" in bold magenta, and send it to STDERR do this (or
see C<colorstring>):

    hilite --warn -c magenta/bold --text hello

See my `colorNames.md` for details on my naming conventions for colors and
effects. Terminal colors are discussed under C<sjdUtils.pm> and C<terminfo>.
The color and effect names are described under I<--color> below, and
in the documentation for the I<ColorManager> package.

You can colorize XML tags pretty accurately with the following command
(or just use the `--xml` option):

    hilite -c blue -e "</?\w.*?>" myFile.txt -c green -e "<\!--.*?-->"
        -c red -e "<\?.*?\?>" -c white/black -e "&.*?;"

You can generate HTML markup that applies colors, instead of raw ANI terminal
color-control sequences, using I<--useHtml> (though for the moment, only the
same colors are available).

B<Note>: There are many pre-defined sets of regexes available as options,
listed separately below under L<Predefined regex sets>.


=head1 General Options

See also the following section, L<Predefined regex sets>.

=over

=item * B<--color> I<colorName> OR B<-c> I<colorName>

A color name to use for following match expressions (I<-e>). Default: red.
The specified color remains in effect for following I<-e> expressions,
unless/until reset by another use of I<--color>.
See also I<--useHtml> and I<--listColors>.

The basic colors are as provided by the I<ColorManage> package (q.v.):
'black', 'red', 'green', 'yellow', 'blue',
'magenta', 'cyan', 'white' (typically, this is light gray), and 'default'.
These are ANSI terminal codes 30 to 37 and 0, respectively.

For background colors, prefix C</>.
These are ANSI terminal codes 40 to 47 and 0 (?), respectively.

For combined foreground/background color, use C<fg/bg>.

To add an effect, include its name as a third token, for example
C<fg/bold>, C<fg/bg/bold>, or C<fg/bg/bold>.

The known effect names are:
"blink", "bold", "faint", "fblink" (fast-blink),
"inverse" (fg and bg swapped),
"invisible", "italic", "plain" (the default, which need not be specified),
"strike", and "ul" (underscore or underline).

The effect names may be prefixed by "!" to turn the effect off.
Terminal programs vary wildly in what effects they support, and
in the appearance of some combinations (such as setting fg and bg the same).
The "bold" effect may yield true or simulated boldface, or brighter colors.

You can use the I<--listColors> option
or the equivalent C<colorstring --list> command to see
what colors and effects your terminal program supports.

=item * B<--changeColor> I<color1=color2>

B<Experimental>.
Remap the color named I<color1>, to show as I<color2> instead. For example,
with I<--changeColor red=blue>, any requests for red show up as blue instead.
Repeatable.

=item * B<--columns> I<colspec>

Highlight the fixed column positions identifier by I<colspec>, which must be
of a form such as: C<4>, C<-8>, C<12->, or C<16-20>. Repeatable.
Unfinished.

=item * B<-e> I<regex>

A (Perl-style) regex to highlight matches of. Repeatable.
Matches use the latest value for I<--color>. For example:

    hilite -c red -e 'error' -c yellow -e 'warning'

=item * B<--file> I<file>

Load a I<file>, treating lines as if specified via I<-e>.
Two types of lines are special. Lines starting with (optional whitespace and) "#" are ignored.
Lines starting with "!COLOR" should have a following space(s) and then color name, which
is applied to following expressions (until overridden). For example:

    # Pat's color table.
    !COLOR red/white
    the
    !COLOR green/italic
    and

=item * B<--ignoreCase> or B<-i>

Ignore case for all regex matching (not just following I<-e> instances).
Default: on. Turn off with I<-r> (q.v.).

=item * B<--lines> OR B<--wholeLines> OR B<-l>

Highlight whole lines, not just the matched part(s).

=item * B<--listColors>

Print a list of the known color names, then exit.
The list is generated by running the C<colorstring --list> command (q.v.),
which is also useful for determining what colors and effects your terminal
program supports.

=item * B<--only>

Only show lines that have highlighting.

=item * B<--quiet> OR B<-q>

Suppress most messages.

=item * B<-r> OR B<--regardCase>

Regard case. Default is to ignore case. See also I<-i>.

=item * B<--test>

Show expressions to match, but do nothing.

=item * B<--untabify> I<n> OR B<-u> I<n> OR B<expand> I<n>

Expand tabs, assuming tabstops every I<n> columns. Default: 0 (off).
Not yet supported.

=item * B<--useHtml>

B<Experimental>.

Instead of inserting ANSI terminal escape sequences to create color, insert HTML markup.
This inserts outermost "html" and "body" tags (but no DOCTYPE), and inserts "span" tags
with inline "style" attributes to get colors and effects.
It does not even attempt the "blink", "fblink", or "reverse" effects.
However, it does accept #RGB colors as well as the usual ANSI names (it does I<not>
know the other HTML color names, however).
It doesn't XML-escape the text, so take care of "<" and "&" first if needed.

This only applies to coloring a file or STDIN, not to special options like I<--list>.

=item * B<--version>

Display version info and exit.

=item * B<--waitFor> I<pat>

Suppress all output until a match to regex I<pat> is seen (applyigng the
I<--ignoreCase> setting). If I<pat> is '*', waitq for the first highlightable match.

=back


=head1 Predefined regex sets

You can use these in combination with I<-e>, and/or specify more than one
of these options, but you can't delete expressions from them.
The colors set for these work best with a dark background.
See also I<--changeColor>, above.

=over

=item * B<--aname> I<e>

XML attribute with (entire) name matching I<e>.
This doesn't do a full-fledged XML parse, so can be fooled by CDATA, comments, etc.

=item * B<--avalue> I<e>

XML attribute with (entire) value matching e.
This doesn't do a full-fledged XML parse, so can be fooled by CDATA, comments, etc.

=item * B<--css>

CSS style property names.

=item * B<--cvsstatus>

Output from cvs status, highlighting problems.

=item * B<--diff>

Show lines (I<-l>) with differences found by I<diff -y> (side-by-side).

=item * B<--fixBars>

Use before I<--diff> to try to do better detection of '|' flag.

=item * B<--entities>

XML entity references.
This doesn't do a full-fledged XML parse, so can be fooled by CDATA, comments, etc.

=item * B<--error>

Words like 'error' in red, 'warning' in yellow, 'info' in cyan, etc.
(this one is really nice!)

=item * B<--gi>

Start and end XML tags but only the element type name.

=item * B<--htmlstyle>

CSS style attributes in HTML.

=item * B<--man>

Man pages (experimental).

=item * B<--nonascii>

Strings of non-ASCII characters.
The C<showInvisibles> command has a I<--color> option that is similar.
However, it is not effective to pipe C<hilite> into C<showInvisibles>.
That is because C<showInvisibles> also effects the escape sequences
inserted by I<hilite>.

=item * B<--ns> I<name>

XML tags with explicit namespace prefix I<name> (repeatable).

=item * B<--regex>

Special characters in regular expressions (experimental).

=item * B<--tag> I<name>

XML tags of the specified I<name>
(repeatable).
This doesn't do a full-fledged XML parse, so can be fooled by CDATA, comments, etc.

=item * B<--uri>

http uris (not finished).

=item * B<--xml>

XML tags, XSL tags, comments, declarations, etc.
This doesn't do a full-fledged XML parse, so can be fooled by CDATA, comments, etc.

=back


=head1 Related commands

C<ColorManager.pm>: This library provides the underlying color-name handling
and escape generation. It can also be run as a standalone command, in which case
it displays a short list of test swatches, or with I<-h> for help.

C<ColorManager.pm>: does much the same for Python.

C<colorstring>: provides terminal color control strings, etc.

C<mathAlphanumerics.py>: Provides access to many Unicode "Mathematical" and other
variations on Latin and Greek letters and Arabic digits.

C<sjdUtils.pm>: provides access to the same color escapes for Perl code.

C<sjdUtils.py>: does much the same for Python.

Linux C<enscript>: can highlight program syntax, or I<diff> output; this
program may eventually support color printing via I<enscript>.

Linux C<colorit> seems to do something similar.


=head1 Known bugs and limitations

To see highlighting with the I<more> command on some systems, use I<more --raw>.

Multiple predefined expressions are not necessarily
applied in the order specified.

For XML-related options, constructs that are split across lines are not accurately
highlighted (the code use regexes, not a real parser). This does have
the compensation that non-well-formed files can be highlighted.
See my C<prettyPrintXml.py>, C<normalizeXML>, etc. for more precise treatment.

Overlapping matches may produce interesting highlighting.

The I<--diff> option highlights some lines with ' | ' in them that aren't diffs.


=head1 To do

     Fix problem with --diff setup
     Finish --columns feature
     Finish --useHtml support for factoring out CSS.
     Postscript output so you can color-print? Linux 'enscript' supports
     Option for filename-containing or filename-with-extension
     --aname and --avalue should use lookbehind so only match target portion
     Option to hilite all occurrences of any words from file F (-f enough?)
     hilite given fields by name or number
     Add support for svn and git status?


=head1 History

    2006-11-03: Written by Steven J. DeRose.
    2008-03-23 sjd: Rewrite.
    2008-04-03 sjd: Add --aname, --avalue, --file options.
    2008-04-11 sjd: Fix doc.
    2010-11-29 sjd: Cleanup. Finish --file.
    2011-12-14 sjd: Integrate sjdUtils.pm.
    2013-08-25: Add --useHtml: Insert HTML highlighting.
    2014-08-14: Break out setup methods.
    2014-09-01: Add --nonascii, --changeColor, --listColors.
    2016-04-08: Switch from *nix 'expand' to Perl Text::Tabs::expand
(which helps portability, and supports Unicode). Drop --tabInterval
in favor of value for --untabify. Add --expand synonym. Fix STDIN.
    2016-10-25: Hook to ColorManager via hColor, eliminating colorHash.
Drop --useHtml. Redo --changeColor locally.
    2017-06-02: Start making options like --color "sticky".
    2018-09-17: Fix accidentally hard-coded 'red'.
    2021-06-25: New layout.
    2022-02-14: Implement most of --useHtml. Add #comment and !COLOR to --file.
    2022-12-16: Move options closer to colorstring to be less confusing.


=head1 Ownership

This work by Steven J. DeRose is licensed under a Creative Commons
Attribution-Share Alike 3.0 Unported License. For further information on
this license, see L<http://creativecommons.org/licenses/by-sa/3.0/>.

For the most recent version, see L<http://www.derose.net/steve/utilities/>.


=cut


###############################################################################
# Options
#
my @changeColor = ();        # Color-name remapping.
my @columns     = ();        # Array of [ color, colspec ] pairs.
my $curColor    = "red";
my %exprs       = ();        # expressions to hilite => colorName
my $file        = "";        # load expressions from a file
my $fixBars     = 0;         # try harder to detect real diff change-lines
our $ignoreCase = 1;         # ignore case for all -e exprs.
my $listColors  = 0;
my $only        = 0;
my $quiet       = 0;
my $test        = 0;
my $text        = 0;         # Take input from command line, not file/stdin.
my $untabify    = 0;
my $useHtml     = 0;
my $verbose     = 0;
my $waitFor     = "";
my $warn        = 0;         # Output to stderr
my $wholeLines  = 0;

# Option flags for pre-defined expression sets
#
my $predefined_cvsstatus = 0;
my $predefined_diff      = 0;
my $predefined_entities  = 0;
my $predefined_err       = 0;
my $predefined_gi        = 0;
my $predefined_htmlstyle = 0;
my $predefined_man       = 0;
my $predefined_ns        = 0;
my $predefined_nonascii  = 0;
my $predefined_pylint    = 0;
my $predefined_regex     = 0;
my $predefined_tag       = 0;
my $predefined_uri       = 0;
my $predefined_xml       = 0;
my $predefined_css       = 0;
my @predefined_aname     = ();
my @predefined_avalue    = ();

#alogging::vMsg(1, "hilite calling sjdUtils::setColors().");
sjdUtils::setColors(1);
ColorManager::setColors(1);
my %hColorOverrides = ();

# Set up color table before handling options, so we can error-check "-c".
#
my $testColor = "red/italic";
my $testStart = hColor($testColor);
my $cStart = hColor($curColor);
my $cEnd = hColor("default");
($cStart && $cEnd) || die "Unable to set up test color '$testColor'.\n";
Getopt::Long::Configure ("ignore_case");
my $haveSeenColorOption = 0;

my $result = GetOptions(
    # Predefined target expressions
    "aname=s"      => \@predefined_aname,
    "avalue=s"     => \@predefined_avalue,
    "css!"         => \$predefined_css,
    "cvsstatus!"   => \$predefined_cvsstatus,
    "diff!"        => \$predefined_diff,
    "entities!"    => \$predefined_entities,
    "err|error!"   => \$predefined_err,
    "gi!"          => \$predefined_gi,
    "htmlstyle!"   => \$predefined_htmlstyle,
    "man!"         => \$predefined_man,
    "nonascii!"    => \$predefined_nonascii,
    "ns=s"         => \$predefined_ns,
    "pylint!"      => \$predefined_pylint,
    "regex!"       => \$predefined_regex,
    "tag=s"        => \$predefined_tag,
    "uri!"         => \$predefined_uri,
    "xml!"         => \$predefined_xml,

    # Other options
    "c|color=s"    => sub {
        $haveSeenColorOption = 1;
        $curColor = $_[1];
        my $try = hColor($curColor);
        if (!$try) {
            warn "Unknown color '$curColor'..\n";
        }
        else {
            $cStart = $try;
        }
    },
    #"changeColor=s"        => \@changeColor,
    "cols|columns=s"       => sub { push(@columns, [ $curColor, $_[1] ]); },
    "e|expr=s"             => sub {
        $exprs{$_[1]} = $curColor;
        ($verbose && !$haveSeenColorOption) &&
            warn "Did you want to set --color before --expr? Defaulting to '$curColor'.\n";
    },
    "file=s"               => sub {
        (-f $_[1]) || die "Can't find file of expressions for -f $_[1].\n";
        open F, "<$_[1]";
        my $nf = 0;
        while (my $rec = <F>) {
            $nf++;
            $exprs{$rec} = $curColor;
        }
        close F;
        ($verbose) && warn "Loaded $nf expressions from file '$_[1]'.\n";
    },
    "fixBars!"             => \$fixBars,
    "h|help|?"             => sub { system "perldoc $0"; exit; },
    "i|ignoreCase!"        => \$ignoreCase,
    "l|lines|wholeLines!"  => \$wholeLines,
    "listColors!"          => \$listColors,
    "only!"                => \$only,
    "q|quiet!"             => \$quiet,
    "r|regardCase!"        => sub { $ignoreCase = 0; },
    "test|dry-run|dryrun!" => \$test,
    "text|msg|message!"    => \$text,
    "u|untabify|expand=o"  => \$untabify,
    "useHtml!"             => \$useHtml,
    "v|verbose+"           => \$verbose,
    "version"              => sub {
        die "Version of $VERSION_DATE, by Steven J. DeRose.\n";
    },
    "waitFor=s"            => \$waitFor,
    "warn!"                => \$warn,
);

alogging::setLogVerbose($verbose);

sub hColor {
    my ($name) = @_;
    #alogging::vMsg(2, "hColor: got request for color '$name'.");
    if ($hColorOverrides{$name}) {
        alogging::vMsg(2, "Using override for '$name'.\n");
        return $hColorOverrides{$name};
    }
    my $cstr = ColorManager::getColorString($name); # undef on fail
    #vMsg(2, "Setting up color '$name', got " . sjdUtils::showInvisibles($cstr) . ".\n");
    return $cstr;
}

if ($verbose) {
    warn("\n******* Color/expression map *******\n");
    for my $e (keys(%exprs)) {
        warn(sprintf("%-20s /%s/\n", $exprs{$e}, $e));
    }
}

###############################################################################
# HTML support
#
my %colors = (
    "black"   => 0,
    "red"     => 1,
    "green"   => 2,
    "yellow"  => 3,
    "blue"    => 4,
    "magenta" => 5,
    "cyan"    => 6,
    "white"   => 7,
    "default" => 9,
);

my %effects = (
    "bold"       => 1,  # aka 'bright'
    "faint"      => 2,  #
    "italic"     => 3,  # (rare)
    "underline"  => 4,  # aka 'ul'
    "blink"      => 5,  #
    "fblink"     => 6,  # aka 'fastblink' (rare)
    "reverse"    => 7,  # aka 'inverse'
    "concealed"  => 8,  # aka 'invisible' or 'hidden'
    "strike"     => 9,  # aka 'strikethru' or 'strikethrough'
    "plain"      => 0,  # (can be used to express "no special effect")
);


###############################################################################
# TODO: Move this into ColorManager?
#
my $colorExpr = '^#([0-9a-f][0-9a-f]){1,3}$';  # 2/4/6 hex digits for RGB
#
sub getStartTag {
    my ($colorName) = @_;
    my $css = "";
    my @c = split(/\//, $colorName);
    if ($c[0] && (defined $colors{$c[0]} or $c[0] =~ m/$colorExpr/i)) {
        $css .= "color:" . $c[0] . ";";
    }
    if ($c[1] && (defined $colors{$c[1]} or $c[1] =~ m/$colorExpr/i)) {
        $css .= "background:" . $c[1] . ";";
    }
    if ($c[2] && defined $effects{$c[2]}) {
        if ($c[2]    eq "bold")      { $css .= "font-weight:bold;"; }
        elsif ($c[2] eq "faint")     { $css .= "font-weight:lighter;"; }
        elsif ($c[2] eq "italic")    { $css .= "font-style:italic;"; }
        elsif ($c[2] eq "underline") { $css .= "font-decoration:underline;"; }
        #elsif ($c[2] eq "blink")     { $css .= ""; }
        #elsif ($c[2] eq "fblink")    { $css .= ""; }
        # https://stackoverflow.com/questions/17741629/how-can-i-invert-color-using-css
        #elsif ($c[2] eq "reverse")   { $css .= ""; }
        elsif ($c[2] eq "concealed") { $css .= "display:none;"; }
        elsif ($c[2] eq "strike")    { $css .= "text-decoration-line:line-through;"; }
        elsif ($c[2] eq "plain")     {
            $css .= "font-weight:medium; font-style:normal; font-decoration:none;";
        }
    }

    my $tag = sprintf('<span style="%s">', $css);
    ($verbose) && warn(sprintf("Mapping '%s' to %s\n", $colorName, $tag));
    return $tag;
}

sub getEndTag {
    return "</span>"
}


###############################################################################
# Main
#
($verbose) && warn sprintf("TEST %s%s%s (cur color %s%s%s).\n",
     $testStart, $testColor, $cEnd, $cStart, $curColor, $cEnd);

#if ($untabify) {
#    ($untabify > 1) ||
#        die "hilite: Bad --untabify interval '$untabify'.\n";
#    $Text::Tabs::tabstop = $untabify;  # For Text::Tabs
#}

($result) || die "hilite: Bad options.\n";

my @files = ();
if ($ARGV[0]) {
    @files = @ARGV;
}
else {
    $files[0] = "--stdin--";
}

for (my $i=0; $i<scalar(@changeColor); $i++) {
    $hColorOverrides{$1} = hColor($2);
}

if ($listColors) {
    print "Known color names are:\n";
    system "colorstring --list";
    exit;
}
setup_predefined();

# Read regexes from a file
#
if ($file) {
    open(XLIST, "<$file") || die "Could not open expression file '$file'.\n";
    binmode(XLIST, ":encoding(utf8)");
    my $curColor = "red";
    my $c = hColor("red");
    while (my $x = <XLIST>) { # Could 'or' together instead...
        chomp $x;
        if ($x =~ m/^\s*#/) {
            next;
        }
        elsif ($x =~ m/^!COLOR\s/) {
            $curColor = $x;
            $curColor =~ s/!COLOR\s+//;
        }
        else {
            $exprs{$x} = $curColor;
        }
    }
    close XLIST;
    ($verbose) && warn
        "Expressions loaded from '$file': " . scalar(keys %exprs) . ".\n";
}

# Map the colorNames to ANSI or HTML.
# TODO: Shorten up the start-tags by putting the CSS in head/style and use via @class.
#
my $headCSS = "<style type=\"text/css\"><!--\n";
my %color2esc = ();
if ($useHtml) {
    $cEnd = getEndTag();
    for my $e (keys %exprs) {
        my $className = my $colorName = $exprs{$e};
        $className =~ s/[^\w]/_/g;
        my $stag = getStartTag($colorName);
        $color2esc{$colorName} = $stag;
        $stag =~ s/<span style="(.*)">/$1/;
        $headCSS .= sprintf("    span.%-20a { %s }\n", $className, $stag);
        ($verbose) && warn("%-20s %s\n", $colorName, $color2esc{$colorName});
    }
    $headCSS .= "--></style>\n";
}
else {
    $cEnd = hColor("default");
    for my $e (keys %exprs) {
        my $colorName = $exprs{$e};
        $color2esc{$colorName} = hColor($colorName);
        ($verbose) && warn("%-20s %s\n", $colorName, $color2esc{$colorName});
    }
}

if (!$quiet) { showSetup(); }

($verbose) && warn "Files to do: " . scalar(@files) . ".\n";
my $tfile = "/tmp/hilite.txt_" . int(rand(100000));

my $doneWaitingFor = 0;
($useHtml) && issueLine("<html>\n<head>\n<title></title>\n$headCSS</head>\n<body>");

if ($text) {
    doLine(join(" ", @ARGV) . "\n");
}
else {
    while (my $rec = <>) {
        doLine($rec);
    }
}

($useHtml) && issueLine("</body>\n</html>");

system "rm $tfile 2>/dev/null";

exit;


###############################################################################
#
sub issueLine {
    my ($line) = @_;
    if ($warn) { warn $line . "\n"; }
    else { print $line . "\n"; }
}

sub doLine {
    my ($line) = @_;
    my $gotit = 0;

    chomp $line;

    if (0 && $untabify) {
        $line = Text::Tabs::expand($line);
    }

    if (scalar %exprs == 0) {
        issueLine("$cStart$line$cEnd");  # $color2esc{$curColor}
        return;
    }

    if ($wholeLines) { # stop at first match per line
        foreach my $e (keys %exprs) {
            if (($ignoreCase  && $line =~ m/$e/i) ||
                (!$ignoreCase && $line =~ m/$e/)) {
                $gotit = 1;
                if (!defined $exprs{$e}) {
                    alogging::eMsg(0, "Undefined expr '" . $exprs{$e} . "'. Known:");
                    alogging::eMsg(-1, join("\n", sort keys %exprs));
                }
                $line = $color2esc{$exprs{$e}} . $line . $cEnd;
                last;
            }
        }
    }
    elsif (scalar(@columns)>0) {
        $line = colorColumns($line, \@columns);
    }
    else { # find all matches per line
        foreach my $e (keys %exprs) {
            # screwy case is where match contains $cEnd already....
            $gotit ||= ($line =~ m/$e/);
            my $cOn = $color2esc{$exprs{$e}};  # Use the right color
            if ($ignoreCase) { $line =~ s/($e)/$cOn$1$cEnd/gi; }
            else             { $line =~ s/($e)/$cOn$1$cEnd/g;  }
        }
    }
    if (!$doneWaitingFor) {
        if ($waitFor eq "*") {
            if ($gotit) { $doneWaitingFor = 1; }
        }
        elsif ((!$ignoreCase && $line =~ m/$waitFor/) ||
               ( $ignoreCase && $line =~ m/$waitFor/i)) {
            $doneWaitingFor = 1;
        }
        else { next; }
    }
    if ($only == 0 || $gotit) {
        issueLine $line;
    }
}


###############################################################################
# Report the setup (*** fix to show colors, too ***)
#
sub showSetup {
    ((scalar keys %exprs) > 0) || ($test) || ($quiet) ||
        warn "hilite: No regexes specified (see -e and -f).\n";

    if ($verbose || $test) {
        my @ekeys = keys %exprs;
        my $nexprs = scalar @ekeys;
        warn "hilite: The $nexprs expression(s) to match:\n";
        my $i = 1;
        for my $e (@ekeys) {
            if ($i < 10) { $i = " $i"; }
            warn "    $i:\t /$e/\n";
            $i++;
        }
        warn "hilite:     Case will be " .
            (($ignoreCase) ? "ignored":"regarded") . ".\n";
        ($test) && exit;
    }
}


###############################################################################
#
sub colorColumns {
    my ($line, $columns) = @_;
    for (my $i=0; $i<scalar(@{$columns}); $i++) {
        my $color = $columns->[$i]->[0];
        my $cols  = $columns->[$i]->[1];
        my $fr = 0; my $to = 0;
        if ($cols =~ m/^-(\d+)$/) {
            $fr = 0; $to = int($1);
        }
        elsif ($cols =~ m/^(\d+)-$/) {
            $fr = int($1); $to = 0;
        }
        elsif ($cols =~ m/^(\d+)-(\d+)$/) {
            $fr = int($1); $to = int($2);
        }
        elsif ($cols =~ m/^(\d+)$/) {
            $fr = int($1); $to = int($1);
        }
        # Colorize from right to left so col numbers don't break.
    }
}


###############################################################################
# Implement predefined target expressions
#
sub setup_predefined {
    my $e = "";

    if (scalar @predefined_aname > 0) {
        foreach my $a (@predefined_aname) {
        $e = "<\\w[-_:.\\w\\d]*[^>]*\\s*(" . $a . ")\\s*=\\s*\"[^\"]*\"";
        $exprs{$e} = hColor("red");
        $e = "<\\w[-_:.\\w\\d]*[^>]*\\s*(" . $a . ")\\s*=\\s*'[^']*'";
        $exprs{$e} = hColor("red");
        }
    }

    if (scalar @predefined_avalue > 0) {
        foreach my $a (@predefined_avalue) {
        $e = "<\\w[-_:.\\w\\d]*[^>]*=\\s*\"(" . $a . ")\"";
        $exprs{$e} = hColor("red");
        $e = "<\\w[-_:.\\w\\d]*[^>]*=\\s*\'(" . $a . ")\'";
        $exprs{$e} = hColor("red");
        }
    }

    if ($predefined_cvsstatus) {
        $wholeLines = 1;
        $exprs{"Up-to-date"}          = hColor("green");
        $exprs{"Locally Added"}       = hColor("green");
        $exprs{"Locally Modified"}    = hColor("cyan");
        $exprs{"Needs Patch"}         = hColor("yellow");
        $exprs{"Needs Checkout"}      = hColor("yellow");
        $exprs{"Unknown"}             = hColor("yellow");
        $exprs{"Needs Merge"}         = hColor("red");
        $exprs{"had conflicts"}       = hColor("red");
        $exprs{"Unresolved Conflict"} = hColor("red");
        $exprs{"Invalid|Error"}       = hColor("/red");
        # Following is added by ~deroses/bin/mods...
        $exprs{"but repository has"}  = hColor("/red");
        $wholeLines = 1;
    }

    # Require 2 spaces before "|" so we don't catch it in content.
    if ($predefined_diff) {
        $wholeLines = 1;
        if ($fixBars) {
            warn "hilite: --fixBars may not work due to tabs.\n";
            my $w = $ENV{COLUMNS};
            if ($w eq "") { $w = 80; }
            my $centerColumn = int(($w+1) / 2.0);
            my $min = int($w/2) - 2; my $max = $min + 4; # look for "|" here
            $exprs{"^\\s+>\\s"}  = hColor("green");
            $exprs{"\\s<\\s+\$"} = hColor("cyan/bold");
            my $e = "^.{$min,$max}\\|";
            ($verbose) && warn "***** min $min, max $max, expr '$e'.\n";
            $exprs{$e}           = hColor("blue");
        }
        else {
            $exprs{"^\\s+>\\s"}     = hColor("green");
            $exprs{"\\s<\\s+\$"}    = hColor("cyan/bold");
            $exprs{"\\s\\s\\|\\s"}  = hColor("blue");
            warn "Consider --fixBars to improve '|' detection.\n";
            #$exprs{"\\s\\s\\|\\s"}  = hColor("yellow"); # problem
        }
    }

    if ($predefined_entities) {
        $e = "&[^;]*;";
        $exprs{$e} = $cStart;
    }

    if ($predefined_err) {
        $ignoreCase = 1;
        # RED messages
        $exprs{"(syntax )?ERROR( occurred at|s)?"} =
        $exprs{"(element|attribute)? *('\\w+')? *(is)? *(INVALID|NOT VALID)"} =
        $exprs{"CANNOT|CAN'T"}               =
        $exprs{"MUST NOT|MAY NOT"}           =
        $exprs{"UNKNOWN( file| command| variable| directory)?(, )?"} =
        $exprs{"FATAL"}                      =
        $exprs{"TERMINAT(e|ed|ing)"}         =
        $exprs{"( \\w+)?EXCEPTION"}          =
        $exprs{"FAIL(ed)?"}                  = hColor("red");
        # would be nice to exclude "(0 failed)":
        # $exprs{"([1-9]+0\s+|[^0]\s+)FAIL(ed)?"}

        # YELLOW messages
        $exprs{"WARNINGs? *[:=]? *[0-9]*"}   =
        $exprs{"SHOULD NOT"}                 =
        $exprs{"<<<= check the source here"} = hColor("yellow");

        # CYAN messages
        $exprs{"INFO"}                       =
        $exprs{"(XSLT )?MESSAGE:"}           =
        $exprs{"processed *(in *[0-9.]+ *sec\\.?)?"} = hColor("cyan");

        # GREEN messages
        $exprs{"SUCCESS(ful)?(ly)?"}         =
        $exprs{"SUCCEED(ed|ing)?"}           =
        $exprs{"START(ed|ing) *(file|document)*"} =
        $exprs{"RUNNING:? *(file|document)*"} =
        $exprs{"FINISH(ed|ing)"}             =
        $exprs{"COMPLET(e|ed|ing)?"}         =
        $exprs{"DONE"}                       = hColor("green");

        $exprs{"(/[-\\w]*\\.x.l,)? *line [0-9]+((, )(column|offset) [0-9]+)"}
                                             = hColor("bold");
    } # --err

    if ($predefined_gi) {
        $e = "<[-\\w]+\\s";
        $exprs{$e} = $cStart;
        $e = "</[-\\w]+";
        $exprs{$e} = $cStart;
    }

    if ($predefined_htmlstyle) {
        my @names = split(/\s+/, htmlStyleAttributeList());
        push @names, "style";
        my $nprops = scalar @names;
        ($quiet) || warn "hilite: Scanning for $nprops HTML style attributes.\n";
        # ($verbose) && print "Properties: @names.\n";
        for my $a (@names) {
            $e = "\\s+$a=(\"[^\"]*\"|'[^']*')";
            $exprs{$e} = hColor("yellow");
        }
    }

    if ($predefined_man) {
        $e = "\\W-+\\w+";
        $exprs{$e} = hColor("magenta");
        $e = "\\(see \\W+[^)]*\)";
        $exprs{$e} = hColor("green");
        $e = "^[A-Z ]+\\\$";
        $exprs{$e} = hColor("blue");
        $e = "^ {3,4}[A-Z].*";
        $exprs{$e} = hColor("blue");
        $e = "^ {14}\\S.*";
        $exprs{$e} = hColor("magenta/bold");
    }

    if ($predefined_nonascii) {
        $e = "[^ -~]+";
        $exprs{$e} = hColor("blue");
    }

    if ($predefined_ns) { # fix
        my $theNS = $predefined_ns;
        $theNS =~ s/:.*$//;
        $e = "<\\/?$theNS:" . "[^>]*>";
        $exprs{$e} = hColor("blue");
    }

    if ($predefined_pylint) {
        $e = "^E:";
        $exprs{$e} = hColor("red");
        $e = "^W:";
        $exprs{$e} = hColor("yellow");
        $wholeLines = 1;
    }

    if ($predefined_tag) {
        my $t = $predefined_tag;
        ($t =~ m/^\w[-:\w]*$/) || die "hilite: Invalid --tag type '$t'.\n";
        $e = "<\\/?$t(\\s[^>]*|\\/)?>";
        $exprs{$e} = $cStart;
    }

    if ($predefined_uri) {
        $e = "http"; # RFC 1738
        # $e = "https?:[-_\$.+!*,()&a-zA-Z0-9]*"; # RFC 1738
        $exprs{$e} = $cStart;
    }

    if ($predefined_xml) {
        $e = "<[^!][^>]*(>|\$)";
        $exprs{$e} = hColor("magenta");
        $e = "<![^>]*(>|\$)";
        $exprs{$e} = hColor("yellow");
        $e = "<\\/?xsl:[^>]*(>|\$)";
        $exprs{$e} = hColor("cyan");
        $e = "&[^;]*;";
        $exprs{$e} = hColor("green");
    }

    if ($predefined_regex) {
        $e = "[{}]";
        $exprs{$e} = hColor("magenta");
        $e = "[^\\\\][()?*+]";
        $exprs{$e} = hColor("red");
        $e = "\\\\";
        $exprs{$e} = hColor("yellow");
        $e = "[\\[\\]][-^]";
        $exprs{$e} = hColor("green");
    }

    if ($predefined_css) {
        my @names = split(/\s+/, cssPropertyList());
        my $nprops = scalar @names;
        ($quiet) || warn "hilite: Scanning for $nprops css property names.\n";
        # ($verbose) && print "Properties: @names.\n";
        for my $a (@names) {
            $e = "[^-\\w]$a" . "[^-\\w]";
            $exprs{$e} = hColor("yellow");
        }
    }
} # setup_predefined


###############################################################################
# CSS property names so we can colorize them.
#
sub cssPropertyList {
    return (
    "accelerator azimuth background background-attachment background-color
    background-image background-position background-position-x
    background-position-y background-repeat behavior border border-bottom
    border-bottom-color border-bottom-style border-bottom-width
    border-collapse border-color border-left border-left-color
    border-left-style border-left-width border-right border-right-color
    border-right-style border-right-width border-spacing border-style
    border-top border-top-color border-top-style border-top-width
    border-width bottom caption-side clear clip color content counter-increment
    counter-reset cue cue-after cue-before cursor direction display
    elevation empty-cells filter float font font-family font-size
    font-size-adjust font-stretch font-style font-variant font-weight
    height ime-mode include-source
    layer-background-color layer-background-image
    layout-flow layout-grid layout-grid-char layout-grid-char-spacing
    layout-grid-line layout-grid-mode layout-grid-type left letter-spacing
    line-break line-height list-style list-style-image list-style-position
    list-style-type margin margin-bottom margin-left margin-right
    margin-top marker-offset marks max-height max-width min-height
    min-width orphans outline outline-color outline-style outline-width
    overflow overflow-X overflow-Y padding padding-bottom padding-left
    padding-right padding-top page page-break-after page-break-before
    page-break-inside pause pause-after pause-before pitch pitch-range
    play-during position quotes richness right ruby-align ruby-overhang
    ruby-position scrollbar-3d-light-color scrollbar-arrow-color
    scrollbar-base-color scrollbar-dark-shadow-color scrollbar-face-color
    scrollbar-highlight-color scrollbar-shadow-color scrollbar-track-color
    size speak speak-header speak-numeral speak-punctuation speech-rate
    stress table-layout text-align text-align-last text-autospace
    text-decoration text-indent text-justify text-kashida-space
    text-overflow text-shadow text-transform text-underline-position
    top unicode-bidi vertical-align visibility voice-family volume
    white-space widows width word-break word-spacing word-wrap
    writing-mode z-index zoom");
}


###############################################################################
# Any missing?
#
sub htmlStyleAttributeList {
    return (
    "align axis background bgcolor
    border
    cellpadding cellspacing
    clear color
    face frameborder
    height hspace marginheight marginwidth nowrap rules size
    style
    valign vspace width");
}