Copyright (C) 2016, 2017, 2018, 2019  Stefan Vargyas

This file is part of Json-Type.

Json-Type is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

Json-Type is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with Json-Type.  If not, see <http://www.gnu.org/licenses/>.

--------------------------------------------------------------------------------

1. What is Json-Litex
2. Json-Litex's Literal Expressions
3. Json-Litex's Text Path Libraries
4. Json-Litex's Compiled Path Libraries
5. Querying Github API at the Bash Command Line
6. Appendix: Json-Litex's Builtins
7. Appendix: Using 'json-litex.so'
8. Appendix: Using 'test-expr'


1. What is Json-Litex
=====================

Json-Litex is a Json-Type filter library in the sense defined by the document
'doc/filter-libs.txt' and is implemented by the library 'lib/json-litex.so'.
The purpose of Json-Litex is to enforce constraints -- specified by the user
in the form of so-called path libraries -- on the literal values of the JSON
input of 'json' program.

The constraints which the user specifies are boolean expressions made of mixing
constant numbers, constant strings, a number of builtin functions and PCRE2 [1]
regexes. The syntax of the boolean expressions is defined by 'lib/json-litex.g'.

Important to emphasise is that Json-Litex is not here to replace the principal
feature of Json-Type -- that of type checking of JSON texts. Json-Litex may well
function in a filter library pipeline that does not include Json-Type's type
checker class. As to be seen further below, Json-Litex was concieved to only
augment the main function of Json-Type.


2. Json-Litex's Literal Expressions
===================================

As already noted above, Json-Litex's literal expressions are boolean expressions
made up of constants -- numbers and strings --, builtin functions and PCRE2 [1]
regexes, that are conforming to the specifications in 'lib/json-litex.g'.

The meaning of the syntactical constructs in 'lib/json-litex.g' are for the most
part self-evident. Only a couple of notations need to be explicitely defined --
for the case of syntactical category 'prim':

  -----------------------  ------------------------------------------------
   `# STR'                  the meaning if `#' is counting; the expression
                            `# STR' means the number of occurences of 'STR'
                            in the JSON literal against which `# STR' gets
                            evaluated
  -----------------------  ------------------------------------------------
   `# REX'                  similarly with `# STR', the expression `# REX'
                            means the number of occurrences of PCRE2 'REX'
                            in the JSON literal against which `# REX' gets
                            evaluated
  -----------------------  ------------------------------------------------
   `STR' by itself          a `STR' by itself -- i.e. when not preceded by
                            the counting operator `#' -- means matching:
                            true if and only if 'strcmp' applied to 'STR'
                            and the JSON literal against which `STR' gets
                            evaluated returns 0
  -----------------------  ------------------------------------------------
   `REX' by itself          a `REX' by itself -- i.e. when not preceded by
                            the counting operator `#' -- means matching:
                            true if and only if 'pcre2_match' applied to
                            'REX' and the JSON literal against which `REX'
                            gets evaluated returns successfully
  -----------------------  ------------------------------------------------
   `ID [ "(" args ")" ]'    the meaning of this expression is simple: call
                            the builtin function `ID' passing to it actual
                            arguments as given; any builtin function that
                            requires no arguments, may simply be called by
                            an expression with no parentheses: `ID'. The
                            builtin functions receive as a hidden argument
                            the JSON literal against which the expression
                            is evaluated and each produces either a number
                            or a boolean value. Further below, the section
                            6, Appendix: Json-Litex's Builtins, describes
                            each builtin function currently implemented
  -----------------------  ------------------------------------------------

The regex options defined by terminal symbol 'OPT' in 'lib/json-litex.g' do have
the meaning shown by the table below. The rightmost column of the table indicates
what are the PCRE2 option constants that correspond to the regex options letters
of Json-Litex:

  -----  --------------------  --------------------
   `I'    ignore letter case    PCRE2_CASELESS
  -----  --------------------  --------------------
   `S'    dot `.' match all     PCRE2_DOTALL
  -----  --------------------  --------------------
   `U'    no UTF-8 check        PCRE2_NO_UTF_CHECK
  -----  --------------------  --------------------
   `X'    extended pattern      PCRE2_EXTENDED
  -----  --------------------  --------------------

An interesting feature of Json-Litex's expression parser is the following: it is
accepting as valid identifiers all unique prefixes of builtin names. Therefore,
one may use the shortened identifier 'num' for to refer to the builtin function
'number'. However, the identifier 'nu' will be rejected by the parser since 'nu'
is prefix to both builtin names 'null' and 'number'.

For to accommodate oneself with Json-Litex's literal expressions, or, otherwise,
for apprehending the behavior of 'json-litex.so' with respect to certain literal
expressions, one may use the test program 'lib/test-expr' -- which Json-Litex's
test suite is using itself for the purpose of testing the implementation.

If not already built as a result of building Json-Litex, 'test-expr' may well be
obtained by the following command issued within 'lib' directory:

  $ make -f test-expr.mk

Note that for the above 'make' command to succeed is needed that GCC be able to
find the development package of PCRE2 library. The makefile 'test-expr.mk' can
be instructed to look up in a non-standard place for PCRE2 binaries and header
files, precisely the same way as Json-Litex is. All details are given in section
3, Building and Testing Json-Type, of the 'README' file.

Upon a successful build of 'test-expr' binary, there is yet of an issue one needs
to be aware of for running this program: the system's dynamic loader must be able
to find the PCRE2 shared library that the program has been linked with. For this
to come true, is usually enough to set properly the LD_LIBRARY_PATH environment
variable.

The section 8 below, Appendix: Using 'test-expr', gives all the relevant details
about using 'test-expr'. Here are couple of simple 'test-expr' command lines:

  $ alias test-expr=lib/test-expr

  $ test-expr 'len > 3' false
  test-expr: error: executor error: function 'len' applied to 'boolean'

  $ test-expr 'len > 3' '"foo"'
  0

  $ test-expr 'len > 3' '"fooo"'
  1

  $ test-expr 'nu' -v
  test-expr: error:1:1: compiler error: unknown builtin
  test-expr: error:1:1: compiler error: nu
  test-expr: error:1:1: compiler error: ^

  $ test-expr 'num' '"bar"'
  0

  $ test-expr 'num' 012345678901234567890123456789
  1

  $ test-expr 'uint(64)' 012345678901234567890123456789
  0

  $ test-expr '/^a+b+$/ && #`a` < #`b`' '"ab"'
  0

  $ test-expr '/^a+b+$/ && #`a` < #`b`' '"abb"'
  1

One must be aware of the rules of shell quoting when issuing commands like the
ones above. If the encompassing shell is a POSIX shell and the current locale is
'C', then the following command should work as shown:

  $ test-expr 'date(c_locale)' "\"`date`\""
  1

The counting operator `#' applied to a string or a regex computes the number of
occurrences of the respective operand within the given JSON literal. Note that
`#' counts all *non-overlapping* occurrences, therefore the output produced by
the following two 'test-expr' commands is correcty `2' and not `3':

  $ test-expr '#`aa`' '"aaaa"'
  2

  $ test-expr '#/aa/' '"aaaa"'
  2

Counting regex matchings anchored with `^' may be confusing at a first glance:

  $ test-expr '#/^ab/' '"ababac"'
  2

But one have to consider the meaning of an expression as `#/^ab/': match regex
`/^ab/' succesively against the following substrings of `"ababac"': the string
itself, `"abac"' (the first regex matching was at position 0 of `"ababac"' and
of lenght 2) and, lastly, `"ac"' (the second regex matching was at position 0
of `"abac"' and of length 2). Than the result `2' of 'test-expr' is indeed OK.

Instead of evaluating literal expressions against JSON literals, 'test-expr' is
able to produce a text representation of the internal executable code generated
for input expressions:

  $ test-expr '/^a+b+$/ && #`a` < #`b`' -C
  0 match_rex("^a+b+$")
  1 jump_false(eos)
  2 count_str("a")
  3 count_str("b")
  4 cmp_op(lt)

The op codes seen above are part of a slightly longer list which can be seen --
in a C language representation -- as the enum 'json_litex_expr_node_type_t' in
header file 'lib/json-litex.h'.


3. Json-Litex's Text Path Libraries
===================================

Json-Litex's path libraries are the means by which the user of Json-Litex does
specify completely the constraints he needs be enforced on the JSON literals of
input JSON texts.

A path library can be of two varieties: either a JSON text or a shared library
encoding in binary form a JSON text path library -- the so-called compiled path
libraries. The latter kind of path libraries are described in the next section;
the meaning of these depends on that of the former ones.

A text path library is defined by the following syntactical rules:

  (1) a text path library consists of JSON strings, objects or arrays of the
      type described further;

  (2) the JSON strings are expressions in the sense of section 2 above;

  (3) the JSON objects may have any number of arguments; their argument names
      have no constraints, but each argument value must have recursively the
      type of any text path library;

  (4) the JSON arrays are closed, having one or two elements; these elements
      must be recursively either text path library strings or objects; when
      arrays have two elements, these cannot be both strings nor can they be
      both objects.

The meaning of the constructs defined by rules (1) - (4) above is as resulting
from the definitions that follow:

  (A) a path sequence of a text path library is by definition the ordered
      sequence of JSON object argument names connecting the top level node
      of the path library tree and a node of the path tree that is labeled
      by a literal expression; therefore each path sequence of a text path
      library is associated uniquely with a literal expression;

  (B) the path location of a JSON literal instance within a JSON text is by
      definition the ordered sequence of JSON keys -- i.e. of JSON object
      argument names -- that connects the root JSON object with that JSON
      literal instance within the given JSON text;

  (C) a JSON literal instance of a JSON text satisfies by definition the
      constraints of a pair of a path sequence and a literal expression if
      and only if the path location of that JSON literal instance within
      the given JSON text is element by element equal with the given path
      sequence and if the evaluation of the literal expression against the
      JSON literal instance gets the boolean value true;

  (D) a JSON text satisfies by definition the constraints of a path library
      if and only if for each JSON literal instance contained in that JSON
      text, there exists a path sequence in the path library such that the
      pair of that path sequence and its associated literal expression is
      satisfied by the respective JSON literal instance.

An important consequence of definition (D) is that a condition for a text path
library to be satisfied by a JSON text is the following: the path library must
be structurally similar with the respective JSON text, modulo the arrays that
are contained in the JSON text.

Note also that the path location notion of definition (B) doesn't include any
information with respect to the JSON arrays that a given JSON text may contain.
E.g., all JSON literals below are at path location `"foo", "bar"':

  { "foo": [ { "bar": 0 }, [ { "bar": 1 } ] ] }

  { "foo": [ { "bar": 2 },   { "bar": 3 }   ] }

Consequently, one must remark that structurally different JSON texts may well
satisfy the contraints of the same path library:

  { "foo": { "bar": "number && /^[0-3]$/" } }

This example illustrates the fact that Json-Litex is not capturing -- with its
notion of path library -- the exact structure of input JSON texts. This happens
by design: Json-Litex's purpose is not to replace the structural type checking
of JSON texts done by Json-Type's library -- but to extend that function only.

Here is a more elaborated example of a text path library:

  $ text-path-lib() { echo \
     '{
          "a": {
              "b": [
                  "/^[[:ascii:]]*$/ && #`\n` <= 1",
                  {
                      "c": {
                          "d": "null || /^x+y+$/ && #`x` < #`y`"
                      }
                  }
              ]
          },
          "c": {
              "d": "null || date(`%Y-%m-%d`)"
          }
      }'
  }

All path sequences and associated literal expressions of this path library are
listed below. The notation of form "/a/b/c/d" represents of the following path
sequence: `"a", "b", "c", "d"'.

  "/a/b"     : "/^[[:ascii:]]*$/ && #`\n` <= 1"
  "/a/b/c/d" : "null || /^x+y+$/ && #`x` < #`y`"
  "/c/d"     : "null || date(`%Y-%m-%d`)"

To verify whether 'text-path-lib' is a valid text path library is sufficient to
invoke 'json-litex.so' with action options `-V|--validate':

  $ text-path-lib|json -VF -- json-litex.so -V && echo OK
  OK

Note that for 'json' program to be able to successfully load the shared library
'json-litex.so' the system's dynamic loader must succeed finding and loading the
PCRE2 shared library that 'json-litex.so' has been linked with. Ordinarly it is
enough to set a proper LD_LIBRARY_PATH environment variable to avoid the failure
of a command like the one below:

  $ json -VF -- json-litex.so --version
  json: error: json-litex.so: filter library: failed loading library: \
  dlopen failed: libpcre2-8.so.0: cannot open shared object file: \
  No such file or directory

Back to the sample path library, if changing 'text-path-lib' such that to be
erroneous, then its validation will fail:

  $ text-path-lib|sed '4s/ascii/foo/'|json -VF -e 16 -- json-litex.so -V
  json: error: <stdin>:4:25: filter library: expression error: \
  pcre2 compile: unknown POSIX class name
  json: error: <stdin>:4:25:           "/^[[:foo:]]*$/ && #`\\
  json: error: <stdin>:4:25:                 ^

Before continuing, let 'json-litex' be a shortcut for a longer 'json' command
line that calls for 'json-litex.so' on path library 'text-path-lib':

  $ json-litex() {
    json -VOf -- json-litex.so -p <(text-path-lib) "$@" &&
    echo OK
  }

Now let apply some input JSON texts to 'text-path-lib' throught shell function
'json-litex':

  $ json-litex <<< '{}'
  OK

  $ json-litex <<< '{"a":null}'
  json: error: <stdin>:1:6: filter library: path check error: \
  unexpected JSON literal
  json: error: <stdin>:1:6: {"a":null}
  json: error: <stdin>:1:6:      ^

  $ json-litex <<< '{"e":null}'
  json: error: <stdin>:1:6: filter library: path check error: \
  unexpected JSON literal
  json: error: <stdin>:1:6: {"e":null}
  json: error: <stdin>:1:6:      ^

  $ json-litex <<< '{"c":{}}'
  OK

  $ json-litex <<< '{"c":{"d":null}}'
  OK

  $ json-litex <<< '{"c":{"d":123}}'
  json: error: <stdin>:1:11: filter library: path check error: expression \
  falsified: value at "/c/d" -> expression "null || date(`%Y-%m-%d`)"
  json: error: <stdin>:1:11: {"c":{"d":123}}
  json: error: <stdin>:1:11:           ^

  $ json-litex <<< '{"c":[{"d":"2018-09-10"},{"d":456}]}'
  json: error: <stdin>:1:31: filter library: path check error: expression \
  falsified: value at "/c/d" -> expression "null || date(`%Y-%m-%d`)"
  json: error: <stdin>:1:31: {"c":[{"d":"2018-09-10"},{"d":456}]}
  json: error: <stdin>:1:31:                               ^

Shouldn't be any surprise that the first command above succeded: the empty JSON
object `{}' satisfies any path library according to definition (D).

The second command failed as required, since at path location "/a" in the input
JSON text `{"a":null}' is not expected a JSON literal.

It was also required that the third command fail: there is no path sequence "/e"
in 'text-path-lib', yet the JSON literal `null' has location "/e" in the input
JSON text `{"e":null}'. When one wishes that such unexpected JSON literals be
tacitly ignored, he may rely on the options `-l|--unexpect-literals=ignore'.

It may be surprising that the fourth command above succeeded, but this behavior
is correct since the input JSON text `{"c":{}}' contains no JSON literals.

The fifth command above succeeded simply because the JSON literal `null' at the
location "/c/d" in the input text `{"c":{"d":null}}' satisfies 'text-path-lib's
literal expression `null || date(`%Y-%m-%d`)'.

The sixth and the seventh commands above failed correctly since the JSON number
literals `123' and respectively `456' do not satisfy the literal expression that
corresponds to path sequence "/c/d" of 'text-path-lib'.


4. Json-Litex's Compiled Path Libraries
=======================================

Json-Litex's compiled path libraries relate to text path libraries exactly as
Json-Type's compiled type libraries relate to Json-Type's text type libraries.
This is to say that a compiled path library is a binary representation of all
internal data structures that are constructed by 'json-litex.so' upon parsing
and validating a given text path library.

As for the case of Json-Type's compiled type libraries, Json-Litex's compiled
path libraries come to good use when the user is developing consistently-sized
text path libraries which are to be used frequently but seldom changed. Using
a compiled library instead of its originating text library has the effect of
'json-litex.so' running faster, since in the case of compiled path libraries
'json-litex.so' is neither parsing text, nor it is constructing and validating
internal data structures.

(To be one hundred percent accurate, 'json-litex.so' has to construct certain
data structures even if loading in a compiled path library: all regexes of such
a path library are compiled and stored in one bundle in a serialized form; thus,
when loading in a compiled path library, 'json-litex.so' has to deserialize that
bundle for to construct PCRE2's internal regex structures! It is quite good that
PCRE2's deserialization function is not expensive, modulo it using 'malloc'.)

With the help of 'json-litex.so's action options `-D|--gen-def', it's very easy
to produce compiled path libraries out of text ones:

Extending the illustrating procedure in section 4.d of the 'README' file, let's
say that the user's text path library is named 'jsonrpc-litex.json' and that he
intents to obtain 'jsonrpc-litex.so' in a directory of his choice, which already
hosts 'jsonrpc-litex.json'.

The procedure for obtaining 'jsonrpc-litex.so' is very simple, as follows:

  Step 1: obtain 'jsonrpc-litex.mk' and 'jsonrpc-litex.c'
  -------------------------------------------------------
  Json-Litex provides the generic makefile 'json-litex-module.mk' and the generic
  C source file 'json-litex-module.c' in $JSON_TYPE_HOME/lib. ($JSON_TYPE_HOME is
  the path to the directory were the source tree of Json-Type is located.)

  These files need no manual changes, but only to be copied in the user's path
  library directory:

  $ export JSON_TYPE_HOME=...

  $ cp $JSON_TYPE_HOME/lib/json-litex-module.mk jsonrpc-litex.mk

  $ cp $JSON_TYPE_HOME/lib/json-litex-module.c jsonrpc-litex.c

  Step 2: generate 'jsonrpc-litex.def' out of 'jsonrpc-litex.json'
  ----------------------------------------------------------------
  The file 'jsonrpc-litex.def' will contain C definitions for all the structures
  used by Json-Litex for to work in accordance with the specifications contained
  in 'jsonrpc-litex.json'.

  $ json -F -- json-litex.so -D jsonrpc-litex.json > jsonrpc-litex.def

  Step 3: make 'jsonrpc-litex.so'
  -------------------------------
  Upon having generated 'jsonrpc-litex.def', obtaining 'jsonrpc-litex.so' is a
  matter of simply issuing 'make':

  $ make -f jsonrpc-litex.mk

  Note that $JSON_TYPE_HOME needs to be defined prior the call to 'make'. If it
  is not defined in the current environment, then pass it in explicitly:

  $ make -f jsonrpc-litex.mk JSON_TYPE_HOME=...

If everything went OK -- one should expect to get neither warnings nor errors
out of 'make' --, the shared library 'jsonrpc-litex.so' is ready for its first
use:

  $ ./jsonrpc-litex.so
  jsonrpc-litex.so: litex library version: x.y.z

As seen, the library is executable by itself, producing on output the version
numbers 'x.y.z' of the 'json-litex.so' library that produced it.

Another immediate use of 'jsonrpc-litex.so' would be that of asking Json-Litex
to verify that it accepts the freshly built path library:

  $ json -F -- json-litex.so -V ./jsonrpc-litex.so && echo OK
  OK

Important note: similarly to the case of 'json.so' library and its compiled type
libraries, it is imposed that the pair of major and minor version numbers of a
'json-litex.so' library running a given compiled path library must match exactly
the pair of major and minor version numbers of the 'json-litex.so' library that
made that compiled path library.

  $ json-litex.so
  json-litex.so: version a.b.c ...
  ...

Therefore, for the above 'jsonrpc-litex.so' to match the above 'json-litex.so',
the numbers obtained must be such that 'a' be equal with 'x' and 'b' with 'y'.


5. Querying Github API at the Bash Command Line
===============================================

This section's purpose is to extend the type checking example shown in section
2, Querying Github API at the Bash Command Line, of file 'doc/type-samples.txt',
by adding literal expression contraints checking to the JSON output of Github.

The file 'doc/github-litex.json' is a text path library file which can be used
in tandem with 'doc/github.json' for querying one of Github's web services API.
(This is true for at least today, Wed Sep 12 10:22:08 EEST 2018.)

First, check whether Github's JSON output corresponding to the most recent 100
commits of GCC that this web site is mirroring satisfies the specifications of
type library 'doc/github.json' and of path library 'doc/github-litex.json':

  $ set +o pipefail

  $ wget-gcc-commits() {
    [[ "$1" =~ ^$|^[1-9][0-9]*$ ]] &&
    wget -qO- 'https://api.github.com/repos/gcc-mirror/gcc/commits?per_page=100&page='"${1:-1}"
  }

  $ github-json() {
    json -t doc/github.json:commits "$@" -Vf -- \
    json-litex.so -p doc/github-litex.json:commits
  }

  $ wget-gcc-commits|github-json -O && echo OK
  OK

Note that the path library 'doc/github-litex.json' is defined similarly as the
type library 'doc/github.json': both have their respective specification objects
defined under the name `"commits"'. As a result, the argument of `-p|--path-lib'
specifies `commits' to be the inner path object with which 'json-litex.so' will
validate its input.

Now, see the list of the most recent 100 commit messages obtained using 'json's
action option `-J|--json2':

  $ filter-msgs() {
    sed -nr 's|^\s*$\|^/commit/committer/(name=\|date=)\|^/commit/(mess)age(=)|\1\2\3|p'
  }

  $ wget-gcc-commits|github-json -J|filter-msgs
  name=aldyh
  date=2018-09-12T06:50:34Z
  mess=   * gimple-ssa-warn-alloca.c
  mess=   (alloca_type_and_limit::alloca_type_and_limit): Initialize limit
  mess=   field for ALLOCA_BOUND_*_LARGE.
  mess=
  mess=git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@264227 138bc75d-0d04-0410-961f-82ee72b054a4
  ...


6. Appendix: Json-Litex's Builtins
==================================

The table below lists the builtins functions and constants of Json-Litex, along
with a short description attached. Note that each function has an argument that
is not specified, namely the value of the JSON literal that the encompassing
expression is supposed to be evaluated against. All the builtin functions below
return a boolean value, excepting the function 'len', which produces a number.

    ------------  ------------------  ---------------------------------------
        Name             Type                       Description
    ------------  ------------------  ---------------------------------------
     null          function()          true iff the JSON literal is `null'
    ------------  ------------------  ---------------------------------------
     boolean       function()          true iff the JSON literal is boolean
    ------------  ------------------  ---------------------------------------
     number        function()          true iff the JSON literal is a number
    ------------  ------------------  ---------------------------------------
     string        function()          true iff the JSON literal is a string
    ------------  ------------------  ---------------------------------------
     len           function()          gets the length of the JSON literal
                                       if that is a number or a string;
                                       otherwise produces an error
    ------------  ------------------  ---------------------------------------
     int           function(number)    the argument is allowed to be either
                                       8, 16, 32 or 64; returns true iff
                                       the JSON literal is a number or
                                       a string that represents a decimal
                                       integer of specified bit precision
    ------------  ------------------  ---------------------------------------
     uint          function(number)    same as 'int', but the JSON literal
                                       must represent a decimal unsigned
                                       integer of specified bit precision
    ------------  ------------------  ---------------------------------------
     date          function(string)    the argument is a format string to be
                                       passed on to the C library function
                                       'strptime'; gets true iff 'strptime'
                                       parses successfully and completely
                                       the JSON literal number or string;
                                       an error is produced when the JSON
                                       literal is neither a number nor it
                                       is a string
    ------------  ------------------  ---------------------------------------
     timestamp     string              `%s`
    ------------  ------------------  ---------------------------------------
     long_iso      string              `%Y-%m-%d %H:%M:%S`
    ------------  ------------------  ---------------------------------------
     iso_8601      string              `%Y-%m-%dT%H:%M:%S.%NZ`
    ------------  ------------------  ---------------------------------------
     full_iso      string              `%Y-%m-%d %H:%M:%S.%N %z`
    ------------  ------------------  ---------------------------------------
     c_locale      string              `%a %b %e %H:%M:%S %Z %Y`
    ------------  ------------------  ---------------------------------------

The strings 'timestamp', 'long_iso', 'iso_8601', 'full_iso' and 'c_locale' are
formatting strings that are suitable as arguments to the builtin function 'date'.

Note that the builtin function 'date' augments the C library function 'strptime'
such a way that it may be invoked with an actual argument containing the format
specifiers '%N', '%z' and '%Z' -- even if the actual implementation of 'strptime'
does not support some of these specifiers.


7. Appendix: Using 'json-litex.so'
==================================

Since 'json-litex.so' is a Json-Type filter library in the sense specified by
'doc/filter-libs.txt', it functions either as a filter component in a filter
pipeline:

  $ json -f [-- FILTER_SPEC]... -- json-litex.so [OPTION]... [-- FILTER_SPEC]...

or, may run alone, acting solely on the command line options received:

  $ json -F -- json-litex.so [ACTION|OPTION]... [FILE]

These two modes of operation of 'json-litex.so' are discriminated by 'json'
program's command line options `-f|--filter-libs' and `-F|--filterlib-exec'.

In the latter mode of operation, the 'exec' mode, one immediately useful command
line is that which produces a text about the usage of 'json-litex.so':

  $ json -F -- json-litex.so --help
  usage: json-litex.so [ACTION|OPTION]... [FILE]
  where actions are specified as:
    -V|--validate                check the validity of input (default)
    -P|--print-struct            print out the constructed structure
    -A|--print-attr              print out the constructed structure
                                   along with generated attributes
    -D|--gen-def                 generate the litex def code for the
                                   constructed structure
  the options are:
    -d|--path-def STR            take the input path structure in form of text
    -p|--path-lib FILE[:NAME]      string specified by STR, or in form of text
    -n|--path-name [NAME]          file or compiled library specified by FILE;
                                   the optional NAME specifies which structure
                                   to use in case the def/library is made of a
                                   set of inner structures; if neither options
                                   `-d|--path-def' nor `-p|--path-lib' nor a
                                   standalone FILE argument were given then
                                   take the input path structure from stdin;
                                   note that when the library runs in filter
                                   mode input cannot come from stdin
    -c|--[expr-]compiler NAME    when compiling literal expressions to internal
                                   executable code, use the named expression
                                   compiler: flat or ast; the flat compiler is
                                   faster compared to the other one, while the
                                   ast compiler produces better executable code;
                                   which compiler is used by default depends on
                                   path library's running mode: in filter mode,
                                   that is flat; in exec mode is ast; note that
                                   all literal expressions of a compiled path
                                   library are already compiled: therefore this
                                   option has no influence on running compiled
                                   path libraries
    -l|--unexpect-literals ACT   specify program's behavior when running into
                                   unexpected JSON literals; ACT can be either
                                   'error' for terminating the execution with
                                   error, or 'ignore' for ignoring the literal;
                                   the default action is 'error'; option `-l'
                                   cuts short `--unexpect-literals=ignore'
    -t|--trie-path TYPE          when generating trie lookup functions produce
                                   trie path matching code of specified type:
                                   expanded or function (default)
    -q|--path-equality TYPE      when given `-t|--trie-path=function', do use the
                                   specified type of path equality matching code:
                                   either 'strcmp' for the named library function
                                   or, otherwise, 'builtin' for using a builtin
                                   function (default)
       --sizes-NAME VALUE        assign the named sizes parameter the given value;
                                   use `--help-sizes' option to obtain the list of
                                   all the sizes parameters along with the minimum,
                                   the maximum and the default values and a short
                                   description attached
       --debug-CLASS [LEVEL]     do print some debugging output for the named
                                   class, which can be any of: expr, lib or obj;
                                   the level can be [0-9], 1 being the default
       --no-debug                do not print debugging output at all (default)
       --dump-options            print parsed options and exit
       --help-sizes              print info about the sizes parameters and exit
    -v|--version                 print version numbers and exit
    -?|--help                    display this help info and exit

A 'json-litex.so' command line may contain a standalone `FILE' argument. Such an
argument is equivalent with options `-p|--path-lib=FILE'.

The info options -- these are: `--dump-options', `--help-sizes', `-v|--version'
and `-?|--help' -- and the action options are permitted only when the library
was invoked in 'exec' mode:

  $ json -f -- json-litex.so --help
  json: error: json-litex.so: filter library: info options not allowed \
  when library mode is 'filter'

  $ json -f -- json-litex.so --validate
  json: error: json-litex.so: filter library: action options not allowed \
  when library mode is 'filter'

In 'filter' mode it only makes sense to pass to 'json-litex.so' options that are
influencing its running as a filter. Since 'json-litex.so' expects to get a path
library upon which to act, one is required to specify at least one of the options
`-d|--path-def' or `-p|--path-lib' along with proper option arguments. (If more
than one such option is given, then only the last one counts.)

  $ json -f -- json-litex.so
  json: error: json-litex.so: filter library: input cannot be stdin \
  when library mode is 'filter'

The command above shows that 'json-litex.so' complains about the expected path
library coming from 'stdin'. This is an adequate behaviour, since when running
in 'filter' mode, the 'stdin' file may already be attached to the 'json's input
class that is responsible for reading in the input JSON text.

However, the path library options are not strictly required in 'exec' mode, since
in this mode 'json' is not running any of its input classes at all; consequently,
'json-litex.so' is free to use the 'stdin' file.

The role of the command line options `-l|--unexpect-literals=ACT' is to control
the behavior of 'json-litex.so' when JSON literals of the input JSON text occur
at unexpected locations with respect to the specified path library. These JSON
literals are those that have a path location in the input JSON text which does
not match any path sequence of the input path library.

When `ACT' is 'error', 'json-litex.so' functions precisely as prescribed by the
definition (D) of section 3 above. If `ACT' is 'ignore', then the condition of
definition (D) is relaxed to the extend that any such unexpected JSON literal
is simply ignored. This latter use-case is put to good use when one is needing
to have constraints enforced only on a strict subset of the JSON literals his
JSON texts contain.


8. Appendix: Using 'test-expr'
==============================

The program 'lib/test-expr' was devised as a tool to be employed in testing
Json-Litex's literal expressions outside the elaborated framework of filter
libraries that encloses 'json-litex.so'.

It is a standalone program that depends on PCRE2 shared library and it functions
in a straightforward manner: expect a literal expression, together with a JSON
literal value if it is the case and act in three possible ways:

  * compile the given expression literal, printing out the internal executable
    code generated for that expression;

  * execute the given expression against the given JSON literal value: compile
    the expression and than evaluate it against that JSON literal value;

  * only print out the bytecodes that PCRE2 generated upon compiling the given
    expression -- thus upon compiling all the regexes contained in it.

The command line options of 'test-expr' are shown below. There are three action
options -- which were anticipated by the list above. Also note that the literal
expressions and the input JSON literals may be given as arguments to designated
command line options as well as standalone command line arguments.

  $ alias test-expr=lib/test-expr

  $ test-expr --help
  usage: test-expr [ACTION|OPTION]... [EXPR [LITERAL]]
  where the actions are:
    -C|--compile-expr          compile given literal expression
    -E|--execute-expr          execute given literal expression on given
                                 JSON literal value (default)
    -B|--dump-bytecodes        print out the PCRE2 bytecodes of given
                                 literal expression
  and the options are:
    -e|--expression=EXPR       input literal expression
    -l|--[json-]literal=VALUE  input JSON literal value against which
                                 the input literal expression is evaluated
    -a|--alpha-values=LIST     when executing literal expressions containing
                                 references to alpha builtins (i.e. if using
                                 options `-b|--builtins-space=alpha'), assign
                                 the given values to the one-letter function
                                 builtins; LIST is a comma-separated list of
                                 decimal non-negative integers; LIST defines
                                 the value to be returned by the corresponding
                                 one-letter function: 'a' and 'A' return the
                                 value at position 0, 'b' and 'B' return the
                                 value at position 1, and so on; note that the
                                 two-character functions return 0, N1 is 123,
                                 N2 is 456, S1 is `foo` and S2 is `bar`
    -b|--builtins-space=NAME   when compiling literal expressions containing
                                 references to builtins, use the specified
                                 namespace: either default or alpha; the alpha
                                 builtins are the following:
                                   [a-z]   function: num()
                                   [A-Z]   function: bool()
                                   [ABC]1  function: bool(num)
                                   [AB]2   function: bool(num, num)
                                   A3      function: bool(num, num, num)
                                   N[12]   constant: num
                                   S[12]   constant: str
    -c|--[expr-]compiler=NAME  when compiling literal expressions to internal
                                 executable code, use the named expression
                                 compiler: flat or ast; the flat compiler is
                                 faster compared to the other one, while the
                                 ast compiler produces better executable code;
                                 flat is the default compiler used
    -n|--null-char=CHAR        replace all occurrences of CHAR with NUL chars in
       --no-null-char            the input JSON literal if that is a string
    -r|--[no-]relative-offs    when printing out literal expression code, in case
                                 of jump instructions let offsets be relative, or
                                 otherwise do not (default)
       --sizes-NAME=VALUE      assign the named sizes parameter the given value;
                                 use `--help-sizes' option to obtain the list of
                                 all the sizes parameters along with the minimum,
                                 the maximum and the default values and a short
                                 description attached
    -v|--[no-]verbose          be verbose or not (default not)
       --dump-options          print options and exit
       --help-sizes            print info about the sizes parameters and exit
       --version               print version numbers and exit
    -?|--help                  display this help info and exit

Important note: 'test-expr' does not parse input JSON literal strings according
to JSON's rules. This program regards any text that starts and ends with double
quote characters as a valid JSON string literal.

Since the strings of literal expressions start and end with the character '`',
better not surround literal expressions to be given to 'test-expr' with double
quote characters '"', but with single quote characters '\''. The double quote
characters tell the encompassing shell program to interpret a command line like
the one that follows with a 'date' command substitution implied:

  $ test-expr "`date`" -v
  test-expr: error:1:1: compiler error: unknown builtin
  test-expr: error:1:1: compiler error: Wed Sep 12 12:00:00 EEST 2018
  test-expr: error:1:1: compiler error: ^

The expressions quoted with single quote characters work without surprise:

  $ test-expr '`date`' -C
  0 match_str("date")

  $ test-expr '`date`' '"date"'
  1


References:
-----------

[1] PCRE -- Perl Compatible Regular Expressions
    https://www.pcre.org/


