Copyright (C) 2016, 2017, 2018, 2019, 2020, 2021  Stefan Vargyas

This file is part of Json-Type.

Json-Type is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

Json-Type is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with Json-Type.  If not, see <http://www.gnu.org/licenses/>.

--------------------------------------------------------------------------------

1. Type Definitions
2. Querying Github API at the Bash Command Line


1. Type Definitions
===================

The specifications of the type objects in 'doc/type-spec.txt' are kind of dry.
Therefore a few illustrating type definitions should be more than welcomed.

  * open array of numbers or strings:

    {
        "type": "array",
        "args": {
            "type": "list",
            "args": [
                "number",
                "string"
            ]
        }
    }

  * closed array of one argument of type number or string:

    {
        "type": "array",
        "args": [
            {
                "type": "list",
                "args": [
                    "number",
                    "string"
                ]
            }
        ]
    }

  * open array of numbers or open array of strings:

    {
        "type": "list",
        "args": [
            {
                "type": "array",
                "args": "number"
            },
            {
                "type": "array",
                "args": "string"
            }
        ]
    }

  * an object of three arguments:
    - the 1st argument have its  name `"foo"' and its type `"type"',
    - the 2nd argument must have its name either `"bar"' or `"bav"' and
      its type `"number"',
    - the 3rd argument is of name `"baz"' and of type `"string"':

    {
        "type": "list",
        "args": [
            {
                "type": "object",
                "args": [
                    {
                        "name": "foo",
                        "type": "type"
                    },
                    {
                        "name": "bar",
                        "type": "number"
                    },
                    {
                        "name": "baz",
                        "type": "string"
                    }
                ]
            },
            {
                "type": "object",
                "args": [
                    {
                        "name": "foo",
                        "type": "type"
                    },
                    {
                        "name": "bav",
                        "type": "number"
                    },
                    {
                        "name": "baz",
                        "type": "string"
                    }
                ]
            }
        ]
    }

Here are the answers of 'json' for inputs which are invalid with respect to the
type definition above:

  $ json -OV -d ... <<< '{"foo":{},"zzz":false,"bay":0}'
  json: error: <stdin>:1:11: type check error: invalid argument name: expected "bar" or "bav"
  json: error: <stdin>:1:11: {"foo":{},"zzz":false,"baz":0}
  json: error: <stdin>:1:11:           ^

  $ json -OV -d ... <<< '{"foo":{},"bar":false,"bay":0}'
  json: error: <stdin>:1:17: type check error: type mismatch: expected a value of type `"number"'
  json: error: <stdin>:1:17: {"foo":{},"bar":false,"baz":0}
  json: error: <stdin>:1:17:                 ^

  $ json -OV -d ... <<< '{"foo":{},"bar":false,"bay":0}'
  json: error: <stdin>:1:19: type check error: invalid argument name: expected "baz"
  json: error: <stdin>:1:19: {"foo":{},"bar":1,"bay":0}
  json: error: <stdin>:1:19:                   ^

  $ json -OV -d ... <<< '{"foo":{},"bar":1,"baz":0}'
  json: error: <stdin>:1:25: type check error: type mismatch: expected a value of type `"string"'
  json: error: <stdin>:1:25: {"foo":{},"bar":1,"baz":0}
  json: error: <stdin>:1:25:                         ^

The type definition below specifies any object having at most three arguments,
of which names are `"foo"', `"bar"' and `"baz"' and types are, respectively,
`"boolean"', `"number"' and `"string"', objects of which arguments can be given
in any order, but without repeated names:

    {
        "type": "dict",
        "args": [
            {
                "name": "foo",
                "type": "boolean"
            },
            {
                "name": "bar",
                "type": "number"
            },
            {
                "name": "baz",
                "type": "string"
            }
        ]
    }

With respect to this type definition, below are the answers obtained from 'json'
upon feeding it with a couple of invalid JSON objects:

  $ json -OV -d ... <<< '{"baz":"","zzz":false}'
  json: error: <stdin>:1:11: type check error: invalid argument name: expected "foo" or "bar"
  json: error: <stdin>:1:11: {"baz":"","zzz":false}
  json: error: <stdin>:1:11:           ^

  $ json -OV -d ... <<< '{"baz":"","baz":false}'
  json: error: <stdin>:1:11: type check error: duplicate argument name: "baz"
  json: error: <stdin>:1:11: {"baz":"","baz":false}
  json: error: <stdin>:1:11:           ^

  $ json -OV -d ... <<< '{"baz":"","foo":false,"bar":null}'
  json: error: <stdin>:1:29: type check error: type mismatch: expected a value of type `"number"'
  json: error: <stdin>:1:29: {"baz":"","foo":false,"bar":null}
  json: error: <stdin>:1:29:                             ^

  $ json -OV -d ... <<< '{"baz":"","foo":false,"bar":0,"zzz":null}'
  json: error: <stdin>:1:30: type check error: too many arguments
  json: error: <stdin>:1:30: {"baz":"","foo":false,"bar":0,"zzz":null}
  json: error: <stdin>:1:30:                              ^

The "dict" type objects accepts also the so-called constraining expressions on
the keys of a maching JSON object. Simply put, by specifying an attached `"expr"'
argument of "string" type in the "dict" type, one is able to impose the presence
of needed keys in matching JSON objects:

  $ json-dict() { echo ' 
        {
            "type": "dict",
            "args": [
                {
                    "name": "foo",
                    "type": "boolean"
                },
                {
                    "name": "bar",
                    "type": "number"
                },
                {
                    "name": "baz",
                    "type": "string"
                }
            ],
            "expr": "'"$1"'"
        }'
    }

The three-argument object defined above with no additional constraints:

  $ json -OV -t <(json-dict '0') <<< '{}' && echo OK
  OK

Constrain the input object to have all its specified keys present:

  $ json -OV -t <(json-dict '1') <<< '{}'
  json: error: <stdin>:1:2: type check error: "dict" expression falsified: missing required args: "foo", "bar" and "baz"
  json: error: <stdin>:1:2: {}
  json: error: <stdin>:1:2:  ^

  $ json -OV -t <(json-dict '1') <<< '{"baz":""}'
  json: error: <stdin>:1:10: type check error: "dict" expression falsified: missing required args: "foo" and "bar"
  json: error: <stdin>:1:10: {"baz":""}
  json: error: <stdin>:1:10:          ^

  $ json -OV -t <(json-dict '1') <<< '{"baz":"","foo":false}'
  json: error: <stdin>:1:22: type check error: "dict" expression falsified: missing required args: "bar"
  json: error: <stdin>:1:22: {"baz":"","foo":false}
  json: error: <stdin>:1:22:                      ^

  $ json -OV -t <(json-dict '1') <<< '{"baz":"","foo":false,"bar":0}' && echo OK
  OK

Constrain the input object to have the key 'foo' present:

  $ json -OV -t <(json-dict 'foo') <<< '{}'
  json: error: <stdin>:1:2: type check error: "dict" expression falsified: missing required args: "foo"
  json: error: <stdin>:1:2: {}
  json: error: <stdin>:1:2:  ^

  $ json -OV -t <(json-dict 'foo') <<< '{"baz":""}'
  json: error: <stdin>:1:10: type check error: "dict" expression falsified: missing required args: "foo"
  json: error: <stdin>:1:10: {"baz":""}
  json: error: <stdin>:1:10:          ^

  $ json -OV -t <(json-dict 'foo') <<< '{"foo":false}' && echo OK
  OK

Constrain the input object to have the keys 'foo' and 'bar' present:

  $ json -OV -t <(json-dict 'foo|bar') <<< '{}'
  json: error: <stdin>:1:2: type check error: "dict" expression falsified: missing required args: "foo" and "bar"
  json: error: <stdin>:1:2: {}
  json: error: <stdin>:1:2:  ^

  $ json -OV -t <(json-dict 'foo|bar') <<< '{"baz":""}'
  json: error: <stdin>:1:10: type check error: "dict" expression falsified: missing required args: "foo" and "bar"
  json: error: <stdin>:1:10: {"baz":""}
  json: error: <stdin>:1:10:          ^

  $ json -OV -t <(json-dict 'foo|bar') <<< '{"foo":false}'
  json: error: <stdin>:1:13: type check error: "dict" expression falsified: missing required args: "bar"
  json: error: <stdin>:1:13: {"foo":false}
  json: error: <stdin>:1:13:             ^

  $ json -OV -t <(json-dict 'foo|bar') <<< '{"foo":false,"bar":0}' && echo OK
  OK

Constrain the input object to have all but the key 'baz' be present:

  $ json -OV -t <(json-dict '~baz') <<< '{}'
  json: error: <stdin>:1:2: type check error: "dict" expression falsified: missing required args: "foo" and "bar"
  json: error: <stdin>:1:2: {}
  json: error: <stdin>:1:2:  ^

  $ json -OV -t <(json-dict '~baz') <<< '{"baz":""}'
  json: error: <stdin>:1:10: type check error: "dict" expression falsified: missing required args: "foo" and "bar"
  json: error: <stdin>:1:10: {"baz":""}
  json: error: <stdin>:1:10:          ^

  $ json -OV -t <(json-dict '~baz') <<< '{"foo":false}'
  json: error: <stdin>:1:13: type check error: "dict" expression falsified: missing required args: "bar"
  json: error: <stdin>:1:13: {"foo":false}
  json: error: <stdin>:1:13:             ^

  $ json -OV -t <(json-dict '~baz') <<< '{"bar":0,"foo":false}' && echo OK
  OK

Constrain the input object to have all but the keys 'foo' and 'bar' be present
(equally: have the key 'baz' be present):

  $ json -OV -t <(json-dict '~foo ~bar') <<< '{}'
  json: error: <stdin>:1:2: type check error: "dict" expression falsified: missing required args: "baz"
  json: error: <stdin>:1:2: {}
  json: error: <stdin>:1:2:  ^

  $ json -OV -t <(json-dict '~foo ~bar') <<< '{"foo":false}'
  json: error: <stdin>:1:13: type check error: "dict" expression falsified: missing required args: "baz"
  json: error: <stdin>:1:13: {"foo":false}
  json: error: <stdin>:1:13:             ^

  $ json -OV -t <(json-dict '~foo ~bar') <<< '{"bar":0,"foo":false}'
  json: error: <stdin>:1:21: type check error: "dict" expression falsified: missing required args: "baz"
  json: error: <stdin>:1:21: {"bar":0,"foo":false}
  json: error: <stdin>:1:21:                     ^

  $ json -OV -t <(json-dict '~foo ~bar') <<< '{"baz":""}' && echo OK
  OK

Constrain the input object to have all but the key 'foo' or all but the key 'bar'
be present:

  $ json -OV -t <(json-dict '~foo||~bar') <<< '{}'
  json: error: <stdin>:1:2: type check error: "dict" expression falsified: missing required args: 1st case: "bar" and "baz"; 2nd case: "foo" and "baz"
  json: error: <stdin>:1:2: {}
  json: error: <stdin>:1:2:  ^

  $ json -OV -t <(json-dict '~foo||~bar') <<< '{"foo":false}'
  json: error: <stdin>:1:13: type check error: "dict" expression falsified: missing required args: 1st case: "bar" and "baz"; 2nd case: "baz"
  json: error: <stdin>:1:13: {"foo":false}
  json: error: <stdin>:1:13:             ^

  $ json -OV -t <(json-dict '~foo||~bar') <<< '{"bar":0}'
  json: error: <stdin>:1:9: type check error: "dict" expression falsified: missing required args: 1st case: "baz"; 2nd case: "foo" and "baz"
  json: error: <stdin>:1:9: {"bar":0}
  json: error: <stdin>:1:9:         ^

  $ json -OV -t <(json-dict '~foo||~bar') <<< '{"baz":""}'
  json: error: <stdin>:1:10: type check error: "dict" expression falsified: missing required args: 1st case: "bar"; 2nd case: "foo"
  json: error: <stdin>:1:10: {"baz":""}
  json: error: <stdin>:1:10:          ^

  # satisfying '~foo' alternative:
  $ json -OV -t <(json-dict '~foo||~bar') <<< '{"baz":"","bar":0}' && echo OK
  OK

  # satisfying '~bar' alternative:
  $ json -OV -t <(json-dict '~foo||~bar') <<< '{"baz":"","foo":false}' && echo OK
  OK


2. Querying Github API at the Bash Command Line
===============================================

Nowadays it is quite common that web services provide APIs which can be called
in through and provide output of the form of JSON text.

The file 'doc/github.json' contains an incipient set of type definitions which
can be used for querying one of the web service APIs of 'github'. (This is true
for at least today, Sun Jul  4 16:05:20 EEST 2021.)

For starters, let's type-check the JSON output of the most recent 100 commits
of GCC that 'github' is mirroring:

  $ set +o pipefail

  $ wget-gcc-commits() { [[ "$1" =~ ^$|^[1-9][0-9]*$ ]] && wget -qO- 'https://api.github.com/repos/gcc-mirror/gcc/commits?per_page=100&page='"${1:-1}"; }

  $ wget-gcc-commits|json -t doc/github.json -OV
  json: error: doc/github.json: type lib error: library error: type name not specified

  $ wget-gcc-commits|json -t doc/github.json:commits -OV && echo OK
  OK

The first of the two invocations of 'json' is reported to be incorrect. The type
library 'doc/github.json' is made of an array of "name" objects. Therefore, the
argument to `-t|--type-lib' has to specify the name of the type definition with 
which 'json' validates the input.

The second invocation of 'json' does give the name of the respective definition:
that is 'commits'. Note that instead of using `-t doc/github.json:commits', the
initial command line can be made to work by only adding the option `-ncommits'.

Next, let's see the list of the most recent 100 commit messages obtained using
the action option `-J|--json2':

  $ filter-msgs() { sed -nr 's|^\s*$\|^/commit/committer/(name=\|date=)\|^/commit/(mess)age(=)|\1\2\3|p'; }

  $ wget-gcc-commits|json -t doc/github.json:commits -VJ|filter-msgs
  name=marxin
  date=2016-05-25T09:10:16Z
  mess=Fix PR tree-optimization/71239.
  mess=
  mess=   * g++.dg/pr71239.C: New test.
  mess=   PR tree-optimization/71239
  mess=   * tree.c (array_at_struct_end_p): Do not call operand_equal_p
  mess=   if DECL_SIZE is NULL.
  mess=
  mess=
  mess=git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@236696 138bc75d-0d04-0410-961f-82ee72b054a4
  
  name=rguenth
  date=2016-05-25T08:52:22Z
  mess=2016-05-25  Richard Biener  <rguenther@suse.de>
  mess=
  mess=   * timevar.def (TV_TREE_LOOP_IFCVT): Add.
  mess=   * tree-if-conv.c (pass_data_if_conversion): Use it.
  mess=
  mess=
  mess=git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@236695 138bc75d-0d04-0410-961f-82ee72b054a4

  ...

The action option `-J|--json2' flattens the hierarchical nested structure of the
JSON input to a series of text lines of form `/PATH/NAME=VALUE'. Upon flattening,
the classical Unix power-tools (such as 'awk', 'sed' or 'grep') can conveniently
be used for filtering and transforming further the text obtained from 'json'.


