Vely logo install examples
documentation about

14.2.0 released on Nov 29, 2022

Example: json



DESCRIPTION:


This example demonstrates parsing JSON text. The text parsed contains information about cities. JSON document includes an array of countries, which includes an array of states, each having an array of cities. The JSON data is also not fixed - some array members contain data that others don't. So the example is fairly involved.

Use of UTF8 Unicode data is also shown - some cities have such characters in their names.

JSON text comes from two different kinds of client requests: from an HTML form and also from Javascript/fetch(). The first one uses URL-query POST from a form. The second one demonstrates a request with "application/json" content type and use of request-body to get the HTTP request body.

Each JSON data node is retrieved using built-in hash statements (see new-json).

The JSON parsing demonstrates two ways to do it:

In a nutshell: 5 source files, 160 lines of code; no database; web browser; Nginx; Unix sockets;

Screenshots of application


This is the form where you can enter JSON text to be parsed:

Vely

This is the result of submitting as 'Extract all data', where all data nodes are shown, along with their normalized names (that includes hierarchy and arrays), their values and types:

Vely

The following is the output of pushing 'Extract specific data' button. Specific data nodes are found, taking into account the hierarchy of data, and their names and values displayed:

Vely

You can call Vely code from Javascript fetch() - this is the output from post.html (see Access application... on how to run it):

Vely


Setup prerequisites


Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because it is used in this example, you will need to install Nginx as a web server.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m


Get the source code


The source code is a part of Vely installation. It is a good idea to create a separate source code directory for each application (and you can name it whatever you like). In this case, unpacking the source code will do that for you:

tar xvf $(vv -o)/examples/json.tar.gz
cd json


Setup application


The very first step is to create an application. The application will be named "json", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) json

This will create a new application home (which is "/var/lib/vv/json") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Build application


Use vv utility to make the application:

vv -q


Start your application server


To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 json

This will start 3 daemon processes to serve the incoming requests. You can also start an adaptive server that will increase the number of processes to serve more requests, and gradually reduce the number of processes when they're not needed:

vf json

See vf for more options to help you achieve best performance.

To stop your application server:

vf -m quit json


Setup web server


This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf


Add the following in the "server {}" section ("/json" is the application path (see request_URL) and "json" is your application name):

location /json { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/json/sock/sock; }

Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx


Access application server from the browser


Use the following URL(s) to access your application server from a web client like browser (see request_URL). Use actual IP or web address instead of 127.0.0.1 if different:

#Enter JSON and send for parsing 
http://127.0.0.1/json/json_form

Copy the contents of file cities.json (see under FILES) to a text area in the form.

Run from Javascript via fetch method
You can also test JSON parsing via Javascript/fetch mechanism. First, copy the file to your web server in a separate directory:

sudo mkdir /var/www/html/velytest
sudo cp post.html /var/www/html/velytest

Test it (use your web address instead of 127.0.0.1 if not testing locally):

http://127.0.0.1/velytest/post.html


Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

FILES:


You are now done with the example! What follows are the source files in this project so you can examine how it works:

JSON document (cities.json)


This is the JSON document that you enter in the form for parsing. It contains countries, states and cities along with their population.

{ "country": [
    {
        "name": "USA",
        "state": [
            {
                "name": "Arizona",
                "city": [
                    {
                        "name" : "Phoenix",
                        "population": 5000000
                    } ,
                    {
                        "name" : "Tuscon",
                        "population": 1000000
                    }

                ]
            } ,
            {
                "name": "California",
                "city": [
                    {
                        "name" : "Los Angeles",
                        "population": 19000000
                    },
                    {
                        "name" : "Irvine"
                    }
                ]
            }
        ]
    } ,
    {
        "name": "Mexico",
        "state": [
            {
                "name": "Veracruz",
                "city": [
                    {
                        "name" : "Xalapa-Enríquez",
                        "population": 8000000
                    },
                    {
                        "name" : "C\u00F3rdoba",
                        "population": 220000
                    }
                ]
            } ,
            {
                "name": "Sinaloa",
                "city": [
                    {
                        "name" : "Culiac\u00E1n Rosales",
                        "population": 3000000
                    }
                ]
            }
        ]
    }
    ]
}


Call from Javascript (post.html)


You can call your Vely code from Javascript via fetch(). This HTML file will do that as soon as it loads (no buttons to push), you can of course change it to fit your needs. This also demonstrates the use of POST method with Content-Type of "application/json" to talk to your server-side Vely code.

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Vely + JavaScript/Fetch + POST/PUT/PATCH + JSON</title>
</head>
<body>
    <h1 class="align">Example: Vely + JavaScript/Fetch + POST/PUT/PATCH + JSON</h1>
    <script>
        fetch('/json/json_process',{
            method: 'POST',
            headers: {'content-type': 'application/json'},
            body:  '{ "country": [ \
    {  \
        "name": "USA", \
        "state": [ \
            {  \
                "name": "Arizona", \
                "city": [ \
                    { \
                        "name" : "Phoenix", \
                        "population": "5000000" \
                    } , \
                    { \
                        "name" : "Tuscon", \
                        "population": "1000000" \
                    }  \
 \
                ] \
            } , \
            {  \
                "name": "California", \
                "city": [ \
                    { \
                        "name" : "Los Angeles", \
                        "population": "4000000" \
                    }, \
                    { \
                        "name" : "Irvine" \
                    } \
                ] \
            }  \
        ]  \
    } , \
    {  \
        "name": "Mexico", \
        "state": [ \
            {  \
                "name": "Veracruz", \
                "city": [ \
                    { \
                        "name" : "Xalapa-Enríquez", \
                        "population": "8000000" \
                    }, \
                    { \
                        "name" : "C\u00F3rdoba", \
                        "population": "220000" \
                    } \
                ] \
            } , \
            {  \
                "name": "Sinaloa", \
                "city": [ \
                    { \
                        "name" : "Culiac\u00E1n Rosales", \
                        "population": "3000000" \
                    } \
                ] \
            }  \
        ]  \
    } \
    ] \
}'
        })
        .then((result) => { return result.text(); })
        .then((content) => { document.getElementById("json_output").innerHTML = content; });
    </script>
    <div id='json_output'></div>
</body>
</html>


Enter JSON (json_form.vely)


This is a simple HTML form where you can enter JSON document. Since the code in json_process.vely and json_all.vely parses a list of cities as described, the text to enter is given in cities.json file.

#include "vely.h"

void json_form () {

    out-header default

    @<h2>Enter JSON</h2>

    @<form action="<<p-path>>/json_process" method="POST">

    @    <label for="json_text">JSON text:</label><br>
    @    <textarea name="json_text" rows="8" columns="70"></textarea><br/>

    @    <button type="submit">Extract specific data</button>
    @    <button type="submit" formaction="<<p-path>>/json_all">Extract all data</button>
    @ </form>

}


Parse all JSON data in a loop (json_all.vely)


This parses the document that you may not know the structure of. Each data node is obtained you can examine it in your code.

#include "vely.h"

void json_all() {
    out-header default

    input-param json_text

    // Parse json text and display any error and the position of it
    new-json define json from json_text status define st error-text define etext error-position define epos
    if (st != VV_OKAY) {
        @Could not parse JSON! Error [<<p-out etext>>] at position <<p-num epos>>.
        exit-request
    }

    // Traverse JSON document, node by node, display as a table of all data nodes
    read-json json traverse begin
    @<table border='1'>
    while (1)
    {
        read-json json traverse key define k value define v type define t status define s
        if (s != VV_OKAY) break;
        // Display name, value and type (ignore boolean and type since we don't have them)
        @<tr>
            @<td><<p-out k>></td> <td><<p-out v>></td>
            @<td><<p-out t==VV_JSON_TYPE_NUMBER?"Number":(t==VV_JSON_TYPE_STRING?"String":"Other")>></td>
        @</tr>
    }
    @</table>

}


Parse JSON by looking up specific elements (json_process.vely)


Parse the JSON document. This shows parsing the document that you know a structure of, but does not have a fixed structure, so each element is retrieved based on its normalized name (see read-json).

#include "vely.h"

void json_process() {
    out-header default

    // If JSON data sent via URL-encoded GET or POST
    input-param json_text
    // If JSON data sent in the request body (application/json), use that JSON data 
    request-body json_body
    get-req content-type to define ctype
    if (!strcmp(ctype, "application/json")) json_text=json_body;

    // Parse json text and display any error and the position of it
    new-json define json from json_text status define st error-text define etext error-position define epos
    if (st != VV_OKAY) {
        @Could not parse JSON! Error [<<p-out etext>>] at position <<p-num epos>>.
        exit-request
    }

    @Cities found<hr/>

    num country_count;
    num state_count;
    num city_count;

    // Start displaying a list
    @<ul>
    // Look for countries, states and then cities
    // Data is organized in hashed arrays, for example
    // country[0].state[1].city[0]
    // and each can have sub-nodes, such as
    // country[0].name
    // etc.
    for (country_count = 0; ; country_count++) {

        // First, build key prefix for a country
        (( define json_key_country
        @"country"[<<p-num country_count>>]
        ))

        // Search for a country name
        (( define json_key_country_name
        @<<p-out json_key_country>>."name"
        ))

        // Search for a country name under index [country_count]
        read-json json key json_key_country_name value define country_name  status st
        if (st != VV_OKAY) break;

        // Country found
        @<li>Country: <<p-out country_name>><br/>
        @<ul>

        // Look for states under this country
        for (state_count = 0; ; state_count++) {

            // Build key prefix for a state
            (( define json_key_state
            @<<p-out json_key_country>>."state"[<<p-num state_count>>]
            ))

            // Search for state name
            (( define json_key_state_name
            @<<p-out json_key_state>>."name"
            ))

            // Search for a state name as: country[countr_count].state[state_count]
            read-json json key json_key_state_name value define state_name  status st
            if (st != VV_OKAY) break;

            // State found
            @<li>State: <<p-out state_name>><br/>
            @<ul>

            // Look for cities under state
            for (city_count = 0; ; city_count++) {

                // Build key prefix for city
                (( define json_key_city
                @<<p-out json_key_state>>."city"[<<p-num city_count>>]
                ))

                // Search for city name
                (( define json_key_city_name
                @<<p-out json_key_city>>."name"
                ))

                // Search for a city name as: country[countr_count].state[state_count].city[city_count]
                read-json json key json_key_city_name value define city_name  status st
                if (st != VV_OKAY) break;

                // Found city, get its population by building a key for it
                (( define json_key_city_population
                @<<p-out json_key_city>>."population"
                ))

                // Get city population
                read-json json key json_key_city_population value define city_population  status st
                if (st != VV_OKAY) city_population="unknown";

                // Display city name and its population
                @<li>City:<<p-out city_name>> (<<p-out city_population>>)</li>
            }
            @</ul>
            @</li>
        }
        @</ul>
        @</li>
    }
    @</ul>

}


SEE ALSO:


Examples ( example_cookies   example_create_table   example_docker   example_file_manager   example_form   example_hello_world   example_json   example_multitenant_SaaS   examples   example_sendmail   example_stock   example_utility   example_write_report  )  SEE ALL (documentation)



Copyright (c) 2022 DaSoftver LLC. Vely is a trademark of Dasoftver LLC. The software and information herein are provided "AS IS" and without any warranties or guarantees of any kind. Some icons copyright Paweł Kuna licensed under MIT. Vely elephant logo copyright DaSoftver LLC. This web page is licensed under CC-BY-SA-4.0.