18.4.0 released Sep 25, 2023
|
JSON parsing and searching
This example demonstrates parsing JSON text. The text parsed contains information about cities. JSON document includes an array of countries, which includes an array of states, each having an array of cities. Use of UTF8 Unicode data is also shown - some cities have such characters in their names. JSON text comes from two different kinds of client requests: from an HTML form and also from Javascript/fetch(). This example shows how to deal with a generic JSON document where structure is known but can change, as well as if you don't know its structure and search for what interests you along the way.
In a nutshell: web browser; Nginx; Unix sockets; 5 source files, 139 lines of code.
Screenshots of application
This is the form where you can enter JSON text to be parsed:
This is the result of submitting as 'Extract all data', where all data nodes are shown, along with their normalized names (that includes hierarchy and arrays), their values and types:
The following is the output of pushing 'Extract specific data' button. Specific data nodes are found, taking into account the hierarchy of data, and their names and values displayed:
You can call Vely code from Javascript fetch() - this is the output from post.html (see Access application... on how to run it):
Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.
Because it is used in this example, you will need to install Nginx as a web server.
After installing Vely, turn on syntax highlighting in vim if you're using it:
The source code is a part of Vely installation. It is a good idea to create a separate source code directory for each application (and you can name it whatever you like). In this case, unpacking the source code will do that for you:
tar xvf $(vv -o)/examples/json.tar.gz
cd json
The very first step is to create an application. The application will be named "json", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:
sudo vf -i -u $(whoami) json
This will create a new application home (which is "/var/lib/vv/json") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.
Use vv utility to make the application:
Start your application server
To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):
This will start 3 daemon processes to serve the incoming requests. You can also start an adaptive server that will increase the number of processes to serve more requests, and gradually reduce the number of processes when they're not needed:
See vf for more options to help you achieve best performance.
If you want to stop your application server:
This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.
- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:
sudo vi /etc/nginx/sites-enabled/default
while on Fedora and other systems it might be at:
sudo vi /etc/nginx/nginx.conf
Add the following in the "server {}" section ("/json" is the application path (see request-URL) and "json" is your application name):
location /json { include /etc/nginx/fastcgi_params; fastcgi_pass unix:///var/lib/vv/json/sock/sock; }
- Step 2:
Finally, restart Nginx:
sudo systemctl restart nginx
Note: you must not have any other URL resource that starts with "/json" (such as for example "/json.html" or "/json_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/json", see request-URL.
Access application server from the browser
Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.
# Enter JSON and send for parsing
http://127.0.0.1/json/json_form
Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.
Copy the contents of file cities.json (see under Source files) to a text area in the form.
- Run from Javascript via fetch method
You can also test JSON parsing via Javascript/fetch mechanism. First, copy the file to your web server in a separate directory:
- For Apache or older versions of Nginx:
sudo mkdir /var/www/html/velytest
sudo cp post.html /var/www/html/velytest
Test it (use your web address instead of 127.0.0.1 if not testing locally):
http://127.0.0.1/velytest/post.html
- For newer versions of Nginx:
sudo mkdir /usr/share/nginx/html/velytest
sudo cp post.html /usr/share/nginx/html/velytest
Test it (use your web address instead of 127.0.0.1 if not testing locally):
http://127.0.0.1/velytest/post.html
The following are the source files in this application:
- JSON document (cities.json)
This is the JSON document that you enter in the form for parsing. It contains countries, states and cities along with their population.
{ "country": [
{
"name": "USA",
"state": [
{
"name": "Arizona",
"city": [
{
"name" : "Phoenix",
"population": 5000000
} ,
{
"name" : "Tuscon",
"population": 1000000
}
]
} ,
{
"name": "California",
"city": [
{
"name" : "Los Angeles",
"population": 19000000
},
{
"name" : "Irvine"
}
]
}
]
} ,
{
"name": "Mexico",
"state": [
{
"name": "Veracruz",
"city": [
{
"name" : "Xalapa-Enríquez",
"population": 8000000
},
{
"name" : "C\u00F3rdoba",
"population": 220000
}
]
} ,
{
"name": "Sinaloa",
"city": [
{
"name" : "Culiac\u00E1n Rosales",
"population": 3000000
}
]
}
]
}
]
}
- Call from Javascript (post.html)
You can call your Vely code from Javascript via fetch(). This HTML file will do that as soon as it loads (no buttons to push), you can of course change it to fit your needs. This also demonstrates the use of POST method with Content-Type of "application/json" to talk to your server-side Vely code.
<!DOCTYPE html>
<html lang="en">
<head>
<title>Vely + JavaScript/Fetch + POST/PUT/PATCH + JSON</title>
</head>
<body>
<h1 class="align">Example: Vely + JavaScript/Fetch + POST/PUT/PATCH + JSON</h1>
<script>
fetch('/json/json_process',{
method: 'POST',
headers: {'content-type': 'application/json'},
body: '{ "country": [ \
{ \
"name": "USA", \
"state": [ \
{ \
"name": "Arizona", \
"city": [ \
{ \
"name" : "Phoenix", \
"population": "5000000" \
} , \
{ \
"name" : "Tuscon", \
"population": "1000000" \
} \
\
] \
} , \
{ \
"name": "California", \
"city": [ \
{ \
"name" : "Los Angeles", \
"population": "4000000" \
}, \
{ \
"name" : "Irvine" \
} \
] \
} \
] \
} , \
{ \
"name": "Mexico", \
"state": [ \
{ \
"name": "Veracruz", \
"city": [ \
{ \
"name" : "Xalapa-Enríquez", \
"population": "8000000" \
}, \
{ \
"name" : "C\u00F3rdoba", \
"population": "220000" \
} \
] \
} , \
{ \
"name": "Sinaloa", \
"city": [ \
{ \
"name" : "Culiac\u00E1n Rosales", \
"population": "3000000" \
} \
] \
} \
] \
} \
] \
}'
})
.then((result) => { return result.text(); })
.then((content) => { document.getElementById("json_output").innerHTML = content; });
</script>
<div id='json_output'></div>
</body>
</html>
- Enter JSON (json_form.vely)
This is a simple HTML form where you can enter JSON document. Since the code in json_process.vely and json_all.vely parses a list of cities as described, the text to enter is given in cities.json file.
#include "vely.h"
request-handler /json_form
out-header default
@<h2>Enter JSON</h2>
@<form action="<<p-path>>/json_process" method="POST">
@ <label for="json_text">JSON text:</label><br>
@ <textarea name="json_text" rows="8" columns="70"></textarea><br/>
@ <button type="submit">Extract specific data</button>
@ <button type="submit" formaction="<<p-path>>/json_all">Extract all data</button>
@ </form>
end-request-handler
- Parse all JSON data in a loop (json_all.vely)
This parses the document that you may not know the structure of. Each data node is obtained you can examine it in your code.
#include "vely.h"
request-handler /json-all
out-header default
input-param json_text
new-json define json from json_text status define st error-text define etext error-position define epos
if (st != VV_OKAY) {
@Could not parse JSON! Error [<<p-out etext>>] at position <<p-num epos>>.
exit-request
}
read-json json traverse begin
@<table border='1'>
while (1)
{
read-json json traverse key define k value define v type define t status define s
if (s != VV_OKAY) break;
@<tr>
@<td><<p-out k>></td> <td><<p-out v>></td>
@<td><<p-out t==VV_JSON_TYPE_NUMBER?"Number":(t==VV_JSON_TYPE_STRING?"String":"Other")>></td>
@</tr>
}
@</table>
end-request-handler
- Parse JSON by looking up specific elements (json_process.vely)
Parse the JSON document. This shows parsing the document that you know a structure of, but does not have a fixed structure, so each element is retrieved based on its normalized name (see read-json). Note the usage of "key-list" clause in order to build a key from key fragments, which is especially useful in hierarchical structures; it's also extremely fast because there is no copying of keys.
#include "vely.h"
request-handler /json-process
out-header default
input-param json_text
request-body json_body
get-req content-type to define ctype
if (!strcmp(ctype, "application/json")) json_text=json_body;
new-json define json from json_text status define st error-text define etext error-position define epos
if (st != VV_OKAY) {
@Could not parse JSON! Error [<<p-out etext>>] at position <<p-num epos>>.
exit-request
}
@Cities found<hr/>
num country_count;
num state_count;
num city_count;
@<ul>
010
0
for (country_count = 0; ; country_count++) {
(( define country
@"country"[<<p-num country_count>>]
))
read-json json key-list country, ".\"name\"" value define country_name status st
if (st != VV_OKAY) break;
@<li>Country: <<p-out country_name>><br/>
@<ul>
for (state_count = 0; ; state_count++) {
(( define state
@."state"[<<p-num state_count>>]
))
read-json json key-list country, state, ".\"name\"" value define state_name status st
if (st != VV_OKAY) break;
@<li>State: <<p-out state_name>><br/>
@<ul>
for (city_count = 0; ; city_count++) {
(( define city
@."city"[<<p-num city_count>>]
))
read-json json key-list country, state, city, ".\"name\"" value define city_name status st
if (st != VV_OKAY) break;
read-json json key-list country, state, city, ".\"population\"" value define city_population status st
if (st != VV_OKAY) city_population="unknown";
@<li>City:<<p-out city_name>> (<<p-out city_population>>)</li>
}
@</ul>
@</li>
}
@</ul>
@</li>
}
@</ul>
end-request-handler
Examples
example-client-API
example-cookies
example-create-table
example-distributed-servers
example-docker
example-encryption
example-file-manager
example-form
example-hash-server
example-hello-world
example-how-to-design-application
example-json
example-multitenant-SaaS
example-postgres-transactions
examples
example-sendmail
example-shopping
example-stock
example-uploading-files
example-using-mariadb-mysql
example-utility
example-write-report
See all
documentation
You are free to copy, redistribute and adapt this web page (even commercially), as long as you give credit and provide a link back to this page (dofollow) - see full license at
CC-BY-4.0. Copyright (c) 2019-2023 Dasoftver LLC. Vely and elephant logo are trademarks of Dasoftver LLC. The software and information on this web site are provided "AS IS" and without any warranties or guarantees of any kind.