Single page documentation

19.0.0 released Nov 08, 2023

Vely Single-Page Documentation 19.0.0

This page contains all of Vely documentation topics combined into one. It may be easier to search.

123-hello-world
about-Vely
after-request-handler
application-architecture
application-setup
before-request-handler
begin-transaction
building-URL
call-server
call-web
CGI
close-file
command-line
commit-transaction
connect-apache-tcp-socket
connect-apache-unix-socket
connect-nginx-tcp-socket
connect-nginx-unix-socket
containerize-application
copy-file
copy-string
count-substring
current-row
database-config-file
database-queries
debugging
decode-base64
decode-hex
decode-url
decode-web
decrypt-data
delete-cookie
delete-file
delete-json
delete-mem
delete-query
delete-server
delete-tree
deploying-application
derive-key
diagnostic-messages
documentation
do-once
dot
encode-base64
encode-hex
encode-url
encode-web
encrypt-data
error-code
error-handling
example-client-API
example-cookies
example-create-table
example-develop-web-applications-in-C-programming-language
example-distributed-servers
example-docker
example-encryption
example-file-manager
example-form
example-hash-server
example-hello-world
example-how-to-design-application
example-how-to-use-regex
example-json
example-multitenant-SaaS
example-postgres-transactions
example-sendmail
example-shopping
examples
example-stock
example-uploading-files
example-using-mariadb-mysql
example-using-trees-for-in-memory-queries
example-utility
example-write-report
exec-program
exit-code
exit-request
FastCGI-API
FastCGI-command-line-client
FastCGI
file-position
file-storage
file-uploading
finish-output
flush-output
get-app
get-cookie
get-hash
get-req
get-sys
get-time
getting-URL
get-tree
global-process-data
global-request-data
hash-string
how-vely-works
if-task
inline-code
input-param
json-utf8
license
lock-file
lower-string
manage-memory
match-regex
memory-handling
new-fifo
new-hash
new-json
new-mem
new-server
new-tree
non-request
normalized-URL
num-string
on-error
open-file
OS-conditional-compilation
out-header
output-statement
p-dbl
pf-out
pf-url
pf-web
plain-C-FCGI
p-num
p-out
p-path
prepared-statements
purge-fifo
purge-hash
purge-tree
p-url
p-web
quality-control
query-result
random-crypto
random-string
read-fifo
read-file
read-hash
read-json
read-line
read-server
read-tree
rename-file
rename-files
report-error
request-body
request-handler
request
request-URL
resize-hash
resize-mem
rewind-fifo
rollback-transaction
run-query
SELinux
send-file
set-app
set-cookie
set-input
set-req
silent-header
split-string
startup-handler
statement-APIs
stat-file
syntax-highlighting
task-param
temporary-file
trace-run
trim-string
uniq-file
unlock-file
unused-var
upper-string
use-cursor
utf8-json
vely-architecture
vely-dispatch-request
vely-removal
vely-version
vf
vv
write-fifo
write-file
write-hash
write-string
write-tree

123 hello world

This is a condensed version of Vely Hello World that you can run in minutes. No additional software needs to be installed. For more, see examples.

Step 1: Install Vely

First install Vely.

Step 2: Build it

Create Hello World source file (hello.vely) in a new directory; note it's all one bash command:

echo '#include "vely.h"

request-handler /hello
    out-header default
    @Hello World!
end-request-handler' > hello.vely

Create Hello World application:

sudo vf -i -u $(whoami) helloworld

Make Hello World application:

vv -q

Step 3: Run it

You can run Hello World both as a service and from command line:

As a service, first start your Hello World FastCGI application server:

vf helloworld

then connect to it:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export REQUEST_METHOD=GET
export SCRIPT_NAME="/helloworld"
export PATH_INFO="/hello"
cgi-fcgi -connect /var/lib/vv/helloworld/sock/sock  /

From command line:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export REQUEST_METHOD=GET
export SCRIPT_NAME="/helloworld"
export PATH_INFO="/hello"
/var/lib/vv/bld/helloworld/helloworld

Expected result

The end result looks like this (notice in bold the responses from your web application and the command line):

You have run Hello World with Vely, first by talking directly to a FastCGI server and then from command line. See FastCGI-command-line-client for a similar example that uses TCP sockets.

See also

Quick start
123-hello-world
See all
documentation

About Vely

What is Vely

Vely is a general-purpose framework for rapid development of high-performance software. It is especially well suited for web applications. It's Free Open Source (under the business-friendly Eclipse Public License 2 (EPL-2)).

Vely is declarative and functional, with single-line statements performing entire tasks. It's simple to design, write and maintain applications.

Decades of adding energy-intensive abstractions on top of programming languages led to increase in complexity and decrease in performance. Vely applications are 100% native, high-performance and low-footprint without interpreters, virtual machines, or byte-code schemes.

What's it for

Vely is great for web applications, command-line programs, cloud applications, middleware, distributed systems, database applications, IOT or anything else. Create and manage application servers as quickly as command-line programs.

Vely supports querying databases, file manipulation, network, string manipulation, outputting data, encryption, JSON, REST, distributed computing, time, memory structures like hash and FIFO, program execution, regex, memory management, SSL/TSL, encoding/decoding, error handling, web servers, request handling, daemonizing, web development like cookies, input parameters, uploading and downloading of files, URL parameter parsing etc. In short, lots of very common and useful tasks you need all the time.

Intuitive and practical

Vely statements are easy to read and write, designed to immediately give you a clear idea about what they do, even if you've never seen Vely code. They are more like a natural language than typical programming code. This is important not just if you're starting with Vely, but also for maintenance, when someone works with the code years later. You write Vely statements inside skeleton C code so there is no need to learn anything new about the layer underneath; C is quite simple and well-known.

The scope of Vely statements is typically narrow and the generated code is shallow and direct, similar to what an experienced C programmer would write, incurring virtually no loss of performance. The arguments are specified in any order by naming their purpose, which is important for teams where readability is of importance.

How it works

Vely statements are declarative, descriptive and short, designed with productivity in mind. They are precompiled into C code and then compiled and linked, resulting in a native executable. You don't have to be a C expert because Vely writes C code for you, though you can write as much (or as little) of your own code in C as you like.

Read how Vely works, vely architecture, application architecture, and see examples.

Vely uses well-known standard libraries like cURL, OpenSSL, crypto, FastCGI, PCRE2, native database libraries from MariaDB, PostgreSQL, SQLite, for compliance, performance and reliability. Use simple API to connect to a Vely application server from elsewhere, and use existing libraries in Vely applications.

Performance matters

Many other back-end languages and frameworks are running as a virtual machine, interpreter or some other form of abstraction, or indirect execution. These layers of abstraction by far don't have the best performance, and are energy-inefficient, costing more electricity, water and computing equipment that relies on rare metals, ultimately affecting the environment in a negative way. And sluggish software never makes for good customer experience.

Performance is very important in the Cloud, where smaller and faster means less CPU seconds, less RAM, less money spent, less energy used and less emissions. In addition, there's a bill shock, with cloud costs ranking second right after payroll. Wouldn't it be great to reduce this cost from the ground up, improve customer experience, and help the environment at the same time?.

Safety

When you use Vely statements, you do not need to allocate/free memory, or worry about buffer overruns or memory violations; you also get automatic memory freeing and automatic file closing. Vely is safer than pure C due to its memory handling.

After each request, Vely automatically releases any memory used by it. This makes programming easier and safer, and your application more stable and faster. You can also use unmanaged memory, which is the classic malloc/free.

Aside from memory handling, Vely statements perform many syntax and semantic checks to make sure your thoughts get represented with code as close and as safe as possible. These checks are virtually always done during compile-time and do not affect performance.

Why generate C

C is the shortest route to maximum performance and the smallest memory footprint. C is simple. It also allows usage of virtually any library in existence. And Vely statements are carefully crafted with the goal of generating fast and safe C code on their own. Vely itself is written in C.

Vely's goal is not to write more C code, quite the opposite. By using Vely anywhere possible, C code can be used as a supporting mechanism for Vely statements, such as declaring variables, conditional statements, program flow and usage of external libraries. This means the important and difficult parts are done by single-line Vely statements.

Moore's law and C

In the past several decades, less-efficient languages and frameworks have proliferated because of hardware advances, even to the point of negating those benefits (see this). In other words, there's lots of bloated and slow languages and frameworks out there, relying on layer upon layer of abstractions to work.

And while Moore's law may or may not be failing, it may take significantly more time and funding to keep these hardware advances going, and at some point it may no longer. C can help put any hardware to more efficient use.

The reason why you wouldn't have used C in the past may have been buffer overwrites, memory management and low-level coding. Vely changes that by turning C into a rapid software development platform that's safer and easier.

Quality

Each Vely release must pass rigorous continuous tests on every platform where it's available before it's released. There are currently 2186 such tests, see quality-control. See release-notes for what's new in each release.

Report bugs, request features, contact

Contact at vely@vely.dev - send questions, bugs or suggest new features.

Download and install

Download and install Vely here from pre-built packages with apt, dnf, zypper or pacman; or install from source.

Author

Vely's author is Sergio Mijatovic (LinkedIn, Twitter, dev.to, fosstodon, GitHub); follow Vely updates on any of these social networks.

See also

General
about-Vely
application-architecture
deploying-application
how-vely-works
quality-control
rename-files
SELinux
vely-architecture
vely-removal
vely-version
vf
vv
See all
documentation

After request handler

Purpose: Execute your code after a request is handled.

void _after ( ) ...

Every Vely request goes through a request dispatcher (i.e. vely-dispatch-request() function). In order to specify your code to execute after a request is handled, create a source file "_after.vely" and implement a function "void _after()", which will be automatically picked up and compiled with your application.

If no request executes (for example if your application does not handle a given request), after-request handler does not execute either. If you use exit-request to exit current request handling, after-request handler still executes.

Examples

Here is a simple implementation of after-request handler that just outputs "Hi there!!":

#include <vely.h>

void _after()
{
     @Hi there!!
}

See also

Requests
after-request-handler
before-request-handler
building-URL
getting-URL
global-request-data
non-request
normalized-URL
request
request-handler
request-URL
startup-handler
vely-dispatch-request
See all
documentation

Application architecture

Application, request, source code

An "application" (or a "program") is a single executable program created by vv utility; it can process requests. This executable can either run as:

a server (or "daemon"), which means as any number of processes staying resident in memory, either permanently or based on a workload (see FastCGI),
as a program that runs from a command line, a script etc. (see command-line).

The source code for an application is contained in a flat directory. Each request handler is represented by a namesake .vely file. For example, a request "customer" is always entirely contained in file "customer.vely". Typically, requests handled in an application are connected in some way that makes it advantageous to group them together, be it logically, via common dependencies in code, because of reliance on common infrastructure (such as a database for instance), based on performance etc.

An application contains all request handlers in it, and so can handle any request. Thus, when it runs as a server, any of its processes can handle any request that an application serves. See vely-dispatch-request on how a request is served within an application.

Project

A "project" is a set of applications that are related in some way; the relationship between them may be purely logical, i.e. only in terms of some kind of business, functional or other commonality. Your project can start being a single program. It may stay that way forever, or it may be split into multiple applications. A split is made much easier by the fact that each request is a single .vely file.

Splitting an application

Separation of a single application into multiples may be as simple as placing its .vely files into separate directories, creating new applications with vf and building with vv. Note that you may have non-request source files implementing code that is used in more than one place, i.e. shared among request handlers. Such files can stay in one program's source directory, and other programs may simply use those files as soft-linked (using Linux's ln). Access to basic services, such as databases, files or a network, does not change with such a separation, assuming shared infrastructure.

In this scenario, each application will now have its own application path (see request-URL); thus, if any of the newly created applications build URLs that would point to another application, they must be changed. Whether your application is micro, mini or a macro service (macro service being a "monolith"), or a combination thereof, it isn't immutable, and can change over time.

Mode of execution

The request's input and output are the same regardless of the program's mode of execution, i.e. whether a program runs as an application server or from command line (or both); a request is always served as an HTTP request; in addition, a command-line program can suppress HTTP header output. An application may need both modes of execution for different aspects of its functioning. For example, much of the web interface would run as application server(s), and perhaps data conversion and periodic cron jobs would be better served by command-line programs. In many cases, the same code may serve both, such as when the same tasks are performed as a batch job and as a web request.

Testability

Requests are always valid HTTP requests, and operate as such. This makes testing/mocking easier regardless of the mode of execution (i.e. application server or command line). Testing can be performed via:

executing a program on a command-line,
or to test server execution, by using FastCGI-command-line-client, cURL utility, or with a FastCGI-API for any programming language that has C linkage (virtually all do).

For example, REST interface can be easily tested by using a command-line program, without the need for a web interface.

Databases

Access to databases is provided via statements like run-query, which are independent of the data source; different databases can be swapped without changing the code, even between different vendors (save for any differences in SQL dialects). A separate data model (i.e. data abstraction over actual queries) may or may not be needed; here are some reasons why you can go without it for simpler design/development and better maintainability:

run-query is designed to provide a readable interface (i.e. with input and output data) in single statement, thus playing a role of a data-accessor. Query text may be used directly, or from a another source (i.e. an array of queries, defined elsewhere, obtained from a function etc.).
Too much data abstraction may result in a disconnect between application logic and data when it comes to readability of the design and the code, while the abstraction itself may present a significant overhead.
It may be beneficial for queries to convey application logic and be close to their functional place of use in order to maximize performance and maintainability.
Using the same database structure directly in development and testing reduces side effects that affect performance and functionality often seen with substitute data sources.

Session management, stickyness

A "session" is any data connected to a particular end-user who is communicating with application(s). An end-user would login to your application(s) and during such a session any data exchanged would be:

secure, i.e. no other end-user could eavesdrop or alter the data exchanged, and
relevant and separate from other end-users.

Vely application servers run as a separate layer, i.e separately from web server(s) for performance, safety, scalability and usability; they can be accessed in a number of ways, with web servers being just one. Session information should never be kept in any particular web server instance or application process; rather it resides in a database layer, which can be:

a database like MariaDB or PostgreSQL,
a caching solution (like Redis),
Vely application using a database (persistent, in-memory such as SQLite, etc.),
or other high performance data stores.

This makes application design easier and more robust from the start, because it allows for proper session store that scales without having to worry about "sticking" to a particular end-point web server or process. Rather, stickyness is achieved by keeping the session information in a data layer; such session information can then be accessed from any process of any application by simply querying it.

For performance considerations, typically there are three components to a database design in regards to session management:

Credentials data,
Session data, and
User data

The minimal information you'd need for any kind of session management scenario to work is:

User ID - a unique identifier assigned to each end-user.
Session ID - a unique session identifier assigned to each session of each end-user.

User ID would be obtained during login (based on credentials such as user name and password for instance), in order to grant access to application(s); this is what Credentials data is for, and during such login a Session ID is created.

The subsequent requests from a logged-on user would be based on both User ID and Session ID, which would be used to verify that end-user has the permission to access data, and which is initially provided to the end-user and then passed back to application(s) via secure cookies. After that, User ID is used to perform the actual requests, while Session ID is used to update the Session data with whatever information application(s) require. All such data manipulations are performed via queries (not necessarily SQL queries, though it may be the most common kind).

With this architecture, the stickyness of an end-user's session is achieved regardless of which web server(s), application(s) or application's processes are handling the request; any architecture may want to prioritize this independence from the underlying infrastructure and physical implementation. In addition, for better performance and scaling up, the Credentials, Session and User data can be separated. In most cases, all three of those are contained in a single database (see multitenant SaaS example of this). When you need to scale up, you can separate Credentials and Session data, as well as User data (i.e. transactional database that contains the actual useful end-user's data) in their own physical databases.

Note that this kind of separation can be across different CPUs on a same server, or on different servers connected to a high-speed local network, or some other form of separation. You may also choose to have session data in-memory only to speed up updates and queries - this decision is about business requirements and allowable risks in case such database needs to be restarted for whatever reason. If better reliability is needed (in case database(s) go down), a high-availability database solution may be used, like clustering/mirroring/failover etc. The strategy used should generally avoid process synchronization on an application or caching level, as it tends to eventually slow down the application and grow in complexity.

These concepts are shown here:

Services (local and remote) in a functional and declarative model

Vely is functional and declarative, with basic services provided to you via statements; that's their major purpose. For example, data sanitation and database access including connection handling, input, output, distributed computing, files, encryption, pattern matching and other such basic components are built-in - these are provided by Vely statements. You can write non-request code to create higher-value components of your own that can be shared between requests in the same program, or even between different programs.

A request can be viewed as an HTTP function, where input is provided and output is made available via HTTP protocol to the caller, though this is decoupled from the web (and network in general): it can function as a web service or a command-line program execution. When running on a server, a request is suitable for a wide variety of methodologies (REST-like, generic Remote-Call type of processing, etc.) by calling it with:

call-server, for accessing remote services via fast and direct binary protocol on secure networks, or
call-web, for using remote services on the web with secure SSL/TSL connection.

Tasks

A request may service either a singular purpose (such as with a microservice) or be divided into tasks (and then subtasks). Tasks (or subtasks) do not have to be exlusively separated in terms of functionality, i.e. differents tasks may perform overlapping functions via shared code. Any input parameter (see task-param) can indicate which task a request should perform. Whatever services your application provides, each such service can be identified with:

a request and
an optional task within a request.
an optional subtask within a task, etc.

For an example of tasks and subtasks, see if-task. Your application should not have (sub)tasks that are too many or too deep. Often, the best solution may be for a request to perform a single task (i.e. it would not need a task-param). Tasks should generally serve a request's purpose and such purpose should be as elementary as possible. The delineation of where "elementary" ends isn't a hard rule; rather it's best to remember that a request should be a logical action that (for whatever reason) is not conducive to further simplification by dividing it.

Interpreting and handling task(s) in as little code as close together as possible is preferrable. This means if determining which task to execute and actually executing its functionality is possible without any additional layers of abstraction, it is likely to be more readable and easier to maintain.

In general, the execution flow of a program is:

Within a request, if tasks are used, the support for tasks is semantic and self-documenting:

Without using tasks, you would likely have request paths such as "/customer/add" or "/customer/update" served by source files "customer__add.vely" or "customer__update.vely" (see request-URL). It is up to you to decide which method better serves your purpose: using tasks within a request, or using separate requests each handling a single task. In general you might want to keep the number of tasks per requests small, likely no more than 3 or 4; if you have more, it may be better to split them into separate requests. However, depending on your application design, this isn's a hard rule: your application logic, and requirements about how it is designed and maintained may override such guidelines.

Request design

To begin with, the application design should start with a question: what are the requests my application will process? The answer to this question may be known in advance for small applications. For medium-sized or large applications, the answer is a process in itself, typically revealed during the prototyping and the lifecycle of application's design and use.

There are a few important considerations to take into account when deciding what a request is, i.e. what is its purpose and its input and output, and how to write it:

Its purpose: understanding any application is easier to design and implement if its components are chosen to logically represent the way end-users consume it, such as roles and functionality on one hand, and resources on the other. "End-users" doesn't necessarily mean humans; many applications's end-users are other machines, be it servers in a multi-layered design, API endpoints in user-interfacing devices (such as browsers), end-point consumer electronics (such as thermostats or cars), etc.
Its scope: a request should be the simplest that serves a purpose you had in mind. Dividing it into simpler ones just for the sake of division isn't a good strategy, just as "packing" a request with more functionality than it warrants isn't a good strategy either.
Performance: creating too many layers of an "onion" or too many layers of abstraction may lower performance to unacceptable levels. In addition, since request names are constants, Vely optimizes request dispatching to the point where the cost of such dispatching (within a process) is constant, regardless of the number of requests (i.e. number of .vely files) your application serves. Don't be afraid to have as many requests as it makes sense to you.
Ease of understanding: future maintainers of your applications may not appreciate too many layers of abstraction either.
Sharing and extensibility: is a request standalone, without shared code or data source dependencies (meaning shared with other requests)? Sharing of code can be local via non-requests, or remote via call-server; it may be better to start with local sharing (i.e. non-request code) and use remote services when necessary, especially if horizontal scaling is needed (i.e. adding more computer instances to handle the load). For example, two or more requests can overlap in functionality: you may have a request that creates an entity, and a request that updates it. You can also have a request that creates and updates an entity at the same time to avoid performance issues of multiple calls. A non-request code would be used to reuse the functionality across such requests without duplicating code.

See also

General
about-Vely
application-architecture
deploying-application
how-vely-works
quality-control
rename-files
SELinux
vely-architecture
vely-removal
vely-version
vf
vv
See all
documentation

Application setup

Initialize application

A Vely application must be initialized first. This means creating a directory structure owned by application owner, which can be any Operating System user. To initialize application <app name> while logged-in as application owner:

sudo vf -i -u $(whoami) <app name>

Setup database(s)

If your application does not use database(s), you can skip this part.

You can setup your database(s) in any way you see fit, and this includes creating the database objects (such as tables or indexes) used by your application; all Vely needs to know is the connection parameters, which include database login information (but can include other things as well). For each database in use, you must provide database-config-file along with your source code. This file contains the database connection parameters - these parameters are database-specific. For example, if your code has statements like:

run-query @mydb = ...

or

begin transaction @sales_db

then you must have files "mydb" and "sales_db" present. For example, MariaDB config file might look like:

[client]
user=velyuser
password=pwd
database=velydb
protocol=TCP
host=127.0.0.1
port=3306

or for PostgreSQL:

user=myuser password=mypwd dbname=mydb

Make application

To compile and link the application that doesn't use database(s):

vv -q

When you have database(s) in use, for instance assuming in above example that "mydb" is MariaDB database, "sales_db" is PostgreSQL, and "contacts" is SQLite database:

vv -q --db="mariadb:mydb postgres:sales_db sqlite:contacts"

See vv for more options.

Start application

Stop the application first in case it was running, then start the application:

vf -m quit <app name>
vf <app name>

See vf for more details.

Running application

You can run your application as FastCGI, CGI or command-line.

See also

Running application
application-setup
CGI
command-line
containerize-application
FastCGI
plain-C-FCGI
See all
documentation

Before request handler

Purpose: Execute your code before a request is handled.

void _before ( ) ...

Every Vely request goes through a request dispatcher (i.e. vely-dispatch-request() function), which is auto-generated. In order to specify your code to execute before a request is handled, create a source file "_before.vely" and implement a function "void _before()", which will be automatically picked up and compiled with your application.

If no request executes (for example if your application does not handle a given request), before-request handler does not execute either.

Examples

Here is a simple implementation of before-request trigger that just outputs "Getting Started!!":

#include <vely.h>

void _before()
{
    out-header default
    @Getting Started!!
}

See also

Begin transaction

Purpose: Begins database transaction.

begin-transaction [ <options> ] [ @<database> ] [ on-error-continue | on-error-exit ] [ error [ define ] <error> ] [ error-text [ define ] <error text> ]

This statement begins a database transaction. <options> is any additional options to database's BEGIN/START you wish to supply and must immediately follow begin-transaction.

Once you start a transaction with begin-transaction, you must either commit it with commit-transaction or rollback with rollback-transaction. If you do neither, your transaction will be rolled back once the request has completed and your program will stop with an error message. This is because opening a transaction and leaving without committing or a rollback is a bug in your program.

You must use begin-transaction, commit-transaction and rollback-transaction instead of calling the BEGIN/COMMIT/END through run-query.

Database

Optional <database> is specified in "@" clause and is the name of the database-config-file.

Error handling

The error code is available in <error> variable in optional "error" clause - this code is always "0" if successful. The <error> code may or may not be a number but is always returned as a string value. <error> is allocated memory. In case of error, error text is available in optional "error-text" clause in <error text>, which is allocated memory.

"on-error-continue" clause specifies that request processing will continue in case of error, whereas "on-error-exit" clause specifies that it will exit. This setting overrides database-level on-error for this specific statement only. If you use "on-error-continue", be sure to check the error code.

<error> and <error text> can be created with optional "define".

Note that if database connection was lost, and could not be reestablished, the request will error out (see error-handling).

Examples

begin-transaction @mydb
run-query @mydb="insert into employee (name, dateOfHire) values ('%s', now())" : "Terry" no-loop
commit-transaction @mydb

See also

Database
begin-transaction
commit-transaction
current-row
database-config-file
database-queries
delete-query
on-error
prepared-statements
query-result
rollback-transaction
run-query
See all
documentation

Building URL

Building absolute URLs

Use p-path to create the absolute URL path to refer back to your application so you can issue requests to it.

For example, this is a link that specifies request "show-notes":

@<a href="<<p-path>>/show-notes?date=yesterday">Show Notes</a>

If you are building HTML forms, you can add a note with:

@<form action="<<p-path>>/add-note" method="POST">
@<input type="text" name="note" value="">
@</form>

See request-URL for more on URL structure.

See also

Call server

Purpose: Make a remote server request.

call-server ( <server> [ ,... ] ) | \
                ( <server> [ array-count <array count> ] ) \
    [ status [ define ] <status> ]  \
    [ started [ define ] <started> ] \
    [ finished-okay [ define ] <finished okay> ]

call-server will make FastCGI call(s) as described in a single <server>, a list of <server>s, or an array of <server>s. Unless only a single <server> is specified, each call will execute in parallel with others (as multiple threads).

<server> is a pointer to a variable of type "vv_fc". This variable must have been created with new-server statement.

A <server> call is made to a remote server. "Remote server" means a process accepting requests that is not the same process executing call-server; it may be running on the same or a different computer, or it may be a different process started by the very same application.

- Multiple server calls in parallel
Executing multiple <server> calls in parallel is possible in two ways:

Specify a list of <server>s separated by a comma, or

Specify an array <server> variable containing <array count> elements (specified with "array-count" clause). Each element of <server> array must be a pointer to a variable of type "vv_fc"; each such variable must have been created by new-server. This is convenient if you have a great many parallel server calls to make (for example dozens, hundreds or even thousands). <server> array must be indexed from 0. Here's an example, where an array of server calls ("srv_arr") is created via new-server in a loop, and executed with a single call-server, then the results gathered via read-server - this is the same example as in new-server with the exception that it's using an array of <server> variables:

#include "vely.h"
void srv() {
    out-header default
    char *upay[3]; // list of URL payloads
    vv_fc *srv_arr[3]; // array of servers
    num i;
    for (i = 0; i < 3; i++) {
        // build URL payload string
        (( upay[i]
        @/op/add/key/key_<<p-num i>>/data/data_<<p-num i>>"
        ))
        // Create resources (prepare) for 3 server calls
        new-server srv_arr[i] location "/var/lib/vv/hash/sock/sock" \
            method "GET" app-path "/hash" request-path "/server" \
            url-payload upay[i]
    }
    // Make 3 server calls in parallel, each in it own thread
    call-server srv_arr array-count 3 status define st \
        started define start \
        finished-okay define fok
    // Check if all calls were completed
    if (st == VV_OKAY) {
        @No errors from call-server
    }
    if (start == 3) {
        @All three server calls started.
    }
    if (fok == 3) {
        @All three server calls finished.
    }
    // Display output data from server calls
    for (i = 0; i < 3; i++) {
        read-server srv_arr[i] data define rdata
        p-out rdata
        @
    }
}

There is no limit on how many <server>s you can call at the same time; it is limited only by the underlying Operating System resources, such as threads/processes and sockets.

- Call status
Optional <status> (in "status" clause) will be VV_OKAY if all <server> calls have each returned VV_OKAY; this means all have started and all have finished with a valid message from the server; or VV_ERR_FAILED if at least one did not (for example if the server could not be contacted, if there was a network error etc.). Note that VV_OKAY does not mean that the reply is considered a success in any logical sense; only that the request was made and a reply was received according to the server protocol.

- Request(s) status
Note that the actual application status for each <server>, as well as data returned and any application errors can be obtained via "request-status", "data"/"data-length" and "error"/"error-length" clauses of read-server statement, respectively.

- Request(s) duration
call-server will wait for all <server> requests to finish. For that reason, it is a good idea to specify "timeout" clause in new-server for each <server> used, in order to limit the time you would wait. Use read-server to detect a timeout, in which case "request-status" clause would produce VV_FC_ERR_TIMEOUT.

- How many calls started and finished
Optional <started> (in "started" clause) will be the number of server calls that have started. Optional <finished okay> (in "finished-okay" clause) is the number of calls that have finished with return value of VV_OKAY as described above. By using <status>, <started> and <finished okay> you may surmise whether the results of call-server meet your expectations.

<status>, <started> and <finished okay> variables can be created with optional "define".

- Performance, security
call-server is faster than call-web because it does not use HTTP protocol in addition to FastCGI; rather it only uses small and binary FastCGI protocol, which is extremenly fast, especially when using Unix sockets on the same machine (see new-server). Note that FastCGI protocol does not have any inherent security built-in; that is part of the reason why it is fast. As such, it is very well suited for remote server calls on the same machine or between networked machines on a secure network.

Examples

This example will connect to local Unix socket file "/var/lib/vv/app_name/sock/sock" (a Vely application named "app_name"), and make a request named "server" (i.e. it will be processed by source code file "server.vely") with URL path of "/op/add/key/2" (meaning with input parameters "op=add" and "key=2"). Then, server reply is read and displayed.

out-header default
// Create single call
new-server define srv location "/var/lib/vv/app_name/sock/sock" \
    method "GET" app-path "/app_name" request-path "/server" \
    url-payload "/op/add/key/2"
// Call single server call
call-server srv finished-okay define sfok
// Get results of a remote server call
read-server srv data define rdata
// Display results
@Data from server is <<p-out rdata>>

If you are connecting to a server via TCP (and not with a Unix socket like in the example above), the "location" clause in new-server might be:

new-server define srv location "192.168.0.28:2400" \
    method "GET" app-path "/app_name" request-path "/server" \
    url-payload "/op/add/key/2"

In this case, you are connecting to another server (running on IP "192.168.0.28") on port 2400. See vf on how to start a server that listens on a TCP port. You would likely use TCP connectivity only if a server you're connecting to is on a different computer.

See also new-server.

See also

Distributed computing
call-server
delete-server
new-server
read-server
See all
documentation

Call web

Purpose: Get content of URL resource (call a web address).

call-web <URL> \
    response [ define ] <result> \
    [ response-code [ define ] <response code> ] \
    [ response-headers [ define ] <headers> ] \
    [ status [ define ] <status> ] \
    [ method <request method> ] \
    [ request-headers \
        [ content-type <content type> ] \
        [ content-length <content length> ] \
        custom <header name>=<header value> [ , ... ] ] \
    [ request-body \
        ( [ fields <field name>=<field value> [ , ... ] ] \
            [ files <file name>=<file location> [ , ... ] ] ) \
        | \
        ( content <body content> ) \
    ] \
    [ error [ define ] <error> ] \
    [ cert <certificate> | no-cert ] \
    [ cookie-jar <cookie jar> ] \
    [ timeout <timeout> ]

With call-web, you can get the content of any accessible URL resource, for example web page, image, PDF document, XML document, REST API etc. It allows you to programmatically download URL's content, including the header. For instance, you might want to obtain (i.e. download) the source code of a web page and its HTTP headers. You can then save such downloaded items into files, analyze them, or do anything else.

<URL> is the resource locator, for example "https://some.web.page.com" or if you are downloading an image (for instance) it could be "https://web.page.com/image.jpg". Anything you can access from a client (such as web browser), you can also obtain programmatically. You can specify any URL parameters, for example "https://some.web.page.com?par1=val1&par2=val2".

Response and headers

The result is obtained via "response" clause into variable <result>, and the length (in bytes) of such response is obtained via "status" clause in <status> variable. <result> is allocated memory.

The response code (such as 200 for "OK", 404 for "Not Found" etc.) is available via optional "response-code" clause in number variable <response code>; the default value is 0 if response code is unavailable (due to error for instance).

The optional "response-headers" clause allows for retrieval of response headers (such as HTTP headers) in <headers> variable, as a single string variable. <headers> is allocated memory.

Each of <result>, <response code>, <headers> and <status> can be created with the corresponding optional "define".

Request method

You can specify the request method using "method" clause. <method> has a string value of the request method, such as "GET", "POST", "PUT", "PATCH", "DELETE" or any other.

Status

In case of error, <status> is negative, and has value of VV_ERR_FAILED (typically indicating system issue, such as lack of memory, library or system issue or local permissions), VV_ERR_WEB_CALL (error in accessing URL or obtaining data) - otherwise <status> is the length in bytes of the response (0 or positive). Optionally, you can obtain the error message (if any) via "error" clause in <error> variable (which can also be created with optional "define"). Error is an empty string ("") if there is no error. <error> is allocated memory.

Timeout

If "timeout" clause is specified, call-web will timeout if operation has not completed within <timeout> seconds. If this clause is not specified, the default timeout is 120 seconds. If timeout occurs, <status> will be VV_ERR_WEB_CALL and <error> will indicate timeout. Timeout cannot be negative nor greater than 86400 seconds.

HTTPS and certificates

You can call any valid URL that uses protocol supported by the underlying library (cURL). If you're using https protocol (or any other that requires a SSL/TSL certificate), you can either use the local CA (certificate authority) issued, specify the location of a certificate with "cert" clause, or if you do not want it checked, use "no-cert". By default, the locally installed certificates are used; if the URL you are visiting is not trusted via those certificates, and you still want to visit it, use "no-cert"; and if you do have a no-CA (i.e. self-signed certificate) for that URL, use "cert" to provide it as a file name (either a full path or a name relative to current working directory, see how-vely-works).

If you'd like to obtain cookies (for example to maintain session or examine their values), use "cookie-jar" clause. <cookie jar> specifies the location of a file holding cookies. Cookies are read from this file (which can be empty or non-existent to begin with) before making a call-web and any changes to cookies are reflected in this file after the call. This way, multiple calls to the same server maintain cookies the same way browser would do. Make sure the same <cookie jar> file is not used across different application spaces, meaning it should be under the application home directory (see how-vely-works), which is the most likely method of implementation.

Binary result

The result of call-web (which is <result>) can be a text value or a binary value. If it is a binary value (for example if getting "JPG", "PNG", "PDF" or other documents), then <status> is the number of bytes in a buffer that holds the value. If it is a string value (such as if downloading "HTML" document as a text), then <status> is the string length.

Request body, sending files and arbitrary content

In order to include request body, for instance to send files, use "request-body" clause. Request body is typically used with POST, PUT or PATCH methods. Even though not common, you can also use it with GET, DELETE or any other custom method, such as for example if the resource you wish to identify requires binary data; perhaps a disposable image is used to identify the resource.

- Structured content
Use "fields" and/or "files" subclauses to send a structured body request in the form of name/value pairs, the same as sent from an HTML form. To do that, you can specify fields with "fields" subclause in the form of <field name>=<field value> pairs separated by a comma. For instance, here two fields are set (field "source" with value "web" and field "act" with value "setcookie"):

call-web "http://website.com/app_name/some_request" response resp response-code rc status len \
    request-body fields "source"="web","act"="setcookie"

To include files, use "files" subclause in the form of <file name>=<file location> separated by commas. For example, here "file1" is the file name sent to the client (which can be anything), and local file "uploadtest.jpg" is the file whose contents is sent; and "file23" is the file name sent to the client (which can be anything), and "fileup4.pdf" is the actual local file read and sent to the client. In this case files are in the application home directory (see how-vely-works), but in general you can specify a relative or absolute path:

call-web "http://website.com" response resp response-code rc status len \
    request-body files "file1"="uploadtest.jpg", "file23"="fileup4.pdf"

You can specify both "files" and "fields" fields, for instance (along with getting error text and status):

call-web "http://website.com/app_name/some_request" response resp response-code rc status len
    request-body fields "source"="web","act"="setcookie" \
        files "file1"="uploadtest.jpg", "file23"="fileup4.pdf" \
    status define st error err

There is no limit on the number of files and fields you can specify, other than of the underlying HTTP protocol.

- Non-structured content
To send any arbitrary (non-structured) content in the request body, such as JSON text for example, use "content" subclause:

call-web "https://website.com" response resp \
    request-headers content-type "application/json" \
    request-body content "{ \
        \"employee\": { \
            \"name\":       \"sonoo\", \
            \"salary\":      56000, \
            \"married\":    true \
        } \
    }"

Optional "content-length" subclause (in "request-headers" clause) can be specified to denote the length of body content:

read-file "somefile" to define file_contents status define file_length
call-web "https://website.com" response resp \
    request-headers content-type "image/jpeg" \
    request-headers content-length file_length \
    request-body content file_contents

If "content-length" is not used, then it is assumed to be the length of string <content>.

Request headers

If your request has a body (i.e. "request-body" clause is used), you can set the content type with "content-type" subclause of a request-headers clause:

call-web "https://<web address>/resource" \
    request-headers content-type "application/json" \
    request-body content some.json

Note that using "content-type" without the request body may be ignored by the server processing your request or may cause it to consider the request invalid. If "content-type" is not used, the default is "multipart/form-data" if "fields" or "files" subclause(s) are used with "body-request" clause. Otherwise, if you use "content" subclause to send other types of data, you must set content type explicitly via "content-type" subclause of "request-headers" clause.

You can also specify custom request headers with "request-headers" clause, using "custom" subclause with a list of <header name>=<header value> pairs separated by a comma. For example, here custom header "Vely-header" has value of "Some_ID", and "Another-Header" a value of "New_ID":

call-web "http://website.com/<app name>/<request name>?act=get_file" response resp response-code rc status len \
    request-headers custom "Vely-header"="Some_ID", "Another-Header"="New_ID"

On the receiving side you can get any such custom header by using "header" clause of the get-req statement:

get-req header "Vely-header" to define hvh0
get-req header "Another-Header" to define hvh1

Examples

Get the web page and print it out:

call-web "https://website.com/page.html" response define resp
p-out resp

Get the "JPG" image from the web and save it to a file "pic.jpg":

call-web "https://website.com/images/someimg.jpg" status define wlen response define resp
write-file "pic.jpg" from resp length wlen

See also

Web
call-web
out-header
send-file
silent-header
See all
documentation

CGI

You can run Vely application as a CGI (Common Gateway Interface) program, if your web server supports CGI. This is not recommended in general, as CGI programs do not exhibit great performance. However in some cases you may need to use CGI, such as when performance is not of critical importance.

To run your application with CGI, use Vely command-line program. Since Vely applications require running in the security context of the user who owns the application, you must use "suexec" (or similar feature) of your web server.

The following script sets up an application named "func_test" (any kind of application will do) to run as CGI (after it's been compiled with vv) on Apache web server running on Ubuntu 18 and up. For other web servers/distros, consult their documentation on how to setup CGI for a program.

#prep repos
    sudo apt update

#enable CGI on apache
    sudo a2enmod cgid
    sudo service apache2 restart

#Install suexec-custom for Apache
    sudo apt-get -y install apache2-suexec-custom
    sudo a2enmod suexec
    sudo service apache2 restart

#setup a "vv" directory under cgi-bin where your application can run
    sudo mkdir -p /usr/lib/cgi-bin/vv
    sudo chown $(whoami):$(whoami)  /usr/lib/cgi-bin/vv
    sudo sed -i '1c\/usr/lib/cgi-bin/vv' /etc/apache2/suexec/www-data

#copy your program to "vv" directory
    sudo mv /var/lib/vv/bld/func_test/func_test /usr/lib/cgi-bin/vv
    sudo chown $(whoami):$(whoami)  /usr/lib/cgi-bin/vv/func_test
    sudo chmod 700  /usr/lib/cgi-bin/vv/func_test

#add user/group of Vely application user to suexec    
    sudo sed -i "/SuexecUserGroup/d" /etc/apache2/sites-enabled/000-default.conf
    sudo sed -i "s/<\/VirtualHost>/SuexecUserGroup $(whoami) $(whoami)\n<\/VirtualHost>/g" /etc/apache2/sites-enabled/000-default.conf
    sudo service apache2 restart

#the application is at http://127.0.0.1/cgi-bin/vv/func_test?...
#substitute 127.0.0.1 for your web address

See also

Running application
application-setup
CGI
command-line
containerize-application
FastCGI
plain-C-FCGI
See all
documentation

Close file

Purpose: Close file.

close-file file-id <file id> \
    [ status [ define ] <status> ]

close-file closes file <file id> previously opened with open-file, where <file id> is an open file identifier.

You can obtain the status of file closing via optional <status> (in "status" clause), which can be created with optional "define". The <status> is VV_OKAY if file is closed, or VV_ERR_CLOSE if could not close file.

If you do not close a file opened with open-file, Vely will automatically close it when the request ends.

Examples

See open-file.

See also

Files
close-file
copy-file
delete-file
file-position
file-storage
file-uploading
lock-file
open-file
read-file
read-line
rename-file
stat-file
temporary-file
uniq-file
unlock-file
write-file
See all
documentation

Command line

Command-line programs

A Vely application can run as a web application or a command line program, or both - such as when some requests can be either fulfilled through web interface or the command line. Note that Vely produces two separate executables: a FastCGI server one and a command-line one - they are different because command-line program does not need the FastCGI library and thus is smaller.

The name of the command-line executable is the same as the application name, and its path is (assuming <app name> is the application name):

/var/lib/vv/bld/<app name>/<app name>

Output

A command line program works the same way as a FastCGI executable, and the output is the same, except that it is directed to stdout (standard output) and stderr (standard error).

Exit code

To specify the exit code of a command line program, use exit-code. To exit the program, use exit-request, or otherwise the program will exit when it reaches the end of a request.

Executing a request

A command line program must obtain its request URL via environment variables, for example if the application name is "stock", here is how to execute a request "add-stock" with input-parameters "name" having a value of "ABC" and "price" a value of "300":

export REQUEST_METHOD=GET
export SCRIPT_NAME="/stock"
export PATH_INFO="/add-stock"
export QUERY_STRING="name=ABC&price=300"

/var/lib/vv/bld/stock/stock

Note that you if specify parameters as part of the path, you could write the above the same way as in a URL:

export REQUEST_METHOD=GET
export SCRIPT_NAME="/stock"
export PATH_INFO="/add-stock/name/ABC"
export QUERY_STRING="price=300"

/var/lib/vv/bld/stock/stock

You can generate the shell code like the above by using "-r" option of vv utility, for example here you'd specify the request path and URL payload (see request-URL):

vv -r --app="/stock" --req="/add-stock/name/ABC"

Including a request body

You can include a request body when executing a command-line program. It is always included as the standard input (stdin) to the program.

You must provide the length of this input (as CONTENT_LENGTH environment variable), the type of input (as CONTENT_TYPE), as well as REQUEST_METHOD (such as POST, PUT, PATCH, GET, DELETE or any other).

Here is an example of using a request body to make a POST request on the command line - the application name is "json" and request name is "process". File "prices.json" is sent as request body:

#length of the request body (of file prices.json). Use ${#VAR} for length if the content is in bash variable $VAR. 
export CONTENT_LENGTH=$(stat -c%s prices.json)

#content type (in this case application/json)
export CONTENT_TYPE="application/json"

#state the URL
export SCRIPT_NAME="/json"
export PATH_INFO="/process"
export QUERY_STRING="act=get_total&period=YTD"

#request method (POST, PUT, PATCH, GET, DELETE...)
export REQUEST_METHOD=POST

#provide request body as the standard input to the command-line program by piping a file into it
cat prices.json | /var/lib/vv/bld/ordering/ordering

You can generate the shell code like the above by using "-r" option of vv utility, for example here you'd specify:

vv -r --app=/json --req='/process?act=get_total&period=YTD' --method=POST --content=prices.json --content-type=application/json

Note that you can also include any other headers as environment variables by using the "HTTP_" convention, where custom headers are capitalized with use of underscore for dashes and prefixed with "HTTP_", for example header "Vely-Header" would be set as:

export HTTP_VELY_HEADER="some value"

Suppressing HTTP header output for the entire application

If you wish to suppress the output of HTTP headers for all requests, set environment variable VV_SILENT_HEADER to "yes" before executing the program:

export VV_SILENT_HEADER=yes

This will suppress the effect of out-header, or for any other case where headers are output. This has the same effect as silent-header, the only difference is that it applies to the entire application.

URL-encoding the input

Any data in QUERY_STRING or PATH_INFO must be formatted to be a valid URL; for example, data that contains special characters (like "&" or "?") must be URL-encoded, for instance:

export QUERY_STRING="action=show&data=a%3Fb"

In this case, field "data" has value of "a?b", where special character "?" is encoded as "%3F".

To make sure all your input parameters are properly URL-encoded, use Vely's v1 code processor:

$($(vv -l)/v1 -urlencode '<your data>')

For instance, to encode "a?=b" as a parameter:

export QUERY_STRING="act=show&data=$($(vv -l)/v1 -urlencode 'a?=b')"

CGI

You can also use a command line program with CGI (Common Gateway Interface). Note that CGI programs generally exhibit much lower performance; use CGI only when warranted by a specific situation.

See also

Running application
application-setup
CGI
command-line
containerize-application
FastCGI
plain-C-FCGI
See all
documentation

Commit transaction

Purpose: Commits a database transaction.

commit-transaction [ @<database> ] [ on-error-continue | on-error-exit ] [ error [ define ] <error> ] [ error-text [ define ] <error text> ]

Database transaction started with begin-transaction is committed with commit-transaction.

Database

Optional <database> is specified in "@" clause and is the name of the database-config-file.

Error handling

Examples

begin-transaction @mydb
run-query @mydb="insert into employee (name, dateOfHire) values ('Terry', now())"
run-query @mydb="insert into payroll (name, salary) values ('Terry', 100000)"
commit-transaction @mydb

See also

Connect apache tcp socket

This shows how to connect your application listening at TCP port <port number> (started with "-p" option in vf) to Apache web server.

- Step 1:
To setup Apache as a reverse proxy and connect your application to it, you need to enable FastCGI proxy support, which generally means "proxy" and "proxy_fcgi" modules - this is done only once:

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/<app path>" is the application path, see request-URL):

ProxyPass "/<app path>" fcgi://127.0.0.1:<port number>/

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/<app path>" (such as for example "/<app path>.html" or "/<app path>_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/<app path>", see request-URL.

See also

Web servers
connect-apache-tcp-socket
connect-apache-unix-socket
connect-nginx-tcp-socket
connect-nginx-unix-socket
See all
documentation

Connect apache unix socket

This shows how to connect your application listening on a Unix socket (started with vf) to Apache web server.

- Step 1:
To setup Apache as a reverse proxy and connect your application to it, you need to enable FastCGI proxy support, which generally means "proxy" and "proxy_fcgi" modules - this is done only once:

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/<app path>" is the application path (see request-URL) and "<app name>" is your application name):

ProxyPass "/<app path>" unix:///var/lib/vv/<app name>/sock/sock|fcgi://localhost/<app path>

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

See also

Web servers
connect-apache-tcp-socket
connect-apache-unix-socket
connect-nginx-tcp-socket
connect-nginx-unix-socket
See all
documentation

Connect nginx tcp socket

This shows how to connect your application listening at TCP port <port number> (started with "-p" option in vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/<app path>" is the application path, see request-URL):

location /<app path> { include /etc/nginx/fastcgi_params; fastcgi_pass  127.0.0.1:<port number>; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

See also

Web servers
connect-apache-tcp-socket
connect-apache-unix-socket
connect-nginx-tcp-socket
connect-nginx-unix-socket
See all
documentation

Connect nginx unix socket

This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/<app path>" is the application path (see request-URL) and "<app name>" is your application name):

location /<app path> { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/<app name>/sock/sock; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

See also

Web servers
connect-apache-tcp-socket
connect-apache-unix-socket
connect-nginx-tcp-socket
connect-nginx-unix-socket
See all
documentation

Containerize application

This is a short tutorial on building a base Vely image, and an application image (regardless of what your application does) that can be deployed anywhere, even on non-Linux systems. This is one way to deploy your Vely application. See also example-docker for an example you can run. You can use docker or podman (substitute podman for docker).

The application container has a Vely application installed, and your reverse proxy web server and your databases must be created (see application-setup for more information), as they will likely either run on the host machine, or be containers themselves.

The base image used here is "vely" (created in the previous step). User "vely" is created and assigned limited sudo privileges for Vely only. Source code of your application is in "docker" directory on the host, and copied to the container, where application is compiled and linked, then source code is deleted, making the container ready for deployment (if you wish to keep the source, you certaintly can, just remove the lines that delete it). Finally, the entry point will start application - that's what runs when the container is deployed.

The following shows how to create a deployment image and a subsequent running container. The application name is "velydemo" - change it as needed.

#
# Create base vely image. Install Vely on top of Ubuntu 20.04
#
FROM ubuntu:20.04
ENV TZ=America/Phoenix
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt update
RUN apt install -y apt-transport-https ca-certificates wget sudo
RUN wget  -qO - https://vely.dev//pkg/OPENPGP|sudo tee /usr/share/keyrings/vely.asc >/dev/null
ARG arch
RUN sudo bash -c "echo 'deb [signed-by=/usr/share/keyrings/vely.asc] https://vely.dev//pkg/ubuntu_20_$arch/latest ubuntu_20_$arch vely' >/etc/apt/sources.list.d/vely.list"
RUN sudo apt update
RUN sudo apt install -y vely

Create an image with - use --no-cache in order to update the image when new Vely release is available:

sudo docker build --no-cache -t vely .

In some cases your application is stateless, i.e. all the data is through to database(s) and other applications. In other cases, you may want to have file storage that persists even when containers recycle - typically that means persisting Vely directory (which is /var/lib/vv, see how-vely-works). In that case, you can use a container storage option for that purpose; in case of docker, that would be volumes, for instance:

sudo docker volume create velyhome

For example, if your application uploads/downloads files, you may use Vely file-storage (which is under /var/lib/vv); or you may store other files that your application produces. In such cases using a volume (or similar) may be a good idea; in other cases you may want a stateless container and offload such functionality elsewhere (for instance to databases, other applications etc.).

Next is velyapp.dockerfile, which is used to create an image for an application. In this case it is a demo application with source code in "docker" directory (under the current working directory):

#
# Create image that will be used to compile and link an application. Source code is copied from host,
# application is setup, and the source code is deleted (remove this step if you want to ship the source code).
# After that the image is ready to run in a container. Here, the application name is "velydemo"; change it to
# fit your application. The port used in 2300, you can change that as well.
# Customize the ENTRYPOINT command to fit your runtime.
#
FROM vely
# create vely user and give it limited sudo powers
RUN useradd -ms /bin/bash vely && echo "vely:vely" | chpasswd
RUN echo "vely ALL=(ALL) NOPASSWD: /usr/bin/vf" >> /etc/sudoers
# default user
USER vely
# default working dir
WORKDIR /home/vely
EXPOSE 2300
# copy over source code and make app
COPY ./docker/* /home/vely/
# this is to run app with docker run
ENTRYPOINT [ "./runit" ]

The "runit" script will create an application in the container, build it, and start it in the foreground. This script is in the "docker" directory with your application source code:

#!/bin/bash
# Create Vely application and run it in foreground for docker
# demout has a log of execution
sudo vf -i -u $(whoami) velydemo
vv -c
vv -q --db=mariadb:db -s
vv -c
vf -f -w3 -p2300 velydemo > demout

Stop and remove any current containers/images:

sudo docker stop velyapp || true
sudo docker rmi velyapp || true

Build an application image:

sudo docker build --no-cache -t velyapp -f velyapp.dockerfile .

Assuming you have setup web server and any database(s), start the container (using "host" network interface for simplicity):

sudo docker run --init --name velyapp -d  --network="host" --rm velyapp velyapp

If your application uses volumes for data persistence, you might use:

sudo docker run --init --name velyapp -d -v velyhome:/var/lib/vv  --network="host" --rm velyapp

Note that you can persist any storage you want, however at the minimum you would likely persist Vely directory (/var/lib/vv) which is stored on the host under volume "velyhome".

Now you can use your application through web server:

http://127.0.0.1/velydemo/<request name>...

See also

Running application
application-setup
CGI
command-line
containerize-application
FastCGI
plain-C-FCGI
See all
documentation

Copy file

Purpose: Copies one file to another.

copy-file <source file> to <target file> [ status [ define ] <status> ]

File <source file> is copied into <target file>, which is created if it does not exist.

Status can be obtained in <status> variable, which is VV_ERR_OPEN if cannot open source file, VV_ERR_CREATE if cannot create target file, VV_ERR_READ if cannot read source file, VV_ERR_WRITE if cannot write target file, or number of bytes copied (including 0) on success.

Examples

copy-file "/home/user/source_file" to "/home/user/target_file" status define st

See also

Copy string

Purpose: Copies string to another string.

copy-string <source string> to [ define ] <dest string> \
    [ ( length [ define ] <length> ) \
        | ( [ bytes-written [ define ] <bytes written> ) ]

Use copy-string to copy <source string> to <dest string> (which can be created with optional "define"). <dest string> is allocated memory.

Without "length" clause, <source string> is treated as a null-terminated string and the number of bytes to copy is computed as its length. With "length" clause, exactly <length> bytes are copied into <dest string>, regardless of whether there is a null-character within <source string>, i.e. you can copy binary data this way. Regardless of whether "length" clause is used or not, <dest string> will always have a null-character at the end.

If "bytes-written" clause is used, then the number of bytes copied is stored into <bytes written>, which can be created with optional "define". "bytes-written" can be used only if "length" is not used, for obvious reason, as the two are equal otherwise. <bytes written> does not include the null-character placed at the end.

You can copy a string to itself. In this case, the original string remains and the new string points to a copy:

char *str = "original string"; // string to change

char *orig = str; // points to original copy of the string to change

copy-string str to str // make a copy of string to change and assign it to itself

str[0] = 'O'; // change the copy

// Now "str" points to "Original string" 
// and "orig" points to "original string"

Examples

After copy-string below, "my_str" will point to a copy of string "some value", and "bw" will be 10:

char *other_string="some value";
copy-string other_string to define my_str bytes-written define bw

Copy certain number of bytes, the result in "my_str" will be "som":

char *other_string="some value";
copy-string other_string to define my_str length 3

See also

Strings
copy-string
count-substring
lower-string
num-string
split-string
trim-string
upper-string
write-string
See all
documentation

Count substring

Purpose: Count substrings.

count-substring <substring> in <string> to [ define ] <count> [ case-insensitive [ <case-insensitive> ] ]

count-substring counts the number of occurrences of <substring> in <string> and stores the result in <count> (specified in "to" clause), which can be created with "define" if it does not exist. By default, search is case-sensitive. If you use "case-insensitive" clause without boolean expression <case-insensitive>, or if <case-insensitive> evaluates to true, then the search is case-insensitive.

If <substring> is empty ("") or NULL, <count> is 0.

Examples

In the following example, 1 occurrence will be found after the first count-substring, and 2 after the second (since case insensitive search is used there):

char sub[] = "world";
char str[] = "Hello world and hello World!";

count-substring sub in str to define num_occ
pf-out "Found %lld occurrences!\n", num_occ

count-substring sub in str to num_occ case-insensitive
pf-out "Found %lld occurrences!\n", num_occ

See also

Strings
copy-string
count-substring
lower-string
num-string
split-string
trim-string
upper-string
write-string
See all
documentation

Current row

Purpose: Get or print out the row number of a current row in the result-set of a query.

current-row [ to [ define ] <current row> ]

Without "to" clause, current-row will print out the current row number. First row is numbered 1. With "to" clause, the row number is stored into variable <current row>, which is created if "define" is used. current-row must be within a run-query loop, and it always refers to the most inner one.

Examples

Display row number before a line with first and last name for each employee:

run-query @mydb="select firstName, lastName from employee"
    @Row #<<current-row>><br/>
    query-result firstName
    @,
    query-result lastName
    @<br/>
end-query

See also

Database config file

Vely application can use any number of databases, with each specified by a database configuration file. This file provides database name, login and connection settings and preferences.

When making a Vely application, you specify a database vendor and the database configuration file for each database your application uses (see vv), for instance:

vv ... --db="mariadb:db1 postgres:db2 sqlite:db3"  ...

in which case there are three database configuration files (db1, db2 and db3), with db1 being MariaDB, db2 being PostgreSQL and db3 being SQLite database.

You must create a database configuration file for each database your application uses, and this file must be placed with your source code. Such file will be copied to locations specified and used by Vely to connect to database(s) (see how-vely-works).

Each such file contains connection information and authentication to a database, which Vely uses to login. The names of these configuration files are used in queries. There is no limit on how many databases can be used in your application and those from different vendors can be used in the same application.

An example of database configuration file (in this case MariaDB):

[client]
user=mydbuser
password=somepwd
database=mydbname
protocol=TCP
host=127.0.0.1
port=3306

Using in your queries

Database statements that perform queries (such as run-query) must specify the database configuration file used, unless your application uses only a single database. Such configuration is given by "@<database config file>" (for instance in run-query or begin-transaction). For example, in:

run-query @mydb="select name from employees"
...
end-query

the query "myquery" is performed on a database specified by the configuration file "mydb", as in (assuming it's PostgreSQL database):

vv ... --db="postgres:mydb"  ...

You do not need to manually connect to the database; when your application uses it for the first time, a connection is automatically established, and lost connection is automatically re-established when needed.

If a database is the only one used by your application, you can omit it, as in:

run-query ="select name from employees"
...
end-query

Connection settings

The contents of a configuration file depends on the database used:

MariaDB: configuration file is written as a MariaDB client options file. You can see the parameters available at
```
https://mariadb.com/kb/en/configuring-mariadb-connectorc-with-option-files/#options
```
Most of the time, though, you would likely use only a few of those options, as in (for local connection):
```
[client]
user=myuser
password=mypwd 
database=mydb
socket=/run/mysqld/mysqld.sock
```
The above file has fields "user" (MariaDB user), "password" (the password for MariaDB user), "database" (the MariaDB database name) and MariaDB communication "socket" location (assuming your database is local to your computer - if the database is across network you would not use sockets!).

If you use passwordless MariaDB login (such as when the MariaDB user name is the same as your Operating System user name and where unix socket plugin is used for authentication), the password would be empty.

To get the location of the socket, you might use:
```
sudo mysql -u root -e "show variables like 'socket'"
```
Postgres: configuration file has a Postgres connection string. You can see the parameters available at
```
https://www.postgresql.org/docs/14/libpq-connect.html#LIBPQ-CONNSTRING
```
Most of the time, though, you may be using only a few of those options, as in:
```
user=myuser password=mypwd dbname=mydb
```
The above file has parameters "user" (Postgres user), "password" (the password for Postgres user), "dbname" (the Postgres database name). If you use peer-authenticated (i.e. passwordless) login, then omit "password" - this is when the Postgres user name is the same as your Operating System user name and where local unix domain socket is used for authentication.
SQLite: configuration file should contain a single line of text, and it must be the full path to the SQLite database file used, for example (if you keep it in Vely's application directory):
```
/var/lib/vv/<app name>/app/<db name>.db
```

Substituting environment variables

You can use environment variables in database configuration files by means of substitution, in the form of "${VAR_NAME}". For example in file "mydb":

[client]
user=${DB_USER}
password=${DB_PWD}
database=${DB_NAME}
protocol=TCP
host=127.0.0.1
port=${DB_PORT}

Here, environment variables DB_USER, DB_PWD, DB_NAME and DB_PORT are used. They must be defined in the shell environment prior to calling vv to make your application (if not defined the value will be empty):

#Define environment variables for a configuration file
export DB_USER="my_user"
export DB_PWD="my_password"
export DB_NAME="my_database"
export DB_PORT="3307"

#Make application using the above database configuration file with the environment variables specified
vv -q --db=mariadb:mydb

which results in file /var/lib/vv/<app name>/app/db/mydb:

[client]
user=my_user
password=my_password
database=my_database
protocol=TCP
host=127.0.0.1
port=3307

Besides making application deployment easier, this also adds to its security as the information such as above (including the database password) does not need to be a part of source code and reside in source code control system (such as git).

Your environment variables can have any names, except that they cannot start with an underscore ("_") or be prefixed by "VV_", because those variable names are reserved by Vely.

Note that if your data actually has a dollar sign and is a part of the configuration file, then you can create a variable for it:

export DOLLAR_SIGN='$'

and in the configuration file:

..
database=my${DOLLAR_SIGN}database
..

In this case the database name is "my$database".

See also

Database queries

Databases

Your application can use any number of databases, specified by database-config-files. If only a single database is used, you may skip specifying a database with "@" clause. You can use different database vendors at the same time.

Query interface, SQL injection

Vely has the same query interface for all databases (see for example run-query). It provides means to execute dynamic as well as static queries; regardless, query input parameters are always separated from the query logic. This separation is in some cases emulated (see run-query), and in others native, such as with prepared statements (see run-query). In either case, Vely provides protection against SQL injections.

Input and output parameters

Input parameters and output resuls are always null-delimited strings. In web applications, more often than not, it is string values that are sought as output or provided as input, and conversion to other types rarely happens; there is surely a number of applications where the opposite may be true, but for the majority of applications this approach may work better. If data conversion is needed, it can be performed pre-query or post-query, depending on the purpose. The placeholder for input parameter is chosen to represent this approach, using single quotes (as text literals are single quoted in SQL) and %s (a C convention for string placeholders), thus '%s' is used for input parameters.

Binary data

Binary data can be handled for storage or retrieval via hexadecimal strings (see encode-hex, decode-hex) or Base64 (see decode-base64, encode-base64); this is reasonable for smaller amounts of data. Note that storing large amounts of binary data in the database is generally less desirable than storing such data (for instance PDF or JPG documents) in the file system (see file-storage), as manipulating and retrieving binary data is generally easier and faster that way.

Prepared statements

Prepared statements typically provide better performance but may not be ideal in every circumstance (see prepared-statements).

Multiple statements

Multiple statements in one query can be executed in PostgreSQL, where only the last statement's result set is available, which is simpler and probably more manageable. MariaDB multiple statement execution is disabled by default, and so is for SQLite. Multiple statement execution isn't ideal as generally the number of statements may not be known in advance and retrieving multiple results sets can be challenging. Using stored procedures that return a single result set, or multiple queries to execute multiple statements are both better approach.

Prepared queries, regardless of the database, can never be multiple statements, which is a limitation that exists in all databases.

Connection persistence and reuse, transactions

A single Vely process (see how-vely-works and vely-architecture) keeps a single connection open to the database, which is shared between all requests a process will handle. Each request will either commit or rollback a transaction; if the transaction is neither committed nor rollback-ed, it will be rollback-ed with error (see begin-transaction). If a database connection is lost, Vely will automatically re-establish it if possible. This allows for very fast query execution and essentially unlimited reuse of prepared statements.

Autocommit

The autocommit feature is enabled by default on all supported databases. Do not disable it, as that would cause unpredictable and inconsistent behavior. Whenever you need a transactional behavior, use commit-transaction to begin a transaction and commit-transaction/rollback-transaction to end it. Also, in general, multiple DML statements (such as INSERT) are faster within a transaction block because the data is not flushed with each one, but rather once per block.

See also

Debugging

Several techniques on debugging Vely applications are discussed here.

Tracing and Backtrace file

To see any errors reported by Vely, use -e option of vv and check backtrace file. For example, to see the last 3 error messages:

vv -e 3

You can use trace-run statement to create run-time traces (see how-vely-works for directory location). To quickly find the location of recently written-to trace files, use -t option of vv, for example for 5 most recently used trace files:

vv -t 5

Use get-req to get the trace file location at run-time from your application.

Output from FastCGI application without web server

Use FastCGI-command-line-client to send request and receive reply from your FastCGI server processes from command line. This is useful in debugging issues and automating tests.

Issues in starting vf

vf starts your web application, running as FastCGI processes. If you're having issues with vf, check out its log. Assuming your application name is "app_name", the log file is:

/var/lib/vv/app_name/vflog/log

Web server error log

Check web server's error log, which would store the error messages emitted to the client. Typically, such files are in the following location:

/var/log/<web server>

(for example /var/log/apache2), but the location may vary - consult your web server's documentation.

Using gdb debugger

In order to use gdb debugger, you must make your application with "--debug" flag (see vv). Do not use "--debug" in any other case, because performance will be considerably affected.

Ultimately, you can attach a debugger to a running Vely process. If your application name is "app_name", first find the PID (process ID) of its process(es):

ps -ef|grep app_name.fcgi

Note that you can control the number of worker processes started, and thus have only a single worker process (or the minimum necessary), which will make attaching to the process that actually processes a request easier (see vv).

Use gdb to load your program:

sudo gdb /var/lib/vv/bld/app_name/app_name.fcgi

and then attach to the process (<PID> is the process ID you obtained above):

att <PID>

Once attached, you can break in the request you're debugging:

br <request name>

or in general Vely request dispatcher:

br vely_dispatch_request

which would handle any request to your application.

Error containing "vely_statement_must_be_within_code_block_here"

If you get an error like:

error: ‘_vely_statement_must_be_within_code_block_here_1’ undeclared (first use in this function);

then you are attempting to use a Vely statement in a single-statement implicit block, as in:

if (some condition)
    exec-program ....

Since each Vely statement expands into one or more lines, if a Vely statement is the sole functional code used in an implicit code block, you must place it in an explicit block:

if (some condition)  {
    exec-program ....
}

See example-hello-world for an example in debugging and tracing.

See also

Debugging
debugging
trace-run
See all
documentation

Decode base64

Purpose: Base64 decode.

decode-base64 <data> to [ define ] <output data> \
    [ input-length <input length> ] \
    [ output-length [ define ] <output length> ]

decode-base64 will decode string <data> into <output data>, which can be binary data. <output data> is allocated memory.

If "input-length" clause is used, then <input length> is the number of bytes decoded, otherwise <data> is treated as a null-terminated string and its length determined.

The result is stored in <output data> (in "to" clause), which can be created with optional "define". The length of the decoded string can be obtained via "output-length" clause in <output length>, which can be created with optional "define".

<output data> will always have an extra null byte appended - this null byte is not included in length and is appended only as a safety precaution.

Note that the string to decode can have whitespaces before it (such as spaces or tabs), and whitespaces and new lines after it, which will be ignored for the purpose of decoding.

Examples

See encode-base64.

See also

Base64
decode-base64
encode-base64
See all
documentation

Decode hex

Purpose: Decode hexadecimal string into data.

decode-hex <data> to [ define ] <output> \
    [ input-length <input length> ] \
    [ output-length [ define ] <output length> ]

decode-hex will decode hexadecimal string <data> to string <output> given in "to" clause, which may be any binary data. <output> is allocated memory.

<data> must consist of an even number of digits 0-9 and letters A-F or a-f. The length of <data> may be given by optional <input length> in "input-length" clause, otherwise it is assumed to be the string length of <data>. The output string <output> is allocated and its length is given in the optional <output length> in "output-length" clause.

Optional "define" subclause can be used to create <output> string and a number variable <output length> if they do not already exist.

Examples

Get binary data from a hexadecimal string "hexdata". The output string "binout" is created and so is its length "olen" variable of type "num":

char *hexdata = "0041000F414200";
decode-hex hexdata to define binout output-length define olen

The value of "binout" will be binary data equal to this C literal:

char *binout = "\x00""A""\x00""\xF""AB""\x00""\x04";

See also

Hex encoding
decode-hex
encode-hex
See all
documentation

Decode url

Purpose: Decode URL-encoded string.

decode-url <string> [ output-length [ define ] <decoded length> ] [ input-length <length> ]

decode-url will decode <string> (created by encode-url or other URL-encoding software) and store the result back into it (i.e. it will be modified). An optional "output-length" clause lets you get the length of decoded string (which is always lesser or equal than the length of the encoded string) in <decoded length>, which can be created with optional "define". <length> in optional "input-length" clause specifies the number of bytes to decode; if omitted or negative, it is the string length of <string>.

All encoded values (starting with %) are decoded, and "+" (plus sign) is converted to space. If there is an error (for example hexadecimal value following % is invalid), the decoding stops and whatever was decoded up to that point is the result, and the length reflects that.

Note that <string> must not be a constant because decode-url will write into it. If it is a constant, make a copy of it first with copy-string. See encode-url.

Examples

Decode URL-encoded string "str", after which it will hold a decoded string. "dec_len" will be the length (in bytes) of that string:

decode-url str output-length define dec_len

See also

URL encoding
decode-url
encode-url
See all
documentation

Decode web

Purpose: Decode web(HTML)-encoded string.

decode-web <string> [ output-length [ define ] <decoded length> ] [ input-length <length> ]

decode-web will decode <string> (created by encode-web or other web-encoding software) and store the result back into it (i.e it will be modified). "output-length" clause lets you get the length of the decoded string (which is always lesser or equal than the length of the encoded string) in <decoded length>, which can be created with optional "define". To decode only a number of leading bytes in <string>, use "input-length" clause and specify <length>.

See encode-web.

Examples

Decode web-encoded string "str", after which it will hold a decoded string. "dec_len" will be the length (in bytes) of that string:

decode-web str output-length define dec_len

See also

Web encoding
decode-web
encode-web
See all
documentation

Decrypt data

Purpose: Decrypt data.

decrypt-data <data> to [ define ] <result> \
    [ input-length <input length> ] \
    [ output-length [ define ] <output length> ] \
    [ binary [ <binary> ] ] \
    ( password <password> \
        [ salt <salt> [ salt-length <salt length> ] ] \
        [ iterations <iterations> ] \
        [ cipher <cipher algorithm> ] \
        [ digest <digest algorithm> ]
        [ cache ]
        [ clear-cache <clear cache> ) \
    [ init-vector <init vector> ]

decrypt-data will decrypt <data> which must have been encrypted with encrypt-data, or other software using the same algorithms and clauses as specified.

If "input-length" clause is not used, the data to decrypt is considered to be a string, i.e. null-terminated, otherwise, if specified, then exactly <input length> bytes are decrypted. Password used for decryption is <password> (in "password" clause) and it must match the password used in encrypt-data. If "salt" clause is used, then string <salt> must match the salt used in encryption. If "init-vector" clause is used, then string <init vector> must match the IV (initialization vector) used in encryption. If "iterations" clause is used, then <iterations> must match the number used in encryption.

"output-length" clause lets you obtain the number of bytes in decrypted data in <output length>, which can be created with optional "define". The result of decryption is in <result> (in "to" clause) and can be created with optional "define". <result> is allocated memory.

If data was encrypted in binary mode (see encrypt-data), you must decrypt it with the same, and if it wasn't, then you must not use it in decrypt-data either. The reason for this is obvious - binary mode of encryption is encrypted data in its shortest form, and character mode (without "binary" or if <binary> evaluates to false) is the same data converted to a hexadecimal string - thus decryption must first convert such data back to binary before decrypting.

The cipher and digest algorithms (if specified as <cipher algorithm> and <digest algorithm> in "cipher" and "digest" clauses respectively) must match what was used in encrypt-data.

"cache" clause is used to cache the result of key computation, so it is not computed each time decryption takes place, while "clear-cache" allows key to be re-computed every time <clear cache> evaluates to boolean true. For more on "cache" and "clear-cache" clauses, as well as safety of encrypting/decrypting, see "Caching key" and "Safety" in encrypt-data.

Examples

See encrypt-data.

See also

Encryption
decrypt-data
derive-key
encrypt-data
hash-string
random-crypto
random-string
See all
documentation

Delete cookie

Purpose: Deletes a cookie.

delete-cookie <cookie name> [ path <cookie path> ] [ status [ define ] <status> ] [ secure <secure> ]

delete-cookie marks a cookie named <cookie name> for deletion, so it is sent back in the reply telling the client (such as browser) to delete it.

Newer client implementations require a cookie deletion to use a secure context if the cookie is considered secure, and it is recommended to use "secure" clause to delete such a cookie. This is the case when either "secure" clause is used without optional boolean expression <secure>, or if <secure> evaluates to true.

<cookie name> is a cookie that was either received from the client as a part of the request, or was added with set-cookie.

A cookie can be deleted before or after sending out a header (see out-header). However a cookie must be deleted prior to outputting any actual response (such as with output-statement or p-out for example), or the request will error out (see error-handling).

<status> (in the optional "status" clause) is the integer variable (which can be created with optional "define") that will be -1 if the cookie did not exist, or 0 or greater if it did.

The same cookie name may be stored under different URL paths. You can use the optional "path" clause to specify <cookie path> to ensure the desired cookie is deleted.

Examples

delete-cookie "my_cookie"
bool is_secure = true;
delete-cookie "my_cookie" path "/path" secure is_secure

See also

Cookies
delete-cookie
get-cookie
set-cookie
See all
documentation

Delete file

Purpose: Deletes a file.

delete-file <file location> [ status [ define ] <status var> ]

File specified with <file location> is deleted. <file location> can be given with an absolute path or relative to the application home directory (see vv).

If "status" is specified, the status is stored into <status var>. The status is VV_OKAY on success or if the file did not exist, and VV_ERR_DELETE on failure. If "define" is specified, <status var> variable is created.

Examples

delete-file "/home/user/some_file" status define st

See also

Delete json

Purpose: Delete memory associated with JSON parsing.

delete-json <json>

delete-json will delete memory associated with variable <json> created to parse a JSON string with new-json.

See memory-handling for more on when (not) to delete memory explicitly like this; the same rules apply as for delete-mem.

Examples

delete-json jv

See also

JSON
delete-json
new-json
read-json
See all
documentation

Delete mem

Purpose: Free memory.

delete-mem <memory> status [ define ] <status>

delete-mem frees memory allocated by Vely. Do not use it on memory allocated by any C-library functions (such as malloc(), calloc() or realloc()).

delete-mem will attempt to handle memory violations or invalid pointers, and may not error out if there is an issue; if the error is too severe, it will error out. If it doesn't error out, you can obtain the status of freeing memory in variable <status> by using "status" clause (you can create this number variable with "define" if it does not exist). The reason for this is that freeing memory is in many cases unnecessary as Vely will automatically do so at the end of each request and problems in freeing memory will not affect doing so in the end.

If <status> is VV_OKAY, the memory has been freed successfully. If <status> is VV_ERR_MEMORY, there was a problem; most likely <string> was not a valid memory pointer allocated by any of Vely statements; or it was a bad memory pointer for any number of reasons (such as for example it was already freed once).

Examples

Allocate and free memory:

new-mem define mystr size 300
delete-mem mystr status define st
if (st != VV_OKAY) {
    @Error in memory release!
}

Free memory allocated by write-string (consisting of 100 "Hello World" statements):

write-string define ws
    num i;
    for (i = 0; i < 100; i++) {
        @Hello World
    }
end-write-string
delete-mem ws

See also

Memory
delete-mem
manage-memory
memory-handling
new-mem
resize-mem
See all
documentation

Delete query

Purpose: Delete memory associated with a query.

delete-query <query name> [ skip-data ]

A named query (see clause "name" in run-query) can be deleted with delete-query by specifying the <query name>. Deletion means that all memory allocated for a query will be freed. It means query results will be freed as well. Do not delete query if its results are still referenced afterwards.

However, if "skip-data" clause is used, then the query results will not be deleted; rather everything else will be. This means the results of query-result statement or "output define" clause in run-query statement will not be freed. This is useful if you want to keep the results of a query and otherwise release all of its resources. Note that the results will remain allocated even if you do not obtain them via "query-result" or "output define" clause, for instance.

Make sure not to delete anything twice. For instance, do not use delete-mem to delete query data such as error message, column name or results, while using delete-query as well. By the same token, if you use "skip-data", you can use delete-mem to free query results later.

See memory-handling for more on when (not) to delete memory explicitly like this; the same rules apply as for delete-mem.

Examples

run-query @db="drop table if exists test" no-loop name drop_query
delete-query drop_query

See also

Delete server

Purpose: Delete resources for a server call.

delete-server <server>

delete-server will delete resources allocated for a call to <server> (see new-server). If you have used a server array (see "array-count" in call-server), then delete-server would be used on each element of the array.

Internal Vely memory associated with <server> is deleted. The actual data/errors returned (such as with read-server) are also deleted; do not attempt to use them after delete-server, since all resources allocated with a server call are released.

See memory-handling for more on when (not) to delete memory explicitly like this; the same rules apply as for delete-mem.

Examples

See new-server.

See also

Distributed computing
call-server
delete-server
new-server
read-server
See all
documentation

Delete tree

Purpose: Delete a node from a tree.

delete-tree <tree> key <key> \
    [ status [ define ] <status> ] \
    [ value [ define ] <value> ] \
    [ old-key [ define ] <old key> ]

delete-tree will search for <key> and if found, delete its node and set optional <status> (in "status" clause) to VV_OKAY. If <key> is not found, <status> will be VV_ERR_EXIST.

The key of the deleted node can be obtained in optional "old-key" clause in <old key> string; while its value in <value> in optional "value" clause, which can be a pointer of any type. Even though the value of <old key> will match <key> when deletion is successful, the two pointers may differ. Thus you may want to obtain <old key> if you need to release memory for the key, though in many cases you don't need to (see memory-handling). If <status> is not VV_OKAY, <value> and <old key> are unchanged.

Note that <status>, <value> and <old key> can be created with optional "define".

Examples

Delete node with key "123", and obtain its value and (the original) key.

char *k = "123";
delete-tree mytree key k value define val old-key define old_k status define st
if (st != VV_OKAY){
   @Could not find key <<p-out k>>
   exit-request
}
// delete the original key and value
delete-mem old_k
delete-mem val
@Deleted key <<p-out k>> with value <<p-out val>>

See also

Tree search
delete-tree
get-tree
new-tree
purge-tree
read-tree
use-cursor
write-tree
See all
documentation

Deploying application

You can deploy Vely applications in different ways.

Containers

Using containers (such as with docker or podman) is an easy, reliable and portable way of deploying a Vely application. See containerize-application.

This method is also applicable for deployment to Internet of Things (IOT) systems, such as for example Raspberry PI Operating Systems often used for devices. For IOT, Vely can be deployed both on servers and devices.

Installing from source

You can also deploy a Vely application by installing it from source; see application-setup. After installation, you may delete the source code and remove generated C code:

vv -c

Installing from binary

The binary for your application can be copied to another computer with the same Operating System and with Vely installed, provided the Operating System has compatible libraries; in practice this means the same major Operating System version and the same or higher minor version. The same is true for Vely version; you must have the same major Vely version and the same or higher minor version.

You must install Vely first on the target machine and then create the application, for instance:

sudo vf -i -u $(whoami) <app name>

Next you must copy the binaries for your application from source to target, to the same location. Assuming application name is <app name>, the binaries are:

#Fast CGI binary:
/var/lib/vv/bld/<app name>.fcgi

#command line and CGI binary:
/var/lib/vv/bld/<app name>

For an application that does not use any database(s) and does not need any additional files, this is enough to deploy the application to a target machine.

If you use database(s), you must install such database(s) and migrate any needed data, and copy the database-config-files, meaning the following directory:

/var/lib/vv/<app name>/app/db

If you use SQLite, you can just copy the database file as it is portable; generally such file would be in the application's home directory. Since the application's home directory is:

/var/lib/vv/<app name>/app

you would copy any custom files and directories you have created and that your application may need. Note that all files and directories copied must be owned by the application owner, i.e. the user used in application creation command above (see vf).

Multi-user and multi-application environment

Each application owner (i.e. the user used in application creation with -u flag, see vf) can have any number of applications. At the same time, different application owners can create any number of applications of their own; the application directory for each application is owned by its respective owner with 700 permissions, meaning different users cannot access application files of other owners (see how-vely-works). Similarly, application servers (i.e. FastCGI servers) run under the application owner permissions, and can only be managed by its owner.

See also

General
about-Vely
application-architecture
deploying-application
how-vely-works
quality-control
rename-files
SELinux
vely-architecture
vely-removal
vely-version
vf
vv
See all
documentation

Derive key

Purpose: Derive a key.

derive-key [ define ] <key> from <source> length <length> \
    [ binary [ <binary> ] ] \
    [ from-length <source length> ] \
    [ digest <digest algorithm> ] \
    [ salt <salt> [ salt-length <salt length> ] ] \
    [ iterations <iterations> ]

derive-key derives <key> (which can be created with optional "define") from string <source> in "from" clause. <key> is allocated memory. If <source length> in "from-length" clause is specified, exactly <source length> bytes of <source> are used, regardless of null bytes. Otherwise, the length of a null-terminated <source> string is used as the number of bytes.

The length of derived key is given by <length> in "length" clause. The method for key generation is PBKDF2. By default the digest used is "SHA256". You can use a different <digest algorithm> in "digest" clause (for example "SHA3-256"). To see a list of available digests:

#get digests
openssl list -digest-algorithms

The optional salt for key derivation can be given with <salt> in "salt" clause. If "salt-length" clause is not used, then the salt is null-terminated, otherwise its length is <salt length>.

The number of iterations is given by <iterations> in "iterations" clause. The default is 1000 per RFC 8018, though depending on your needs and the quality of <source> you may choose a different value.

By default, the derived key is produced as a null-terminated string in a hexadecimal form, where each byte is encoded as two-character hexadecimal characters, so its length is 2*<length>. If "binary" clause is used without optional boolean expression <binary>, or if <binary> evaluates to true, then the output is a binary string of <length> bytes, and a null byte is placed after it.

Key derivation is often used when storing password-derivatives in the database (with salt), and also for symmetrical key generation.

Examples

Derived key is in variable "mk":

random-string to define rs9 length 16
derive-key define mk from "clave secreta" digest "sha-256" salt rs9 salt-length 10 iterations 2000 length 16

See also

Encryption
decrypt-data
derive-key
encrypt-data
hash-string
random-crypto
random-string
See all
documentation

Diagnostic messages

Since Vely generates C code from your Vely statements and build directives, you can get diagnostic messages about Vely source code files, generated C code files or both (the default). Messages inidicating build errors and warnings are output from Vely's build utility (see vv).

By default, diagnostic messages include source code indication from both Vely source code and the generated C code. The line number in .vely file is displayed, as well as the actual source code of that line (or the last line if it's split into multiple lines); the exact part of C generated code is shown, along with carets visually pointing to the code where an issue is found. This information (if available) is followed by the underlying compiler diagnostic.

The root diagnostic messages are displayed in bold blue and prefixed with "***"; the associated Vely statement or source C code is in red and prefixed with ">>>"; and generated C code in green and prefixed with "###". The column where error was reported in C code (original or generated) is marked with three carets (i.e. "^^^") Any other diagnostic messages from the underlying C compiler are in plain default coloring. This kind of visual output helps with quickly identifying and correcting errors.

In the example below, you can see diagnostics for type mismatches, undeclared variables, a syntax error in a Vely statement, and a missing block closure (i.e. "}"):

Prior to version 17.1, Vely's diagnostic output did not include as much detail or color coding; it still included all the relevant information, such as source line numbers in .vely files and other diagnostic messages. You can switch back to this kind of output with "--plain-diag" option of vv.

If you use "--c-lines" option of vv, the above color-coded and additional diagnostics will not be displayed; rather it will show the the diagnostic from the underlying compiler relating to generated C code only.

Note that you can increase the number of errors shown with "--max-error" option of vv.

See also

Diagnostics
diagnostic-messages
See all
documentation

Documentation

Reference

Note: All the topics below are available as a single-page documentation. You can also download it as a .tar.gz file and use it on your computer, host it on your web site etc.

Application information
get-app
set-app

Base64
decode-base64
encode-base64

Client
FastCGI-API
FastCGI-command-line-client

Cookies
delete-cookie
get-cookie
set-cookie

Database
begin-transaction
commit-transaction
current-row
database-config-file
database-queries
delete-query
on-error
prepared-statements
query-result
rollback-transaction
run-query

Debugging
debugging
trace-run

Diagnostics
diagnostic-messages

Distributed computing
call-server
delete-server
new-server
read-server

Documentation
documentation

Encryption
decrypt-data
derive-key
encrypt-data
hash-string
random-crypto
random-string

Error handling
error-code
error-handling
on-error
report-error

Examples
example-client-API
example-cookies
example-create-table
example-develop-web-applications-in-C-programming-language
example-distributed-servers
example-docker
example-encryption
example-file-manager
example-form
example-hash-server
example-hello-world
example-how-to-design-application
example-how-to-use-regex
example-json
example-multitenant-SaaS
example-postgres-transactions
examples
example-sendmail
example-shopping
example-stock
example-uploading-files
example-using-mariadb-mysql
example-using-trees-for-in-memory-queries
example-utility
example-write-report

FIFO
new-fifo
purge-fifo
read-fifo
rewind-fifo
write-fifo

Files
close-file
copy-file
delete-file
file-position
file-storage
file-uploading
lock-file
open-file
read-file
read-line
rename-file
stat-file
temporary-file
uniq-file
unlock-file
write-file

General
about-Vely
application-architecture
deploying-application
how-vely-works
quality-control
rename-files
SELinux
vely-architecture
vely-removal
vely-version
vf
vv

Hash table
get-hash
new-hash
purge-hash
read-hash
resize-hash
write-hash

Hex encoding
decode-hex
encode-hex

JSON
delete-json
new-json
read-json

Language
dot
inline-code
statement-APIs
syntax-highlighting
unused-var

License
license

Memory
delete-mem
manage-memory
memory-handling
new-mem
resize-mem

Output
finish-output
flush-output
output-statement
p-dbl
pf-out
pf-url
pf-web
p-num
p-out
p-path
p-url
p-web

Process
global-process-data

Program execution
exec-program
exit-code

Program flow
do-once
exit-request

Quick start
123-hello-world

Regex
match-regex

Request information
get-req
if-task
input-param
request-body
set-input
set-req
task-param

Requests
after-request-handler
before-request-handler
building-URL
getting-URL
global-request-data
non-request
normalized-URL
request
request-handler
request-URL
startup-handler
vely-dispatch-request

Running application
application-setup
CGI
command-line
containerize-application
FastCGI
plain-C-FCGI

Strings
copy-string
count-substring
lower-string
num-string
split-string
trim-string
upper-string
write-string

System information
get-sys
OS-conditional-compilation

Time
get-time

Tree search
delete-tree
get-tree
new-tree
purge-tree
read-tree
use-cursor
write-tree

URL encoding
decode-url
encode-url

UTF8
json-utf8
utf8-json

Web
call-web
out-header
send-file
silent-header

Web encoding
decode-web
encode-web

Web servers
connect-apache-tcp-socket
connect-apache-unix-socket
connect-nginx-tcp-socket
connect-nginx-unix-socket

Man pages

Vely documentation is available online, and also as man pages (i.e. manual pages). You can take any of the topics above and type it in man, for example

man run-query

man how-vely-works

man example-shopping

The Vely section is '2vv', so in case of other software having conflicting topic names, you can also type

man 2vv run-query

man 2vv how-vely-works

man 2vv example-shopping

Do once

Purpose: Execute statements only once in a process.

do-once
    <any statements>
    ...
end-do-once

do-once will execute <any statements> only once in a single process regardless of how many requests that process serves. <any statements> end with end-do-once. The first time a process reaches do-once, <any statements> will execute; in all subsequent cases the program control will skip to immediately after end-do-once.

do-once cannot be nested, but otherwise can be used any number of times. <any statements> execute in the same scope as the code prior and after the do-once/end-do-once.

Typical use of do-once may be a one-time setup of variables or making calls that need to be performed only once per process.

Examples

In this example, a process-scoped hash (that is available to multiple requests of a single process) is created in the very first request a process serves and data is written to it; the subsequent requests do not create a new hash but rather just write to it.

...
do-once
new-hash define my_hash size 1024 process-scope
end-do-once
write-hash my_hash key my_key value my_data
...

See also

Program flow
do-once
exit-request
See all
documentation

Dot

Purpose: Write a single line of C code when conflicting with Vely code.

.<C code>;

If Vely code conflicts with C code, you can use the dot (".") statement to write a single line of C code - so each line of such code must be prefixed with "." (a dot). Note there doesn't have to be a space after the dot.

Examples

In this case, valid C expression exec-sql would be confused for "exec-sql" Vely statement:

num exec = 3;
num sql = 2;
printf ("%lld\n",
exec-sql);

You can simply prefix the offending code with the dot:

num exec = 3;
num sql = 2;
printf ("%lld\n",
.exec-sql);

See also

Language
dot
inline-code
statement-APIs
syntax-highlighting
unused-var
See all
documentation

Encode base64

Purpose: Base64 encode.

encode-base64 <data> to [ define ] <output data> \
    [ input-length <input length> ] \
    [ output-length [ define ] <output length> ]

encode-base64 will encode <data> into base64 string <output data>, which is allocated memory.

If "input-length" clause is used, then <input length> is the number of bytes encoded, otherwise <data> is treated as a null-terminated string and its length determined.

The result is stored in <output data> (in "to" clause), which can be created with optional "define". The length of the encoded string can be obtained via "output-length" clause in <output length>, which can be created with optional "define".

Base64-encoded strings are often used where binary data needs to be in a format that complies with certain text-based protocols, such as in attaching documents in email, or embedding binary documents (such as "JPG" files for example) in web pages, such as with images with "data:image/jpg;base64..." specified, etc.

Examples

Simple example that encodes a string and then decodes, and checks if they match:

// Original string, generally this would be binary data in most cases
char dt[]="  oh well  ";

// Encode in base64
encode-base64 dt to define out_dt

decode-base64 out_dt to define new_dt

if (!strcmp (dt, new_dt)) {
    @Success!
} else {
    @Failure!
}

In the next example, "input-length" and "output-length" clauses are used, and only a partial of the input string is encoded. The length of the final decoded string, as well as the string itself is checked against the original:

 // Original string, generally this would be binary data in most cases
char dt[]="  oh well  ";

// Encode in base64, encode only 6 bytes
encode-base64 dt input-length 6 to define out_dt output-length define out_len

decode-base64 out_dt input-length out_len to define new_dt output-length define new_len

if (new_len != 6) {
    @Failure!
} else {
    @Success!
}

if (!strncmp(dt,new_dt,6)) {
   @Success!
} else {
   @Failure!
}

See also

Base64
decode-base64
encode-base64
See all
documentation

Encode hex

Purpose: Encode data into hexadecimal string.

encode-hex <data> to [ define ] <output> \
    [ input-length <input length> ] \
    [ output-length [ define ] <output length> ] \
    [ prefix <prefix> ]

encode-hex will encode string <data> to hexadecimal string <output> given in "to" clause, which is null-terminated and consists of digits "0"-"9" and letters "a"-"f". <output> is allocated memory.

<data> can contain null-bytes and in general is any binary data, the length of which is given by <input length> in "input-length" clause. The output string <output> is allocated and its length is given in <output length> in "output-length" clause. If you wish to prefix the output with a null-terminated string <prefix>, you can specify it in "prefix" clause.

"output-length" clause is optional, and so is "prefix" (in which case no prefix is prepended). "input-length" clause is also optional, and if not specified, <input length> is taken as the string length of <data>, which is in that case assumed to be null-terminated.

Optional "define" subclause can be used to create <output> string and a number variable <output length> if they do not already exist.

Examples

Create hexadecimal string from binary data "mydata" of length 7, prefixed with string "\\\\x" (which is typically needed for PostgreSQL binary input to queries). The output string "hexout" is created and so is its length "olen" variable of type "num":

char *mydata = "\x00""A""\x00""\xF""AB""\x00""\x04";
encode-hex mydata to define hexout input-length 7 output-length define olen prefix "\\\\x"

The value of "hexout" will be:

\\x0041000F414200

See also

Hex encoding
decode-hex
encode-hex
See all
documentation

Encode url

Purpose: URL-encode string.

encode-url <string> to [ define ] <encoded string> \
    [ output-length [ define ] <encoded length> ] \
    [ input-length <length> ]

encode-url URL-encodes <string> and stores the result in <encoded string> which may be created with optional "define". <encoded string> is allocated memory.

Number <length> in the optional "input-length" clause lets you specify the number of bytes in <string> that will be encoded - if not specified or negative, it is the string length. "output-length" clause (also optional) lets you get the length of the encoded string in <encoded length>, which can be created with optional "define".

All bytes except alphanumeric and those from "-._~" (i.e. dash, dot, underscore and tilde) are encoded.

Examples

In this example, a string "str" is URL encoded and the result is in a "result" string variable, and its length is in "len_of_result" variable:

char str[]="  x=y?z&  ";
encode-url str to define result output-length define len_of_result

The "result" is "%20%20x%3Dy%3Fz%26%20%20" and "len_of_result" is 24.

See also

URL encoding
decode-url
encode-url
See all
documentation

Encode web

Purpose: Web(HTML)-encode string.

encode-web <string> to [ define ] <encoded string> \
    [ output-length [ define ] <encoded length> ] \
    [ input-length <length> ]

encode-web encodes <string> so it can be used in a HTML-like markup text (such as a web page or an XML/XHTML document), and stores the result in <encoded string> which may be created with optional "define". <encoded string> is allocated memory.

Optional "output-length" clause lets you get the length of the encoded string in <encoded length>, which can be created with optional "define". You can encode only the first <length> bytes, given by an "input-length" clause.

Examples

In this example, a string "str" will be web-encoded and the result is in "result" variable, with its length in "len_of_result" variable:

char str[]="  x<y>z&\"'  ";
encode-web str to define result output-length define len_of_result

The "result" is " x<y>z&"' " and "len_of_result" is 33.

See also

Web encoding
decode-web
encode-web
See all
documentation

Encrypt data

Purpose: Encrypt data.

encrypt-data <data> to [ define ] <result> \
    [ input-length <input length> ] \
    [ output-length [ define ] <output length> ] \
    [ binary [ <binary> ] ] \
    ( password <password> \
        [ salt <salt> [ salt-length <salt length> ] ] \
        [ iterations <iterations> ] \
        [ cipher <cipher algorithm> ] \
        [ digest <digest algorithm> ]
        [ cache ]
        [ clear-cache <clear cache> ) \
    [ init-vector <init vector> ]

encrypt-data encrypts <data> and stores the ciphertext to <result> specified by "to" clause, which can be created with optional "define".

Cipher and digest

By default, AES-256-CBC encryption and SHA256 hashing is used. You can however specify different cipher and digest algorithms with <cipher algorithm> (in "cipher" clause) and <digest algorithm> (in "digest" clause) as long as OpenSSL supports them, or you have added them to OpenSSL. You can see the available ones by using:

#get list of cipher providers
openssl list -cipher-algorithms

#get list of digest providers
openssl list -digest-algorithms

Note that the default algorithms will typically suffice. If you use different algorithms, you should have a specific reason. If you use a specific cipher and digest for encoding, you must use the same for decoding. The key derivation method is PBKDF2.

Data to be encrypted

If "input-length" clause is missing, then <data> is considered to be a null-terminated string and the number of bytes encrypted is its length. If "input-length" clause is used, then <input length> bytes are encrypted, regardless of whether <data> is a null-terminated string or not.

Password

String <password> (in "password" clause) is the password used to encrypt and it must be a null-terminated string.

Salt

Optional <salt> (in "salt" clause) is the salt used in Key Derivation Function (KDF) when an actual symmetric encryption key is created. If <salt length> (in "salt-length" clause) is not specified, then the salt is null-terminated, otherwise it is a binary value of length <salt length>. See random-string or random-crypto for generating a random salt. If you use the "salt" clause, then you must use the exact same <salt> when data is decrypted with decrypt-data - typically salt values are stored or transmitted unencrypted.

Iterations

The number of iterations used in producing a key is specified in <iterations> in optional "iterations" clause. The default is 1000 per RFC 8018, though depending on your needs and the quality of password you may choose a different value.

Initialization vector (IV)

Different encrypted messages should have a different IV value, which is specified with <init vector> in the "init-vector" clause. See random-string or random-crypto for generating IV values. The decrypting side must use the same IV value to decrypt the message. Just like salt, IV is not a secret and is transmitted in plain text. IV is generally a binary value and each cipher algorithm may require a certain number of bytes.

Encrypted data

The encrypted data is stored in <result> (in "to" clause), which you can create with optional "define". <result> is allocated memory. The encrypted data can be a binary data (if "binary" clause is present without optional boolean expression <binary>, or if <binary> evaluates to true), which is binary-mode encryption; or if not, it will be a null-terminated string, which is character-mode encryption, consisting of hexadecimal characters (i.e. ranging from "0" to "9" and "a" to "f"). Character mode of encryption is convenient if the result of encryption should be a human readable string, or for the purposes of non-binary storage in the database.

If this is a binary-mode encryption, then "output-length" clause can be used to get the length of the binary encrypted data in <output length> , which can be created with optional "define". In any case, if used, <output length> has the length of the encrypted data, which is the exact byte count in binary mode, or the length of encrypted string in character mode (i.e. the number of character bytes excluding the terminating null byte).

Caching key

A key used to actually encrypt/decrypt data is produced by using password, salt, cipher, digest and the number of iterations. Depending on these parameters (especially the number of iterations), computing the key can be a resource intensive and lengthy operation. You can cache the key value and compute it only once (or once in a while) by using "cache" clause. If you need to recompute the key once in a while, use "clear-cache" clause. <clear cache> is a "bool" variable; the key cache is cleared if it is true, and stays if it is false. For example with encrypt-data (the same applies to decrypt-data):

bool clear;
if (q == 0) clear = true; else clear = false;
encrypt-data dt init-vector non password pwd \
    salt rs salt-length 10 iterations iter to \
    define dt_enc cache clear-cache clear

In this case, when "q" is 0, cache will be cleared, with values of password, salt and iterations presumably changed, and the new key is computed and then cached. In all other cases, the last computed key stays the same. Normally, with IV usage (in "init-vector" clause), there is no need to change the key often, or at all.

Note that while "cache" clause is in effect, the values for "password", "salt", "cipher", "digest" and "iterations" clauses can change without any effect. Only when "clear-cache" evaluates to "true" are those values taken into account.

Safety

Unless you are encrypting/decrypting a single message, you should always use IV in "init-vector" clause. Its purpose is to randomize the data encrypted, so that same messages do not produce the same ciphertext.

If you use salt, a random IV is created with each different salt value. However, different salt values without "cache" clause will regenerate the key, which may be computationally intensive, so it may be better to use a different IV instead for each new encryption and keep the salt value the same with the high number of iterations. In practicality this means using "cache" so that key is computed once per process with the salt, and IV changes with each message. If you need to recompute the key occasionally, use "clear-cache".

Each cipher/digest combination carries separate recommendations about the usage of salt, IV and the number of iterations. Please consult their documentation for more details.

Examples

In the following example, the data is encrypted, and then decrypted, producing the very same data:

// Original string to encrypt
char *orig_data="something to encrypt!";

// Encrypted data is in "res" variable
encrypt-data orig_data password "mypass" to define res

// Decrypt what was just encrypted, decrypted data is in "dec_data"
decrypt-data res password "mypass" to define dec_data

// Check that decrypted data matches the original 
if (!strcmp (orig_data, dec_data)) {
    @Success!
} else {
    @Failure!
}

A more involved example below encrypts specific number of bytes (6 in this case). random-string is used to produce salt. The length of data to encrypt is given with "input-length" clause. The encrypted data is specified to be "binary" (meaning not as a human-readable string), so the "output-length" of such binary output is specified. The decryption thus uses "input-length" clause to specify the length of data to decrypt, and also "output-length" to get the length of decrypted data. Finally, the original data is compared with the decrypted data, and the length of such data must be the same as the original (meaning 6):

// Original data (only the first 6 bytes are encrypted)
char *orig_data="something to encrypt!";

// Get 8 random binary bytes to be the salt
random-string to define newsalt length 8 binary

// Encrypt data using salt and produce binary output (meaning it's not a null-terminated character string), with the
// length of such output in "encrypted_len" variable.
encrypt-data orig_data input-length 6 output-length define encrypted_len password "mypass" salt newsalt to define res binary

// Decrypt the data encrypted above. The length of encrypted data is passed in "encrypted_len" variable, and then length of decrypted data
// is obtained in "decrypted_len" variable.
decrypt-data res output-length define decrypted_len password "mypass" salt newsalt to define dec_data input-length encrypted_len binary

// Check if the 6 bytes of the original data matches decrypted data, and if exactly 6 bytes was decrypted
if (!strncmp(orig_data,dec_data, 6) && decrypted_len == 6) {
    @Success!
} else {
    @Failure!
}

An example of using different algorithms:

encrypt-data "some data!" password "mypwd" salt rs1 to encd1 cipher "camellia-256-cfb1" digest "sha3-256"
decrypt-data encd1 password "mypwd" salt rs1 to decd1 cipher "camellia-256-cfb1" digest "sha3-256"

See also

Encryption
decrypt-data
derive-key
encrypt-data
hash-string
random-crypto
random-string
See all
documentation

Error code

Many Vely statements return status with VV_ERR_... error codes, which are generally descriptive to a point. Such status is not as detailed as the operating system "errno" variable, however you can use "errno" clause in get-req statement to obtain the last known errno value from aforementioned statements. You should obtain this value as soon as possible after the statement because another statement may set it afterwards.

In the following example, a directory is attempted to be deleted via delete-file, which will fail with VV_ERR_DELETE - however you can get a more specific code via "errno" (which in this case is "21", or "EISDIR", which means that it cannot delete a directory with this statement):

delete-file "some_directory" status define stc
if (stc == VV_ERR_DELETE) {
    get-req errno to define e
    @Cannot delete file
    pf-out "Error %lld\n", e
}

Note that with some VV_ERR_... codes, the "errno" clause in get-req may return 0. This means the error was detected by Vely and not reported by the operating system.

See also

Error handling
error-code
error-handling
on-error
report-error
See all
documentation

Error handling

When your program errors out

"Erroring out" means your program has encountered insurmountable difficulty and it will end. For instance, it could be out of memory, or the database is permanently down and connection cannot be re-established. These are errors that cannot be handled. If your program is started with vf, it may be automatically restarted.

When report-error is called

You can report an error in your Vely code with report-error, after which:

if it is started with vf (such as a FastCGI application), the program will not exit, but rather it will skip the rest of the request and move on to handle the next request, or
if it is a command-line application, it will exit.

When there is a problem in Vely

If there is a fatal internal error (i.e. error in Vely code itself that cannot be handled), it will be caught by Vely, and the program will end. If your program is started with vf, it may be automatically restarted.

Logging the error

Regardless of the type of error and regardless of whether the program exits or not, the error is logged and the program stack with full source code lines (see vv for including debug information) will be written to backtrace file (use -e option of vv to obtain its location). Note that the program stack is logged only if Vely is built in debugging mode (see "DI=1" option when making Vely from source); otherwise, production code may be slowed down by stack dumping.

You can see the list of last N errors (and the location of file containing backtrace for them) by using vv, for instance to see the last 3 errors:

vv -e 3

See also

Error handling
error-code
error-handling
on-error
report-error
See all
documentation

How to connect to Vely with API: multi-threaded and single-threaded calls

Vely and FastCGI

The way Vely receives client requests is via FastCGI, which is a high-performance binary protocol for communication between servers, clients, and in general programs of all kinds. Originally it was created to rectify shortcomings of CGI (Common Gateway Interface) protocol, in order to dramatically increase its performance. Over time, FastCGI emerged as a great protocol for high performance server applications.

The reason for this is that it allows client requests to be handled without starting up and shutting down server threads or processes; the inter-process communication is especially fast with Unix sockets. In addition, the protocol is binary and very lean, meaning it provides high performance. Typically it's used on secure networks as it doesn't have any security built-in (which is one of the reasons for high performance), such as behind front-facing web server(s), on secure intranets etc.

Here you will learn how to connect to any FastCGI server (including Vely and PHP FPM) from virtually any programming language (that has C linkage, which is most of them), using FastCGI-API; that's the reason examples are written in C.

Prerequisites

This example uses API that comes with Vely framework (at least 17.1.3 or later should be installed to run these examples). The examples are for Ubuntu 22, so you can install Vely with apt packager or from source. You can also install it for other Linux distros.

Alternatively, if you are using API to connect to a non-Vely server (i.e. PHP or some other), you can do so without installing Vely - see "Using API without Vely" in FastCGI-API; in this case you cannot run the Vely server example below. Note that in this case you also must install gcc beforehand.

What will you do here

You will connect to a Vely server (and to PHP FPM too) using API, and run code on those servers on behalf of a client, meaning send data and receive a reply. Both single- and multi-threaded examples are included.

Setup a Vely server

In order to run a client example with a Vely server, you need to setup and run a server first. First, create Vely application named "example" in a new directory:

mkdir client
cd client
sudo vf -i -u $(whoami) example

To see highlighting for Vely code in vim, run this just one time:

vv -m

Then create source Vely file:

vi echo.vely

and copy this:

#include "vely.h"

void echo() {
    out-header default
    input-param par
    @Input is <<p-out par>>
}

Build the application:

vv -q

And start a server, in this case with 5 worker processes:

vf -w 5 example

Once you've done this, you can proceed to build a client and call the server.

Simple example with Vely

Create a file:

vi api.c

and copy this:

#include "vfcgi.h"

void main ()
{
    // Request type vv_fc - create a request variable and zero it out
    vv_fc req = {0};

    req.fcgi_server = "/var/lib/vv/example/sock/sock"; // Unix socket
    req.req_method = "GET"; // GET HTTP method
    req.app_path = "/example"; // application path
    req.req = "/echo"; // request name
    req.url_payload = "par=99";

    // Make a request
    int res = vv_fc_request (&req);

    // Check return status, and if there's an error, display error code and the
    // corresponding error message. Otherwise, print out server response.
    if (res != VV_OKAY) printf ("Request failed [%d] [%s]\n", res, vv_fc_error(&req));
    else printf ("%s", vv_fc_data(&req));

    // Free up resources so there are no memory leaks
    vv_fc_delete(&req);
}

The example is very simple and fairly self-explanatory. A few things you must always have (see FastCGI-API):

"fcgi_server" member of "req" variable (of "vv_fc" type) says how you plan to connect to a FastCGI server. In the example, you'd use a local Unix socket for simplicity and performance; use TCP socket only when connecting to another computer or if you have a good reason otherwise.
"req_method" is the HTTP method, which in this case is "GET", but it could be "POST", "PUT", "DELETE", or anything you want.
"app_path" is the SCRIPT_NAME.
"req" is the request name, or in HTTP talk it's the leading path segment of PATH_INFO. See request-URL for more on application path (i.e. "app_path"), request path (i.e. "req"), and URL payload (i.e. "url_payload").

Also provided is "url_payload", which in terms of HTTP environment variables is really the rest of PATH_INFO (divided by forward slashes "/", or up to "?" if there's one), plus QUERY_STRING (in the form of "name=value" pairs, or after "?", if there's one). Finally "vv_fc_request()" calls the server and you can use "vv_fc_error()" to get any error response and "vv_fc_data()" to get the data reply. Note that error and data may be interwoven; not to worry, the API will separate the two as proper streams. To delete memory used by this call, use "vv_fc_delete()". That's all for a simple example!

To build the client, execute:

gcc -o api api.c $(vv -i)

Run it:

./api

The result is:

Content-type: text/html;charset=utf-8
Cache-Control: max-age=0, no-cache
Pragma: no-cache
Status: 200 OK

Input is 99

This is the expected result.

Multi-threaded example with Vely

The next example is a MT (multi-threaded) client. You will make 100 simultaneous calls to a Vely server. The code that does this is very similar to the simple example, with one addition: you'll pass along an extra environment variable, in this case "VV_SILENT_HEADER" with value "yes", which will suppress HTTP header output from the server. Otherwise, the "url_payload" is dynamically constructed, so that you can display input "0" through "99".

In the "main()" function, you will create 100 threads and call "call_server()" function that many times in parallel, then wait for all of them to finish. The result of each thread (i.e. if a call to the server was successful) is passed to "main()" as a return value of "call_server()".

Create client C file:

vi api_mt.c

Copy the following:

#include "pthread.h"
#include "assert.h"
#include "vfcgi.h"
#define REQ_LEN 200
#define MT_RUNS 100

void *call_server (void *inp)
{
    // Request type vv_fc - create a request variable and zero it out
    vv_fc req = {0};

    req.fcgi_server = "/var/lib/vv/example/sock/sock"; // Unix socket
    req.req_method = "GET"; // GET HTTP method
    req.app_path = "/example"; // application path
    req.req = "/echo"; // request name
    char *env[3];
    env[0]="VV_SILENT_HEADER";
    env[1]="yes";
    env[2]=NULL;
    req.env = env;

    req.url_payload = (char*)malloc (REQ_LEN); assert(req.url_payload);
    snprintf (req.url_payload, REQ_LEN, "par=%ld", (off_t)inp);

    int res = vv_fc_request (&req);

    // Check return status, and if there's an error, display error code and the
    // corresponding error message. Otherwise, print out server response.
    if (res != VV_OKAY) {
        fprintf (stderr, "Request failed [%d] [%s]\n", res, vv_fc_error(&req));
    }
    else {
        printf ( "%s", vv_fc_data(&req));
    }

    // Free up resources so there are no memory leaks
    vv_fc_delete(&req);
    return (void*)(off_t)res;
}

int main ()
{
    // Make a request
    pthread_t thread_id[MT_RUNS+1];
    int i;
    for (i = 0; i < MT_RUNS; i++){
        pthread_create(&(thread_id[i]), NULL, call_server, (void*)(off_t)i);
    }
    int bad = 0;
    void *thread_res[MT_RUNS+1];
    for (i = 0; i < MT_RUNS; i++){
        pthread_join(thread_id[i], &(thread_res[i]));
        int r = (int)(off_t)(thread_res[i]);
        if (r != VV_OKAY) bad++;
    }
    if (bad!=0) {
        fprintf (stderr, "Total [%d] bad\n", bad);
        return -1;
    } else {
        printf ("All okay return value\n");
        return 0;
    }
}

To build the client, execute:

gcc -o api_mt api_mt.c $(vv -i) -lpthread

Run it:

./api_mt

The result is, as expected (the input numbers are somewhat randomly dispersed since all clients work truly in parallel):

Input is 0
Input is 3
Input is 4
Input is 6
Input is 5
Input is 2
Input is 1
Input is 7
Input is 8
...
Input is 71
Input is 74
Input is 75
Input is 78
Input is 77
Input is 70
Input is 65
Input is 76
Input is 67
All okay return value

Setup PHP FPM server

The PHP example is for Ubuntu with Apache, however you can adapt it to your particular distribution. The PHP FPM version used is 8.1; if you're using a different version, replace "8.1" with your own.

First, if you don't already have Apache web server server installed:

sudo apt install apache2

Then, if you don't already have PHP FPM server installed:

sudo apt install php-fpm

In order for your client program to be able to write PHP FPM's Unix socket, you can change the configuration for it - in this case you'd make the group be the same as yours, thus giving you permission to write to the socket. To do this, edit file:

sudo vi /etc/php/8.1/fpm/pool.d/www.conf

and change line with "listen.group" to your group name (which is normally the same as your OS user name):

listen.group = <your login user>

Then restart PHP FPM service:

sudo service php8.1-fpm restart

First create a simple PHP program to just output what the input parameter was. Create file:

sudo vi /var/www/html/example.php

and copy this:

<?php
echo "Input is " . $_GET['par'] . "\n";
?>

Make sure PHP file is accessible to PHP FPM (which runs in the same ownership context as the web server, meaning "www-data" user):

sudo chown www-data:root /var/www/html/example.php

Simple example with PHP FPM

The C program to connect and call the above PHP file is simple. Note that the version of PHP used is "8.1", so if you're using a different one, replace "php8.1" with your own. You're also using a Unix socket to connect, the request method is "GET", and the rest is setup for a very simple invocation of a PHP script. "url_payload" is in the form of a query string, and PHP also expects SCRIPT_FILENAME to be set to where you saved the "example.php" file; this demonstrates usage of environment variables. Overall, you can take this simple example and adjust it to serve your needs.

Create client C file:

vi php_api.c

and copy this:

#include "vfcgi.h"

void main ()
{
    // Request type vv_fc - create a request variable and zero it out
    vv_fc req = {0};

    req.fcgi_server = "/run/php/php8.1-fpm.sock"; // Unix socket
    req.req_method = "GET"; // GET HTTP method
    req.app_path = "/"; // application path
    req.req = "/example.php"; // request name
    req.url_payload = "par=99";

    char *env[3];
    env[0]="SCRIPT_FILENAME";
    env[1]="/var/www/html/example.php";
    env[2]=NULL;
    req.env = env;

    // Make a request
    int res = vv_fc_request (&req);

    // Check return status, and if there's an error, display error code and the
    // corresponding error message. Otherwise, print out server response.
    if (res != VV_OKAY) printf ("Request failed [%d] [%s]\n", res, vv_fc_error(&req));
    else printf ("%s", vv_fc_data(&req));

    // Free up resources so there are no memory leaks
    vv_fc_delete(&req);
}

The example is very similar to Vely one - see the discussion there. To build the client, execute:

gcc -o php_api php_api.c $(vv -i)

Run it:

./php_api

The result is, as expected:

Content-type: text/html; charset=UTF-8

Input is 99

Multithreaded example with PHP FPM

The next example is a MT (multi-threaded) client connecting to PHP FPM. You will make 100 simultaneous calls to PHP FPM server. The code that does this is very similar to the simple example, with the exception that the "url_payload" is dynamically constructed, so that you can display input from "0" through "99".

In the "main()" function, you will create 100 threads and call "call_server()" function that many times in parallel, then wait for all of them to finish. The result of each thread (i.e. if a call to the server was successful) is passed to "main()" as a return value of "call_server()".

Create client C file:

vi php_api_mt.c

Copy the following:

#include "pthread.h"
#include "assert.h"
#include "vfcgi.h"
#define REQ_LEN 200
#define MT_RUNS 100

void *call_server (void *inp)
{
    // Request type vv_fc - create a request variable and zero it out
    vv_fc req = {0};

    req.fcgi_server = "/run/php/php8.1-fpm.sock"; // Unix socket
    req.req_method = "GET"; // GET HTTP method
    req.app_path = "/"; // application path
    req.req = "/example.php"; // request name

    char *env[3];
    env[0]="SCRIPT_FILENAME";
    env[1]="/var/www/html/example.php";
    env[2]=NULL;
    req.env = env;

    req.url_payload = (char*)malloc (REQ_LEN); assert(req.url_payload);
    snprintf (req.url_payload, REQ_LEN, "par=%ld", (off_t)inp);

    int res = vv_fc_request (&req);

    // Check return status, and if there's an error, display error code and the
    // corresponding error message. Otherwise, print out server response.
    if (res != VV_OKAY) {
        fprintf (stderr, "Request failed [%d] [%s]\n", res, vv_fc_error(&req));
    }
    else {
        printf ( "%s", vv_fc_data(&req));
    }

    // Free up resources so there are no memory leaks
    vv_fc_delete(&req);
    return (void*)(off_t)res;
}

int main ()
{
    // Make a request
    pthread_t thread_id[MT_RUNS+1];
    int i;
    for (i = 0; i < MT_RUNS; i++){
        pthread_create(&(thread_id[i]), NULL, call_server, (void*)(off_t)i);
    }
    int bad = 0;
    void *thread_res[MT_RUNS+1];
    for (i = 0; i < MT_RUNS; i++){
        pthread_join(thread_id[i], &(thread_res[i]));
        int r = (int)(off_t)(thread_res[i]);
        if (r != VV_OKAY) bad++;
    }
    if (bad!=0) {
        fprintf (stderr, "Total [%d] bad\n", bad);
        return -1;
    } else {
        printf ("All okay return value\n");
        return 0;
    }
}

To build the client, execute:

gcc -o php_api_mt php_api_mt.c $(vv -i) -lpthread

Run it:

./php_api_mt

The result is, as expected (note that since clients work truly in parallel, the numbers are somewhat randomly dispersed):

Content-type: text/html; charset=UTF-8

Input is 0
Content-type: text/html; charset=UTF-8

Input is 3
Content-type: text/html; charset=UTF-8

Input is 5
Content-type: text/html; charset=UTF-8

Input is 4
Content-type: text/html; charset=UTF-8

Input is 1
Content-type: text/html; charset=UTF-8

Input is 2
Content-type: text/html; charset=UTF-8

Input is 6
Content-type: text/html; charset=UTF-8

...

Input is 93
Content-type: text/html; charset=UTF-8

Input is 80
Content-type: text/html; charset=UTF-8

Input is 95
Content-type: text/html; charset=UTF-8

Input is 97
Content-type: text/html; charset=UTF-8

Input is 99
Content-type: text/html; charset=UTF-8

Input is 98
Content-type: text/html; charset=UTF-8

Input is 96
Content-type: text/html; charset=UTF-8

Input is 94
All okay return value

Conclusion

You have learned how to connect to any FastCGI server (in this case Vely and PHP FPM) from a program with C linkage (which can be used with any programming language that has it), using FastCGI-API. You have also learned how to make many parallel API calls using Linux threads. You can use this knowledge to call server code from any other application and receive results.

Cookies in a web application

A value is entered in the browser and saved as a cookie, then read back later. This example displays a web form. When it is submitted, the input is used to set a cookie in response. Then in a separate page, the cookie value is obtained and displayed. This is the basic mechanism often used in saving web application states and session information.

In a nutshell: web browser; Apache; Unix sockets; 1 source files, 48 lines of code.

Screenshots of application

A web form is displayed where the user can enter a name. Clicking "Submit" sends it to the server:

A reply page when the server receives the information and sets the cookie in the browser:

A request sent to the server will now have the cookie, which the server code will read and display back:

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because it is used in this example, you will need to install Apache as a web server.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

The source code is a part of Vely installation. It is a good idea to create a separate source code directory for each application (and you can name it whatever you like). In this case, unpacking the source code will do that for you:

tar xvf $(vv -o)/examples/cookies.tar.gz
cd cookies

Setup application

The very first step is to create an application. The application will be named "cookies", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) cookies

This will create a new application home (which is "/var/lib/vv/cookies") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Build application

Use vv utility to make the application:

vv -q

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 cookies

This will start 3 daemon processes to serve the incoming requests. You can also start an adaptive server that will increase the number of processes to serve more requests, and gradually reduce the number of processes when they're not needed:

vf cookies

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit cookies

Setup web server

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/cookies" is the application path (see request-URL) and "cookies" is your application name):

ProxyPass "/cookies" unix:///var/lib/vv/cookies/sock/sock|fcgi://localhost/cookies

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/cookies" (such as for example "/cookies.html" or "/cookies_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/cookies", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Enter cookie 
http://127.0.0.1/cookies/cookies?action=enter_cookie

# Query entered cookie 
http://127.0.0.1/cookies/cookies?action=query_cookie

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Source files

The following are the source files in this application:
- Working with cookies (cookies.vely)
Usage of cookies is simple - get-cookie is used to obtain the cookie from a web client (i.e. a browser), and set-cookie to set the cookie in the browser when the response is sent back.

Three tasks (based on "action" task parameter) allow user to enter a value that is sent back to the server (see "enter_cookie" as the "action" parameter), which the server then sends back to the browser in response (see "save_cookie"), and finally with "query_cookie" this value can be obtained and displayed again. This demonstrates how server-side software uses cookies to manipulate and obtain the client state.



#include "vely.h"

request-handler /cookies

    task-param action

    if-task "enter_cookie"

        // Display a form to get cookie value
        out-header default
        @<h2>Enter your name</h2>
        @<form action="<<p-path>>/cookies" method="POST">
        @    <input type="hidden" name="action" value="save_cookie">
        @    <label for="cookie_value">Your name:</label><br/>
        @    <input type="text" name="cookie_value" value=""><br/>
        @    <br/>
        @    <input type="submit" value="Submit">
        @</form>

    else-task "save_cookie"

        // Submittal of form: save the cookie through response to the browser
        input-param cookie_value
        get-time to define cookie_expiration year 1 timezone "GMT"
        set-cookie "customer_name" = cookie_value  expires cookie_expiration  path "/"
        out-header default
        @Cookie sent to browser!
        @<hr/>

    else-task "query_cookie"

        // Web request that delivers cookie value back here (to server); display it.
        get-cookie define name="customer_name"
        out-header default
        @Customer name is <<p-web name>>
        @<hr/>

    else-task other
        out-header default
        @Unrecognized action<hr/>
    end-task

end-request-handler

See also

Using DDL and DML with database

Example of manipulating tables via SQL. Table is dropped, created, data inserted, then queried, and finally it is dropped.

In a nutshell: PostgreSQL; command line; web browser; Nginx; Unix sockets; 2 source files, 43 lines of code.

Screenshots of application

The web output from creating a table, inserting data, selecting and dropping it:

The same output, only when the program runs from the command line. Set VV_SILENT_HEADER to yes to skip the header output:

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because they are used in this example, you will need to install Nginx as a web server and PostgreSQL as a database.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

tar xvf $(vv -o)/examples/create-table.tar.gz
cd create-table

Setup application

The very first step is to create an application. The application will be named "create-table", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) create-table

This will create a new application home (which is "/var/lib/vv/create-table") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create PostgreSQL database "db_create_table". You can change the database name, but remember to change it everywhere here. And then, you will create database objects in the database.

Note the example here uses peer-authentication, which is the default on all modern PostgreSQL installations - this means the database user name is the same as the Operating System user name.

Execute the following in PostgreSQL database as root (using psql utility):

echo "create user $(whoami);
create database db_create_table with owner=$(whoami);
grant all on database db_create_table to $(whoami);
\q
" | sudo -u postgres psql

Next, login to database db_create_table and create the database objects for the application:

psql -d db_create_table -f setup.sql

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_create_table". This name doesn't have to be "db_create_table", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo "user=$(whoami) dbname=db_create_table"  > db_create_table

The above is a standard postgres connection string that describes the login to the database you created. Since Vely uses native PostgreSQL connectivity, you can specify any connection string that your database lets you.

Build application

Use vv utility to make the application:

vv -q --db=postgres:db_create_table

Note usage of --db option to specify PostgreSQL database and the database configuration file name.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 create-table

vf create-table

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit create-table

Setup web server

This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/create-table" is the application path (see request-URL) and "create-table" is your application name):

location /create-table { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/create-table/sock/sock; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

Note: you must not have any other URL resource that starts with "/create-table" (such as for example "/create-table.html" or "/create-table_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/create-table", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Create, use and drop table, show output 
http://127.0.0.1/create-table/create-table

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Access application server from command line

To access your application server from command line (instead through web browser/web server), use this to see the application response:

#Create, use and drop table, show output 
export CONTENT_TYPE=
export CONTENT_LENGTH=
export REQUEST_METHOD=GET
export SCRIPT_NAME='/create-table'
export PATH_INFO='/create-table'
export QUERY_STRING=''
cgi-fcgi -connect /var/lib/vv/create-table/sock/sock /

Note: to suppress output of HTTP headers, add this before running the above:

export VV_SILENT_HEADER=yes

If you need to, you can also run your application as a CGI program.

Run program from command line

Execute the following to run your application from command line (as a command-line utility):

#Create, use and drop table, show output 
vv -r --app='/create-table' --req='/create-table?' --method=GET --exec

You can also omit "--exec" option to output the bash code that's executed; you can then copy that code to your own script. Note: to suppress output of HTTP headers, add "--silent-header" option to the above.
Note: if running your program as a command-line utility is all you want, you don't need to run an application server.

Source files

The following are the source files in this application:
- Setup database objects (setup.sql)
Create a table used in the example:

create table if not exists my_table (col1 bigint);

- Creating, using and dropping a table (create_table.vely)
DDL (Data Definition Language), DML (Data Definition Language) and SELECT SQL is used in this example. A table is dropped (if it exists), created, data inserted, queried, and then table is dropped again:


#include "vely.h"

%% /create-table

    out-header default

    // Drop existing table
    run-query @db_create_table = "drop table if exists my_table"  error define err error-text define err_text no-loop
    if (strcmp (err, "0")) {
        report-error "Trouble dropping table, error [%s], error text [%s]", err, err_text
    }
    @Table dropped!<br/>

    // Create table
    run-query @db_create_table =  "create table if not exists my_table (my_number bigint)"  error err error-text err_text no-loop
    if (strcmp (err, "0")) {
        report-error "Trouble creating table, error [%s], error text [%s]", err, err_text
    }
    @Table created!<br/>

    // Insert data into table
    run-query @db_create_table="insert into my_table (my_number) values ('%s'), ('%s'), ('%s')" : "100", "200", "400" affected-rows define nrows
    end-query
    @Added <<pf-out "%lld", nrows>> rows!<br/>

    // Select data we just inserted
    @Getting data we inserted:<br/>
    run-query @db_create_table="select my_number from my_table" output my_number
        @I got number <<query-result my_number>>!<br/>
    end-query

    // Drop the table again
    run-query @db_create_table =  "drop table if exists my_table" error err error-text err_text no-loop
    if (strcmp (err, "0")) {
        report-error "Trouble creating table, error [%s], error text [%s]", err, err_text
    }
    @Table dropped!<br/>

%%

See also

Develop web applications in C programming language

Vely is a framework and language that generates C code underneath. Because it has lots of functionality, you may not need to write much C code, if any. However, if you need to, you can write as much C code as necessary. The example here demonstrates this.

Your C application

Create new "c-app" application first, in a new directory (you can name it anything you like):

mkdir -p vely_c
cd vely_c

The vf command is a Vely program manager and it will create a new application (see how-vely-works) named "c-app":

sudo vf -i -u $(whoami) c-app

To get vim highlighting of Vely syntax:

vv -m

Create a source code file "using_c.vely":

vi using_c.vely

and copy and paste this to it:

#include <math.h>

int factorial(int num);

%% /using-c
    out-header default
    input-param inp
    int res = factorial (atoi(inp)); // call C function
    @Factorial of <<p-out inp>> is <<p-num res>>!
    double sr = sqrt ((double)res); // call standard C's sqrt
    @And its square root is <<p-dbl sr>>!
%%

// C function to calculate factorial of a number 'num'
int factorial(int num)
{
    int res = 1;
    int i;
    for (i = 2; i <= num; i++) {
        res *= i;
    }
    return res;
}

Note that the source file name ("using_c.vely") should match the request name, which is "using-c". If you're using hyphens (which is useful for web applications), just substitute with underscore. The fact that a request is implemented in a file with the same name helps keep your applications neat, tidy and easy to peruse.

The code here is pretty self-explanatory. You'd get an input-parameter "inp", call a C function "factorial()", then get the square root of it. As you go along, you output the results of computation. Included is C's header file "math.h" since you're using math function here to calculate a square root. Just like in any other C program, you'd declare the function up top. The request itself (between "%%" signs) will translate into a C function with the name that's a decorated version of a request name "/using-c", in this case just "using_c()", see request-handler for more.

Make an executable program

Now, make a native executable:

vv -q --lflag=-lm

Note the use of "--lflag" option that lets you specify additional C linker options, in this case you're using the math library ("-lm"). You can also add C flags for compilations (like -D for defines for instance) using "--cflag".

Execute from command line

You can now run your program! Here's how to do it from command line - you'd specify input parameter "inp" to have value of "5":

vv -r --req="/using-c?inp=5" --exec --silent-header

The result:

Factorial of 5 is 120!
And its square root is 10.954451!

That's a success right there! How did this work? Here you're using vv utility to call a program for convenience, but if you omit "--exec" option:

vv -r --req="/using-c?inp=5" --silent-header

here's what you get:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=yes
export REQUEST_METHOD=GET
export SCRIPT_NAME="/c-app"
export PATH_INFO="/using-c"
export QUERY_STRING="inp=5"
/var/lib/vv/bld/c-app/c-app

This is what's executed with "--exec" option. The above output you can copy and paste to your bash scripts to directly execute your program, which is located at "/var/lib/vv/bld/c-app/c-app". You can see that input parameter "inp" is provided as query string "inp=5". This is all very neat, because this is how web programs work and you can run this program from the web without modifications! That's next.

Run as application server

First, try running your program as an application server. That means a daemon, a resident server process that remains in memory and can serve many requests simultaneously. With Vely, that's a breeze, because it will take care of all the infrastructure (that's why it's a "framework"). Here's how you do that:

vf -w 5 c-app

The above will start 5 application server processes to serve incoming requests (you can also have a dynamic number of processes too, see vf). Testing your server is easy:

vv -r --req="/using-c?inp=5" --exec --server --silent-header

Note the "--server" option. It says to contact a server and execute the same request as before. But now, each of the 5 processes you started is staying resident in memory and serving the incoming requests. This way, your server can serve a large number of concurrent requests in parallel. Because each process stays alive, you get great performance.

Again, you can see what's going on behind scenes by omitting "--exec":

vv -r --req="/using-c?inp=5" --server --silent-header

The result being:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=yes
export REQUEST_METHOD=GET
export SCRIPT_NAME="/c-app"
export PATH_INFO="/using-c"
export QUERY_STRING="inp=5"
cgi-fcgi -connect /var/lib/vv/c-app/sock/sock  /c-app

Here a "cgi-fcgi" client program will contact the server you started, get a response, and print it out. You can make your own client application by using FastCGI-API; this way you can do whatever you want with the response, and you can do so in a multi-threaded application, since Vely's FastCGI API is MT-safe.

Run as web application

Of course, your application server will probably serve web requests. Check out connect-apache-unix-socket on how to connect Apache web server to your application server, or the same for Nginx: connect-nginx-unix-socket. FastCGI is supported widely among web servers, so you can use pretty much any web server of your choice.

Here's a brief intro for Nginx:

Setup web server

This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/c-app" is the application path (see request-URL) and "c-app" is your application name):

location /c-app { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/c-app/sock/sock; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

Note: you must not have any other URL resource that starts with "/c-app" (such as for example "/c-app.html" or "/c-app_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/c-app", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Call your application server to calculate factorial of 5 
http://127.0.0.1/c-app/using-c?inp=5

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

The result is, from the browser:

Conclusion

In this article you've learned how to build web applications in C programming language. The same code makes a command line program, application server and a web application.

See also

How to write distributed applications

What is distributed computing

Distributed computing is two or more servers communicating for a common purpose. Typically, some tasks are divvied up between a number of computers, and they all work together to accomplish it. Note that "separate servers" may mean physically separate computers. It may also mean virtual servers such as Virtual Private Servers (VPS) or containers, that may share the same physical hardware, though they appear as separate computers on the network.

There are many reasons why you might need this kind of setup. It may be that resources needed to complete the task aren't all on a single computer. For instance, your application may rely on multiple databases, each residing on a different computer. Or, you may need to distribute requests to your application because a single computer isn't enough to handle them all at the same time. In other cases, you are using remote services (like a REST API-based for instance), and those by nature reside somewhere else.

In any case, the computers comprising your distributed system may be on a local network, or they may be worldwide, or some combination of those. The throughput (how many bytes per second can be exchanged via network) and latency (how long it takes for a packet to travel via network) will obviously vary: for a local network you'd have a higher throughput and lower latency, and for Internet servers it will be the opposite. Plan accordingly based on the quality of service you'd expect.

How servers communicate

Depending on your network(s) setup, different kinds of communication are called for. If two servers reside on a local network, then they would typically used the fastest possible means of communication. A local network typically means a secure network, because nobody else has access to it but you. So you would not need TSL/SSL or any other kind of secure protocol as that would just slow things down.

If two servers are on the Internet though, then you must use a secure protocol (like TSL/SSL or some other) because your communication may be spied on.

Local network distributed computing

Most of the time, your distributed system would be on a local network. Such network may be separate and private in a physical sense, or (more commonly) in a virtual sense, where some kind of a Private Cloud Network is established for you by the Cloud provider. It's likely that separation is enforced by specialized hardware (such as routers and firewalls) and secure protocols that keep networks belonging to different customers separate. This way, a "local" network can be established even if computers on it are a world apart, though typically they reside as a part of a larger local network.

Either way, as far as your application is concerned, you are looking at a local network. Thus, the example here will be for such a case, as it's most likely what you'll have. A local network means different parts of your application residing on different servers will use some efficient protocol based on TCP/IP. One such protocol is FastCGI, a high-performance binary protocol for communication between servers, clients, and in general programs of all kinds, and that's the one used here. So in principle, the setup will look like this (there'll be more details later):

Prerequisites

To begin with, install Vely (minimum version 17.3), which will be used to create application servers. Note that you can use similar code to call remote PHP FPM services, as it also uses FastCGI protocol!

Next, in theory you should have two servers, however in this example both servers will be on the same localhost (i.e. "127.0.0.1"). This is just for simplicity; the code is exactly the same if you have two different servers on a local network - simply use another IP (such as "192.168.0.15" for instance) for your "remote" server instead of local "127.0.0.1". The two servers do not even necessarily need to be physically two different computers. You can start a Virtual Machine (VM) on your computer and host another virtual computer there. Popular free software like VirtualBox or KVM Hypervisor can help you do that.

In any case, in this example you will start two simple application servers; they will communicate with one another. The first one will be called "local" and the other one "remote" server. The local application server will make a request to the remote one.

Local server

On a local server, create a new directory for your local application server source code:

mkdir local_server
cd local_server

and then create a new file "status.vely" with the following:

#include "vely.h"

request-handler /status
    silent-header
    out-header default
    // input parameter: the IP address of remote server
    input-param server
    // input parameter: number of days to ask the status for
    input-param days

    // Create URL payload for remote server
    // such as "/days/18" to get status for 18 days
    pf-out "/days/%s", days to define payload
    // Create a string describing the remote server
    // so if "server" is "192.168.0.15", then it would
    // be 192.168.0.15:3800, meaning it runs on TCP port 3800
    pf-out "%s:3800", server to define srv_location

    // Create a remote server connection
    new-server define srv location srv_location \
        method "GET" app-path "/server" \
        request-path "/remote_status" \
        url-payload payload \
        timeout 30

    // Call the remote server
    call-server srv
    // Get the results from remote server
    read-server srv data define dt
    // Print out the results
    @Output is: [<<p-out dt>>]
end-request-handler

The code here is very simple. new-server will create a new connection to a remote server, running on IP address given by input parameter "server" (and obtained with input-param) on TCP port 3800. URL payload created in string variable "payload" is passed to the remote server. If it doesn't reply in 30 seconds, then the code would timeout. Then you're using call-server to actually make a call to the remote server (which is served by application "server" and by request handler "remote_status.vely" below), and finally read-server to get the reply from it. For simplicity, error handling is omitted here, but you can easily detect a timeout, any network errors, any errors from the remote server, including error code and error text, etc. See the above statements for more on this.

Make and start the local server

Next, create a local application:

sudo vf -i -u $(whoami) client

Make the application (i.e. compile the source code and build the native executable):

vv -q

Finally, start the local application server:

vf -w 2 client

This will start 2 server instances of a local application server.

Remote server

Okay, now you have a local server. Next, you'll setup a remote server. Login to your remote server and create a new directory for your remote application server:

mkdir remote_server
cd remote_server

Then create file "remote_status.vely" with this code:

#include "vely.h"

request-handler /remote_status
    silent-header
    out-header default
    input-param days

    pf-out "Status in the past %s days is okay", days
end-request-handler

This is super simple, and it just replies that the status is okay; it accepts the number of days to check for status and displays that back. In a real service, you might query a database to check for status (see run-query).

Make and start remote server

First create your application:

sudo vf -i -u $(whoami) server

Then make your program:

vv -q

And finally start the server:

vf -w 2 -p 3800 server

This will start 2 daemon processes running as background servers. They will serve requests from your local server.

Note that if you're running this example on different computers, some Linux distributions come with a firewall, and you may need to use ufw or firewall-cmd to make port 3800 accessible here. Also if you're using SELinux on this server, you may either need to allow binding to port 3800, or make SELinux permissive (with "sudo setenforce 0").

Run distributed calls

There is a number of ways you can call the remote service you created. These are calls made from your local server, so change directory to it:

cd local_server

Here's various way to call the remote application server:

Execute a command-line program on local server that calls remote application server:

To do this, use "-r" option of vv utility to generate shell commands you can use to call your program:
```
vv -r --req "/status/days/18/server/127.0.0.1"
```
Here, you're saying that you want to make a request "status" (which is in source file "status.vely" on your local server). You are also saying that input parameter "days" should have a value of "18" and also that input parameter "server" should have a value of "127.0.0.1" - see input-param statements in the above file "status.vely". If you actually have a different server with a different IP, use it instead of "127.0.0.1". The result of the above might be:
```
export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=no
export REQUEST_METHOD=GET
export SCRIPT_NAME="/client"
export PATH_INFO="/status/days/18/server/127.0.0.1"
export QUERY_STRING=""
/var/lib/vv/bld/client/client
```
Execute this, and the result will be:
```
Output is: [Status in the past 18 days is okay]
```
where the part in between "[..]" comes from the remote server, and the "Output is: " part comes from the command line Vely program you executed.
Call remote application server directly from a command-line program:

You will use FastCGI-command-line-client to do this:
```
export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=no
export SCRIPT_NAME="/server"
export PATH_INFO="/remote_status"
export QUERY_STRING="days=12"
cgi-fcgi -connect "127.0.0.1:3800" /server
```
The result is, as expected:
```
Status in the past 12 days is okay
```
In this case, the output comes straight from the remote server, so the "Output is: " part is missing. "cgi-fcgi" utility simply copies the output from a remote service to the standard output.
Use a command-line utility to contact local application server, which then calls the remote server, which replies back to local application server, which replies back to your command-line utility:

You will use FastCGI-command-line-client to do this:
```
export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=no
export SCRIPT_NAME="/client"
export PATH_INFO="/status/server/127.0.0.1"
export QUERY_STRING="days=10"
cgi-fcgi -connect /var/lib/vv/client/sock/sock  /client
```
The result is:
```
Output is: [Status in the past 10 days is okay]
```
which is what you'd expect. Note that in this case, some of the input parameters are in the path of request-URL ("server" in "PATH_INFO") and some in query string ("days" in "QUERY_STRING"). Also, in this case the "cgi-fcgi" utility first sends a request to your local application server, which sends it to a remote service, so there is "Output is: " output.

You have different options when designing your distributed systems, and this article shows how easy it is to implement them.

See also

Example: docker

The following is a bash script that will create a base Vely image as well as a base image for a Vely application, and setup Apache server and MariaDB database. To run this example, you must have Docker, Apache and MariaDB installed on Ubuntu 20 and up or Red Hat 9 or up. You can use docker or podman (substitute podman for docker).

To run this example on Ubuntu:

tar xvf /usr/share/vely/examples/velydocker.tar.gz
cd velydocker
export DBROOTPWD="<mariadb root database pwd>"
export VV_APACHE_CONFIG="/etc/apache2/apache2.conf"
export VV_APACHE_SERVICE="apache2"
./runvelydocker

DBROOTPWD environment variable should have a MariaDB root database password (or empty if passwordless). VV_APACHE_CONFIG should be the location of Apache configuration file, and VV_APACHE_SERVICE the name of Apache service. You must have sudo privilege to run this example. The settings above are for Ubuntu/Debian, but you can change them for other distros.

The script will create a container with your application installed. You can remove the application source code from the container in "runit" script in order to distribute only the application binaries. You can then run this container on any machine.

Note that Vely demo application source code being containerized is in "docker" directory. You can replace it with your own source code; see application-setup on building application with Vely.

Once the image is built and container started, use the following link(s) to run it, or if you can't, use curl to see the demo application response (the demo is example-stock):

#Add stock ticker 'XYZ' with stock price 450 
http://127.0.0.1/velydemo/docker_stock?action=add&stock_name=XYZ&stock_price=450

#Display list of stock tickers 
http://127.0.0.1/velydemo/docker_stock?action=show

Files

File Dockerfile:

#
# Create base vely image. Install Vely on top of Ubuntu 20.04
#
FROM ubuntu:20.04
ENV TZ=America/Phoenix
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt update
RUN apt install -y apt-transport-https ca-certificates wget sudo
RUN wget  -qO - https://vely.dev//pkg/OPENPGP|sudo tee /usr/share/keyrings/vely.asc >/dev/null
ARG arch
RUN sudo bash -c "echo 'deb [signed-by=/usr/share/keyrings/vely.asc] https://vely.dev//pkg/ubuntu_20_$arch/latest ubuntu_20_$arch vely' >/etc/apt/sources.list.d/vely.list"
RUN sudo apt update
RUN sudo apt install -y vely

File velyapp.dockerfile:

#
# Create image that will be used to compile and link an application. Source code is copied from host,
# application is setup, and the source code is deleted (remove this step if you want to ship the source code).
# After that the image is ready to run in a container. Here, the application name is "velydemo"; change it to
# fit your application. The port used in 2300, you can change that as well.
# Customize the ENTRYPOINT command to fit your runtime.
#
FROM vely
# create vely user and give it limited sudo powers
RUN useradd -ms /bin/bash vely && echo "vely:vely" | chpasswd
RUN echo "vely ALL=(ALL) NOPASSWD: /usr/bin/vf" >> /etc/sudoers
# default user
USER vely
# default working dir
WORKDIR /home/vely
EXPOSE 2300
# copy over source code and make app
COPY ./docker/* /home/vely/
# this is to run app with docker run
ENTRYPOINT [ "./runit" ]

File runvelydocker:

#!/bin/bash

#
# Vely Docker example. 
#

# if you want to rebuild all from scratch, switch CACHE to the commented option
# that is useful when installing new Vely version in the base image
CACHE=""
# CACHE="--no-cache"

set -eE -o functrace
trap 'echo "Error: status $?, $(caller), line ${LINENO}"' ERR


#
# Build vely base image
#
sudo docker build --build-arg arch=$(uname -m) $CACHE -t vely .

#
# Create persistent named volume for /var/lib/vv in the container
#
sudo docker volume create velyhome

#
# Stop current container(s) and remove current images
#
sudo docker stop velyapp || true
sudo docker rmi --force velyapp || true

#
# Build a container for your application
#
sudo docker build $CACHE -t velyapp -f velyapp.dockerfile .

#
# Add ProxyPass to Apache configuration on the host to be able to access the application
# Web server is on the host for simplicity in this example, but it can be in its own container
#
sudo sed -i "/^ProxyPass \"\/velydemo\" .*$/d" $VV_APACHE_CONFIG
echo 'ProxyPass "/velydemo" fcgi://127.0.0.1:2300/' | sudo tee -a $VV_APACHE_CONFIG >/dev/null
sudo a2enmod proxy || true
sudo a2enmod proxy_fcgi || true
sudo service $VV_APACHE_SERVICE restart

#
# Create a MariaDB database on the host, setup the user and create database objects (a table in this case).
# Database is on the host for simplicity in this example, but it can be in its own container
#
MKDB=$(echo "create database if not exists velydb; 
create user if not exists velyuser identified by 'pwd';
grant create,alter,drop,select,insert,delete,update on velydb.* to velyuser;
use velydb;
source docker/setup.sql;")

#
# Execute database setup. 
#
if [ "$DBROOTPWD" == "" ]; then
    echo $MKDB | sudo mysql
else
    echo $MKDB | sudo mysql -u root -p$DBROOTPWD
fi

mkdir -p $HOME/libvv
#
# Now that Vely application container is setup, web server and database setup as well, run it in a container.
# Note that "host" network interface is used for simplicity.
sudo docker run --name velyapp -d -v velyhome:/var/lib/vv  --network="host" --rm velyapp


exit 0

File runit:

#!/bin/bash
# Create Vely application and run it in foreground for docker
# demout has a log of execution
sudo vf -i -u $(whoami) velydemo
vv -c
vv -q --db=mariadb:db -s
vv -c
vf -f -w3 -p2300 velydemo > demout

See also

Encryption: ciphers, digests, salt, IV and a hands-on guide

What is encryption

Encryption is a method of turning data into an unusable form that can be made useful only by means of decryption. The purpose is to make data available solely to those who can decrypt it (i.e. make it usable). Typically, data needs to be encrypted to make sure it cannot be obtained in case of unauthorized access. It is the last line of defense after an attacker has managed to break through authorization systems and access control.

This doesn't mean all data needs to be encrypted, because often times authorization and access systems may be enough, and in addition, there is a performance penalty for encrypting and decrypting data. If and when the data gets encrypted is a matter of application planning and risk assessment, and sometimes it is also a regulatory requirement, such as with HIPAA or GDPR.

Data can be encrypted at-rest, such as on disk, or in transit, such as between two parties communicating over the Internet.

Here you will learn how to encrypt and decrypt data using a password, also known as symmetrical encryption. This password must be known to both parties exchanging information.

Cipher, digest, salt, iterations, IV

To properly and securely use encryption, there are a few notions that need to be explained.

A cipher is the algorithm used for encryption. For example, AES256 is a cipher. The idea of a cipher is what most people will think of when it comes to encryption.

A digest is basically a hash function that is used to scramble and lengthen the password (i.e. the encryption key) before it's used by the cipher. Why is this done? For one, it creates a well randomized, uniform-length hash of a key that works better for encryption. It's also very suitable for "salting", which is the next one to talk about.

The "salt" is a method of defeating so-called "rainbow" tables. An attacker knows that two hashed values will still look exactly the same if the originals were. However, if you add the salt value to hashing, then they won't. It's called "salt" because it's sort of mixed with the key to produce something different. Now, a rainbow table will attempt to match known hashed values with precomputed data in an effort to guess a password. Usually, salt is randomly generated for each key and stored with it. In order to match known hashes, the attacker would have to precompute rainbow tables for great many random values, which is generally not feasible.

You will often hear about "iterations" in encryption. An iteration is a single cycle in which a key and salt are mixed in such a way to make guessing the key harder. This is done many times so to make it computationally difficult for an attacker to reverse-guess the key, hence "iterations" (plural). Typically, a minimum required number of iterations is 1000, but it can be different than that. If you start with a really strong password, generally you need less.

IV (or "Initialization Vector") is typically a random value that's used for encryption of each message. Now, salt is used for producing a key based on a password. And IV is used when you already have a key and now are encrypting messages. The purpose of IV is to make the same messages appear differently when encrypted. Sometimes, IV also has a sequential component, so it's made of a random string plus a sequence that constantly increases. This makes "replay" attacks difficult, which is where attacker doesn't need to decrypt a message; but rather an encrypted message was "sniffed" (i.e. intercepted between the sender and receiver) and then replayed, hoping to repeat the action already performed. Though in reality, most high-level protocols already have a sequence in place, where each message has, as a part of it, an increasing packet number, so in most cases IV doesn't need it.

Prerequisites

This example uses Vely framework.

Note that using custom ciphers and digests, and explicit use of Initialization Vectors and key caching is available since 15.2 - if you're using earlier version, you should install 15.2 or later to run these examples.

Encryption example

To run the examples here, create an application "enc" in a directory of its own (see vf for more on Vely's program manager):

mkdir enc_example
cd enc_example
sudo vf -i -u $(whoami) enc

To encrypt data use encrypt-data statement. The simplest form is to encrypt a null-terminated string. Create a file "encrypt.vely" and copy this:

#include "vely.h"

request-handler /encrypt
    out-header default
    char *str = "This contains a secret code, which is Open Sesame!";
    // Encrypt
    encrypt-data str to define enc_str password "my_password"
    p-out enc_str
    @
    // Decrypt
    decrypt-data enc_str password "my_password" to define dec_str
    p-out dec_str
    @
end-request-handler

You can see the basic usage of encrypt-data and decrypt-data. You supply data (original or encrypted), the password, and off you go. The data is encrypted and then decrypted, yielding the original.

In the source code, a string variable "enc_str" (which is created as a "char *") will contain the encrypted version of "This contains a secret code, which is Open Sesame!" and "dec_str" will be the decrypted data which must be exactly the same.

To run this code from command line, make the application first:

vv -q

Then have Vely produce the bash code to run it - the request path is "/encrypt", which in our case is handled by function "void encrypt()" defined in source file "encrypt.vely". In Vely, these names always match, making it easy to write, read and execute code. Use "-r" option in vv to specify the request path and get the code you need to run the program:

vv -r --req="/encrypt" --silent-header --exec

You will get a response like this:

72ddd44c10e9693be6ac77caabc64e05f809290a109df7cfc57400948cb888cd23c7e98e15bcf21b25ab1337ddc6d02094232111aa20a2d548c08f230b6d56e9
This contains a secret code, which is Open Sesame!

What you have here is the encrypted data, and then this encrypted data is decrypted using the same password. Unsurprisingly, the result matches the string you encrypted in the first place.

Note that by default encrypt-data will produce encrypted value in a human-readable hexadecimal form, which means consisting of hexadecimal characters "0" to "9" and "a" to "f". This way you can store the encrypted data into a regular string. For instance it may go to a JSON document or into a VARCHAR column in a database, or pretty much anywhere else. However you can also produce a binary encrypted data. More on that in a bit.

Note that if you omit "--exec" in the "vv -r" above, you can see the exact bash shell commands to execute your program. So for example:

vv -r --req="/encrypt" --silent-header

might result in:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=yes
export REQUEST_METHOD=GET
export SCRIPT_NAME="/enc"
export PATH_INFO="/encrypt"
export QUERY_STRING=""
/var/lib/vv/bld/enc/enc

These set the standard HTTP environment variables and execute your binary program (/var/lib/vv/bld/enc/enc). You can copy and paste these commands to your shell scripts for any Vely program.

Encrypt data into a binary result

In the previous example, the resulting encrypted data is in a human-readable hexadecimal form. You can also create binary encrypted data, which is not a human-readable string and is also shorter. To do that, use "binary" clause. Replace the code in "encrypt.vely" with:

#include "vely.h"

request-handler /encrypt
    out-header default
    char *str = "This contains a secret code, which is Open Sesame!";
    // Encrypt
    encrypt-data str to define enc_str password "my_password" \
        binary output-length define outlen
    // Save the encrypted data to a file
    write-file "encrypted_data" from enc_str length outlen
    get-app directory to define app_dir
    @Encrypted data written to file <<p-out app_dir>>/encrypted_data
    // Decrypt data
    decrypt-data enc_str password "my_password" \
        input-length outlen binary to define dec_str
    p-out dec_str
    @
end-request-handler

When you want to get binary encrypted data, you should get its length in bytes too, or otherwise you won't know where it ends, since it may contain null bytes in it. Use "output-length" clause for that purpose. In this code, the encrypted data in variable "enc_str" is written to file "encrypted_data", and the length written is "outlen" bytes. When a file is written without a path, it's always written in the application home directory (see how-vely-works), so you'd use get-app to get that directory.

When decrypting data, notice the use of "input-length" clause. It says how many bytes the encrypted data has. Obviously you can get that from "outlen" variable, where encrypt-data stored the length of encrypted data. When encryption and decryption are decoupled, i.e. running in separate programs, you'd make sure this length is made available.

Notice also that when data is encrypted as "binary" (meaning producing a binary output), the decryption must use the same.

Make the application:

vv -q

Run it the same as before:

vv -r --req="/encrypt" --silent-header --exec

The result is:

Encrypted data written to file /var/lib/vv/enc/app/encrypted_data
This contains a secret code, which is Open Sesame!

The decrypted data is exactly the same as the original.

You can see the actual encrypted data written to the file by using "octal dump" ("od") Linux utility:

$ od -c /var/lib/vv/enc/app/encrypted_data
0000000   r 335 324   L 020 351   i   ; 346 254   w 312 253 306   N 005
0000020 370  \t   )  \n 020 235 367 317 305   t  \0 224 214 270 210 315
0000040   # 307 351 216 025 274 362 033   % 253 023   7 335 306 320
0000060 224   #   ! 021 252     242 325   H 300 217   #  \v   m   V 351
0000100

There you have it. You will notice the data is binary and it actually contains the null byte(s).

Encrypt binary data

The data to encrypt in these examples is a string, i.e. null-delimited. You can encrypt binary data just as easily by specifying its length in "input-length" clause, for example copy this to "encrypt.vely":

#include "vely.h"

request-handler /encrypt
    out-header default
    char *str = "This c\000ontains a secret code, which is Open Sesame!";
    // Encrypt
    encrypt-data str to define enc_str password "my_password" \
        input-length 12
    p-out enc_str
    @
    // Decrypt
    decrypt-data enc_str password "my_password" to define dec_str \
        output-length define res_len
    // Output binary data; present null byte as octal \000
    int i;
    for (i = 0; i < res_len; i++) {
        if (dec_str[i] == 0) {
            p-out "\\000"
        } else {
            pf-out "%c", dec_str[i]
        }
    }
    @
end-request-handler

This will encrypt 12 bytes at memory location "enc_str" regardless of any null bytes. In this case that's "This c" followed by a null byte followed by "ontain" string, but it can be any kind of binary data, for example the contents of a JPG file.

On the decrypt side, you'd obtain the number of bytes decrypted in "output-length" clause. Finally, the decrypted data is shown to be exactly the original and the null byte is presented in a typical octal representation.

Make the application:

vv -q

Run it the same as before:

vv -r --req="/encrypt" --silent-header --exec

The result is:

6bea45c2f901c0913c87fccb9b347d0a
This c\000ontai

The encrypted value is shorter because the data is shorter in this case too, and the result matches exactly the original.

Use any cipher or digest

The encryption used by default is AES256 and SHA256 hashing from the standard OpenSSL library, both of which are widely used in cryptography. You can however use any available cipher and digest (i.e. hash) that is supported by OpenSSL (even the custom ones you provide).

To see which algorithms are available, do this in command line:

#get list of cipher providers
openssl list -cipher-algorithms

#get list of digest providers
openssl list -digest-algorithms

These two will provide a list of cipher and digest (hash) algorithms. Some of them may be weaker than the default ones chosen by Vely, and others may be there just for backward compatibility with older systems. Yet others may be quite new and did not have enough time to be validated to the extent you may want them to be. So be careful when choosing these algorithms and be sure to know why you're changing the default ones. That said, here's an example of using Camellia-256 (i.e. "CAMELLIA-256-CFB1") encryption with "SHA3-512" digest. Replace the code in "encrypt.vely" with:

#include "vely.h"

request-handler /encrypt
    out-header default
    char *str = "This contains a secret code, which is Open Sesame!";
    // Encrypt data
    encrypt-data str to define enc_str password "my_password" \
        cipher "CAMELLIA-256-CFB1" digest "SHA3-512"
    p-out enc_str
    @
    // Decrypt data
    decrypt-data enc_str password "my_password"  to define dec_str \
        cipher "CAMELLIA-256-CFB1" digest "SHA3-512"
    p-out dec_str
    @
end-request-handler

Make the application:

vv -q

Run it:

vv -r --req="/encrypt" --silent-header --exec

In this case the result is:

f4d64d920756f7220516567727cef2c47443973de03449915d50a1d2e5e8558e7e06914532a0b0bf13842f67f0a268c98da6
This contains a secret code, which is Open Sesame!

Again, you get the original data. Note you have to use the same cipher and digest in both encrypt-data and decrypt-data!

You can of course produce the binary encrypted value just like before by using "binary" and "output-length" clauses.

If you've got external systems that encrypt data, and you know which cipher and digest they use, you can match those and make your code interoperable. Vely uses standard OpenSSL library so chances are that other software may too.

Using salt

To add a salt to encryption, use "salt" clause. You can generate random salt by using random-string statement (or random-crypto if there is a need). Here is the code for "encrypt.vely":

#include "vely.h"

request-handler /encrypt
    out-header default
    char *str = "This contains a secret code, which is Open Sesame!";
    // Get salt
    random-string to define rs length 16
    // Encrypt data
    encrypt-data str to define enc_str password "my_password" salt rs
    @Salt used is <<p-out rs>>, and the encrypted string is <<p-out enc_str>>
    // Decrypt data
    decrypt-data enc_str password "my_password" salt rs to define dec_str
    p-out dec_str
    @
end-request-handler

Make the application:

vv -q

Run it a few times:

vv -r --req="/encrypt" --silent-header --exec
vv -r --req="/encrypt" --silent-header --exec
vv -r --req="/encrypt" --silent-header --exec

The result:

Salt used is VA9agPKxL9hf3bMd, and the encrypted string is 3272aa49c9b10cb2edf5d8a5e23803a5aa153c1b124296d318e3b3ad22bc911d1c0889d195d800c2bd92153ef7688e8d1cd368dbca3c5250d456f05c81ce0fdd
This contains a secret code, which is Open Sesame!
Salt used is FeWcGkBO5hQ1uo1A, and the encrypted string is 48b97314c1bc88952c798dfde7a416180dda6b00361217ea25278791c43b34f9c2e31cab6d9f4f28eea59baa70aadb4e8f1ed0709db81dff19f24cb7677c7371
This contains a secret code, which is Open Sesame!
Salt used is nCQClR0NMjdetTEf, and the encrypted string is f19cdd9c1ddec487157ac727b2c8d0cdeb728a4ecaf838ca8585e279447bcdce83f7f95fa53b054775be1bb2de3b95f2e66a8b26b216ea18aa8b47f3d177e917
This contains a secret code, which is Open Sesame!

As you can see, a random salt value (16 bytes long in this case) is generated for each encryption, and the encrypted value is different each time, even though the data being encrypted was the same! This makes it difficult to crack encryption like this.

Of course, to decrypt, you must record the salt and use it exactly as you did when encrypting. In the code here, variable "rs" holds the salt. If you store the encrypted values in the database, you'd likely store the salt right next to it.

Initialization vector

In practice, you wouldn't use a different salt value for each message. It creates a new key every time, and that can reduce performance. And there's really no need for it: the use of salt is to make each key (even the same ones) much harder to guess. Once you've done that, you might not need to do it again, or often.

Instead, you'd use an IV (Initialization Vector) for each message. It's usually a random string that makes same messages appear different, and increases the computational cost of cracking the password. Here is the new code for "encrypt.vely":

#include "vely.h"

request-handler /encrypt
    out-header default
    // Get salt
    random-string to define rs length 16
    // Encrypt data
    num i;
    for (i = 0; i < 10; i++) {
        random-string to define iv length 12
        encrypt-data "The same message" to define enc_str password "my_password" salt rs iterations 2000 init-vector iv cache
        @The encrypted string is <<p-out enc_str>>
        // Decrypt data
        decrypt-data enc_str password "my_password" salt rs iterations 2000 init-vector iv to define dec_str cache
        p-out dec_str
        @
    }
end-request-handler

Make the application:

vv -q

Run it a few times:

vv -r --req="/encrypt" --silent-header --exec
vv -r --req="/encrypt" --silent-header --exec
vv -r --req="/encrypt" --silent-header --exec

The result may be:

The encrypted string is 787909d332fd84ba939c594e24c421b00ba46d9c9a776c47d3d0a9ca6fccb1a2
The same message
The encrypted string is 7fae887e3ae469b666cff79a68270ea3d11b771dc58a299971d5b49a1f7db1be
The same message
The encrypted string is 59f95c3e4457d89f611c4f8bd53dd5fa9f8c3bbe748ed7d5aeb939ad633199d7
The same message
The encrypted string is 00f218d0bbe7b618a0c2970da0b09e043a47798004502b76bc4a3f6afc626056
The same message
The encrypted string is 6819349496b9f573743f5ef65e27ac26f0d64574d39227cc4e85e517f108a5dd
The same message
The encrypted string is a2833338cf636602881377a024c974906caa16d1f7c47c78d9efdff128918d58
The same message
The encrypted string is 04c914cd9338fcba9acb550a79188bebbbb134c34441dfd540473dd8a1e6be40
The same message
The encrypted string is 05f0d51561d59edf05befd9fad243e0737e4a98af357a9764cba84bcc55cf4d5
The same message
The encrypted string is ae594c4d6e72c05c186383e63c89d93880c8a8a085bf9367bdfd772e3c163458
The same message
The encrypted string is 2b28cdf5a67a5a036139fd410112735aa96bc341a170dafb56818dc78efe2e00
The same message

You can see that the same message appears different when encrypted, though when decrypted it's again the same. Of course, the password, salt, number of iterations, and init-vector must be the same for both encryption and decryption.

Note the use of "cache" clause in encrypt-data and decrypt-data. It effectively caches the computed key (given password, salt, cipher/digest algorithms and number of iterations), so it's not computed each time through the loop. With "cache" the key is computed once, and then a different IV (in "init-vector" clause) is used for each message.

If you want to occasionally rebuild the key, use "clear-cache" clause, which supplies a boolean. If true, the key is recomputed, otherwise it's left alone. See encrypt-data for more on this.

Conclusion

You have learned how to encrypt and decrypt data using different ciphers, digests, salt and IV values in Vely. You can also create a human-readable encrypted value and a binary output, as well as encrypt both strings and binary values (like documents).

See also

Web file manager in 160 lines of code

Uploading and downloading files is one of the most common tasks in web applications. This article shows how to build a file manager application in about 160 lines of code.

Files such as JPG, PDF or other can be uploaded, and each file can have a description tag. A list of uploaded files can be displayed. Each file can be viewed or downloaded, and each file can be deleted. Files metadata is kept in the database (name, path, description, size, extension), while the files themselves are kept in Vely file-storage.

In a nutshell: PostgreSQL; web browser; Apache; TCP sockets; 6 source files, 165 lines of code.

Screenshots of application

This is the page where application user always goes first. Here, your application will display a web link that shows all files kept in the database so far, as well as HTML form to upload a new file where you can enter its description and choose a file from your computer or device. Here's a snapshot of what the page will look like - this is a basic layout and you can change the code to style it however you wish:

Files that have been saved will be shown in a list. Next to each file name will be its size, and links to show the file (display it or download it if can't be displayed), and a link to delete the file:

When a file is about to be deleted, a confirmation is asked of user:

Once confirmed, deletion of a file is carried out:

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because they are used in this example, you will need to install Apache as a web server and PostgreSQL as a database.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

tar xvf $(vv -o)/examples/file-manager.tar.gz
cd file-manager

Setup application

The very first step is to create an application. The application will be named "file-manager", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) file-manager

This will create a new application home (which is "/var/lib/vv/file-manager") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create PostgreSQL database "db_file_manager". You can change the database name, but remember to change it everywhere here. And then, you will create database objects in the database.

Note the example here uses peer-authentication, which is the default on all modern PostgreSQL installations - this means the database user name is the same as the Operating System user name.

Execute the following in PostgreSQL database as root (using psql utility):

echo "create user $(whoami);
create database db_file_manager with owner=$(whoami);
grant all on database db_file_manager to $(whoami);
\q
" | sudo -u postgres psql

Next, login to database db_file_manager and create the database objects for the application:

psql -d db_file_manager -f setup.sql

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_file_manager". This name doesn't have to be "db_file_manager", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo "user=$(whoami) dbname=db_file_manager"  > db_file_manager

Build application

Use vv utility to make the application:

vv -q --db=postgres:db_file_manager

Note usage of --db option to specify PostgreSQL database and the database configuration file name.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a TCP socket 2310 to communicate with the web server (i.e. a reverse-proxy).

vf -p 2310 -w 3 file-manager

vf -p 2310 file-manager

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit file-manager

- SELinux and TCP ports
If you are using SELinux and have it enabled, then you must make the TCP socket accessible to the web server and Vely:

sudo semanage port -a -t vvport_t -p tcp 2310

Without this step, the application will appear to not work.

Setup web server

This shows how to connect your application listening at TCP port 2310 (started with "-p" option in vf) to Apache web server.

- Step 1:
To setup Apache as a reverse proxy and connect your application to it, you need to enable FastCGI proxy support, which generally means "proxy" and "proxy_fcgi" modules - this is done only once:

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/file-manager" is the application path, see request-URL):

ProxyPass "/file-manager" fcgi://127.0.0.1:2310/

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/file-manager" (such as for example "/file-manager.html" or "/file-manager_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/file-manager", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Start the file management application 
http://127.0.0.1/file-manager/start

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Source files

The following are the source files in this application:
- Storing file information in the database (setup.sql)
The information about files, such as file names, sizes, extensions, descriptions etc. is stored in the table "files". Where will the files themselves be stored?

Well, you could store them in the database, but that's not the most optimal way. You could store them anywhere on a file system, but Vely has a very simple (and generally transparent and automatic) way of storing files, which is Vely's file-storage. When a file is uploaded, it's automatically stored there; and you can also create files there as well.

create table if not exists files (fileName varchar(100), localPath varchar(300), extension varchar(10), description varchar(200), fileSize int, fileID bigserial primary key);

- Application landing page (start.vely)
This code displays the page where application user always goes first. One thing you'll notice often in Vely code is the use of "@" to output text (in this case HTML page text) - "@" is an output-statement, and it makes it easy and readable. In general, Vely does not use traditional API for its functionality, rather it uses statement-APIs.

#include "vely.h" // must always be here

// Upload and list/download files
request-handler /start

   out-header default

   @<h2>File Manager</h2>

   @To manage the uploaded files, <a href="<<p-path>>?req=list">click here.</a><br/>
   @<br/>

   // Form to upload a file

   @<form action="<<p-path>>" method="POST" enctype="multipart/form-data">
   @    <input type="hidden" name="req" value="upload">

   @    <label for="file_description">File description:</label><br>
   @    <textarea name="filedesc" rows="3" columns="50"></textarea><br/>
   @    <br/>
   @    <label for="filename">File:</label>
   @    <input type="file" name="file" value=""><br><br>

   @    <input type="submit" value="Submit">
   @</form>

end-request-handler

- Uploading files (upload.vely)
The next source code file is "upload.vely", which will upload a file when a user selects a file and clicks "Submit" button, which is in "start.vely" file. When you upload a file into Vely application, it's just an input-parameter, just like it would be with any GET or POST request parameter. In addition, Vely will automatically store the uploaded file into file directory (see file-storage) which is optimized for speed of access, and you can either keep it there, move it elsewhere or do whatever you want. All of this make is easy to work with files.

The above code will get the information about the file uploaded (such as file description, as well as file name, location, size, extension) - which is provided by Vely as input parameters. Then, use run-query statement to insert such information into the database table you created. From now on, the information about a file is stored in the database (and that includes the location where it was uploaded), and the actual file is in the file system. This allows fast access to files and easier management.

And finally, the message will be sent back to web client (such as a browser) that you've saved the file.

#include "vely.h"

request-handler /upload
   out-header default

   input-param filedesc      // file description from the upload form
   input-param file_filename // file name
   input-param file_location // the path to uploaded file
   input-param file_size     // size in bytes
   input-param file_ext      // the file extension

   @<h2>Uploading file</h2>

   // insert the information about the file into the database
   run-query @db_file_manager= \
        "insert into files (fileName, localPath, extension, description, fileSize) \
            values ('%s', '%s', '%s', '%s', '%s')" \
        input file_filename, file_location, file_ext, filedesc, file_size
   end-query

   @File <<p-web file_filename>> of size <<p-web file_size>> \
        is stored on server at <<p-web file_location>>. \
        File description is <<p-web filedesc>>.<hr/>
end-request-handler

- Listing files (list.vely)
For files that have been saved, this code shows them in a list. Next to each file name will be its size, and links to show the file (display it or download it if can't be displayed), and a link to delete the file. To produce the list of files, the table "files" is queried, and the list of files displayed as an HTML table.

#include "vely.h" // must always be here

// List files
request-handler list
    // List current files in the database
    out-header default
    @<h2>List of files</h2>
    @To add a file, <a href="<<p-path>>?req=start">click here</a><br/><br/>
    @<table border="1">
    @<tr>
    @    <td>File</td><td>Description</td><td>Size</td><td>Show</td><td>Delete</td>
    @</tr>

    // get the list of files from the database
    run-query @db_file_manager= \
        "select fileName, description, fileSize, fileID from files order by fileSize desc" \
        output define file_name, description, file_size, file_ID

        // construct table output with links to Show and Delete files
        @<tr>
        @    <td><<p-web file_name>></td><td><<p-web description>><td><<p-web file_size>></td>
        @    <td><a href="<<p-path>>?req=download&amp;file_id=<<p-url file_ID>>">Show</a></td>
        @    <td><a href="<<p-path>>?req=delete&amp;action=confirm&amp;file_id=<<p-url file_ID>>">Delete</a></td>
        @</tr>

    end-query
    @</table>
end-request-handler

- Downloading files (download.vely)
When a link to download a file is clicked in the list of files, the following code is called. The ID of a file requested is queried in table "files", the file path on server is found, and the file is sent to the client using send-file statement.

#include "vely.h"

// Download a file
request-handler /download
    // Show or download a file (its ID is in the database)
    input-param file_id

    // get the local path and extension of the file
    run-query @db_file_manager= \
        "select localPath,extension from files where fileID='%s'" \
        input file_id \
        output define local_path, ext \
        row-count define num_files

            // display JPG or PDF files in the browser, or download any other kind
            if (!strcmp (ext, ".jpg")) {
                send-file local_path headers content-type "image/jpg"
            } else if (!strcmp (ext, ".pdf")) {
                send-file local_path headers content-type "application/pdf"
            } else {
                send-file local_path headers content-type "application/octet-stream" download
            }
    end-query

    // check we can find the file
    if (num_files != 1) {
        out-header default
        @Cannot find this file!<hr/>
        return;
    }

end-request-handler

- Deleting files (delete.vely)
Finally, deleting a file (by clicking on delete link in the list of files) is handled in the following code.

When a file is about to be deleted, a confirmation is asked of user - task parameter "action" is "confirm" in this case.

Once confirmed, deletion of a file is carried out; task parameter "action" is "delete" then (you can name parameter "action" anything you want).

Note that deletion is done as a transaction, only if both the record in table "files" is deleted and the actual file deleted. If the file cannot be deleted, transaction is rollback-ed.

#include "vely.h" // must always be here

// Delete the file
request-handler /delete
   out-header default
   @<h2>Delete a file</h2>
   task-param action
   input-param file_id
   char *file_name = NULL;
   char *desc = NULL;
   char *local_path = NULL;

   // Get file information from the database
   run-query @db_file_manager="select fileName, localPath, description  from files where fileID='%s'" output fileName, localPath, description : file_id
       query-result fileName to file_name
       query-result description to desc
       query-result localPath to local_path
   end-query

   if-task "confirm" // get file information to confirm what will be deleted
       @Are you sure you want to delete file <<p-web file_name>> (<<p-web desc>>)? Click <a href="<<p-path>>?req=delete&amp;action=delete&amp;file_id=<<p-url file_id>>">Delete</a> or click the browser's Back button to go back.<br/>

   else-task "delete"  // actual delete file, once confirmed
       begin-transaction @db_file_manager
       run-query @db_file_manager= "delete from files where fileID='%s'" : file_id error define err no-loop
       if (atol(err) != 0) {
           @Could not delete the file (error <<p-web err>>)
           rollback-transaction @db_file_manager
       } else {
           delete-file local_path status define st
           if (st == VV_OKAY) {
               commit-transaction @db_file_manager
               @File deleted. Go back to <a href="<<p-path>>?req=start">start page</a>
           } else {
               rollback-transaction @db_file_manager
               @File could not be deleted, error <<pf-web "%lld", st>>
           }
       }
   else-task other
       @Unrecognized action <<p-web action>>
   end-task
end-request-handler

See also

HTML form for guest database

This application is a guest tracking database. The user can input first and last name, which are added to a database table. A web page shows the list of names queried from the database.

In a nutshell: PostgreSQL; web browser; Apache; Unix sockets; 4 source files, 83 lines of code.

Screenshots of application

Users can sign the guest book via HTML form:

When a Submit button is clicked, entered data is added to the guest book:

This shows the entire guest book:

Setup prerequisites

vv -m

Get the source code

tar xvf $(vv -o)/examples/form.tar.gz
cd form

Setup application

The very first step is to create an application. The application will be named "form", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) form

This will create a new application home (which is "/var/lib/vv/form") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create PostgreSQL database "db_form". You can change the database name, but remember to change it everywhere here. And then, you will create database objects in the database.

Note the example here uses peer-authentication, which is the default on all modern PostgreSQL installations - this means the database user name is the same as the Operating System user name.

Execute the following in PostgreSQL database as root (using psql utility):

echo "create user $(whoami);
create database db_form with owner=$(whoami);
grant all on database db_form to $(whoami);
\q
" | sudo -u postgres psql

Next, login to database db_form and create the database objects for the application:

psql -d db_form -f setup.sql

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_form". This name doesn't have to be "db_form", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo "user=$(whoami) dbname=db_form"  > db_form

Build application

Use vv utility to make the application:

vv -q --db=postgres:db_form

Note usage of --db option to specify PostgreSQL database and the database configuration file name.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 form

vf form

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit form

Setup web server

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/form" is the application path (see request-URL) and "form" is your application name):

ProxyPass "/form" unix:///var/lib/vv/form/sock/sock|fcgi://localhost/form

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/form" (such as for example "/form.html" or "/form_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/form", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Display form 
http://127.0.0.1/form/form_display

# Show data entered 
http://127.0.0.1/form/display_data

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Source files

The following are the source files in this application:
- Setup database objects (setup.sql)
Create table to hold the first and last name of the guest:

create table if not exists names (firstName varchar(30), lastName varchar(40));

- Web form (form_display.vely)
This is a simple HTML web form where first and last name is collected and sent to the server via POST mechanism. Note the path segment after the application path (set with p-path), which is "/form_post", meaning this request will be handled in source file "form_post.vely":


#include "vely.h" // must always be here


// 
// Display a web form
//
request-handler /form_display

     out-header default

    @<h2>Enter your name</h2>

    @<form action="<<p-path>>/form_post" method="POST">

    @    <label for="first_name">First name:</label><br>
    @    <input type="text" name="first_name" value=""><br>

    @    <label for="last_name">Last name:</label><br>
    @    <input type="text" name="last_name" value=""><br><br>

    @    <input type="submit" value="Submit">
    @ </form>
end-request-handler

- Submitting web form (form_post.vely)
input-param is used to obtain input parameters sent here from the web form created in "form_display.vely". The data is inserted into the database and the status message output back to web client (i.e. browser):


#include "vely.h" // must always be here


// 
// Post input from a web form
//
request-handler /form_post

    out-header default

    input-param first_name
    input-param last_name
    run-query @db_form="insert into names(firstName, lastName) values ('%s','%s')" : first_name, last_name error define err
    end-query

    if (strcmp(err, "0")) {
        @Could not add name, error [<<p-out err>>]
        exit-request
    }

    @Name added to the database<hr/>
end-request-handler

- Display database data in table format in browser (display_data.vely)
This shows typical web output from querying a database. SELECT statement in run-query produces a result set, which is displayed in a tabular format in the browser:


#include "vely.h"


// 
// Display table data
//
request-handler /display_data

    out-header default

    @<h2>List of names</h2>

    @<table>
        run-query @db_form="select firstName, lastName from names order by lastName,firstName" output firstName,lastName
    @        <tr>
    @            <td>
                    query-result lastName
    @            </td>
    @            <td>
                    query-result firstName
    @            </td>
    @        </tr>
        end-query
    @</table>

end-request-handler

See also

Hashed key/value server

You will create your own hash server in this example. A REST API will enable end-user to add key/data pairs, query and delete them. The data is in memory-only; a more involved example could be constructed to persist the data in some form (such as with SQLite). This is an extremely fast, single-process hash server.

In a nutshell: web browser; Apache; REST API; Unix sockets; 2 source files, 47 lines of code.

Setup prerequisites

vv -m

Get the source code

tar xvf $(vv -o)/examples/hash-server.tar.gz
cd hash-server

Setup application

The very first step is to create an application. The application will be named "hash-server", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) hash-server

This will create a new application home (which is "/var/lib/vv/hash-server") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Build application

Use vv utility to make the application:

vv -q

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 1 hash-server

This will start 1 daemon process to serve the incoming requests.

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit hash-server

Setup web server

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/hash-server" is the application path (see request-URL) and "hash-server" is your application name):

ProxyPass "/hash-server" unix:///var/lib/vv/hash-server/sock/sock|fcgi://localhost/hash-server

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/hash-server" (such as for example "/hash-server.html" or "/hash-server_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/hash-server", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Add key "key_1" with data "data_1" 
http://127.0.0.1/hash-server/server/op/add/key/key_1/data/data_1

# Query key "key_1" 
http://127.0.0.1/hash-server/server/op/query/key/key_1

# Delete key "key_1" 
http://127.0.0.1/hash-server/server/op/delete/key/key_1

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

- Test hash server with a bash script
To test the hash server, you can use "test_hash" bash script (see source code):

./test_hash

This will insert 1000 key/data value pairs, query them, delete and then query again. The result is:

Keys added
Keys queried
Keys deleted
Keys queried
Hash server test successful

- Use REST API from command line
Here is the REST API for your application.

Substitute the loopback "localhost" with the IP or web address of your server.

Note that in these REST calls, the application path is "/hash-server", and the request name is "/server", followed by URL payload (see request-URL).

The request method used is based on REST methodology, i.e. POST to create resource, GET to obtain, DELETE to delete it, etc. This is how you can use the API:

Here is an URL payload to add a key/data pair; use the "add" operation (i.e. "add" is the value for "op" parameter, written as path segment "/op/add"), and (as an example) specify "key_1" for a key (written as path segment "/key/key_1") and "data_1" for data (written as path segment "/data/data_1"). The entire URL is:
```
curl -X POST  "http://localhost/hash-server/server/op/add/key/key_1/data/data_1"
```
The result:
```
Added [key_1]
```
To query the hash server, use the "query" operation (i.e. "query" is the value for "op" parameter, written as path segment "/op/query"), and (as an example) specify "key_1" for a key (written as path segment "/key/key_1"):
```
curl -X GET "http://localhost/hash-server/server/op/query/key/key_1"
```
The result:
```
Value [data_1]
```
To delete a key/data pair, use the "delete" operation (i.e. "delete" is the value for "op" parameter, written as path segment "/op/delete"), and (as an example) specify "key_1" for a key (written as path segment "/key/key_1"):
```
curl -X DELETE "http://localhost/hash-server/server/op/delete/key/key_1"
```
The result:
```
Deleted [data_1]
```

Source files

The following are the source files in this application:
- Hash server (server.vely)
This is the hash server. Because the data kept in hash needs to exist beyond a single request, you'd use "process-scope" clause (see new-hash). This way hash data stays allocated and available for the life of the server process. The creation of a new hash is done within do-once statement, so it executes only once for the life of the process.

Next, you get the input parameters (see input-param); in this case task-param "op" (operation requested), as well as "key" (value of a key) and "data" (value of data). Then, depending on the operation requested, the data is added, deleted and retrieved (i.e. queried). Note that because input parameter values are not allocated memory, they are copied into new memory before being stored in the hash; this way they survive for the life of the process - this is done automatically due to "process-scope" clause when hash was created.

%% /server
    out-header default

    do-once
    new-hash define h size 1024 process-scope
    end-do-once

    // Get input parameters
    task-param op
    input-param key
    input-param data

    if-task "add" // Add data to hash, 
        // Make a copy of input params are they are request-scoped
        copy-string key to define c_key
        copy-string data to define c_data
        write-hash h key c_key value c_data \
            old-value define old_data old-key define old_key \
            status define st
        if (st == VV_ERR_EXIST) {
            // delete old key/value if replaced by new
            delete-mem old_key
            delete-mem old_data
        }
        @Added [<<p-out key>>]
    else-task "delete" // Delete data and obtain the value deleted
        read-hash h key (key) value define val \
            old-key define okey \
            delete \
            status define st
        if (st == VV_ERR_EXIST) {
            @Not found [<<p-out key>>]
        } else {
            // If found, then delete key and value
            @Deleted [<<p-out val>>]
            delete-mem val
            delete-mem okey
        }
    else-task "query" // Query hash based on key value
        read-hash h key (key) value define val status define st
        if (st == VV_ERR_EXIST) {
            @Not found, queried [<<p-out key>>]
        } else {
            @Value [<<p-out val>>]
        }
    end-task
%%

- bash testing script (test_hash)
Test hash server by adding 1000 key/data pairs, querying to make sure the data is correct, deleting all of them and then querying to make sure they were all deleted.

#Restart hash-server server for this test
vf -m restart hash-server

#Add 1000 key/data pairs. Key value is 0,1,2... and data values are "data_0", "data_1", "data_2" etc.
for i in {1..1000}; do
    if [ "$(curl -s "http://localhost/hash-server/server/op/add/key/$i/data/data_$i")" != "Added [$i]" ]; then
        echo "Error adding key [$i]"
        exit -1
    fi
done

echo "Keys added"

#Query all 1000 keys and check that values retrieved are the correct ones.
for i in {1..1000}; do
    if [ "$(curl -s "http://localhost/hash-server/server/op/query/key/$i")" != "Value [data_$i]" ]; then
        echo "Error querying key [$i]"
        exit -1
    fi
done

echo "Keys queried"

#Delete all 1000 keys
ERR="0"
for i in {1..1000}; do
    if [ "$(curl -s "http://localhost/hash-server/server/op/delete/key/$i")" != "Deleted [data_$i]" ]; then
        echo "Error deleting key [$i]"
        exit -1
    fi
done

echo "Keys deleted"

#Query all 1000 keys and check that values retrieved are the correct ones.
for i in {1..1000}; do
    if [ "$(curl -s "http://localhost/hash-server/server/op/query/key/$i")" != "Not found, queried [$i]" ]; then
        echo "Error querying key [$i] after deletion."
        exit -1
    fi
done

echo "Keys queried"

echo "Hash server test successful"

See also

Example: hello world

This is a simple Hello World example. It explains basics of making applications as well as tracing and debugging them.

In a nutshell: command line; web browser; Nginx; Unix sockets; 2 source files, 6 lines of code.

Screenshots of application

Hello World output:

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because it is used in this example, you will need to install Nginx as a web server.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

tar xvf $(vv -o)/examples/hello-world.tar.gz
cd hello-world

Setup application

The very first step is to create an application. The application will be named "hello-world", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) hello-world

This will create a new application home (which is "/var/lib/vv/hello-world") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Build application

Use vv utility to make the application:

vv -q

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 hello-world

vf hello-world

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit hello-world

Setup web server

This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/hello-world" is the application path (see request-URL) and "hello-world" is your application name):

location /hello-world { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/hello-world/sock/sock; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

Note: you must not have any other URL resource that starts with "/hello-world" (such as for example "/hello-world.html" or "/hello-world_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/hello-world", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Hello world 
http://127.0.0.1/hello-world/hello

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Access application server from command line

To access your application server from command line (instead through web browser/web server), use this to see the application response:

#Hello world 
export CONTENT_TYPE=
export CONTENT_LENGTH=
export REQUEST_METHOD=GET
export SCRIPT_NAME='/hello-world'
export PATH_INFO='/hello'
export QUERY_STRING=''
cgi-fcgi -connect /var/lib/vv/hello-world/sock/sock /

Note: to suppress output of HTTP headers, add this before running the above:

export VV_SILENT_HEADER=yes

If you need to, you can also run your application as a CGI program.

Run program from command line

Execute the following to run your application from command line (as a command-line utility):

#Hello world 
vv -r --app='/hello-world' --req='/hello?' --method=GET --exec

Source files

The following are the source files in this application:
- Hello World (hello.vely)
Hello World example outputs HTTP header and a simple message back to the browser.

The file name ("hello.vely") must match the function name implemented in it, so the function's signature is "void hello()". Call it with a URL like:

http://your-web-site/hello-world/hello

Note "/hello-world" path segment - that's the application path and by default the same as application name (see how-vely-works and request-URL). The following path segment is "/hello", which is the request name, and it must match the function name that implements it.

Note the "@" is used to do the output, it's the output-statement. You'll probably use that one a lot.

In this case there's only one request file. In a real world application, there'd likely be quite a few. Vely will pick up all .vely files and automatically make them into an application. While at it, it will generate a simple request dispatcher that routes an incoming request to the appropriate code; in this case a "hello" request will be routed to your "void hello()" function.

#include "vely.h"

request-handler /hello
   out-header default
   @Hello World!
end-request-handler

Changing your code

When you change your source code and recompile, vf will automatically pick up that the executable has changed and reload (see vf if you wish to change this behavior), making your web application live instantly. For example, change hello.vely to this:

#include "vely.h"

void hello()
{
   out-header default
   @Hello World! And a good day to you too!
}

and then recompile:

vv -q

Now try again from the browser:

http://127.0.0.1/hello-world/hello

The result:

Generated C code, errors, tracing and debugging

Generated C code is located at:

/var/lib/vv/bld/hello-world/__hello.o.c

It's useful if you'd like to see it or debug your code.

If you are using your Vely application as a command line program, you can debug it simply by using gdb.

Note that in order to see the debugging symbols (such as variable names), you should compile your application with --debug flag:

vv -q --debug

For FastCGI processes, you'd want to start your application as a single process, generally as few as possible, for example, stop your application first and then start as a single process:

vf -m quit hello-world
vf -w 1 hello-world

First, find out the PID of your process:

ps -ef|grep hello-world.fcgi

Then you can load gdb with your application and attach to it - assuming the PID is 12345:

sudo gdb /var/lib/vv/bld/hello-world/hello-world.fcgi
... att 12345

To break in your code (in this case in function hello()), do this in gdb:

br hello

If you'd like to break in Vely's request dispatcher, i.e. just before any request is handled:

br vely_dispatch_request

From then on, you can set breakpoints, continue, stop and do anything else to debug your program.

When you debug Vely applications in gdb, each Vely statement is a single execution unit. This is very useful when reporting errors, since they are reported referencing lines in .vely file, in this case hello.vely. Sometimes you'd want to use line numbers of the generated C file, in which case you'd compile with --c-lines flag:

vv -q --c-lines

Another way to debug your application is to use tracing (see trace-run). To enable tracing, compile your application with --trace flag:

vv -q --debug --trace

Trace files are in the trace directory, see how-vely-works. In our case, trace file would be in this directory:

/var/lib/vv/hello-world/app/trace

and trace file name for the process you're debugging would be (with timestamp being of the time it's written):

trace-12345-2022-05-17-22-46-54

In addition to this, learn more about debugging Vely applications.

See also

How to design an application

This article will show how to get started with any Vely application, not just when it comes to creating one, but making the first steps in its design. Specifically, the focus here is requests an application serves and the code that handles them.

One important takeaway is that Vely automates things for you when it comes to technical side of things: parsing URLs and environment variables, letting you use various methods of specifying requests, deciding what code to run based on URL paths and parameters, getting input parameters in code, outputting HTTP headers (unless you supress them), managing server processes etc. The idea is to focus on application design and not the technical details; and more importantly, those details should be taken care of fast and with maximum performance.

Create an application

When starting a new application, create it first. To do this, first create a new directory for your source code, and then create your application. For instance:

mkdir cars_app
cd cars_app
sudo vf -i -u $(whoami) cars

This will create a directory structure for application "cars" under "/var/lib/vv/cars" directory. The above is the only Vely command that needs "sudo", because it will setup privileges for directories that make your application secure (see how-vely-works). "-i" means to create an application, and "-u" specifies who will own it - in this case it's your own logged-on user. And finally, the application name is "cars".

What is a request?

A request is data that's sent to your program so it can do something useful. In practical terms, it is a URL, the same kind that any web server or application works with.

Your program can run as an application server, or as a command-line utility.

If it's an application server, then it could be a web application in which case a web server (such as Apache, Nginx etc.) is probably used to talk to it (see for instance here for Apache setup or here for Nginx). Your application server could also serve as a middleware, talking to all kinds of clients (see here on building clients with API, or a simple utility that can talk to your server). It could also be any kind of server you may need.

If it's a command-line program, then it's like any other program you run. It can query databases, read and write files, process text, or just about anything else.

Regardless of how it runs, your Vely program works the same. It takes the same input and produces the same output. That's neat because you don't have to worry about two ways of writing a program; it also lets you test web programs on the command line.

Request name

Each request has a name. The name is typically the first path segment in the URL, right after application name. For example, in this URL:

https://webserver.com/cars/shop

"cars" is the application name, because it's the first part of the path. Request name is "shop" because it follows right after.

Consequently, "/cars" is the application path, and "/shop" is the request path.

Input parameters

While a request can be without any parameters, typically there are some parameters your program needs to fulfill the request. These parameters are passed in a URL right after the request name, for instance:

https://webserver.com/cars/shop/lot/used/doors/4

In this case, there are 2 parameters: "lot" with the value of "used", and "doors" with the value of "4". You could also pass them as a query string:

https://webserver.com/cars/shop?lot=used&doors=4

Or you can mix them up as both a path segment and a query string:

https://webserver.com/cars/shop/lot/used?doors=4

Request handler

Once parameters are passed to your program, you need to get them in order to use them. To do that, you'd use input-param statement. For the example here, this may be a very simple implementation of "shop" request in file "shop.vely" - note that a file name where you implement a request must be named after the request itself:

#include "vely.h"

request-handler /shop
    out-header default

    input-param lot
    input-param doors

    @Hello, you are interested in a <<p-out lot>> car with <<p-out doors>> doors!
end-request-handler

This request will have the value of parameter "lot" in variable "lot", and that of parameter "doors" in variable "doors". The names of parameters from the URL/environment and the names of variables in input-param statement must always match; this makes it easy to read and write programs. The request then prints out a message with both parameters. That's simple enough.

Make your program

To make this into an application, use vv with "-q" option:

vv -q

This will pick up any .vely files in the current directory. In this case, there's only one, but if you had 20, or 200, they would be used. Vely will use a "make" mechanism, meaning it will only compile files that need it.

Run your program

For command-line programs, you need to specify an application path and a request path. However, since a command-line program doesn't take URL like the above, you'd pass that information through the environment before executing the program. That's a more flexible and safer way to pass parameters. The way you do this is to use vv utility with "-r" option like this:

vv -r --req="/shop/lot/used/doors/4" --exec

The result is:

Content-Type: text/html;charset=utf-8
Cache-Control: max-age=0, no-cache
Pragma: no-cache
Status: 200 OK

Hello, you are interested in a used car with 4 doors!

Since command-line programs work exactly the same as web applications, you will see the HTTP header output (i.e. the Content-Type above and rest of it). You can easily skip that with --silent-header option:

vv -r --req="/shop/lot/used/doors/4" --exec --silent-header

and the output is then just:

Hello, you are interested in a used car with 4 doors!

Input parameters can be specified in many different ways. You could have done this to test a query string:

vv -r --req="/shop/lot/used?doors=4" --exec --silent-header

or this:

vv -r --req="/shop?lot=used&doors=4" --exec --silent-header

The result is the same.

Take a look at vv utility. You can specify more than just input parameters. For instance, you can attach file(s), use different request methods (such as for REST APIs, like PUT, POST, DELETE etc.). You can pretty much supply any information a web application would have; this is great for testing web applications: you can test them in a batch, automated mode in a simple test script.

If you skip the "--exec" option, you can see how to set environment variables and execute your program manually (and faster, since you'd be doing it directly):

vv -r --req="/shop/lot/used/doors/4"

This will display all the environment variables you might want to set along with the command to execute your program, for example this may be the result:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=no
export REQUEST_METHOD=GET
export SCRIPT_NAME="/cars"
export PATH_INFO="/shop/lot/used/doors/4"
export QUERY_STRING=""
/var/lib/vv/bld/cars/cars

You can copy and paste this to execute in command line or a shell script.

Run your application server

When you built your application (with "vv -q" above), both command-line and application-server executables were created. They are different, even though they serve the same requests, taking the same input and producing the same output. The application server executable runs as a FastCGI server, so its internal request-processing mechanism is different than for a command-line program.

So, to run an application server, simply use vf, which is Vely FastCGI program manager:

vf -w 3 cars

Here, you'd start 3 processes to serve incoming requests in parallel. You can start any number of them. If you want Vely to start as many processes as needed to handle the load, simply do:

vf cars

In this case, when there's no requests, there will be no server processes staying resident in memory. When there are requests coming in, the number of processes will increase as the load increases, and decrease as it goes down. See vf for a number of parameters you can fine tune this.

Now that you have an application server running, you can talk to it. In a web application, this would mean that your web server will talk to your application server; that's how web requests get fullfilled. But you can test your newly minted application server without having a web server; here's how: you can use a client command-line utility, like in this example:

vv -r --req="/shop/lot/used?doors=4" --exec --silent-header --server

Note the "--server" option above. It means your application server will be called. If you'd like to see what's executed, just omit "--exec":

vv -r --req="/shop/lot/used?doors=4" --silent-header --server

and the result might be like this:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=yes
export REQUEST_METHOD=GET
export SCRIPT_NAME="/cars"
export PATH_INFO="/shop/lot/used"
export QUERY_STRING="doors=4"
cgi-fcgi -connect /var/lib/vv/cars/sock/sock  /cars

Just like before, you can copy this and either execute or run in a script. Here, a "cgi-fcgi" utility is called. It will use all the variables set and send them to your server as a request, it will receive a reply from it, and then display it. The result is of course the same:

Hello, you are interested in a used car with 4 doors!

However, this was very different from what you've done before. Previously, you started a command-line program that ran a request, and then exited. Here, you have an application server that has resident (daemon) processes running in the background, and you sent a request to one of them, which then replied and voila, you have it. Many clients can simultaneously call the server this way and get a reply. So this is a totally different beast here, even though it looks similar.

Note that "cgi-fcgi" is connecting to your application server via a Unix socket "/var/lib/vv/cars/sock/sock", which is automatically created by Vely. A Unix socket is an extremely fast method of inter-process communication, and is used by default. If your application server(s) run on different computers, then you'd use TCP sockets. That's easy too; see vf.

Setup web server and access from browser

Okay, so now that you know all this, you can use a web server to talk to your application server. This is how real-world web applications work. End-users (typically through their browsers) will talk to your web server(s), and in turn, they will talk to your application server(s). The reply goes back in the same order.

The advantages of this kind of architecture are many. You're separating your presentation/transfer layer (i.e. web server) from your application/business layer (i.e. your application server). This is good because it increases security, data safety, performance and scalability. This is a topic of its own, but suffice it to say you're on the right track.

Now, set up your web server. To do this, here are the instructions for Apache and Nginx. Virtually all web servers support FastCGI, so if you're using another web server, check out how to reverse proxy FastCGI requests. In a nutshell, aside from a one-time setup described in the links here, the most important thing you need to do is to provide a "directive" to your web server, so it knows where is the Unix socket for your application server. That way, they can talk. For instance, for Nginx, this directive would be in the Nginx configuration file (under "server" section):

location /cars { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/cars/sock/sock; }

and for Apache, the following would be in Apache's configuration file:

ProxyPass "/cars" unix:///var/lib/vv/cars/sock/sock|fcgi://localhost/cars

That's about it. After you do this and restart the web server, you should be able to try it from a web browser with something like the following - here the web address is your local computer (i.e. "127.0.0.1"), but in general if you're working off of a computer with a different IP or name, just replace it:

http://127.0.0.1/cars/shop/lot/used/doors/4

Just like in the above examples, you can vary your URL to use any combination of a path segment and a query string, as in:

http://127.0.0.1/cars/shop?lot=used&doors=4

The result will be the same.

Tasks

A task is some action your request executes. When you created "shop.vely" request handler, you've done so with the intention of handling a task. So a request that performs a single action is a task in itself.

As you design your application, you might want to keep each request to perform a single task (in which case you don't need to use tasks at all!), or you may want to perform different (but related) actions within the same request handler. It's a matter of how you'd like to structure your code. You may have tasks in only some of your request handlers (maybe those that are a bit more involved but you still want them in a single handler), and the rest of your handlers would not use tasks (since they serve just a single purpose).

Here's an example of using tasks. In this case you might want to have two particular actions related to cars: buy and lease. To do that, you might add an input parameter "type" (meaning a type of shopping) with the intent to have two possible values: "buy" and "lease". Copy and paste this to file "shop.vely":

#include "vely.h"

%% /shop
    out-header default

    input-param lot
    input-param doors

    task-param type
    if-task "buy"
        @Buying a car
    else-task "lease"
        @Leasing a car
    end-task

    @Hello, you are interested in a <<p-out lot>> car with <<p-out doors>> doors!
%%

Note usage of "%%" - you can use them instead of "request-handler" and "end-request-handler" statements for brevity.

To recompile, use vv with "-q" option again:

vv -q

A task-param is like input parameter, but it's used semantically as a value to choose what task to perform. A subsequent if-task refers to the task-param that last executed prior to it. This way a program structure is more streamlined, avoiding complicated conditionals while making it easy to follow the code logic. That's what tasks do. Here, you're just outputting informational messages; in a real application you'd be doing things that implement the intented functionality.

The URL for this might be:

http://127.0.0.1/cars/shop/type/lease/lot/used?doors=4

You can have any number of tasks. A sub-task is just a task that's evaluated within another task - so basically, it's a task-param within an if-task. Hence, the distinction between a task and a subtask is purely logical, meaning it depends on your own vision of task hierarchy. For example, you might add a subtask "financing" that only applies if the car is bought (i.e. not leased), and it would have values of "loan" and "cash"; in this case there's an additional input parameter "interest_rate" but only if "financing" subtask is "loan". Here's what that looks like in file "shop.vely":

#include "vely.h"

%% /shop
    out-header default

    input-param lot
    input-param doors

    task-param type
    if-task "buy"
        task-param financing
        if-task "loan"
            input-param interest_rate
            @Buying a car using a loan with interest rate of <<p-out interest_rate>>%.
        else-task "cash"
            @Buying a car using cash.
        end-task
    else-task "lease"
        @Leasing a car.
    else-task other
        @Unknown car shopping type.
    end-task

    @Hello, you are interested in a <<p-out lot>> car with <<p-out doors>> doors!
%%

To recompile, use vv with "-q" option again:

vv -q

You can see how a clean hierarchy of tasks can be built to naturally reflect the intent here.

To execute from command line:

vv -r --req="/shop/lot/used/doors/4/type/buy/financing/loan/interest-rate/3"  --exec --silent-header

The URL for this might be in financing case:

http://127.0.0.1/cars/shop/type/buy/lot/new/financing/loan?interest_rate=5&doors=4

or in case of buying cash:

http://127.0.0.1/cars/shop/type/buy/lot/new/financing/cash?doors=4

Requests without tasks

Another way to design an application is to have single-task requests. It means that a request will perform just a single task; any input parameters it takes only apply to this task.

In the application here, it might look like the following. Note that now, the request path will be a hierarchical path, so this is great for implementing REST APIs, but also any other hierarchy-based methodology.

One important thing to note is that the file names for each of these request handlers are "decorated". It means any inner forward slash is substituted for a double underscore, and any hyphen for a single underscore. Okay, here's that in practice:

First, a request to buy a car with a loan in file "shop__buy__loan.vely" - note that the request path is "/shop/buy/loan", and that forward slashes in it are double underscores in the source file name:

#include "vely.h"

%% /shop/buy/loan
    out-header default

    input-param lot
    input-param doors
    input-param interest_rate
    @Buying a car using a loan with interest rate of <<p-out interest_rate>>%.

    @Hello, you are interested in a <<p-out lot>> car with <<p-out doors>> doors!
%%

Next, a request to buy a car with cash in file "shop__buy__cash.vely":

#include "vely.h"

%% /shop/buy/cash
    out-header default

    input-param lot
    input-param doors
    @Buying a car using cash.

    @Hello, you are interested in a <<p-out lot>> car with <<p-out doors>> doors!
%%

Finally, a request to lease a car in file "shop__lease.vely":

#include "vely.h"

%% /shop/lease
    out-header default

    input-param lot
    input-param doors

    @Leasing a car.
    @Hello, you are interested in a <<p-out lot>> car with <<p-out doors>> doors!
%%

To make this into an application, use vv with "-q" option:

vv -q

Note that Vely will compiles all .vely files currently in the directory. You can keep "shop.vely" file from before, and be able to use either kind of request URL, or you can remove it. This is just an example, so here it stays.

As you can see, you have 3 source files ("shop__buy__loan.vely", "shop__buy__cash.vely" and "shop__lease.vely"), but each is now much simpler. And the way to call these request handlers is also very intuitive, using the hierarchy path they are defined with. The only thing you need to remember is that a request path in a URL must start and end with an underscore. For instance, here are the URLs to call the above request handlers, written in different notations as far as input parameters go. First to buy with a loan:

http://127.0.0.1/cars/_shop/buy/loan_/lot/new/interest-rate/3?doors=4

Note the underscore at the start and the end of request path "/_shop/buy/loan_/". You could also write that as "/_/shop/buy/loan/_/" if it looks better to you.

Then to buy for cash:

http://127.0.0.1/cars/_shop/buy/cash_/lot/new/doors/4

and to lease:

http://127.0.0.1/cars/_shop/lease_/lot/new/doors/4

Or from command line, for example:

vv -r --req="/_shop/buy/loan_/lot/new/interest-rate/3?doors=4"  --exec --silent-header

Custom application path

Note that the application path doesn't necessarily have to be just the application name, but it has to end with it. For instance, it could be "/api/v2/cars". To have a custom application path like this, you'd specify this path when building your application with vv as a "--path" option:

vv -q --path="/api/v2/cars"

In this case you'd change the web server reverse proxying. For Nginx:

location /api/v2/cars { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/cars/sock/sock; }

and for Apache, the following would be in Apache's configuration file:

ProxyPass "/api/v2/cars" unix:///var/lib/vv/cars/sock/sock|fcgi://localhost/api/v2/cars

The URL then might be for instance:

http://127.0.0.1/api/v2/cars/_shop/lease_/lot/new/doors/4

Often, this is done to version the API for an application; to get started you probably won't need this, but perhaps you might in the future.

Structuring of requests and tasks

By "structuring", it's meant the answer to a question: should I use tasks, or write a request for each task, or use a combination?

Tasks are meant to provide a bit of depth to a request, but they shouldn't be too deep. A request should still remain a relatively simple set of related actions in a logical sense; the actual implementation may be lengthy or complex (reflecting the nature of actions requested), however there shouldn't be too many tasks/sub-tasks within it, perhaps up to 3 or 4 (though there's no hard and fast rule). If there are too many (sub)tasks, then it's likely that a request should be split into multiple ones that handle the same logic in a simpler and more manageable way. Tasks should only be there to serve a request, not to substitute it. A request should be the simplest and smallest action that logically makes sense, while tasks help implement their internals.

On the other hand, single-task requests (i.e. not using task-params at all), are easier to write, read and call with a hierarchical structure, though you'll have more source files.

Ultimately it's up to you how you structure your application. It may end up being a combination of approaches, and it may change over time as you keep enhancing your design.

Conclusion

In this article, you've learned how to get started writing a Vely application. Requests and request-handlers are the basics of it, and this should give you a better idea about designing your own. For more, see application-architecture, vely-architecture and how-vely-works.

See also

How to use regular expressions

Regular expression (or regex) is a way to search and replace text with very little code. Typically, regex is used when a free format text is obtained and you wish to analyze it or transform it in some way.

Regex is considered a standard technology, because you can use it pretty much across the board: in shell scripts, and in various languages and frameworks.

For example, you might want to find word "cats" and replace it with "dogs". Or you might need to find all words that start with some prefix and remove them. Other times, you'd be interested in doing something like this but only if the target text is preceded by another word. Or you may want to look only for a certain repetition in the text. There's lots of ways to make use of regex.

Tiny application with regex in it

Your Vely applications can use regex with match-regex statement. Here's a simple example. Create a new application called "regex" in a new directory:

mkdir -p regex
cd regex
sudo vf -i -u $(whoami) regex

The vf command is a Vely program manager and it will create a new application (see how-vely-works). Then make a source code file "regex_example.vely":

vi regex_example.vely

and copy and paste this to it (use "vv -m" for syntax highlighting):

#include "vely.h"

%% /regex-example
    out-header default
    input-param str
    match-regex "[a-zA-Z]{3}[0-9]{1,3}" \
        in str \
        replace-with "XXXX" \
        result define res \
        cache
   @Result is <<p-out res>>
%%

Note that the source file name ("regex_example.vely") should match the request name, which is "regex-example". If you're using hyphens (which is useful for web applications), just substitute with underscore. The fact that a request is implemented in a file with the same name helps keep your applications neat, tidy and easy to peruse.

What does it do

Your match-regex statement will search for a pattern that consists of 3 characters followed by a number between 1 and 3 digits long. This search for a pattern will be performed on string variable "str", and if the pattern is found, it will be replaced with string "XXXX". The result will be stored in string variable "res". Finally, "cache" clause is here for performance, and what it means is that the parsing of a pattern will be done only once and then the whole process of matching and substituting cached, so that repeated executions are very fast.

How does it work

Take a look at a pattern in match-regex. This is what you want to be matched. Start with matching a single character:

[a-zA-Z]

This means match any character ranging from 'a' to 'z', and from 'A' to 'Z' (to cover both lower and upper case letters). Square brackets "[" and "]" simply state to match characters or ranges within them; that's called a "character class". Next, you'd want this character class to repeat 3 times:

[a-zA-Z]{3}

Repetition is a powerful way to specify not just a simple number of repeated patterns, but also a range of repetitions as well, so for example you could say {2-4} to allow 2, 3 or 4 repetitions. And you'd use just that in the next part of your regex pattern:

[a-zA-Z]{3}[0-9]{1,3}

Example of a matching string would be "Hey777" or "ABC1" or "XYZ23". You can see even in a simple example like this that regex can be very useful to quickly specify what you want done and just do it.

Create executable application

Now, make a native executable:

vv -q

Run it

You can now run your program! First provide a matching string, in this case "Trying Hey777 string":

vv -r --req="/regex-example/str/Trying Hey777 string" --exec --silent-header

The result:

Trying XXXX string

That was a success because Hey777 was replaced with XXXX just like you specified. Now try a non-matching string:

vv -r --req="/regex-example/str/Trying H777 string" --exec --silent-header

The result:

Trying H777 string

Since H777 does not match the pattern you have in match-regex, there was no substitution, and that's how it should be.

Note that vv is used here to run a program for convenience. If you skip "--exec" option, it will output a bash script you can copy and paste to run your program's executable directly.

Choosing different regex flavors

Regex is by default parsed as a PCRE2 (Perl-compatible Regular Expressions v2) regular expression, but you can also use use Posix ERE (or extended regex syntax) if you use "--posix-regex" when building your application.

Writing regex with Vely

Since Vely is based on C programming language, the only character you need to escape is the backslash. For example, in this regex:

match-regex "(good).*(day)" \
    in "Hello, good day!" \
    replace-with "\\2 \\1" \
    result res

you're using what's called "subexpressions". That's a very nifty feature of regex, where you place a sub-pattern in parenthesis (such as (good) for instance), and then you can reference those in a replacement string with \1, \2 etc., where \1 is the first such subexpression, \2 is the second etc. So in the above example, the transformation of string "Hello, good day" would give you:

Hello, day good!

Notice the use of \\ instead of \ in the code. That's because \ has a special meaning in a C string, so you'd escape it and use \\ instead. That's it!

Caching and performance

Caching is important with regex. If you have no capability to cache it, then each time you search for a pattern, the regular expression is parsed anew and a plan of parsing created. If your application serves as a server, that's inefficient, so in that case you can use "cache" clause in match-pattern, as it's done here. This will parse the pattern only once and use the saved execution plan every single time in the future. This greatly increases performance (about 5x in testing), and it comes close to theoretical limits of parsing speed.

Run as background processes

Speaking of application servers, you can run your program as a native-executable application server just by doing:

vf -w 5 regex

This will start 5 application server processes to serve incoming requests (you can also have a dynamic number of processes too, see vf). Testing your server is easy:

vv -r --req="/regex-example/str/This is BCD312 account" --exec --silent-header --server

Note the "--server" option. It says to contact a server and execute the same request as before. But now, each of the 5 processes you started is staying resident in memory and serving the incoming requests. This way, your server can serve a large number of concurrent requests in parallel. Because each process stays alive, so does the cache for your regex pattern matching. As a result, you get a better performance for string parsing and manipulation in your applications.

Run from the web

Learn regex

Regex is a pretty large topic, even if you'd typically use short regular expressions, like the one in this example. Keep in mind, there's a huge number of things you can do with it, and so describing regex in full in one article is impossible. Try regex tutorials, like this one. Then you can use what you learned here to make use of regex in your future applications.

See also

JSON parsing and searching

This example demonstrates parsing JSON text. The text parsed contains information about cities. JSON document includes an array of countries, which includes an array of states, each having an array of cities. Use of UTF8 Unicode data is also shown - some cities have such characters in their names. JSON text comes from two different kinds of client requests: from an HTML form and also from Javascript/fetch(). This example shows how to deal with a generic JSON document where structure is known but can change, as well as if you don't know its structure and search for what interests you along the way.

In a nutshell: web browser; Nginx; Unix sockets; 5 source files, 139 lines of code.

Screenshots of application

This is the form where you can enter JSON text to be parsed:

This is the result of submitting as 'Extract all data', where all data nodes are shown, along with their normalized names (that includes hierarchy and arrays), their values and types:

The following is the output of pushing 'Extract specific data' button. Specific data nodes are found, taking into account the hierarchy of data, and their names and values displayed:

You can call Vely code from Javascript fetch() - this is the output from post.html (see Access application... on how to run it):

Setup prerequisites

vv -m

Get the source code

tar xvf $(vv -o)/examples/json.tar.gz
cd json

Setup application

The very first step is to create an application. The application will be named "json", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) json

This will create a new application home (which is "/var/lib/vv/json") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Build application

Use vv utility to make the application:

vv -q

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 json

vf json

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit json

Setup web server

This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/json" is the application path (see request-URL) and "json" is your application name):

location /json { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/json/sock/sock; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

Note: you must not have any other URL resource that starts with "/json" (such as for example "/json.html" or "/json_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/json", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Enter JSON and send for parsing 
http://127.0.0.1/json/json_form

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.
Copy the contents of file cities.json (see under Source files) to a text area in the form.

- Run from Javascript via fetch method
You can also test JSON parsing via Javascript/fetch mechanism. First, copy the file to your web server in a separate directory:

For Apache or older versions of Nginx:
```
sudo mkdir /var/www/html/velytest
sudo cp post.html /var/www/html/velytest
```
Test it (use your web address instead of 127.0.0.1 if not testing locally):
```
http://127.0.0.1/velytest/post.html
```

For newer versions of Nginx:

sudo mkdir /usr/share/nginx/html/velytest
sudo cp post.html /usr/share/nginx/html/velytest

Test it (use your web address instead of 127.0.0.1 if not testing locally):

http://127.0.0.1/velytest/post.html

Source files

The following are the source files in this application:
- JSON document (cities.json)
This is the JSON document that you enter in the form for parsing. It contains countries, states and cities along with their population.

{ "country": [
    {
        "name": "USA",
        "state": [
            {
                "name": "Arizona",
                "city": [
                    {
                        "name" : "Phoenix",
                        "population": 5000000
                    } ,
                    {
                        "name" : "Tuscon",
                        "population": 1000000
                    }

                ]
            } ,
            {
                "name": "California",
                "city": [
                    {
                        "name" : "Los Angeles",
                        "population": 19000000
                    },
                    {
                        "name" : "Irvine"
                    }
                ]
            }
        ]
    } ,
    {
        "name": "Mexico",
        "state": [
            {
                "name": "Veracruz",
                "city": [
                    {
                        "name" : "Xalapa-Enríquez",
                        "population": 8000000
                    },
                    {
                        "name" : "C\u00F3rdoba",
                        "population": 220000
                    }
                ]
            } ,
            {
                "name": "Sinaloa",
                "city": [
                    {
                        "name" : "Culiac\u00E1n Rosales",
                        "population": 3000000
                    }
                ]
            }
        ]
    }
    ]
}

- Call from Javascript (post.html)
You can call your Vely code from Javascript via fetch(). This HTML file will do that as soon as it loads (no buttons to push), you can of course change it to fit your needs. This also demonstrates the use of POST method with Content-Type of "application/json" to talk to your server-side Vely code.

<!DOCTYPE html>
<html lang="en">
<head>
    <title>Vely + JavaScript/Fetch + POST/PUT/PATCH + JSON</title>
</head>
<body>
    <h1 class="align">Example: Vely + JavaScript/Fetch + POST/PUT/PATCH + JSON</h1>
    <script>
        fetch('/json/json_process',{
            method: 'POST',
            headers: {'content-type': 'application/json'},
            body:  '{ "country": [ \
    {  \
        "name": "USA", \
        "state": [ \
            {  \
                "name": "Arizona", \
                "city": [ \
                    { \
                        "name" : "Phoenix", \
                        "population": "5000000" \
                    } , \
                    { \
                        "name" : "Tuscon", \
                        "population": "1000000" \
                    }  \
 \
                ] \
            } , \
            {  \
                "name": "California", \
                "city": [ \
                    { \
                        "name" : "Los Angeles", \
                        "population": "4000000" \
                    }, \
                    { \
                        "name" : "Irvine" \
                    } \
                ] \
            }  \
        ]  \
    } , \
    {  \
        "name": "Mexico", \
        "state": [ \
            {  \
                "name": "Veracruz", \
                "city": [ \
                    { \
                        "name" : "Xalapa-Enríquez", \
                        "population": "8000000" \
                    }, \
                    { \
                        "name" : "C\u00F3rdoba", \
                        "population": "220000" \
                    } \
                ] \
            } , \
            {  \
                "name": "Sinaloa", \
                "city": [ \
                    { \
                        "name" : "Culiac\u00E1n Rosales", \
                        "population": "3000000" \
                    } \
                ] \
            }  \
        ]  \
    } \
    ] \
}'
        })
        .then((result) => { return result.text(); })
        .then((content) => { document.getElementById("json_output").innerHTML = content; });
    </script>
    <div id='json_output'></div>
</body>
</html>

- Enter JSON (json_form.vely)
This is a simple HTML form where you can enter JSON document. Since the code in json_process.vely and json_all.vely parses a list of cities as described, the text to enter is given in cities.json file.

#include "vely.h"

request-handler /json_form

    out-header default

    @<h2>Enter JSON</h2>

    @<form action="<<p-path>>/json_process" method="POST">

    @    <label for="json_text">JSON text:</label><br>
    @    <textarea name="json_text" rows="8" columns="70"></textarea><br/>

    @    <button type="submit">Extract specific data</button>
    @    <button type="submit" formaction="<<p-path>>/json_all">Extract all data</button>
    @ </form>
end-request-handler

- Parse all JSON data in a loop (json_all.vely)
This parses the document that you may not know the structure of. Each data node is obtained you can examine it in your code.

#include "vely.h"

request-handler /json-all
    out-header default

    input-param json_text

    // Parse json text and display any error and the position of it
    new-json define json from json_text status define st error-text define etext error-position define epos
    if (st != VV_OKAY) {
        @Could not parse JSON! Error [<<p-out etext>>] at position <<p-num epos>>.
        exit-request
    }

    // Traverse JSON document, node by node, display as a table of all data nodes
    read-json json traverse begin
    @<table border='1'>
    while (1)
    {
        read-json json traverse key define k value define v type define t status define s
        if (s != VV_OKAY) break;
        // Display name, value and type (ignore boolean and type since we don't have them)
        @<tr>
            @<td><<p-out k>></td> <td><<p-out v>></td>
            @<td><<p-out t==VV_JSON_TYPE_NUMBER?"Number":(t==VV_JSON_TYPE_STRING?"String":"Other")>></td>
        @</tr>
    }
    @</table>

end-request-handler

- Parse JSON by looking up specific elements (json_process.vely)
Parse the JSON document. This shows parsing the document that you know a structure of, but does not have a fixed structure, so each element is retrieved based on its normalized name (see read-json). Note the usage of "key-list" clause in order to build a key from key fragments, which is especially useful in hierarchical structures; it's also extremely fast because there is no copying of keys.

#include "vely.h"

request-handler /json-process
    out-header default

    // If JSON data sent via URL-encoded GET or POST
    input-param json_text
    // If JSON data sent in the request body (application/json), use that JSON data 
    request-body json_body
    get-req content-type to define ctype
    if (!strcmp(ctype, "application/json")) json_text=json_body;

    // Parse json text and display any error and the position of it
    new-json define json from json_text status define st error-text define etext error-position define epos
    if (st != VV_OKAY) {
        @Could not parse JSON! Error [<<p-out etext>>] at position <<p-num epos>>.
        exit-request
    }

    @Cities found<hr/>

    num country_count;
    num state_count;
    num city_count;

    // Start displaying a list
    @<ul>
    // Look for countries, states and then cities
    // Data is organized in hashed arrays, for example
    // country[0].state[1].city[0]
    // and each can have sub-nodes, such as
    // country[0].name
    // etc.
    for (country_count = 0; ; country_count++) {

        // First, build key prefix for a country
        (( define country
        @"country"[<<p-num country_count>>]
        ))

        // Search for a country name under index [country_count]
        read-json json key-list country, ".\"name\"" value define country_name  status st
        if (st != VV_OKAY) break;

        // Country found
        @<li>Country: <<p-out country_name>><br/>
        @<ul>

        // Look for states under this country
        for (state_count = 0; ; state_count++) {

            // Build key prefix for a state
            (( define state
            @."state"[<<p-num state_count>>]
            ))

            // Search for a state name as: country[countr_count].state[state_count]
            read-json json key-list country, state, ".\"name\"" value define state_name  status st
            if (st != VV_OKAY) break;

            // State found
            @<li>State: <<p-out state_name>><br/>
            @<ul>

            // Look for cities under state
            for (city_count = 0; ; city_count++) {

                // Build key prefix for city
                (( define city
                @."city"[<<p-num city_count>>]
                ))

                // Search for a city name as: country[countr_count].state[state_count].city[city_count]
                read-json json key-list country, state, city, ".\"name\"" value define city_name  status st
                if (st != VV_OKAY) break;

                // Get city population
                read-json json key-list country, state, city, ".\"population\"" value define city_population  status st
                if (st != VV_OKAY) city_population="unknown";

                // Display city name and its population
                @<li>City:<<p-out city_name>> (<<p-out city_population>>)</li>
            }
            @</ul>
            @</li>
        }
        @</ul>
        @</li>
    }
    @</ul>

end-request-handler

See also

Multitenant SaaS with MariaDB and Apache

Cloud applications typically run as Software-as-a-Service (SaaS). This article will demonstrate typical functionality you need to have for a minimal functioning SaaS, and how to achieve it. The example shown here is a complete application you can run on virtually any Linux computer.

The "notes" is a multi-tenant web application that you can run on the Internet as SaaS. Each user has a completely separate data space from any other. This web application will let user sign up for Notes service - a place where a user can create notes, and then view and delete them.

In a nutshell: MariaDB; web browser; Apache; Application path; Unix sockets; 7 source files, 315 lines of code.

Screenshots of application

Create new user by specifying an email address and password (you can style these anyway you like, such as with CSS):

Verifying user's email:

User logs in with own user name and password:

Adding a note once the user logged in:

Listing notes:

Ask for confirmation to delete a note:

After user confirms, delete a note:

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because they are used in this example, you will need to install Apache as a web server and MariaDB as a database.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

tar xvf $(vv -o)/examples/multitenant-SaaS.tar.gz
cd multitenant-SaaS

Setup application

The very first step is to create an application. The application will be named "multitenant-SaaS", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) multitenant-SaaS

This will create a new application home (which is "/var/lib/vv/multitenant-SaaS") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create MariaDB database "db_multitenant_SaaS" owned by user "vely" with password "your_password". You can change any of these names, but remember to change them everywhere here. And then, you will create database objects in the database.

Execute the following logged in as root in mysql utility:

--Create velydb database hosting application data (if it doesn't exist):
create database if not exists db_multitenant_SaaS;
create user if not exists vely identified by 'your_password';
grant create,alter,drop,select,insert,delete,update on db_multitenant_SaaS.* to vely;
-- Create database objects needed for the application (eg. tables, indexes):
use db_multitenant_SaaS;
source setup.sql;
exit

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_multitenant_SaaS". This name doesn't have to be "db_multitenant_SaaS", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo '[client]
user=vely
password=your_password
database=db_multitenant_SaaS
protocol=TCP
host=127.0.0.1
port=3306' > db_multitenant_SaaS

The above is a standard mariadb client options file. Vely uses native MariaDB database connectivity, so you can specify any options that a given database lets you.

Build application

Use vv utility to make the application:

vv -q --db=mariadb:db_multitenant_SaaS --path="/api/v2/multitenant-SaaS"

Note usage of --db option to specify MariaDB database and the database configuration file name.

--path is used to specify the application path, see request-URL.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 multitenant-SaaS

vf multitenant-SaaS

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit multitenant-SaaS

Setup web server

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/api/v2/multitenant-SaaS" is the application path (see request-URL) and "multitenant-SaaS" is your application name):

ProxyPass "/api/v2/multitenant-SaaS" unix:///var/lib/vv/multitenant-SaaS/sock/sock|fcgi://localhost/api/v2/multitenant-SaaS

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/api/v2/multitenant-SaaS" (such as for example "/api/v2/multitenant-SaaS.html" or "/api/v2/multitenant-SaaS_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/api/v2/multitenant-SaaS", see request-URL.

Setup local mail

This example uses email as a part of its function. If your server already has capability to send email, you can skip this.

Otherwise, you can use local mail, and that means email addresses such as "myuser@localhost". To do that, install postfix (or sendmail). On Debian systems (like Ubuntu):

sudo apt install postfix
sudo systemctl start postfix

and on Fedora systems (like RedHat):

sudo dnf install postfix
sudo systemctl start postfix

When the application sends an email to a local user, such as <OS user>@localhost, then you can see the email sent at:

sudo vi /var/mail/<OS user>

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Get started 
http://127.0.0.1/api/v2/multitenant-SaaS/notes/action/begin

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Source files

The following are the source files in this application:
- SQL setup (setup.sql)
The two tables created are: "users", which contains information about each user; and "notes" which contains notes entered by the user.

Each user in "users" table has its own unique ID ("userId" column) along with other information such as email address and whether it's verified. There's also a hashed password - an actual password is never stored in plain text (or otherwise), rather a one-way hash is used to check the password.

The "notes" table contains the notes, each along with "userId" column that states which user owns them. The "userId" column's value matches the namesake column from "users" table. This way, every note clearly belongs to a single user.

create table if not exists notes (dateOf datetime, noteId bigint auto_increment primary key, userId bigint, note varchar(1000));
create table if not exists users (userId bigint auto_increment primary key, email varchar(100), hashed_pwd varchar(100), verified smallint, verify_token varchar(30), session varchar(100));
create unique index if not exists users1 on users (email);

- Run-time data (login.h)

In order to properly display the Login, Sign Up and Logout links, you will need some flags that are available anywhere in the application. Also, the application uses cookies to maintain a session, so this needs to be available anywhere, for example to verify that the session is valid. Every request sent to the application is verified that way. Only requests that come with cookies we can verify are permitted.

So to that effect, you will have a global-request-data type "reqdata" (request data) and in it there's "sess_userId" (ID of user) and "sess_id" (user's current session ID). You'll also have rather self-explanatory flags that help render pages.

#ifndef _VV_LOGIN
#define _VV_LOGIN

typedef struct s_reqdata {
    bool displayed_logout; // true if Logout link displayed
    bool is_logged_in; // true if session verified logged-in
    char *sess_userId; // user ID of current session
    char *sess_id; // session ID
} reqdata;

void login_or_signup ();

#endif

- Session checking and session data (_before.vely)

Vely has a notion of a before-request-handler. It's the code you write that executes before any other code that handles a request. To do this, all you need is to write this code in file that's named "_before.vely" and the rest will be automatically handled.

That's very useful here. Anything that a SaaS application does, such as handling requests sent to an application, must be validated for security. This way, the application knows if the caller has permissions needed to perform an action.

So, this checking of permission will be done here in a before-request handler. That way, whatever other code you have handling a request, you will have the information about session already provided.

To keep session data (like session ID and user ID) available anywhere in your code, you'll use global-request-data. It's just a generic pointer (void*) to memory that any code that handles a request can access. This is perfect for handling sessions.

#include "vely.h"
#include "login.h"

// _before() is a before-request-handler. It always executes before
// any other code that handles a request. It's a good place for any
// kind of request-wide setting or data initialization
void _before() {
    // Output HTTP header
    out-header default
    reqdata *rd; // this is global request data, see login.h
    // allocate memory for global request data, will be automatically deallocated
    // at the end of request
    new-mem rd size sizeof(reqdata)
    // initialize flags
    rd->displayed_logout = false;
    rd->is_logged_in = false;
    // set the data we created to be global request data, accessible
    // from any code that handles a request
    set-req data rd
    // check if session exists (based on cookies from the client)
    // this executes before any other request-handling code, making it
    // easier to just have session information ready
    _check_session ();
}

- Checking if session is valid (_check_session.vely)

One of the most important tasks in a multi-tenant SaaS application is to check (as soon as possible) if the session is valid. This means to check if a user is logged in. It's done by getting the session ID and user ID cookies from the client (i.e. web browser), and checking these against the database where sessions are stored.

#include "vely.h"
#include "login.h"


// Check if session is valid
void _check_session () {
    // Get global request data
    reqdata *rd;
    get-req data to rd
    // Get cookies from user browser
    get-cookie rd->sess_userId="sess_userId"
    get-cookie rd->sess_id="sess_id"
    if (rd->sess_id[0] != 0) {
        // Check if session ID is correct for given user ID
        char *email;
        run-query @db_multitenant_SaaS = "select email from users where userId='%s' and session='%s'" output email : rd->sess_userId, rd->sess_id row-count define rcount
            query-result email to email
        end-query
        if (rcount == 1) {
            // if correct, set logged-in flag
            rd->is_logged_in = true;
            // if Logout link not display, then display it
            if (rd->displayed_logout == false) {
                get-app path to define upath
                @Hi <<p-out email>>! <a href="<<p-out upath>>/login/actions/logout">Logout</a><br/>
                rd->displayed_logout = true;
            }
        } else rd->is_logged_in = false;
    }
}

- Signing up, Logging in, Logging out (login.vely)
The basis of any multi-tenant system is the ability for a user to sign up, and once signed up, to log in and log out.

Typically, signing up involves verifying email address, and more often than not, the very same email address is used as a user name. That will be the case here.

There are several tasks implemented here that are necessary to perform the functionality. Each has its own "URL request signature" or a URL path.

When Signing Up a new user, display the HTML form to collect the information. The URL request signature for this is "/login/actions/newuser"
As a response to Sign Up form, create a new user. The URL request signature is "/login/actions/createuser". input-param is used to obtain "email" and "pwd" POST form fields (i.e. query string). Password is one-way hashed and an email verification token is created as a random 5-digit number. These are inserted into "users" table, creating a new user. A verification email is sent out, and the user is prompted to read the email and enter the code.
Verify email by entering verification code sent to that email. The URL request signature is "/login/actions/verify"
Display a Login form for user to login. The URL request signature is "/login" (i.e. "action" is empty)
Login by verifying email address (i.e. user name) and password. The URL request signature is "/login/actions/login"
Logout at user's request. The URL request signature is "/login/actions/logout"
Landing page for application. The URL request signature is "/login/actions/begin"
If the user is currently logged in, go to the application's landing page.

#include "vely.h"
#include "login.h"

// Handle session maintenance, login, logout, session verification
// for any multitenant Cloud application
%% /login
    // Get URL input parameter "actions"
    task-param actions

    // Get global request data, we record session information in it, so it's handy
    reqdata *rd;
    get-req data to rd

    // If session is already established, the only reason why we won't proceed to
    // application home is if we're logging out
    if (rd->is_logged_in) {
        if (strcmp(actions, "logout")) {
            _show_home();
            exit-request
        }
    }

    // Application screen to get started. Show links to login or signup and show
    // home screen appropriate for this
    if-task "begin"
        _show_home();
        exit-request

    // Start creating new user. Ask for email and password, then proceed to create user
    // when this form is submitted.
    else-task "newuser"
        @Create New User<hr/>
        @<form action="<<p-path>>/login/actions/createuser" method="POST">
        @<input name="email" type="text" value="" size="50" maxlength="50" required autofocus placeholder="Email">
        @<input name="pwd" type="password" value="" size="50" maxlength="50" required placeholder="Password">
        @<input type="submit" value="Sign Up">
        @</form>

    // Verify code sent to email by user. The code must match, thus verifying email address    
    else-task "verify"
        input-param code
        input-param email
        // Get verify token based on email
        run-query @db_multitenant_SaaS = "select verify_token from users where email='%s'" output db_verify : email
            query-result db_verify to define db_verify
            // Compare token recorded in database with what user provided
            if (!strcmp (code, db_verify)) {
                @Your email has been verifed. Please <a href="<<p-path>>/login">Login</a>.
                // If matches, update user info to indicate it's verified
                run-query @db_multitenant_SaaS no-loop = "update users set verified=1 where email='%s'" : email
                exit-request
            }
        end-query
        @Could not verify the code. Please try <a href="<<p-path>>/login">again</a>.
        exit-request

    // Create user - this runs when user submits form with email and password to create a user     
    else-task "createuser"
        input-param email
        input-param pwd
        // create hashed (one-way) password
        hash-string pwd to define hashed_pwd
        // generate random 5 digit string for verify code
        random-string to define verify length 5 number
        // create user: insert email, hashed password, verification token. Current verify status is 0, or not verified
        begin-transaction @db_multitenant_SaaS
        run-query @db_multitenant_SaaS no-loop = "insert into users (email, hashed_pwd, verified, verify_token, session) values ('%s', '%s', '0', '%s', '')" : email, hashed_pwd, verify affected-rows define arows error define err on-error-continue
        if (strcmp (err, "0") || arows != 1) {
            // if cannot add user, it probably doesn't exist. Either way, we can't proceed.
            login_or_signup();
            @User with this email already exists.
            rollback-transaction @db_multitenant_SaaS
        } else {
            // Create email with verification code and email it to user
            write-string define msg
                @From: vely@vely.dev
                @To: <<p-out email>>
                @Subject: verify your account
                @
                @Your verification code is: <<p-out verify>>
            end-write-string
            exec-program "/usr/sbin/sendmail" args "-i", "-t" input msg status define st
            if (st != 0) {
                @Could not send email to <<p-out email>>, code is <<p-out verify>>
                rollback-transaction @db_multitenant_SaaS
                exit-request
            }
            commit-transaction @db_multitenant_SaaS
            // Inform the user to go check email and enter verification code
            @Please check your email and enter verification code here:
            @<form action="<<p-path>>/login/actions/verify" method="POST">
            @<input name="email" type="hidden" value="<<p-out email>>">
            @<input name="code" type="text" value="" size="50" maxlength="50" required autofocus placeholder="Verification code">
            @<button type="submit">Verify</button>
            @</form>
        }

    // This runs when logged-in user logs out.    
    else-task "logout"
        // Update user table to wipe out session, meaning no such user is logged in
        if (rd->is_logged_in) {
            run-query @db_multitenant_SaaS = "update users set session='' where userId='%s'" : rd->sess_userId no-loop affected-rows define arows
            if (arows == 1) {
                rd->is_logged_in = false; // indicate user not logged in
                @You have been logged out.<hr/>
            }
        }
        _show_home();

    // Login: this runs when user enters user name and password
    else-task "login"
        input-param pwd
        input-param email
        // create one-way hash with the intention of comparing with user table - password is NEVER recorded
        hash-string pwd to define hashed_pwd
        // create random 30-long string for session ID
        random-string to rd->sess_id length 30
        // Check if user name and hashed password match
        run-query @db_multitenant_SaaS = "select userId from users where email='%s' and hashed_pwd='%s'" output sess_userId : email, hashed_pwd
            query-result sess_userId to rd->sess_userId
            // If match, update user table with session ID
            run-query @db_multitenant_SaaS no-loop = "update users set session='%s' where userId='%s'" : rd->sess_id, rd->sess_userId affected-rows define arows
            if (arows != 1) {
                @Could not create a session. Please try again. <<.login_or_signup();>> <hr/>
                exit-request
            }
            // Set user ID and session ID as cookies. User's browser will return those to us with every request
            set-cookie "sess_userId" = rd->sess_userId path "/"
            set-cookie "sess_id" = rd->sess_id path "/"
            // Display home, make sure session is correct first and set flags
            _check_session();
            _show_home();
            exit-request
        end-query
        @Email or password are not correct. <<.login_or_signup();>><hr/>

    // Login screen, asks user to enter user name and password    
    else-task ""
        login_or_signup();
        @Please Login:<hr/>
        @<form action="<<p-path>>/login/actions/login" method="POST">
        @<input name="email" type="text" value="" size="50" maxlength="50" required autofocus placeholder="Email">
        @<input name="pwd" type="password" value="" size="50" maxlength="50" required placeholder="Password">
        @<button type="submit">Go</button>
        @</form>
    else-task other
        // do not display anything
    end-task
%%

// Display Login or Sign Up links
void login_or_signup() {
        @<a href="<<p-path>>/login">Login</a> &nbsp; &nbsp; <a href="<<p-path>>/login/actions/newuser">Sign Up</a><hr/>
}

- General-purpose application (_show_home.vely)

With this tutorial you can create any multitenant SaaS application you want. The multitenant-processing module above (login.vely) calls _show_home() function, which can house any code of yours. In here, you'll have Notes application, but it can be anything. _show_home() simply calls any code you wish, and is a general-purpose multitenant application plug-in.

#include "vely.h"

void _show_home() {
    notes();
    exit-request
}

- Notes application (notes.vely)

The application will be able to add notes, list them, and delete any given note. It operates under a request "notes", and the tasks it serves are:

/notes/subreqs/add_note - add a note,
/notes/subreqs/list - list notes,
/notes/subreqs/delete_note - delete a note

while a few other tasks are user-interface-only, such as:

/notes/subreqs/add_note - display an HTML form to add a note,
/notes/subreqs/delete_note_ask - ask user to confirm intent to delete a note

#include "vely.h"
#include "login.h"

// Notes application in a multitenant Cloud 
%% notes
    // get global request data
    reqdata *rd;
    get-req data to rd

    // If session invalid, display Login or Signup
    if (!rd->is_logged_in) {
        login_or_signup();
    }
    // Greet the user
    @<h1>Welcome to Notes!</h1><hr/>
    // If not logged in, exit - this ensures security verification of user's identity
    if (!rd->is_logged_in) {
        exit-request
    }
    // Get URL parameter that tells Notes what to do
    task-param subreqs
    // Display actions that Notes can do (add or list notes)
    @<a href="<<p-path>>/notes/subreqs/add">Add Note</a> <a href="<<p-path>>/notes/subreqs/list">List Notes</a><hr/>


    // List all notes for this user
    if-task "list"
        // select notes for this user ONLY
        run-query @db_multitenant_SaaS = "select dateOf, note, noteId from notes where userId='%s' order by dateOf desc" : rd->sess_userId output dateOf, note, noteId
            query-result dateOf to define dateOf
            query-result note to define note
            query-result noteId to define noteId
            (( define web_enc
            p-web note
            ))
            // change new lines to <br/> with fast cached Regex
            match-regex "\n" in web_enc replace-with "<br/>\n" result define with_breaks status define st cache
            if (st == 0) with_breaks = web_enc; // nothing was found/replaced, just use original
            // Display a note
            @Date: <<p-out dateOf>> (<a href="<<p-path>>/notes/subreqs/delete_note_ask?note_id=<<p-out noteId>>">delete note</a>)<br/>
            @Note: <<p-out with_breaks>><br/>
            @<hr/>
        end-query

    // Ask to delete a note
    else-task "delete_note_ask"
        input-param note_id
        @Are you sure you want to delete a note? Use Back button to go back,\
            or <a href="<<p-path>>/notes/subreqs/delete_note?note_id=<<p-out note_id>>">delete note now</a>.

    // Delete a note
    else-task "delete_note"
        input-param note_id
        // Delete note
        run-query @db_multitenant_SaaS = "delete from notes where noteId='%s' and userId='%s'" : note_id, rd->sess_userId \
            affected-rows define arows no-loop error define errnote
        // Inform user of status
        if (arows == 1) {
            @Note deleted
        } else {
            @Could not delete note (<<p-out errnote>>)
        }

    // Add a note
    else-task "add_note"
        // Get URL POST data from note form
        input-param note
        // Insert note under this user's ID
        run-query @db_multitenant_SaaS = "insert into notes (dateOf, userId, note) values (now(), '%s', '%s')" : rd->sess_userId, note \
            affected-rows define arows no-loop error define errnote
        // Inform user of status
        if (arows == 1) {
            @Note added
        } else {
            @Could not add note (<<p-out errnote>>)
        }

    // Display an HTML form to collect a note, and send it back here (with subreqs="add_note" URL param)
    else-task "add"
        @Add New Note
        @<form action="<<p-path>>/notes/subreqs/add_note" method="POST">
        @<textarea name="note" rows="5" cols="50" required autofocus placeholder="Enter Note"></textarea>
        @<button type="submit">Create</button>
        @</form>
    else-task other
        // do not display anything
    end-task
%%

See also

What are ACID transactions and how to use them with files: a PostgreSQL example

Files and transactions

While general operating system files are not a part of database systems, you can commit a database transaction and write a file so that a file is assured to be written if the database record is committed. Note this isn't the same as file being a part of the transaction - you would have to use database's facilities for that; however it's a way to practically tie files to database records. For example you might be writing files and inserting their information into a database. You will do just that with PostgreSQL in this example, but in general you can use other databases that support transactions - more on this next.

PostgreSQL

PostgreSQL (or just Postgres) is a popular Open Source database that uses SQL to create and manipulate data, and which is also ACID compliant. This means Atomicity, Consistency, Isolation and Durability, which are properties that a database should comply with to process transactions reliably. This is very important, as without it, data used in virtually any business might get lost, duplicated or become just plain wrong. Imagine if banks didn't implement transactions correctly - your check deposit or a simple transfer between accounts might not go through as you expect.

So what does ACID mean exactly?

Atomicity means that everything within a transaction succeeds or nothing does. So if, during the transaction, some part of it is complete, and others are not, and the transaction is interrupted (by a system crash for example), then partially completed items will be rolled back to where they were at the beginning of the transaction. Hence, you will see SQL commands like BEGIN and COMMIT that signify the beginning and the ending of a transaction, with ROLLBACK used to cancel the transaction.
Consistency means that data will be in accordance with the rules like uniqueness, predetermined relationships (like foreign keys), data range constraints etc. In other words, if a transaction is about to violate any of the rules established for the database, then it must rollback and the data cannot be left in a state contrary to the database rules.
Isolation means that each transaction is like an island unto itself, and cannot be affected by any changes going on in any other concurrent transaction. That doesn't mean transactions can't happen at the same; they can, as long as they wouldn't impact one another. So those transactions that would operate on the same data in a way that would mean they could produce bad or inconsistent results will be serialized. Different databases employ different methods to ensure this.
Durability means that once the database informs you it has completed a transaction, not even a system crash immediately following will erase its effects.

These qualities are guaranteed by PostgreSQL when you use transactions. Other databases (such as SQLite or MariaDB with InnoDB engine) are also ACID-compliant. For the most part, you can write the same SQL statements that would work with all of these databases, and you can use transactions in the same fashion.

Connecting to database

Working with PostgreSQL is the fastest with a native C library. Here I will use Vely, which uses this kind of library. It also keeps the connection alive, which avoids wasting time on connect/disconnect cycles. In addition, you can use prepared SQL statements, where the server will parse a SQL statement once and then use the parsed statement tree in the future, without having to do it again. Persistent connections and parsed statements go hand in hand, because a parsed statement is only valid within a single session. So if you were to use connection method that isn't persistent, then you would get very little out of prepared statements. Vely will automatically re-establish the connection if it gets lost - for example that can happen if the PostgreSQL server is restarted.

Prerequisites

To begin with, install Postgres. Also, install Vely, which will be used to create a native executable for this example.

Create a new directory for this example:

mkdir postgres
cd postgres

Setup the database

First, you will login as root to "psql" utility and setup database objects:

echo "create user $(whoami);
create database db_items with owner=$(whoami);
grant all on database db_items to $(whoami);
\q
" | sudo -u postgres psql

Here, you've created database "db_items" and user named after your OS Linux user, creating a passwordless Postgres user. The reason for this is because it gives you better and easier security - only you, logged in as your current OS user, can access the database, and thus you don't need a password. Then you'll give database user the permissions to basically own database "db_items" and be able to create objects, data etc.

Finally, create table "item_list" in this database:

echo "create table item_list (item_id bigserial primary key, item_name varchar(30), item_desc varchar(100))" | psql -d db_items

This will create table "item_list" which contains item names and descriptions, as well as auto-generated primary key as an ID.

Access the database

In order to access the database, you will need a database-config-file for it. This file specifes the database user name and password and any other connection string parameters. Consult Postgres documentation to see all the parameters available. Here, create database configuration file named "items". You can call this file anything, but its name is used in the code to reference the database, so if you change it, then also change it in the code below. Create file "items" with this bash code:

echo "user=$(whoami) dbname=db_items"  > items

This is actually a native PostgreSQL client configuration file, so learning the format of it may help you elsewhere as well. You'd specify Postgres user name (which is the name of your OS user, or the result of $(whoami) bash expression), and the database name is "db_items" - again that's the database we created already. There's no password because the login is passwordless, as explained above.

The code

Create file "add_item.vely" and copy this to it (note that the name of this file always matches a function name implemented in it; see how-vely-works):

#include "vely.h"

request-handler /add_item
    out-header default

    input-param name
    input-param desc

    // Start transaction
    begin-transaction @items
    // Insert data
    run-query @items = "insert into item_list (item_name, item_desc) values ('%s', '%s') returning (item_id)" \
            output define item_id : name, desc  \
            error define err_code error-text define err_text affected-rows define rows
        // Check if no error and a row is actually inserted
        if (!strcmp (err_code, "0") && rows == 1) {
            // Construct file name
            write-string define item_file
            @item_added_<<p-out item_id>>
            end-write-string
            // Write file
            write-file item_file from item_id status define write_st
            if (write_st < 0) {
                // Could not write file, even if insert okay
                rollback-transaction @items
                @Could not write to file, status <<p-num write_st>>
            } else {
                // Both write file and insert okay
                commit-transaction @items
                @SUCCESS, item added to database and written to a file (<<p-out item_file>>)
            }
        } else {
            // Could not insert
            rollback-transaction @items
            @Could not insert to database, error <<p-out err_text>>, error code <<p-out err_code>>
        }
    end-query
end-request-handler

This code is a request handler - it handles a request, such as an HTTP(S) request, or a request from command line. Here's how it works:

First, the data you will insert into database comes as input parameters. Note input-param at the beginning where you get "name" and "desc" variables - these will come from the program's caller. This is regardless of whether your program is called from the web (i.e. browser) or from the command line.
Next, begin transaction (begin-transaction) with database specified by "@items", where "items" is the database configuration file you created that describes the database.
Insert data with run-query: a query is specified using '%s' as a placeholder for parameters following a colon (":") - Vely will guard against SQL injection, so you don't need to worry about that. The output is a unique ID of a row created (in "output" clause and created with "define" subclause), and also any error information and the number of rows inserted (in "error", "error-text" and "affected-rows" clauses respectively). All this information is useful later, be in error checking or functionally.
Since the error code and error text are obtained, you would check for errors and make sure the row was actually inserted. If not, rollback the transaction, and finish.
If data was inserted okay, write a file. This file's name is based on this ID and its contents are the same, however both the name and the contents of the file are up to you - this is just for demonstration purposes. Use write-string to create a file name, and write-file to write data to a file.
If writing to a file succeeds, commit-transaction. If it fails, roll it back with rollback-transaction.
There's a minuscule chance that committing the transaction might fail, in which case the file will remain. This is typically not a problem, since it is the record in the database that is always looked up first, i.e. you won't have a record without the file written - so error handling for this is not included in the code.

You can also use prepared SQL statements by using "run-prepared-query" statement instead of "run-query" in the code above.

And the "@" output-statement sends the data to standard output, which can be the actual "stdout" stream if this is going to be a command-line program, or to the browser if this is a web application. The nice thing is, it works the same for both. p-out statement outputs a string, and when placed in between << and >> it is "inlined" into an output statement.

Create and Make the application

When you get started on a Vely application, you have to create it first with the vf program manager:

sudo vf -i -u $(whoami) items_app

"-i" option says to create an application. "-u" option says which user will own it, in this case it's "$(whoami)" which is Linix-speak for the "currently logged in user". And finally the application name is "items_app".

To make your application, use vv tool:

vv -q --db='postgres:items'

This gathers all the .vely files in the current directory (in this case just one), processes all the Vely statements (like run-query) into C code, and then compiles and links it all together into a native application. Note "--db='postgres:items'" option - it states your program uses database named "items" and the database vendor is PostgreSQL. You can have any number of databases and any number of supported vendors.

Two executables will be produced, both in the "/var/lib/vv/bld/items_app" directory. Note the "items_app" subdirectory - it matches your application name created above. This directory is like a scratch-pad for your application, this is where all the generated code goes. One executable created will be "items_app" that you can run from the command line. The other one is "items_app.fcgi" that you can run as a FastCGI application server, which is the web application.

Run from command line

Execute your program:

vv -r --req="/add-item?name=Wifi+Camera&desc=Feature+rich+wifi+camera+for+the+home" --silent-header --exec

You might get:

SUCCESS, item added to database and written to a file (item_added_1)

Verify the data has been added:

echo "select * from item_list" | psql -d db_items

The result is:

 item_id |  item_name  |               item_desc
---------+-------------+---------------------------------------
       1 | Wifi Camera | Feature rich wifi camera for the home
(1 row)

Your request has added data to the database!

Note you can also obtain the direct commands to execute a program by omitting "--exec" option in "vv -r" call above:

vv -r --req="/add-item?name=Wifi+Camera&desc=Feature+rich+wifi+camera+for+the+home" --silent-header

which produces:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=yes
export REQUEST_METHOD=GET
export SCRIPT_NAME="/items_app"
export PATH_INFO="/add-item"
export QUERY_STRING="name=Wifi+Camera&desc=Feature+rich+wifi+camera+for+the+home"
/var/lib/vv/bld/items_app/items_app

You can copy these to your script and execute, to the same result.

Vely is all about standard HTTP requests. So even when you run a command line program, it does so by receiving an HTTP request. That's why there's a request method ("GET"), a script name (which is a path to application name "items_app"), a path info (which is a path to request handler "add_item", i.e. your code above), and a query string containing the input data ("name" and "desc") which match the input parameters in your code.

This makes is very easy to build programs for both command-line and web execution, because they are the same. You don't need to write two code bases and you only debug once. Plus, you can do on command line virtually anything you can on the web, so you can write your program without ever even having a web server setup.

Note the VV_SILENT_HEADER environment variable - it suppresses HTTP header output. If it weren't there, you'd get the HTTP header, the same one that a browser will get.

You can of course run this example as a web service by starting your own application server - see this as an example.

See also

Sending mail

Sending email through web interface demonstrates web input of From, To, Subject and Message fields for an email, and the submittal of the form that sends an email, with confirmation displayed.

In a nutshell: web browser; Apache; Unix sockets; 1 source files, 78 lines of code.

Screenshots of application

An HTML form will accept the necessary information to send an email. Your server must be enabled to send email via Internet; if not, you can send email to localhost (i.e. to users on your own computer) and for that you must have an MTA like sendmail installed:

Once email is sent, a confirmation is sent to the user.:

Setup prerequisites

vv -m

Get the source code

tar xvf $(vv -o)/examples/sendmail.tar.gz
cd sendmail

Setup application

The very first step is to create an application. The application will be named "sendmail", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) sendmail

This will create a new application home (which is "/var/lib/vv/sendmail") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Build application

Use vv utility to make the application:

vv -q

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 sendmail

vf sendmail

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit sendmail

Setup web server

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/sendmail" is the application path (see request-URL) and "sendmail" is your application name):

ProxyPass "/sendmail" unix:///var/lib/vv/sendmail/sock/sock|fcgi://localhost/sendmail

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/sendmail" (such as for example "/sendmail.html" or "/sendmail_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/sendmail", see request-URL.

Setup local mail

sudo apt install postfix
sudo systemctl start postfix

and on Fedora systems (like RedHat):

sudo dnf install postfix
sudo systemctl start postfix

When the application sends an email to a local user, such as <OS user>@localhost, then you can see the email sent at:

sudo vi /var/mail/<OS user>

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Display a send-mail form 
http://127.0.0.1/sendmail/mail?action=show_form

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Source files

The following are the source files in this application:
- Send email on the web (mail.vely)
Here, the user can enter information necessary to send an email, such as the recipient email, the subject and message etc. The form to do this is displayed with task "show_form". When the user clicks Submit, the request is sent back to fulfill task "submit_form" which uses input-param to collect all the user data, construct an email string and then execute "sendmail" program to send email.

This shows usage of "%%" as a shortcut for request-handler, as well as using just the request name ("mail") instead of a request path (which would be "/mail").

Note that you must have an MTA (Mail Transfer Agent) installed (such as postfix or sendmail), and your computer must be authorized to send email on the internet. Otherwise you can test this by sending emails to your localhost.


//
// Send email example
// 

#include "vely.h"

%% mail

    out-header default

    task-param action

    if-task "show_form"

        // Display HTML form to input email details. Here we will set the 'action' parameter
        // to 'submit_form'. This way, when the user submits the form, such a request would
        // come back here and the code under else-if-string checking for 'submit_form' (below)
        // would execute

        @<h2>Enter email and click Send to send it</h2>
        @Note: 'From' field must be the email address from the domain of your server.<br/><br/>

        @<form action="<<p-path>>/mail" method="POST">
        @    <input type="hidden" name="action" value="submit_form">

        @    <label for="from_mail">From:</label><br>
        @    <input type="text" name="from_mail" value=""><br>

        @    <label for="to_mail">To:</label><br>
        @    <input type="text" name="to_mail" value=""><br><br>

        @    <label for="subject_mail">Subject:</label><br>
        @    <input type="text" name="subject_mail" value=""><br><br>

        @    <label for="message">Message:</label><br>
        @    <textarea name="message" rows="3" columns="50"></textarea>

        @    <br/><br/>

        @    <input type="submit" value="Send">
        @</form>

    else-task "submit_form"

        // Send email using data from the form. This code is called from the form above.
        // Effectively, in this file we have the code to display the form and the code
        // to handle its submission (below).

        input-param from_mail
        input-param to_mail
        input-param message
        input-param subject_mail

        write-string define msg
        @From: <<p-out from_mail>>
        @To: <<p-out to_mail>>
        @Subject: <<p-out subject_mail>>
        @
        <<p-out message>>
        end-write-string
        num st;
        exec-program "/usr/sbin/sendmail" args "-i", "-t" input msg status st
        if (st!=0) {
            @Could not send email!
        } else {
            @Email sent!
        }
        @<hr/>

    else-task other
        @Unrecognized action!<hr/>
    end-task

%%

See also

Shopping REST API

This is a shopping web service REST API with basic functions: add customer, add item, create order, add item to order, update order, delete item from order, delete order. The APIs return a valid JSON reply, even if it's just a single string (such as created ID for a customer, item or order). Listing an order returns a JSON document showing the order details. The example demostrates usage of REST methods (POST, PUT, GET, DELETE) as well as construction and use of REST URLs, and the code that implements the API.

In a nutshell: PostgreSQL; web browser; Apache; REST API; Application path; Unix sockets; 12 source files, 203 lines of code.

Setup prerequisites

vv -m

Get the source code

tar xvf $(vv -o)/examples/shopping.tar.gz
cd shopping

Setup application

The very first step is to create an application. The application will be named "shopping", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) shopping

This will create a new application home (which is "/var/lib/vv/shopping") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create PostgreSQL database "db_shopping". You can change the database name, but remember to change it everywhere here. And then, you will create database objects in the database.

Note the example here uses peer-authentication, which is the default on all modern PostgreSQL installations - this means the database user name is the same as the Operating System user name.

Execute the following in PostgreSQL database as root (using psql utility):

echo "create user $(whoami);
create database db_shopping with owner=$(whoami);
grant all on database db_shopping to $(whoami);
\q
" | sudo -u postgres psql

Next, login to database db_shopping and create the database objects for the application:

psql -d db_shopping -f setup.sql

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_shopping". This name doesn't have to be "db_shopping", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo "user=$(whoami) dbname=db_shopping"  > db_shopping

Build application

Use vv utility to make the application:

vv -q --db=postgres:db_shopping --path="/api/v1/shopping"

Note usage of --db option to specify PostgreSQL database and the database configuration file name.

--path is used to specify the application path, see request-URL.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 shopping

vf shopping

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit shopping

Setup web server

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/api/v1/shopping" is the application path (see request-URL) and "shopping" is your application name):

ProxyPass "/api/v1/shopping" unix:///var/lib/vv/shopping/sock/sock|fcgi://localhost/api/v1/shopping

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/api/v1/shopping" (such as for example "/api/v1/shopping.html" or "/api/v1/shopping_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/api/v1/shopping", see request-URL.

Access application server from the browser

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.
Here is REST API for the application.

Substitute the loopback "127.0.0.1" with the IP or web address of your server.

To add a customer, use the POST URL to create new "customer" resource:

curl -X POST  "http://127.0.0.1/api/v1/shopping/customers/first-name/<first name>/last-name/<last name>"

for example:

curl -X POST  "http://127.0.0.1/api/v1/shopping/customers/first-name/Mia/last-name/Beltran"

The return value is a JSON document containing a single value, and that is <customer ID> of a newly created customer.

To add an item to the list of inventory items, use the POST URL to create new "item" resource:

curl -X POST  "http://127.0.0.1/api/v1/shopping/items/name/<item name>?description=<item description>"

for example:

curl -X POST  "http://127.0.0.1/api/v1/shopping/items/name/rice-pudding?description=Delicious"

The return value is a JSON document containing a single value, and that is <item ID> of a newly created item.

To create an order, use the following POST URL to create a new "order" resource, and use <customer ID> previously obtained:

curl -X POST "http://127.0.0.1/api/v1/shopping/orders/customer-id/<customer ID>"

for example:

curl -X POST "http://127.0.0.1/api/v1/shopping/orders/customer-id/1"

The return value is a JSON document containing a single value, and that is <order ID> of a newly created order.

To add an item to an order, use the following POST URL to create a new "order/item" resource, and use <order ID> and <item ID> previously obtained:

curl -X POST "http://127.0.0.1/api/v1/shopping/orders/order-id/<order ID>/item-id/<item ID>?quantity=<item quantity>"

for example:

curl -X POST "http://127.0.0.1/api/v1/shopping/orders/order-id/1/item-id/1?quantity=2"

The return value is a JSON document containing a single value, and that is the number of line items created ("1" or "0").

To update an order (meaning change the number of items), use the following PUT URL, and use <order ID> and <item ID> previously obtained:

curl -X PUT "http://127.0.0.1/api/v1/shopping/orders/order-id/<order ID>/item-id/<item ID>?quantity=<item quantity>"

for example:

curl -X PUT "http://127.0.0.1/api/v1/shopping/orders/order-id/1/item-id/1?quantity=3"

The return value is a JSON document containing a single value, and that is the number of line items updated ("1" or "0"). Per REST methodology, this is an idemopotent operation, i.e. it can be repeated any number of times with the same result.

To list all orders, use the following GET URL:

curl -X GET "http://127.0.0.1/api/v1/shopping/orders"

The return value is a JSON document with all orders, including customer information, as well as items, their descriptions and quantities.

To list a specific order, use the following GET URL:

curl -X GET "http://127.0.0.1/api/v1/shopping/orders/order-id/<order ID>"

for example:

curl -X GET "http://127.0.0.1/api/v1/shopping/orders/order-id/1"

The return value is a JSON document with a specific order, including customer information, as well as items, their descriptions and quantities.

To delete an item from order, use the following DELETE URL, and use <order ID> and <item ID> previously obtained:

curl -X DELETE "http://127.0.0.1/api/v1/shopping/orders/order-id/<order ID>/item-id/<item ID>"

for example:

curl -X DELETE "http://127.0.0.1/api/v1/shopping/orders/order-id/1/item-id/2"

The return value is a JSON document containing a single value, and that is the number of line items delete ("1" or "0").

To delete a specific order, use the following DELETE URL:

curl -X DELETE "http://127.0.0.1/api/v1/shopping/orders/order-id/<order id>"

for example:

curl -X DELETE "http://127.0.0.1/api/v1/shopping/orders/order-id/1"

The return value is a JSON document with a number of orders and number of items deleted.

Source files

The following are the source files in this application:
- These SQL statements will create tables for this application: customers, items, orders and orderItems: (setup.sql)

create table if not exists customers (firstName varchar(30), lastName varchar(30), customerID bigserial primary key);
create table if not exists items (name varchar(30), description varchar(200), itemID bigserial primary key);
create table if not exists orders (customerID bigint, orderID bigserial primary key);
create table if not exists orderItems (orderID bigint, itemID bigint, quantity bigint);

- Customers resource (customers.vely)
This implements REST API for a customer.

#include "vely.h"

request-handler /customers
    get-req method to define req_method

    if (req_method == VV_POST) add_customer ();
end-request-handler

- Orders resource (orders.vely)
This implements REST API for an order.

#include "vely.h"

request-handler /orders
    out-header use content-type "application/json"

    get-req method to define req_method

    if (req_method == VV_POST) {
        input-param item_id
        if (item_id[0] == 0) create_order (); else add_to_order();
    }
    else if (req_method == VV_PUT) update_order ();
    else if (req_method == VV_GET) list_orders ();
    else if (req_method == VV_DELETE) {
        input-param item_id
        if (item_id[0] == 0) delete_order (); else {
            set-input "quantity" = "0"
            update_order ();
        }
    }
end-request-handler

- Items resource (items.vely)
This implements REST API for an item.

#include "vely.h"

request-handler /items
    out-header use content-type "application/json"

    get-req method to define req_method

    if (req_method == VV_POST) add_item ();
end-request-handler

- Add a customer (add_customer.vely)
This will add a new customer. Removing a customer is not included, and it should remove all its orders.

#include "vely.h"

request-handler /add-customer

    out-header use content-type "application/json"

    input-param first_name
    input-param last_name
    // Add a customer SQL
    run-query @db_shopping ="insert into customers (firstName, lastName) \
            values ('%s', '%s') returning customerID" output define customerID : \
            first_name, last_name
        @"<<p-out customerID>>"
    end-query
end-request-handler

- Add an item to inventory (add_item.vely)
Here an item is added to invetory available for sale.

#include "vely.h"

request-handler /add-item
    out-header use content-type "application/json"

    input-param name
    input-param description
    // Add an item to inventory SQL
    run-query @db_shopping ="insert into items (name, description) \
        values ('%s', '%s') returning itemID" output itemID : name, description
        query-result itemID to define item_id
        @"<<p-out item_id>>"
    end-query
end-request-handler

- Create an order (create_order.vely)
This creates a new order, ready to have items added to it.

#include "vely.h"

request-handler /create-order
    out-header use content-type "application/json"

    input-param customer_id
    // SQL to create an order
    run-query @db_shopping ="insert into orders (customerId) \
        values ('%s') returning orderID" output orderID : customer_id
        query-result orderID to define order_id
        @"<<p-out order_id>>"
    end-query
end-request-handler

- Add item to order (add_to_order.vely)
Add an item to existing order.

#include "vely.h"

request-handler /add-to-order
    out-header use content-type "application/json"

    input-param order_id
    input-param item_id
    input-param quantity
    // SQL to add an item to an order
    run-query @db_shopping ="insert into orderItems (orderId, itemID, quantity) values  ('%s', '%s', '%s')" \
        : order_id, item_id, quantity no-loop affected-rows define arows
    @"<<p-num arows>>"
end-request-handler

- Update item quantity in an order (update_order.vely)
If the quantity update is 0, the item is deleted from order, otherwise the quantity is updated.

#include "vely.h"

request-handler /update-order
    out-header use content-type "application/json"

    input-param order_id
    input-param item_id
    input-param quantity
    num arows;
    // If quantity update is 0, issue SQL to delete an item from order, otherwise update 
    if (!strcmp (quantity, "0")) {
        run-query @db_shopping ="delete from orderItems where orderID='%s' and itemID='%s'" \
            : order_id, item_id no-loop affected-rows arows
    } else {
        run-query @db_shopping ="update orderItems set quantity='%s' where orderID='%s' and itemID='%s'" \
            : quantity, order_id, item_id no-loop affected-rows arows
    }
    @"<<p-num arows>>"
end-request-handler

- List orders as JSON (list_orders.vely)
All orders are listed along with customer ID and name, and under each order are the items with their names, descriptions and quantity of items ordered.

#include "vely.h"
#include "shopping.h"

request-handler /list-orders
    out-header use content-type "application/json"

    input-param order_id

    num curr_order = 0;

    // Start JSON output
    @{ "orders": [

    if (order_id[0] != 0) {
        // Query just a specific order
        run-query @db_shopping = "select o.orderID, c.customerID, c.firstName, c.lastName \
                from orders o, customers c \
                where o.customerID=c.customerID and o.orderId='%s'" \
                output define customer_id, first_name, last_name \
                row-count define order_count : order_id
            _json_from_order (order_id, curr_order, order_count, customer_id,
                first_name, last_name);
        end-query
    } else {
        // Query to get all orders
        run-query @db_shopping ="select o.orderID, c.customerID, c.firstName, c.lastName \
                from orders o, customers c \
                where o.customerID=c.customerID order by o.orderId" \
                output define order_id, customer_id, first_name, last_name \
                row-count define order_count
            _json_from_order (order_id, curr_order, order_count, customer_id,
                first_name, last_name);
        end-query
    }

    // Finish JSON output
    @   ]
    @}
end-request-handler

- Represent order as JSON (_json_from_query.vely)
This will take order information from the list of orders (as directed by GET REST API for either all orders or a specific one), and find all items within an order. The JSON text is output.

#include "vely.h"
#include "shopping.h"

void _json_from_order (char *order_id, num curr_order, num order_count,
    char *customer_id, char *first_name, char *last_name)
{
    @   {
    @       "orderID": "<<p-out order_id>>",
    @       "customer":
    @       {
    @           "customerID": "<<p-out customer_id>>",
    @           "firstName": "<<p-out first_name>>",
    @           "lastName": "<<p-out last_name>>"
    @       },
    @       "items": [
    num curr_item = 0;
    // Query to get all items in an order
    run-query @db_shopping ="select i.itemID, t.name, t.description, i.quantity \
            from orderItems i, items t where i.orderID='%s' \
                and t.itemID=i.itemID" \
            output itemID, itemName, itemDescription, itemQuantity : order_id \
            row-count define item_count
        @       {
        @           "itemID": "<<query-result itemID>>",
        @           "itemName": "<<query-result itemName>>",
        @           "itemDescription": "<<query-result itemDescription>>",
        @           "itemQuantity": "<<query-result itemQuantity>>"
        // add a comma if there are more items after this
        @       }<<p-out ++curr_item < item_count ? ",":"">>
    end-query
    @   ]
    // add a comma if there are more orders after this
    @   }<<p-out ++curr_order < order_count ? ",":"">>
}

- Include file (shopping.h)
This file has a function declaration for _json_from_query() so it can be used in multiple .vely files.

#include "vely.h"

void _json_from_order (char *order_id, num curr_order, num order_count,
    char *customer_id, char *first_name, char *last_name);

See also

Examples

Click on each example for instructions.

Stock ticker application:
- Stock ticker example will create a new ticker (stock name and its price) or update an existing one, and display the stock ticker database.
- What's in it: MariaDB; command line; web browser; Nginx; Unix sockets; 3 source files, 53 lines of code.
A hands-on guide on writing applications :
- Basic design of requests and request-handlers, testing in both command-line and web environment.
- What's in it: request design, application design, command-line, web.
Web file manager in 160 lines of code:
- Uploading and downloading files is one of the most common tasks in web applications. This article shows how to build a file manager application in about 160 lines of code.
- What's in it: PostgreSQL; web browser; Apache; TCP sockets; 6 source files, 165 lines of code.
Multitenant SaaS with MariaDB and Apache:
- Cloud applications typically run as Software-as-a-Service (SaaS). This article will demonstrate typical functionality you need to have for a minimal functioning SaaS, and how to achieve it. The example shown here is a complete application you can run on virtually any Linux computer.
- What's in it: MariaDB; web browser; Apache; Application path; Unix sockets; 7 source files, 315 lines of code.
Server communication, distributed computing :
- Application servers running on different computers and communicate back and forth in just a few lines of code.
- What's in it: application servers, distributed computing, call-server statement.
Shopping REST API:
- This is a shopping web service REST API with basic functions: add customer, add item, create order, add item to order, update order, delete item from order, delete order. The APIs return a valid JSON reply, even if it's just a single string (such as created ID for a customer, item or order). Listing an order returns a JSON document showing the order details. The example demostrates usage of REST methods (POST, PUT, GET, DELETE) as well as construction and use of REST URLs, and the code that implements the API.
- What's in it: PostgreSQL; web browser; Apache; REST API; Application path; Unix sockets; 12 source files, 203 lines of code.
PostgreSQL: transactions and files :
- You'll learn about ACID (Atomicity, Consistency, Isolation, Durability) with database, and how write database transactions along with regular Operating System files.
- What's in it: PostgreSQL, command line, writing files.
Uploading files :
- This example shows how to upload files to your web applications. It is one of the most common tasks in web development.
- What's in it: Nginx, application server.
MariaDB/mySQL databases :
- Learn how to use MariaDB and mySQL databases in your projects, both web applications and command-line programs.
- What's in it: MariaDB/mySQL, Apache, command line, application server.
SQLite example: temperature utility:
- In this example, a temperature measuring device will periodically insert temperatures into a database table, along with a timestamp. The purpose is to read this history and output the result from time to time, which can then be piped or sent elsewhere.
- What's in it: SQLite; command line; Unix sockets; 2 source files, 25 lines of code.
Encryption and decryption :
- Here you will learn how to encrypt and decrypt data using a password, also known as symmetrical encryption. This password must be known to both parties exchanging information.
- What's in it: encryption, decryption, OpenSSL.
How to use Regex :
- Use regex for high-performance search and replace in your applications.
- What's in it: match-regex statement, search, replace.
JSON parsing and searching:
- This example demonstrates parsing JSON text. The text parsed contains information about cities. JSON document includes an array of countries, which includes an array of states, each having an array of cities. Use of UTF8 Unicode data is also shown - some cities have such characters in their names. JSON text comes from two different kinds of client requests: from an HTML form and also from Javascript/fetch(). This example shows how to deal with a generic JSON document where structure is known but can change, as well as if you don't know its structure and search for what interests you along the way.
- What's in it: web browser; Nginx; Unix sockets; 5 source files, 139 lines of code.
Cookies in a web application:
- A value is entered in the browser and saved as a cookie, then read back later. This example displays a web form. When it is submitted, the input is used to set a cookie in response. Then in a separate page, the cookie value is obtained and displayed. This is the basic mechanism often used in saving web application states and session information.
- What's in it: web browser; Apache; Unix sockets; 1 source files, 48 lines of code.
HTML form for guest database:
- This application is a guest tracking database. The user can input first and last name, which are added to a database table. A web page shows the list of names queried from the database.
- What's in it: PostgreSQL; web browser; Apache; Unix sockets; 4 source files, 83 lines of code.
Sending mail:
- Sending email through web interface demonstrates web input of From, To, Subject and Message fields for an email, and the submittal of the form that sends an email, with confirmation displayed.
- What's in it: web browser; Apache; Unix sockets; 1 source files, 78 lines of code.
Client API, connect to Vely, PHP FPM :
- Connect to FastCGI servers such as Vely or PHP FPM.
- What's in it: FastCGI, client, server, API.
Using DDL and DML with database:
- Example of manipulating tables via SQL. Table is dropped, created, data inserted, then queried, and finally it is dropped.
- What's in it: PostgreSQL; command line; web browser; Nginx; Unix sockets; 2 source files, 43 lines of code.
Write C programs :
- Shows how easy it is to mix Vely and C code, along with a fully functional example that makes command line program, application server and a web application.
- What's in it: C code, command line, application server, Nginx example.
Hashed key/value server:
- You will create your own hash server in this example. A REST API will enable end-user to add key/data pairs, query and delete them. The data is in memory-only; a more involved example could be constructed to persist the data in some form (such as with SQLite). This is an extremely fast, single-process hash server.
- What's in it: web browser; Apache; REST API; Unix sockets; 2 source files, 47 lines of code.
Hello-world:
- This is a simple Hello World example. It explains basics of making applications as well as tracing and debugging them.
- What's in it: command line; web browser; Nginx; Unix sockets; 2 source files, 6 lines of code.
AVL/B Tree for in-memory key/value server with range searches :
- See how to build in-memory database with super fast tree data structure.
- What's in it: tree statements, cache server, application server, Apache example.
Report from SQL database, file writing:
- Builds a report of employees from a database. Creates a "reports" directory and a file name with a time stamp, then writes the report to a file and also displays the report to a web page.
- What's in it: MariaDB; command line; web browser; Nginx; Unix sockets; 2 source files, 68 lines of code.
Simple Hello World :
- This is a very simple Hello World, designed to get you going in minutes. Demonstrates running Hello World as a standalone program, or an application server. All it does is outputs Hello World.
- What's in it: Hello World, command line, application server.
Docker container for Vely application :
- Here you will build a Vely docker container with a simple stock application. The web server and the database are outside the container. This way any Vely application can easily be containerized and moved/managed anywhere.
- What's in it: Docker, MariaDB, web browser, Apache.

See also

Stock ticker application

Stock ticker example will create a new ticker (stock name and its price) or update an existing one, and display the stock ticker database.

In a nutshell: MariaDB; command line; web browser; Nginx; Unix sockets; 3 source files, 53 lines of code.

Screenshots of application

Stock value is updated:

Showing the stock:

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because they are used in this example, you will need to install Nginx as a web server and MariaDB as a database.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

tar xvf $(vv -o)/examples/stock.tar.gz
cd stock

Setup application

The very first step is to create an application. The application will be named "stock", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) stock

This will create a new application home (which is "/var/lib/vv/stock") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create MariaDB database "db_stock" owned by user "vely" with password "your_password". You can change any of these names, but remember to change them everywhere here. And then, you will create database objects in the database.

Execute the following logged in as root in mysql utility:

--Create velydb database hosting application data (if it doesn't exist):
create database if not exists db_stock;
create user if not exists vely identified by 'your_password';
grant create,alter,drop,select,insert,delete,update on db_stock.* to vely;
-- Create database objects needed for the application (eg. tables, indexes):
use db_stock;
source setup.sql;
exit

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_stock". This name doesn't have to be "db_stock", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo '[client]
user=vely
password=your_password
database=db_stock
protocol=TCP
host=127.0.0.1
port=3306' > db_stock

The above is a standard mariadb client options file. Vely uses native MariaDB database connectivity, so you can specify any options that a given database lets you.

Build application

Use vv utility to make the application:

vv -q --db=mariadb:db_stock

Note usage of --db option to specify MariaDB database and the database configuration file name.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 stock

vf stock

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit stock

Setup web server

This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/stock" is the application path (see request-URL) and "stock" is your application name):

location /stock { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/stock/sock/sock; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

Note: you must not have any other URL resource that starts with "/stock" (such as for example "/stock.html" or "/stock_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/stock", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Add stock ticker 'XYZ' with stock price 450 
http://127.0.0.1/stock/add-stock?name=XYZ&price=450

# Display list of stock tickers 
http://127.0.0.1/stock/show-stock

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Access application server from command line

To access your application server from command line (instead through web browser/web server), use this to see the application response:

#Add stock ticker 'XYZ' with stock price 450 
export CONTENT_TYPE=
export CONTENT_LENGTH=
export REQUEST_METHOD=GET
export SCRIPT_NAME='/stock'
export PATH_INFO='/add-stock'
export QUERY_STRING='name=XYZ&price=450'
cgi-fcgi -connect /var/lib/vv/stock/sock/sock /

#Display list of stock tickers 
export CONTENT_TYPE=
export CONTENT_LENGTH=
export REQUEST_METHOD=GET
export SCRIPT_NAME='/stock'
export PATH_INFO='/show-stock'
export QUERY_STRING=''
cgi-fcgi -connect /var/lib/vv/stock/sock/sock /

Note: to suppress output of HTTP headers, add this before running the above:

export VV_SILENT_HEADER=yes

If you need to, you can also run your application as a CGI program.

Run program from command line

Execute the following to run your application from command line (as a command-line utility):

#Add stock ticker 'XYZ' with stock price 450 
vv -r --app='/stock' --req='/add-stock?name=XYZ&price=450' --method=GET --exec

#Display list of stock tickers 
vv -r --app='/stock' --req='/show-stock?' --method=GET --exec

Source files

The following are the source files in this application:
- Database objects (setup.sql)
Table "stock" with stock name and price is created for this example:

create table if not exists stock (stock_name varchar(100) primary key, stock_price bigint);

- Add stock ticker (add_stock.vely)
This requests (/add-stock) saves the names of stock tickers and their prices. It will obtain the stock name and price from the web client via input-param statement, and then use INSERT SQL to store this data in the database. Note the use of "%%", the short version of request-handler and end-request-handler statements.


#include "vely.h"

%% /add-stock
    out-header default
    @<html>
        @<body>
        input-param name
        input-param price
        // Add data to stock table, update if the stock exists
        run-query @db_stock = "insert into stock (stock_name, stock_price) values ('%s', '%s') on duplicate key update stock_price='%s'" : name, price, price error define err no-loop
        if (strcmp (err, "0")) {
            report-error "Cannot update stock price, error [%s]", err
        }
        @<div>
            @Stock price updated!
        @</div>
        @</body>
    @</html>
%%

- Show stock tickers (show_stock.vely)
You can view stock tickers in a list. SELECT SQL is used to get all the stocks saved so far, and display them in a table using run-query and then query-result to get the query results.


#include "vely.h"

%% /show-stock
    out-header default
    @<html>
        @<body>
        // Show stock names and values
            @<table>
                @<tr>
                    @<td>Stock name</td>
                    @<td>Stock price</td>
                @</tr>
                run-query @db_stock = "select stock_name, stock_price from stock" output stock_name, stock_price
                    @<tr>
                        @<td>
                        query-result  stock_name
                        @</td>
                        @<td>
                        query-result  stock_price
                        @</td>
                    @</tr>
                end-query
            @</table>
        @</body>
    @</html>
%%

See also

How to upload files

Uploading files on the web

One of the most common tasks a developer faces when building web applications is uploading files from a client (typically a web browser). Whether it's a small or a large application, chances are you will need to upload images, PDF documents and other kinds of files. These files are then typically stored on the server and indexed in the database, so they can be served up on demand.

Note that the kind of upload I am talking about here is through HTML forms, or using the same method as HTML forms, and is the most common kind. When people talk about uploading files, that's what they are usually referring to.

The mechanics of uploading a file

File upload is usually sent to a server as an HTTP POST request. The body of request will then contain metadata about the files being uploaded from the client, as well as the contents of files themselves. The content sent may look something like this - the first part before the empty line is an HTTP header, followed by the actual upload data:

POST /file_up HTTP/1.1
Host: 127.0.0.1
Content-Type: multipart/form-data; boundary=---------------------------42828225182062843700116470094
Content-Length: 244463

-----------------------------42828225182062843700116470094
Content-Disposition: form-data; name="req"

upload_file
-----------------------------42828225182062843700116470094
Content-Disposition: form-data; name="myfile"; filename="pic7.jpg"
Content-Type: image/jpeg

\377\330...

What's in a message like this?

First of all, there is a uniquely-generated delimiter (in this case "---------------------------37487108932486351905827665017", but in general it would be some other string) that is used to separate the parts of the message. It is generated in such a manner that it does not exist in any part of the message, hence it can be used as a delimiter between the data pieces.

Those pieces tell the server all it needs to know. One kind of information conveyed this way is telling the server that the data comes from a web form ("Content-Disposition: form-data;").

Next, there are name/value pairs that come from the HTML form. For instance when you fill out a form with your name and address, they will be sent as values in such pairs. In this case, there is a "req=upload_file" pair.

Finally, there are the file(s), one or more of them. Here, a file is given with a URL field "myfile", with the actual file name of "pic7.jpg". The file content type is JPG (specified as "image/jpeg"), then followed by the binary content of the file (in this case beginning with "\377\330...").

In the way described, any number of name/value pairs can be specified, as well as any number of files uploaded.

The server must parse this correctly and extract the data in order to get all the name/value pairs and all the file names and their contents. Then these files can be saved, passed on somewhere else, or processed in some way.

This is how web upload works in most cases, and web clients/servers follow this process - that's because it's an Internet standard described in RFC7578.

Virtually all languages and frameworks support file uploads, with varying degrees of complexity. The following is an example of uploading files to the server using Vely framework.

Prerequisites

Nginx is used as a web server, so install it before proceeding. If not already setup, you may need to configure a firewall for it to allow HTTP traffic (see ufw, firewall-cmd etc.).

This example uses Vely application development framework. Install it first.

You will need sudo privilege to install software and to create a web page used for testing.

Upload example

To run the example here, create an application "file_up" in a directory of its own (see vf for more on Vely's program manager):

mkdir upload_example
cd upload_example
sudo vf -i -u $(whoami) file_up

Create file "upload_file.vely" and copy this to it:

#include "vely.h"

request-handler /upload_file
    input-param myfile_filename
    input-param myfile_location
    input-param myfile_ext
    input-param myfile_size

    out-header default
    @File <<p-out myfile_filename>> uploaded to <<p-out myfile_location>> (extension <<p-out myfile_ext>>, size <<p-out myfile_size>>).
end-request-handler

To run this code as an application server, make it first:

vv -q

To start your application server that can upload files:

vf file_up

How does this work

Vely has a built-in functionality for file uploads, and the details described previously are automatically done. When you upload a file, it is treated as any other input parameter. So the code you have on server-side:

input-param myfile_filename
input-param myfile_location
input-param myfile_ext
input-param myfile_size

assumes that your uploaded file will be a URL field named "myfile", and then you can obtain it's actual file name (which is "myfile_filename"), its location where it's uploaded on the server ("myfile_location"), its extension ("myfile_ext") and size in bytes ("myfile_size"). Basically the URL field name is appended with an underscore and the kind of information you want to get.

The last part:

@File <<p-out myfile_filename>> uploaded to <<p-out myfile_location>> (extension <<p-out myfile_ext>>, size <<p-out myfile_size>>).

is an output-statement that begins with "@" and it outputs text to the client. The p-out statement will print a string, and in this case it is inlined by using "<<" and ">>". It's pretty simple (and readable) to output data this way.

Setup web server

The next step is to set up a web server. You have just started an application server, and by definition, it sits behind the web server. This is good for many reasons: performance, safety, separation of layers. You can use any web server that supports FastCGI protocol (the same protocol used by PHP-FPM); in this example I will use Nginx which inherently supports FastCGI. That's nice as there's nothing to configure.

As I mentioned, to start you will need to have Nginx installed. Once done, you will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section - note the "fastcgi_pass" parameter that specifies path to your application's Unix socket:

location /file_up { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/file_up/sock/sock; }
client_max_body_size 20M;

Finally, restart Nginx:

sudo systemctl restart nginx

This will connect the Nginx web server to your application server via a Unix socket, which is a very fast connection indeed, so that requests made via web server are answered by your code above. The "client_max_body_size" directive helps with avoiding "Request Entity Too Large" error, i.e. Nginx by default allows file uploads only up to 2MB in size, which isn't much these days - this way we up the limit to 20MB.

You are now done! Your application is ready to use.

Test by uploading a file

To test, create an HTML file (i.e. a web page) that you can open through your web server. This file should be under the Nginx's root document directory. To see where is this root directory on your system check out the configuration file above and look for "root" directive under the "server{}" section, for instance it might look like:

root   /usr/share/nginx/html/;

in which case the directory is "/usr/share/nginx/html/" - note however it may vary depending on the distribution and release. Create a file "test_upload.html" there, and copy the following into it:

<form action="/file_up" method="POST" enctype="multipart/form-data">
    <input type="hidden" name="req" value="upload_file">

    <label for="myfile">File:</label>
    <input type="file" name="myfile" value=""><br><br>

    <input type="submit" value="Submit">
</form>

The above is a simple HTML form that will call your Vely code. How does it work?

Notice the "action" attribute in a "<form>" element: it is "/file_up", which is the application path of your application (by default it is the name of application preceded by a forward slash), and it is the "location" in the Nginx directive you added.

The request method used is "POST", and the name of the HTML field to upload the file is "myfile" - this is in "<input>" element of type "file" that lets you choose a local file to upload.

Finally, there's a Submit button given as an "<input>" element of type "submit".

This is a very simple setup that you can build upon for your web applications.

To test, go to the browser and (assuming you are doing this on your local computer, otherwise change to your web address):

http://127.0.0.1/test_upload.html

You can now select a file (in this example it's "pic7.jpg"), upload it and you will see the output:

File pic7.jpg uploaded to /var/lib/vv/file_up/app/file/18648/2107645QBsCL (extension .jpg, size 244124).

This means your file has been uploaded to the server and is saved at location "/var/lib/vv/file_up/app/file/18648/2107645QBsCL". This file is in Vely file-storage, and its name is generated to be unique among all uploaded files, even if they are uploaded at the same time. The way files are organized is to make them available for quick retrieval and so you don't have to worry about naming them - after all the name of a file is just a reference you're likely to store somewhere in the database. You can leave the file there (as Vely file storage is meant to keep files for quick access), or you can move it somewhere else by using rename-file statement.

Conclusion

And there you have it. You can now upload files to the server. Of course, the application here is a rudimentary one, but it shows the basics. You can upload multiple files, add other text fields, save the file location to the database and process the files according to your application's needs.

See also

Using MariaDB and mySQL in web applications

MariaDB and mySQL

MariaDB and mySQL are popular Open Source databases that share good amount of functionality and syntax, given their common roots. In recent years, the divergence between them has been growing, though for the most part they are still interchangeable. This article will cover MariaDB, as it is included in virtually all Linux distributions. However, it should also be applicable to mySQL as well.

Connecting to database

There are a number of ways to connect to MariaDB/mySQL, but the fastest connection method is via native C library. There are a few aspects to consider when choosing how will you access MariaDB/mySQL (and not just those):

Are you using only native libraries to connect, as they are faster than those with layers of abstractions on top of it that slow down the access?
Is the database connection persistent? Meaning does the client connection stay open between queries, or does it connect and disconnect each time it needs to access the database?
Are the prepared statements supported? Prepared statements speed up SQL access in many cases because the server does not parse a SQL statement every single time.

When you choose your database access framework, try to find out if you will get at least some positive answers to these questions. Here I will use Vely, for which the answer is "yes" on all three counts above.

Prerequisites

First, install MariaDB. I will also use Apache as a web server to access the database from web browser, so install Apache too - if not already setup, you may need to configure a firewall for it to allow HTTP traffic (see ufw, firewall-cmd etc.). And finally, install Vely, which will be an application server that sits between the web server and the database.

You can also install mySQL and all the steps should be the same.

Create a new directory for this example:

mkdir mariadb
cd mariadb

Setup the database

create database if not exists db_people;
create user if not exists vely identified by 'your_password';
grant create,alter,drop,select,insert,delete,update on db_people.* to vely;
use db_people;
create table if not exists people_list (first_name varchar(30), last_name varchar(40));
exit

Here, you will create database "db_people" and user "vely". You can change their names however you like, but remember to change them everywhere here. Then you'll give user "vely" the permissions to basically own database "db_people" and be able to create objects, data etc.

Finally, table "people_list" is created in this database - it's very simple with just a first and last name. That's it for the database setup.

Access the database

In order to access your database (any database really), you will need a database-config-file for it. This file is simply specifying things like the method of accessing the database, what is the user name and password and so on. To that effect, create a database configuration file "people". You can call this file anything, but keep in mind its name is used in the code to reference the database, so if you change it, then also change it in the code below. Create file "people":

[client]
user=vely
password=your_password
database=db_people
protocol=TCP
host=127.0.0.1
port=3306

This is actually a native MariaDB client configuration file, so learning the format of it may help you elsewhere as well. The "[client]" section signifies this is information that a client needs to connect. Next, you'd specify MariaDB user name (which is "vely" user we created above), then the password ("your_password", of course you can have your own), and the database name is "db_people" - again that's the database we created already.

The rest is pretty much the default method of contacting the database - MariaDB out of the box will listen on TCP port 3306 on localhost. If you changed any of that, then you have to change it here too. Now your code can assess the database.

The code

Create file "list_people.vely" and copy this to it:

#include "vely.h"

%% /list_people
    out-header default

    @List of people:<hr/>

    run-query @people = "select first_name, last_name from people_list" output define f_name, l_name
        @First name <<p-out f_name>>, last name <<p-out l_name>><br/>
    end-query
%%

This will run a query that lists everyone's first and last names. The query will use configuration file "people" that you created previously (note "@people"). And the output columns will go to string variables (i.e. "char *") named "f_name" and "l_name" - these variables are created on the spot with "define" clause of run-query statement. You can also use prepared SQL statements by using "run-prepared-query" statement instead of "run-query" in the code above.

And the "@" output-statement sends the data to standard output, which can be the actual "stdout" stream if this is going to be a command-line program, or to the browser if this is a web application. The nice thing is, it works the same for both. p-out statement outputs a string, and when placed in between << and >> it is "inlined" into an output statement.

Note the use of "%%" as a shortcut for request-handler.

Create and Make the application

When you get started on a Vely application, you have to create it first with the vf program manager:

sudo vf -i -u $(whoami) people_app

"-i" option says to create an application. "-u" option says which user will own it, in this case it's "$(whoami)" which is Linux-speak for the "currently logged in user". And finally the application name is "people_app".

To make your application, use vv tool:

vv -q --db='mariadb:people'

This gathers all the .vely files in the current directory (in this case just one), processes all the Vely statements (like run-query) into C code, and then compiles and links it all together into a native application. Note "--db='mariadb:people'" option - it states your program uses database named "people" and the database vendor is MariaDB or mySQL. You can have any number of databases and any number of supported vendors.

Two executables will be produced, both in the "/var/lib/vv/bld/people_app" directory. Note the "people_app" subdirectory - it matches your application name created above. This directory is like a scratch-pad for your application, this is where all the generated code goes. One executable created will be "people_app" that you can run from the command line. The other one is "people_app.fcgi" that you can run as a FastCGI application server, which is the web application. You'll use both in this article.

Put some data in

To get started, you'll need some data to query. To that effect, insert some data first. Log in to the database with the user credentials created previously (change "your_password" if you changed your password):

mysql -u vely -pyour_password

and then execute this SQL:

use db_people;
insert into people_list (first_name, last_name) values ("Timothy", "McMillan"), ("Tina", "Clark");
commit;
exit;

Now you have two people in your database, Timothy and Tina.

Run from command line

Execute your program:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=yes
export REQUEST_METHOD=GET
export SCRIPT_NAME="/people_app"
export PATH_INFO="/list_people"
export QUERY_STRING=""
/var/lib/vv/bld/people_app/people_app

You'll get:

List of people:<hr/>
First name Timothy, last name McMillan<br/>
First name Tina, last name Clark<br/>

Vely is all about standard HTTP requests. So even when you run a command line program, it does so by receiving an HTTP request. That's why there's a request method ("GET"), a script name (which is a path to application name "people_app") and a path info (which is a path to request handler "list_people", i.e. your code above). This makes is very easy to build programs for both command-line and web execution, because they are the same. You don't need to write two code bases and you only debug once. Plus, you can do on command line virtually anything you can on the web, so you can write your program without ever even having a web server setup.

Note the VV_SILENT_HEADER environment variable - it suppresses HTTP header output. If it weren't there, you'd get the HTTP header, the same one that a browser will get. We'll do that next.

Setup Apache for web access

In this article, you will use Apache web server. Apache has FastCGI support, as most (if not all) major web servers do. Apache is very modular and so this support (like many of its features) has to be enabled. To do that, follow the instructions to setup Apache for Vely access. Note that in Step 2, the <app path> and <app name> are both "people_app" (i.e. the same as application name), so the ProxyPass will be like this:

ProxyPass "/people_app" unix:///var/lib/vv/people_app/sock/sock|fcgi://localhost/people_app

The "ProxyPass" directive tells Apache that any URL path that starts with "/people_app" will be served by your "people_app" application. How does it do that? Note the path "/var/lib/vv/people_app/sock/sock" above - this is a Unix socket file created by Vely to allow other software (like Apache) to communicate with your application. This is a super fast mode of communication, and is typically used by applications running on the same host, as it's directly supported by Linux kernel.

Start your application server

To run from browser, start your application server first:

vf -w 2 people_app

This will start 2 daemon processes to serve requests. These requests come from browser (or another server), pass through Apache web server to your Vely application, and then back. You have many options with vf program manager to run your server efficiently.

Run from browser

Copy this URL to your browser (assuming your server is on your own local computer; if not, replace "127.0.0.1" with your web address):

http://127.0.0.1/people_app/list_people

You should get the exact same response as the above, when you ran it from command line, as in:

Conclusion

Connecting to MariaDB/mySQL isn't difficult. This article explains how to do so in a simple way that you can custom tailor to your own needs.

See also

Using trees for faster in-memory data access

A tree is a data structure that organizes key/value pairs into branches based on some ordering criteria. Each node then branches further down into lower nodes, and the entire structure resembles a tree, hence the name. There are many different kinds of trees, such as AVL, B, B++, Red-Black tree etc. They generally differ in a way the tree is maintained and the way data in it is connected, but most of those widely used today generally provide O(log(N)) access. That means in a tree holding N nodes, you need to access about log(N) of them to find any given key. So for example, if you have 1,000,000 keys/value pairs, it will take about 20 key comparisons to find any of them (since 2^20 is approximately 1M).

As an example of a tree structure, consider this:

In this tree, keys "3", "4", "5", "6", "7", "8" and "9" are stored. An arrow to the left points to a lesser key, while an arrow to the right points to a greater one. You can see how it takes at most 3 key comparisons to find any of the keys, since the tree is 3-level deep. Also, this is a binary tree, as each node has 2 children nodes branching from it (left and right). A tree can have multiple keys per node (typically B-variations), and those are generally well suited for storing data on disk, where access to any data is done in large blocks; such trees work well in-memory as well. However, binary trees are also well suited for in-memory access, if not even more so. Vely uses a modified AVL binary tree that also provides some benefits of various "B" trees, specifically fast range searches; additionally such benefits are optional so you have more options in order to make a tree structure fit your exact needs.

A tree above is a balanced one, which means that the heights of each node's children do not differ by more than one (in this case they are exactly the same). A balanced tree, just like its name suggests, looks "balanced" when shown in its graphical form like it is here. In order to achieve this kind of balance, a tree needs to be maintained with each insert and delete. For instance, if you were to insert keys above in an increasing order, your tree may initially look like this, simply because each insertion goes to the right because each next key is greater:

This binary tree effectively degenerates into a sorted list, as each node is followed by another to the right, due to each being greater than the previous. Searching a tree like this is O(N), meaning you may have to traverse all nodes in worst cases to reach a key/value pair. To balance a tree, connections between the nodes are re-arranged. This is a relatively fast operation, since it doesn't involve copying keys nor values; instead only the direction where a left or right connection goes from a node is changed. For instance:

In this case the tree was rotated to the left around the element with the key of "6". Now, if you right-rotate the left branch around key "4", and if you left-rotate the right branch around key "8", you will get the original balanced tree.

In practicality, these rotations happen when each element is inserted or deleted, and not after you already have a number of key/value pairs in the tree. The above is just to illustrate what could happen if rotations like the above weren't applied, and the effect they have in order to balance the tree. Note that trees can be balanced in different ways; in this case what you're balancing is the height of the tree from any given node.

AVL tree is a binary tree where the difference between a sub-tree on the right and on the left is at most 1. Vely's tree adds (optionally) a linked list that connects all nodes in order from minimum to maxium key; this capability is impotant because it allows very fast range searches; this means starting from a given key and moving to the next lower or greater key until some condition is met. Without this, each next range key would take O(log(N)) access time; with this feature, it's O(1), i.e. the fastest possible access with just a single hop. That's why a Vely tree is "hybrid" AVL/B tree, because its features are found in both varieties; it also makes it high-performance and well-suited for in-memory database or cache.

AVL/B tree cache server application

In this example you will create a cache server that uses fast tree-based data access in order to insert, update and search key/value pairs.

Create new "tree" application first, in a new directory (you can name it anything you like):

mkdir -p tree
cd tree

The vf command is a Vely program manager and it will create a new application (see how-vely-works) named "tree":

sudo vf -i -u $(whoami) tree

To get vim highlighting of Vely syntax:

vv -m

Create a source code file "treesrv.vely":

vi treesrv.vely

and copy and paste this to it:

%% /treesrv
    out-header default

    do-once
    new-tree define t process-scope
    end-do-once

    // Get input parameters
    task-param op
    input-param key
    input-param data

    if-task "add" // Add data to tree
        // Make a copy of key,data so they are Vely-allocated
        copy-string key to define c_key
        copy-string data to define c_data
        write-tree t key c_key value c_data status define st
        if (st == VV_ERR_EXIST) {
            delete-mem c_key
            delete-mem c_data
        }
        @Added [<<p-out key>>]
    else-task "delete" // Delete data and obtain the value deleted
        delete-tree t key (key) value define val old-key define okey status define st
        if (st == VV_ERR_EXIST) {
            @Not found [<<p-out key>>]
        } else {
            // If found, then delete key and value
            @Deleted [<<p-out val>>]
            delete-mem val
            delete-mem okey
        }
    else-task "query" // Query tree based on key value
        read-tree t key (key) value define val status define st
        if (st == VV_ERR_EXIST) {
            @Not found, queried [<<p-out key>>]
        } else {
            @Value [<<p-out val>>]
        }
    end-task
%%

Note that the source file name ("treesrv.vely") should match the request name, which is "treesrv". If you're using hyphens (which is useful for web applications), just substitute with underscore. The fact that a request is implemented in a file with the same name helps keep your applications neat, tidy and easy to peruse.

In do-once statement, a new-tree is created with a process scope; that means all the data in this tree will be kept for as long as the process lives. Normally all memory used by a request in Vely is released at the end of the request. The memory for a new tree here will remain throughout all requests this process will serve. The reason for do-once statement is because you only need to create the tree once for the life of the process.

There are 3 input parameters: "op", "key" and "data". Note that "op" is declared as a task-param (which is a special kind of input-param) because it determines what task (i.e. "op"eration) is being done. The other two, "key" and "data", are used as input parameter values for these tasks.

If the task is "add", a new key/data pair is added to the tree using write-tree statement. Note that copies of "key" and "data" input parameters are created because write-tree expects allocated memory (and the memory from input-param isn't).

If the task is "delete", a tree node with "key" is deleted.

If the task is "query", a value from the node with "key" is returned.

This code is a tree-based cache-server that maintains key/value pairs in memory, and allows adding, deleting and querying.

Make an executable program

Now, make a native executable:

vv -q

Run as application server

"Application server" means a daemon, a resident server process that remains in memory and can serve many requests very fast. Here's how you do that:

vf -w 1 tree

The above will start an application server process to serve incoming requests.

Using server from command line

Testing your server from command line is easy:

vv -r --req="/treesrv/op/add/key/1/data/one" --exec --server --silent-header --app="/tree"

Note the "--server" option. It says to contact a server and execute the same request as before. A process you started is staying resident in memory and serving the incoming requests. This way, your server can serve a large number of concurrent requests because it handles each operation very fast.

Again, you can see what's going on behind scenes by omitting "--exec":

vv -r --req="/treesrv/op/add/key/1/data/one" --server --silent-header --app="/tree"

The result being:

export CONTENT_TYPE=
export CONTENT_LENGTH=
export VV_SILENT_HEADER=yes
export REQUEST_METHOD=GET
export SCRIPT_NAME="/tree"
export PATH_INFO="/treesrv/op/add/key/1/data/one"
export QUERY_STRING=""
cgi-fcgi -connect /var/lib/vv/tree/sock/sock  /treesrv

Test with 100 keys

This is a bash test script to insert 100 keys into your cache server, query them to make sure they are there, delete them, and the query again to verify they're all gone. Create "test_tree" file:

vi test_tree

And copy/paste the following:

#Restart tree server for this test
vf -m restart tree

#Add 100 key/data pairs. Key value is 0,1,2... and data values are "data_0", "data_1", "data_2" etc.
for i in {1..100}; do
   if [ "$(curl -s "http://localhost/tree/treesrv/op/add/key/$i/data/data_$i")" != "Added [$i]" ]; then
       echo "Error adding key [$i]"
       exit -1
   fi
done

echo "Keys added"

#Query all 100 keys and check that values retrieved are the correct ones.
for i in {1..100}; do
   if [ "$(curl -s "http://localhost/tree/treesrv/op/query/key/$i")" != "Value [data_$i]" ]; then
       echo "Error querying key [$i]"
       exit -1
   fi
done

echo "Keys queried"

#Delete all 100 keys
ERR="0"
for i in {1..100}; do
   if [ "$(curl -s "http://localhost/tree/treesrv/op/delete/key/$i")" != "Deleted [data_$i]" ]; then
       echo "Error deleting key [$i]"
       exit -1
   fi
done

echo "Keys deleted"

#Query all 100 keys and check that values retrieved are the correct ones.
for i in {1..100}; do
   if [ "$(curl -s "http://localhost/tree/treesrv/op/query/key/$i")" != "Not found, queried [$i]" ]; then
       echo "Error querying key [$i] after deletion."
       exit -1
   fi
done

echo "Keys queried"

echo "Tree server test successful"

Make sure it's executable:

chmod +x test_tree

and now run it:

./test_tree

The result is this:

Keys added
Keys queried
Keys deleted
Keys queried
Tree server test successful

Run as web application

Setup web server

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/tree" is the application path (see request-URL) and "tree" is your application name):

ProxyPass "/tree" unix:///var/lib/vv/tree/sock/sock|fcgi://localhost/tree

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/tree" (such as for example "/tree.html" or "/tree_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/tree", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Call your application server to add a key 
http://127.0.0.1/tree/treesrv/op/add/key/1/data/one

# Call your application server to query a key 
http://127.0.0.1/tree/treesrv/op/query/key/1

# Call your application server to delete a key 
http://127.0.0.1/tree/treesrv/op/delete/key/1

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.
The result is:

See also

SQLite example: temperature utility

In this example, a temperature measuring device will periodically insert temperatures into a database table, along with a timestamp. The purpose is to read this history and output the result from time to time, which can then be piped or sent elsewhere.

In a nutshell: SQLite; command line; Unix sockets; 2 source files, 25 lines of code.

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because they are used in this example, you will need to install Apache as a web server and SQLite as a database.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

tar xvf $(vv -o)/examples/utility.tar.gz
cd utility

Setup application

The very first step is to create an application. The application will be named "utility", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) utility

This will create a new application home (which is "/var/lib/vv/utility") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, create SQLite database "db_utility". You can change the database name, but remember to change it everywhere here. And then, create database objects in the database.

Execute the following to create database "utility.db" and also setup the database objects needed for the example:

sqlite3 /var/lib/vv/utility/app/utility.db < setup.sql

The SQLite database will be in the application home directory (see how-vely-works):

/var/lib/vv/utility/app

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_utility". This name doesn't have to be "db_utility", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo '/var/lib/vv/utility/app/utility.db' > db_utility

The above in general is a location of SQLite database and is all that's needed to connect to it.

Build application

Use vv utility to make the application:

vv -q --db=sqlite:db_utility

Note usage of --db option to specify SQLite database and the database configuration file name.

Run program from command line

Execute the following to run your application from command line (as a command-line utility):

#Record a temperature to 91F
vv -r --app='/utility' --req='/temphist?action=record&temp=91' --method=GET --exec

#List recorded temperatures
vv -r --app='/utility' --req='/temphist?action=list' --method=GET --exec

Source files

The following are the source files in this application:
- Recording temperature readings in a table (setup.sql)
There is a "temp" column (temperature reading) and "timest" (time stamp of the reading). Table name is "temps" (temperatures).

create table temps (temp integer, timest text primary key);

- Utility for recording temperatures (temphist.vely)
This application will record temperatures, as well as list them. If task parameter "action" is "record", store the temperature into table "temps" and check for any errors. If it is "list", just display all the temperature records in a historical order.

#include "vely.h"

%% /temphist
    out-header default

    task-param action

    if-task "record"
        input-param temp
        run-query @db_utility = "insert into temps (temp, timest) values ('%s', current_timestamp)" : temp \
            affected-rows define rc error-text define er no-loop
        if (rc != 1) {
            @Could not insert temperature reading, error <<p-out er>>.
        } else {
            @Temperature reading stored.
        }
    else-task "list"
        run-query @db_utility = "select temp, timest from temps order by timest" output temp, timest
            @Date: <<query-result  timest>>
            @Temperature: <<query-result  temp>>
        end-query
    end-task
%%

See also

Report from SQL database, file writing

Builds a report of employees from a database. Creates a "reports" directory and a file name with a time stamp, then writes the report to a file and also displays the report to a web page.

In a nutshell: MariaDB; command line; web browser; Nginx; Unix sockets; 2 source files, 68 lines of code.

Screenshots of application

Database report is saved into a file (the path to which is displayed) and the report itself is shown in the web page:

Setup prerequisites

vv -m

Get the source code

tar xvf $(vv -o)/examples/write-report.tar.gz
cd write-report

Setup application

The very first step is to create an application. The application will be named "write-report", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) write-report

This will create a new application home (which is "/var/lib/vv/write-report") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create MariaDB database "db_write_report" owned by user "vely" with password "your_password". You can change any of these names, but remember to change them everywhere here. And then, you will create database objects in the database.

Execute the following logged in as root in mysql utility:

--Create velydb database hosting application data (if it doesn't exist):
create database if not exists db_write_report;
create user if not exists vely identified by 'your_password';
grant create,alter,drop,select,insert,delete,update on db_write_report.* to vely;
-- Create database objects needed for the application (eg. tables, indexes):
use db_write_report;
source setup.sql;
exit

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_write_report". This name doesn't have to be "db_write_report", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo '[client]
user=vely
password=your_password
database=db_write_report
protocol=TCP
host=127.0.0.1
port=3306' > db_write_report

The above is a standard mariadb client options file. Vely uses native MariaDB database connectivity, so you can specify any options that a given database lets you.

Build application

Use vv utility to make the application:

vv -q --db=mariadb:db_write_report

Note usage of --db option to specify MariaDB database and the database configuration file name.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a Unix socket to communicate with the web server (i.e. a reverse-proxy):

vf -w 3 write-report

vf write-report

See vf for more options to help you achieve best performance.

If you want to stop your application server:

vf -m quit write-report

Setup web server

This shows how to connect your application listening on a Unix socket (started with vf) to Nginx web server.

- Step 1:
You will need to edit the Nginx configuration file. For Ubuntu and similar:

sudo vi /etc/nginx/sites-enabled/default

while on Fedora and other systems it might be at:

sudo vi /etc/nginx/nginx.conf

Add the following in the "server {}" section ("/write-report" is the application path (see request-URL) and "write-report" is your application name):

location /write-report { include /etc/nginx/fastcgi_params; fastcgi_pass  unix:///var/lib/vv/write-report/sock/sock; }

- Step 2:
Finally, restart Nginx:

sudo systemctl restart nginx

Note: you must not have any other URL resource that starts with "/write-report" (such as for example "/write-report.html" or "/write-report_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/write-report", see request-URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.

# Run report 
http://127.0.0.1/write-report/write-report

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Access application server from command line

To access your application server from command line (instead through web browser/web server), use this to see the application response:

#Run report 
export CONTENT_TYPE=
export CONTENT_LENGTH=
export REQUEST_METHOD=GET
export SCRIPT_NAME='/write-report'
export PATH_INFO='/write-report'
export QUERY_STRING=''
cgi-fcgi -connect /var/lib/vv/write-report/sock/sock /

Note: to suppress output of HTTP headers, add this before running the above:

export VV_SILENT_HEADER=yes

If you need to, you can also run your application as a CGI program.

Run program from command line

Execute the following to run your application from command line (as a command-line utility):

#Run report 
vv -r --app='/write-report' --req='/write-report?' --method=GET --exec

Source files

The following are the source files in this application:
- Database setup (setup.sql)
For this example, table "employees" is created if it doesn't exist (all its data deleted if it does), and it is populated with some data:

create table if not exists employees (name varchar(50), salary int);
delete from employees;
insert into employees (name, salary) values ('Mike', 50000), ('Joan', 60000), ('Kelly', 70000);

- Database report (write_report.vely)
A result set of a query is stored in a string using write-string with run-query within in. write-string simply redirects any output (that would normally go to the client) into a string. This output is then written to a file using write-file and also sent back to the client.

A few other things are demonstrated here, such as get-req to obtain the database vendor (i.e. which database is used, as any number of databases can be, such as MariaDB/MySQL, PostgreSQL, SQLite). Also getting current time (get-time) is shown to produce a timestamp for the file name, and match-regex to replace spaces with underscores.


#include "vely.h"

%% /write-report

    out-header default

    get-app db-vendor db_write_report to define dbv
    @Database vendor used is <<p-out dbv>><br/>

    // Build report for a query and store it into variable outmsg

    write-string define outmsg
    @Employee report<hr/>

    // Get data from the database
    run-query @db_write_report="select name, salary from employees order by name" output name, salary
       @  -----<br/>
       @  Name: <<query-result name>>
       @  Salary: <<query-result salary>><br/>
    end-query
    @-----<br/>

    @End report.

    end-write-string


    // build time stamp for report file
    get-time to define curr_time format "%A %B %d %Y %l %M %p %Z"
    get-req process-id to define pid
    match-regex " " in curr_time replace-with "_" result define tstamp

    // get application home directory and build the path for reports directory
    get-app directory to define home_dir
    write-string define report_dir
    @<<p-out home_dir>>/reports
    end-write-string

    // make directory, ignore error if it exists
    exec-program "mkdir" args "-p", report_dir status define st
    if (st != VV_OKAY  && st != VV_ERR_EXIST) {
        @Error in creating reports directory
        exit-request
    }

    // build the name of report file
    write-string define report_file
    @<<p-out home_dir>>/reports/report_<<pf-out "%lld", pid>><<p-out tstamp>>
    end-write-string

    // write report file

    write-file report_file from outmsg status st
    if (st<0) {
        @Error in writing report file <<p-out report_file>> (<<pf-out "%lld", st>>)
        exit-request
    } else {
        // if okay, display report file to the web
        @Report file [<<p-out report_file>>] written.<br/>
        @Report file content:<br/>
        p-out outmsg
    }

%%

See also

Exec program

Purpose: Execute a program.

exec-program <program path> \
    [ args <program arg> [ , ... ] ] \
    [ status [ define ] <exit status> ] \
    [ ( input <input string> [ input-length <string length> ] ) \
        | ( input-file <input file> ) ] \
    [ ( output [ define ] <output string> [ output-length [ define ] <output length> ] ) \
        | ( output-file <output file> ) ] \
    [ ( error [ define ] <error string> ) | ( error-file <error file> ) ]

exec-program executes a program specified in <program path>, which can be a program name without path that exists in the path specified by the PATH environment variable; or an absolute path; or a path relative to the application home directory (see how-vely-works).

A program can have input arguments (specified with "args" clause), and if there are more than one, they must be separated by a comma. There is no limit on the number of input arguments, other than of the underlying Operating System. Each argument <program arg> may contain a comma if it is a string (i.e. double-quoted) or it is an expression within parenthesis.

You can specify a status variable <exit status>, which either must exist, or if it doesn't, use "define" to create it - this variable will have program's exit status. Note that if the program was terminated by a signal, <exit status> will have a value of 128+signal_number, so for example if the program was terminated with signal 9 (i.e. KILL signal), <exit status> will be 137 (i.e. 128+9). Any other kind of abnormal program termination (i.e. program termination that did not end in setting the exit code with exit() or similar) will return 126 as <exit code>.

Specifying program input and output is optional. If program has output and you are not capturing it in any way, the output is redirected to a temporary file that is deleted after exec-program completes.

You can specify an <input string> to be passed to program's standard input (stdin) via "input" clause. If "input-length" is not used, the length of this input is the string length of <input string>, otherwise <string length> bytes is passed to the program, allowing binary input. Alternatively, you can specify a file <input file> (via "input-file" clause) to be opened and directed into program's standard input (stdin).

You can redirect the program's output (which is "stdout") to a file <output file> using "output-file" clause. Alternatively, program's output can be captured in <output string> (via "output" clause) with its length in <output length> (via "output-length" clause), both of which can be created with optional "define". <output string> is allocated memory.

To get the program's error output (which is "stderr") to a file <error file>, use "error-file" clause. Alternatively, program's error output can be captured in <error string> (via "error" clause) which can be created with "define". <error string> is allocated memory.

If <input file> cannot be opened, VV_ERR_READ is reported in <exit status>, and if either <output file> or <error file> cannot be opened, the status is VV_ERR_WRITE.

Do not use system() function as it may be unsafe. Use exec-program instead.

Examples

To simply execute a program that is in the path, without any arguments, input or output:

exec-program "myprogram"

Run "grep" program using a string as its standard input in order to remove a line that contains "bad line" in it, and outputting the result into "ovar" variable:

exec-program "grep" args "-v", "bad line" "config" input "line 1\nline 2\nbad line\nline 3" output ovar
p-out ovar

Get the list of files in the application home directory into buffer "ovar" (and its length into variable "olen") and then display it:

exec-program "ls" output define ovar output-length define olen

Similar to the above example of listing files, but output results to a file (which is then read and the result displayed), and provide options "-a -l -s" to "ls" program:

exec-program "ls" args "-a", "-l", "-s" output-file "lsout"
read-file "lsout" to define final_res
p-out final_res

Count the lines of file "config" (which is redirected to the standard output of "wc" program) and store the result in variable "ovar" (by means of redirecting the output of "wc" program to it), and then display:

exec-program "wc" args "-l" input-file "config" output define ovar
p-out ovar

See also

Program execution
exec-program
exit-code
See all
documentation

Exit code

Purpose: Return exit code when running from the command-line.

exit-code <exit code>

exit-code specifies <exit code> (which must be an integer expression) when the program runs from the command-line.

exit-code can be specified anywhere in the code, and does not mean exiting the program. To exit the program, either use exit-request or simply allow request function to reach its end.

When exit-code is not used, the default exit code is 0.

Examples

When the program exits, its exit code will be 12:

exit-code 12
...
exit-request

See also

Program execution
exec-program
exit-code
See all
documentation

Exit request

Purpose: Exit current request processing.

exit-request

Exits current request by transferring control directly after the top-level request dispatcher. If there is an after-request-handler, it will still execute, unless exit-request is called from before-request-handler. exit-request will have no effect in startup-handler because that handler runs before any requests.

exit-request is useful when your request handler has called other functions (i.e. those implemented in non-request source files), which may have called others (etc.), and there is no need to return in the reverse order, nor to pass any further data back to them; in which case returning one step at a time may be cumbersome and error prone.

In other words, exit-request jumps to the top-level request dispatcher, and the stack of functions called to get to exit-request will be bypassed, thus those functions will not get control back; and they will not perform any additional work, rather simply the next request will be processed immediately.

Never use C's exit() function, as it will terminate the server process and prevent exit-code from functioning properly.

Examples

In this example, a request handler "req_handler" calls function "my_function()" which exits the request:

#include "vely.h"

request-handler /req_handler
    ...
    my_function();
    // After exit-request in my_function(), the following code is never executed
    ...
end-request-handler

void my_function () {
    ...
    exit-request
    ...
}

See also

Program flow
do-once
exit-request
See all
documentation

FastCGI API

You can use Vely C API client library to connect to FastCGI application servers, including Vely:

The API has only a few functions, and the main one is "vv_fc_request()", which makes a call to the server.
There is only a single data type used, which is "vv_fc" and it is used to specify a request and its options, as well as to retrieve results.
There is a single include file ("vfcgi.h").
When building your client executable, you can specify build flags by using the result of "vv -i".
It is MT-safe, so you can use it in multi-threaded applications, such as to make many requests in parallel.

See Examples section below for detailed examples.

Sending a request to FastCGI server

The following function is used to make a FastCGI call using C API:

int vv_fc_request (vv_fc *req);

All input and output is contained in a single variable of type "vv_fc", the pointer to which is passed to "vv_fc_request()" function that sends a request to the server. A variable of type "vv_fc" must be initialized to zero before using it (such as with {0} initialization, "memset()" or "calloc()"), or otherwise some of its members may have random values:

// Define and initialize request variable
vv_fc req = {0};
// You could also do:
// memset ((char*)&req, 0, sizeof(vv_fc));
...
// Set members of 'req' variable (see below)
...
// Make FastCGI call
int result = vv_fc_request (&req);

Type "vv_fc" is defined as (i.e. public members of it):

typedef struct {
    const char *fcgi_server; // the IP:port/socket_path to server
    const char *req_method; // request method
    const char *app_path; // application path
    const char *req; // request name
    const char *url_payload; // URL payload (path+query string)
    const char *content_type; // content type
    int content_len; // content len
    const char *req_body; // request body (i.e. content)
    char **env; // environment to pass into request on server side
    int timeout; // timeout for request
    int req_status; // status of request from server
    int data_len; // length of response from server
    int error_len; // length of error from server
    char *errm; // error message when vv_fc_request returns other than VV_OKAY
    vv_fc_out_hook out_hook; // hook to get data output as soon as it arrives
    vv_fc_err_hook err_hook; // hook get error output as soon as it arrives
    vv_fc_done_hook done_hook; // get all data when request is complete
    int thread_id; // custom ID when executing in a multithreaded fashion
    volatile char done; // indicator that the request has completed
    int return_code; // the return code from vv_fc_request()
} vv_fc;

- Mandatory input
The following members of "vv_fc" type must be supplied in order to make a FastCGI call:

"fcgi_server" represents either a Unix socket or a TCP socket, and is:
- for a Unix socket, a fully qualified name to a Unix socket file used to communicate with the server (for a Vely server, it's "/var/lib/vv/<app name>/sock/sock", where <app name> is the application name), or
- for a TCP socket, a host name and port name in the form of "<host name>:<port number>", specifying where is the server listening on (for instance "127.0.0.1:2301" if the FastCGI server is local and runs on TCP port 2301).
"req_method" is a request method, such as "GET", "POST", "PUT", "DELETE" or any other.
"app_path" is an application path (see request-URL). By default it's "/<application name>".
"req" is a request path, i.e. a request name preceded by a forward slash, as in "/<request name>" (see request-URL).

- Request URL payload
"url_payload" is the URL payload, meaning input parameters (as path segments and query string, see request-URL). URL payload can be NULL or empty, in which case it is not used.

- Request body (content)
"req_body" is the request body, which can be any text or binary data. "content_type" is the content type of request body (for instance "application/json" or "image/jpg"). "content_len" is the length of request body in bytes. A request body is sent only if "content_type" and "req_body" are not NULL and not empty, and if "content_len" is greater than zero.

- Passing environment to server
"env" is any environment variables that should be passed along to the request executing on a server. You can access those in Vely via "web-environment" clause of get-sys statement. This is an array of strings, where name/value pairs are specified one after the other, and which always must end with NULL. For example, if you want to use variable "REMOTE_USER" with value "John" and variable "MY_VARIABLE" with value "8000", then it might look like this:

char *env[5];
env[0] = "REMOTE_USER";
env[1] = "John"
env[2] = "MY_VARIABLE";
env[3] = "8000"
env[4] = NULL;

Thus, if you are passing N environment variables to the server, you must size "env" as "char*" array with 2*N+1 elements.

- Timeout
"timeout" is the number of seconds a call to the server should not exceed. For instance if the remote server is taking too long or if the network connection is too slow, you can limit how long to wait for a reply. If there is no timeout, then "timeout" value should be zero. Note that DNS resolution of the host name (in case you are using a TCP socket) is not counted in timeout. Maximum value for timeout is 86400.

Even if timeout is set to 0, a server call may eventually timeout due to underlying socket and network settings. Note that even if your server call times out, the actual request executing on the server may continue until it's done.
- Thread ID
"thread_id" is an integer that you can set and use when your program is multithreaded. By default it's 0. This number is set by you and passed to hooks (your functions called when request is complete or data available). You can use this number to differentiate the data with regards to which thread it belongs to.

- Completion indicator and return code
When your program is multithreaded, it may be useful to know when (and if) a request has completed. "done" is set to to "true" when a request completes, and "return_code" is the return value from vv_fc_request(). In a single-threaded program, this information is self-evident, but if you are running more than one request at the same time (in different threads), you can use these to check on each request executing in parallel (for instance in a loop in the main thread).

Note that "done" is "true" specifically when all the results of a request are available and the request is about to be completed. In a multithreaded program, it means the thread is very soon to terminate or has already terminated; it does not mean that thread has positively terminated. Use standard "pthread_join()" function to make sure the thread has terminated if that is important to you.

Return value

The following are possible return values from "vv_fc_request()" (available in "return_code" member of "vv_fc" type):

VV_OKAY if request succeeded,
VV_FC_ERR_RESOLVE_ADDR if host name for TCP connection cannot be resolved,
VV_FC_ERR_PATH_TOO_LONG if path name of Unix socket is too long,
VV_FC_ERR_SOCKET if cannot create a socket (for instance they are exhausted for the process or system),
VV_FC_ERR_CONNECT if cannot connect to server (TCP or Unix alike),
VV_FC_ERR_SOCK_WRITE if cannot write data to server (for instance if server has encountered an error or is down, or if network connection is no longer available),
VV_FC_ERR_SOCK_READ if cannot read data from server (for instance if server has encountered an error or is down, or if network connection is no longer available),
VV_FC_ERR_PROT_ERR if there is a protocol error, which indicates a protocol issue on either or both sides,
VV_FC_ERR_BAD_VER if either side does not support protocol used by the other,
VV_FC_ERR_SRV if server cannot complete the request,
VV_FC_ERR_UNK if server does not recognize record types used by the client,
VV_FC_ERR_OUT_MEM if client is out of memory,
VV_FC_ERR_ENV_TOO_LONG if the combined length of all environment variables is too long,
VV_FC_ERR_ENV_ODD if the number of supplied environment name/value pairs is incorrect,
VV_FC_ERR_BAD_TIMEOUT if the value for timeout is incorrect,
VV_FC_ERR_TIMEOUT if the request timed out based on "timeout" parameter or otherwise if the underlying Operating System libraries declared their own timeout.

You can obtain the error message in "errm" member of "vv_fc" type.

Server reply

The server reply is split in two. One part is the actual result of processing (called "stdout" or standard output), and that is "data". The other is the error messages (called "stderr" or standard error), and that's "error". If the server is Vely, all its output goes to "data", except from report-error and pf-out/pf-url/pf-web (with "to-error" clause) which goes to "error". Note that "data" and "error" streams can be co-mingled when output by the server, but they will be obtained separately. This allows for clean separation of output from any error messages.

You can obtain server reply when it's ready in its entirety (likely most often used), or as it comes alone bit by bit (see more about asynchronous hooks futher here).

Status of request execution

"req_status" member of "vv_fc" type is the request status when a request had executed; it is somewhat similar to an exit status of a program. This is a part of FastCGI specification, and the particular server software you are using may or may not return this status. A Vely server request returns status by means of exit-code statement. Note that "req_status" is valid only if "vv_fc_request()" returned VV_OKAY (or if "return_code" is VV_OKAY for multi-threaded programs).

Getting data reply (stdout)

Data returned from a request is valid only if "vv_fc_request()" returned VV_OKAY (or if "return_code" is VV_OKAY for multi-threaded programs). In that case, use "vv_fc_data()" function, for example:

// Declare and initialize request variable
vv_fc req = {0};
// Setup the req variable
...
// Execute request
if (vv_fc_request (&req) == VV_OKAY) {
    char *data = vv_fc_data (req); // data response
    int data_len = req->data_len; // length of data response in bytes
}

"data_len" member of "vv_fc" type will have the length of data response in bytes. The reply is always null-terminated as a courtesy, and "data_len" does not include the terminating null byte.

"vv_fc_data()" returns the actual response (i.e. data output) from server as passed to "data" stream. If you are using Vely server, any output will go there, except when "to-error" clause is used in pf-out, pf-url and pf-web - use these constructs to output errors without stopping the request's execution. Additionaly, the output of report-error will also not go to data output.

Getting error reply (stderr)

An error reply returned from a request is valid only if "vv_fc_request()" returned VV_OKAY (or if "return_code" is VV_OKAY for multi-threaded programs). In that case, use "vv_fc_error()" function, for example:

// Declare and initialize request variable
vv_fc req = {0};
// Setup the req variable
...
// Execute request
if (vv_fc_request (&req) == VV_OKAY) {
    char *err = vv_fc_error (req); // error response
    int err_len = req->error_len; // length of error response in bytes
}

"vv_fc_error()" returns any error messages from a server response, i.e. data passed to "error" stream. For a Vely server, it is comprised of any output when "to-error" clause is used in pf-out, pf-url and pf-web, as well as any output from report-error.

"error_len" member (of "vv_fc" type above) will have the length of error response in bytes. The response is always null-terminated as a courtesy, and "error_len" does not include the terminating null byte.

Freeing the result of a request

Once you have obtained the result of a request, and when no longer needed, you should free it by using "vv_fc_delete()":

// Declare and initialize request variable
vv_fc req = {0};
// Setup the req variable
...
// Execute request
vv_fc_request (&req);
// .. Use the result ..
// Free request output (data and error streams)
vv_fc_delete (&req);

If you do not free the result, your program may experience a memory leak. If your program exits right after issuing any request(s), you may skip freeing results as that is automatically done on exit by the Operating System.

You can use "vv_fc_delete()" regardless of whether "vv_fc_request()" returned VV_OKAY or not.

Completion hook

A function you wrote can be called when a request has completed. This is useful in multithreaded invocations, where you may want to receive complete request's results as they are available. To specify a completion hook, you must write a C function with the following signature and assign it to "done_hook" member of "vv_fc" typed variable:

typedef void (*vv_fc_done_hook)(char *recv, int recv_len, char *err, int err_len, vv_fc *req);

"recv" is the request's data output, "recv_len" is its length in bytes, "err" is the request's error output, and "err_len" is its length in bytes. "req" is the request itself which you can use to obtain any other information about the request. In a single threaded environment, these are available as members of the request variable of "vv_fc" type used in the request, and there is not much use for a completion hook.

See an example with asynchronous hooks.

Asynchronous hooks

You can obtain the server's reply as it arrives by specifying read hooks. This is useful if the server supplies partial replies over a period of time, and your application can get those partial replies as they become available.

To specify a hook for data output (i.e. from stdout), you must write a C function with the following signature and assign it to "out_hook":

typedef void (*vv_fc_out_hook)(char *recv, int recv_len, vv_fc *req);

"recv" is the data received and "recv_len" is its length.

To specify a hook for error output (i.e. from stderr), you must write a C function with the following signature and assign it to "err_hook":

typedef void (*vv_fc_err_hook)(char *err, int err_len, vv_fc *req);

"err" is the error received and "err_len" is its length.

"req" (in both hooks) is the request itself which you can use to obtain any other information about the request.

To register these functions with "vv_fc_request()" function, assign their pointers to "out_hook" and "err_hook" members of request variable of type "vv_fc" respectively. Note that the output hook (i.e. hook function of type "vv_fc_out_hook") will receive empty string ("") in "recv" and "recv_len" will be 0 when the request has completed, meaning all server output (including error) has been received.

For example, functions "get_output()" and "get_err()" will capture data as it arrives and print it out, and get_complete() will print the final result:

// Output hook
void get_output(char *d, int l, vv_fc *req)
{
    printf("Got output of [%.*s] of length [%d] in thread [%d]", l, d, l, req->thread_id);
}

// Error hook
void get_err(char *d, int l, vv_fc *req)
{
    printf("Got error of [%.*s] of length [%d], status [%d]", l, d, l, req->req_status);
}

// Completion hook
void get_complete(char *data, int data_len, char *err, int err_len, vv_fc *req)
{
    printf("Got data [%.*s] of length [%d] and error of [%.*s] of length [%d], status [%d], thread [%d]\n", data_len, data, data_len, err_len, err, err_len, req->req_status, req->thread_id);
}

...

vv_fc req = {0};
...
// Register output and error hooks, as well as a completion hook
req.out_hook = &get_output;
req.err_hook = &get_err;
req.done_hook = &get_complete;

Multithreading

The FastCGI client is MT-safe, meaning you can use it both in single-threaded and multi-threaded programs. Note that each thread must have its own copy of "vv_fc" request variable, since it provides both input and output parameters to a request call and as such cannot be shared between the threads.

Usage

Do not use this API directly with Vely - use call-server instead which is made specifically for use in .vely files. Otherwise, you can use this API with any program.

Using API without Vely

You can use FastCGI API without installing Vely. To do that:

get the Vely source code,
copy source files "fcli.c" and "vfcgi.h" to where you need to build your program,
add the content of Vely's "LICENSE" file to your own in order to include the license for these source files,
then replace "$(vv -i)" in the build line with "fcli.c" to include these files, for instance instead of:
```
gcc -o cli cli.c -g $(vv -i)
```
you would write:
```
gcc -o cli cli.c -g fcli.c
```

Note that you do not need to install any other dependencies, as FastCGI API is entirely contained in the aforementioned source files.

Examples

Simple example

The following example is a simple demonstration, with minimum of options used. Copy the C code to file "cli.c":

#include "vfcgi.h"

void main ()
{
    // Request type vv_fc - create a request variable and zero it out
    vv_fc req = {0};

    req.fcgi_server = "/var/lib/vv/helloworld/sock/sock"; // Unix socket
    req.req_method = "GET"; // GET HTTP method
    req.app_path = "/helloworld"; // application path
    req.req = "/hello-simple"; // request name

    // Make a request
    int res = vv_fc_request (&req);

    // Check return status, and if there's an error, display error code and the
    // corresponding error message. Otherwise, print out server response.
    if (res != VV_OKAY) printf("Request failed [%d] [%s]\n", res, req.errm);
    else printf("%s", vv_fc_data(&req));

    // Free up resources so there are no memory leaks
    vv_fc_delete(&req);
}

To make this client application:

gcc -o cli cli.c -g $(vv -i)

In this case, you're using a Unix socket to communicate with the FastCGI server. To test with a Vely request handler, copy the following code to "hello_simple.vely" file:

#include "vely.h"
void hello_simple()
{
   silent-header
   out-header default

   @Hi there!
}

Create and make the Vely server application and run it via local Unix socket:

sudo vf -i -u $(whoami) helloworld
vv -q
vf -m quit helloworld
vf -w 1 helloworld

Run the FastCGI client:

./cli

The output is, as expected:

Hi there!

Example with more options

This example demonstrates using multiple options, including using TCP sockets connecting to a host and port number, environment variables, query string, request body and request execution timeout. It will also show the separation of "data" and "error" (i.e. stdout and stderr) streams from the server.

Copy this to file "cli1.c" - note that in this example FastCGI server will run on localhost (127.0.0.1) and TCP port 2301:

#include "vfcgi.h"

void main ()
{
    // Request type vv_fc - create a request variable
    vv_fc req;
    // Initialize request
    memset (&req, 0, sizeof(req));

    // Add 3 environment variables (in the form of name, value, name, value, ... , NULL)
    char *env[] = { "REMOTE_USER", "John", "SOME_VAR", "SOME\nVALUE", "NEW_VAR", "1000", NULL };

    // Create a request
    // Environment variables to pass to server request
    req.env = env;
    // Server IP and port
    req.fcgi_server = "127.0.0.1:2301";
    // Request method
    req.req_method = "GET";
    // Application path
    req.app_path = "/helloworld";
    // Request
    req.req = "/hello";
    // URL payload (path and query string)
    req.url_payload = "par1=val1&par2=91";
    // Content type
    req.content_type = "application/json";
    // Content (i.e. request body)
    req.req_body = "This is request body";
    // Content length
    req.content_len = strlen (req.req_body);
    // No timeout (set to 0)
    req.timeout = 0;

    // Make a request
    int res = vv_fc_request (&req);

    // Check return status, and if there's an error, display error code and the
    // corresponding error message
    if (res != VV_OKAY) printf("Request failed [%d] [%s]\n", res, req.errm);
    else {
       // If successful, display request results
       // Exit code from the server processing
       printf("Server status %d\n", req.req_status);
       // Length of reply from server
       printf("Len of data %d\n", req.data_len);
       // Length of any error from server
       printf("Len of error %d\n", req.error_len);
       // Reply from server
       printf("Data [%s]\n", vv_fc_data(&req));
       // Any error message from server
       printf("Error [%s]\n", vv_fc_error(&req));
    }

    // Free up resources so there are no memory leaks
    vv_fc_delete(&req);
}

Note that the URL payload (i.e. "req.url_payload") could have been written as a combination of a path segment and query string (see request-URL):

req.url_payload = "/par1/val1?par2=91";

or just as a path segment:

req.url_payload = "/par1/val1/par2/91";

To make this client application:

gcc -o cli1 cli1.c -g $(vv -i)

To test it, you can create a Vely server application. Copy this to "hello.vely" file in a separate directory:

#include "vely.h"

void hello()
{
    silent-header
    out-header default

    // Get request body
    request-body rb

    // Input params
    input-param par1
    input-param par2

    // Get environment variables passed on from the client
    get-sys web-environment "REMOTE_USER" to define ruser
    get-sys web-environment "SOME_VAR" to define somev
    get-sys web-environment "NEW_VAR" to define newv

    // Output, print the environment variables, the PID of server process and the request body received from the client
    @Hello World! [<<p-out ruser>>] [<<p-out somev>>] [<<p-out newv>>] [<<p-out par1>>] [<<p-out par2>>] <<pf-out "%d", getpid()>> <<p-out rb>>

    // Output back a number of lines, generally as "Output line #<num of line>"
    // After line #1418, print out "Line 1419 has an error" to stderr
    // After line #4418, report an error and exit
    // This demostrates outputting data to both stdout and stderr
    num i;
    for (i = 0; i < 8000; i++) {
        @Output line #<<p-num i>>
        if (i == 1419) {
            pf-out to-error "Line %lld has an error\n", i
        }
        if (i == 4419) {
            // Exit code of the server execution
            exit-code 82
            report-error "%s", "Some error!"
        }
    }
}

Create and make the Vely server application and run it on local TCP port 2301 to match the client above:

sudo vf -i -u $(whoami) helloworld
vv -q
vf -m quit helloworld
vf -w 1 -p 2301 helloworld

Run the FastCGI client:

./cli1

The output:

Server status 82
Len of data 78530
Len of error 35
Data [Hello World! [John] [SOME
VALUE] [1000] [val1] [91] 263002 This is request body
Output line #0
Output line #1
Output line #2
Output line #3
Output line #4
Output line #5
Output line #6
Output line #7

...
Output line #4413
Output line #4414
Output line #4415
Output line #4416
Output line #4417
Output line #4418
Output line #4419
]
Error [Line 1419 has an error
Some error!
]

The output shows server exit code (82, see exit-code in the Vely code above), length of data output, and other information which includes environment variables passed to the server from client, the PID of server process, the request body from the client, and then the error output. Note that the data output (stdout) and the error output (stderr) are separated, since FastCGI protocol does use separate streams over the same connection. This makes working with the output easy, while the data transfer is fast at the same time.

See also

Client
FastCGI-API
FastCGI-command-line-client
See all
documentation

FastCGI command line client

You can use cgi-fcgi command-line utility to communicate with your FastCGI application without using a web server. Use it to get responses from the command line. This is useful for shell scripting, debugging, testing etc.

If you have application <app name> running, for example 3 concurrent processes:

vf -w 3 <app name>

then you can send requests to it and receive replies:

export REQUEST_METHOD=GET
export SCRIPT_NAME="/<app name>"
export PATH_NAME="/<request name>"
export QUERY_STRING="par1=val1&par2=val2.."
cgi-fcgi -connect /var/lib/vv/<app name>/sock/sock  /<app name>

If you're running your application on TCP/IP instead of Unix socket (as above), for instance on TCP port 3000:

vf -w 3 -p 3000 <app name>

then you can send requests to it and receive replies:

cgi-fcgi -connect 127.0.0.1:3000  /

Examples

The following example will create an application with 3 Vely request files, and test them as FastCGI application (both using Unix sockets and TCP) and as a command-line program. You can execute below bash commands one by one, or save them in a single file and run:

#
#Create hello_1.vely source file
#
echo '#include "vely.h"

request-handler /hello_1
out-header default
@Hello World #1
end-request-handler' > hello_1.vely

#
#Create hello_2.vely source file
#
echo '#include "vely.h"

request-handler /hello_2
out-header default
@Hello World #2
end-request-handler' > hello_2.vely

#
#Create Vely application named "hithere"
#
sudo vf -i -u $(whoami) hithere

#
#Make the application
#
vv -q

#
#Stop (any) running server processes
#Start 2 workers for your application running as FastCGI server via Unix sockets
#
vf -m quit hithere
vf -w 2 hithere

#
#Send request to execute hello_1() function.
#
export REQUEST_METHOD=GET
export SCRIPT_NAME="/hithere"
export PATH_INFO="/hello_1"
export VV_SILENT_HEADER=no
cgi-fcgi -connect /var/lib/vv/hithere/sock/sock  /hithere

#
#Restart your application as FastCGI server running on TCP port 3000
#Then send request to execute hello_2() function.
#
vf -m quit hithere
vf -p 3000 -w 2 hithere
export REQUEST_METHOD=GET
export SCRIPT_NAME="/hithere"
export PATH_INFO="/hello_2"
export VV_SILENT_HEADER=no
cgi-fcgi -connect 127.0.0.1:3000 /

#
#Run your application as command line program. Supress HTTP headers.
#
export REQUEST_METHOD=GET
export SCRIPT_NAME="/hithere"
export PATH_INFO="/hello_2"
export VV_SILENT_HEADER=no
/var/lib/vv/bld/hithere/hithere

See also

Client
FastCGI-API
FastCGI-command-line-client
See all
documentation

FastCGI

You can run a Vely application as a server by using vf program manager. Your application will communicate with outside world using with FastCGI protocol.

You can access your server application by means of:

A web server (which is probably the most common way). You need to setup a reverse proxy, i.e. a web server that will forward requests and send replies back to clients; see below.
The command line, in which case you can use a FastCGI-command-line-client.
FastCGI-API, which allows any application in any programming language to access your server, as long as it has C linkage (by far most do). This method allows for MT (multithreaded) access to your application, where many client requests can be made in parallel.

Vely server runs as a number of (zero or more) background processes in parallel, processing requests.

Setting up reverse proxy (web server)

To access your application via a reverse proxy (i.e. web server), generally you need to add a proxy directive and restart the web server.

If you use Apache, you need to connect it to your application, see connect-apache-tcp-socket (for using TCP sockets) and connect-apache-unix-socket (for using Unix sockets).

If you use Nginx, you need to connect it to your application, see connect-nginx-tcp-socket (for using TCP sockets) and connect-nginx-unix-socket (for using Unix sockets).

Starting FastCGI server processes

Use vf, for example:

vf <app name>

which in general will start zero or more background resident process(es) (daemons) that process requests in parallel.

Connection timeout

In a heavy-load environment, a client's connection may be rejected by the server. This may happen if the client runs very slowly due to swapping perhaps. Once a client establishes a connection, it has up to 5 seconds by default to send data; if it doesn't, the server will close the connection. Typically, FastCGI clients send data right away, but due to a heavy load, this time may be longer. To set the connection timeout in milliseconds, set the following variable before starting the application server, for instance:

export "LIBFCGI_IS_AF_UNIX_KEEPER_POLL_TIMEOUT"="8000"
vf -w 1 <app name>

In this case, the timeout is set to 8 seconds.

See also

Running application
application-setup
CGI
command-line
containerize-application
FastCGI
plain-C-FCGI
See all
documentation

File position

Purpose: Set a position or get a current position for an open file.

file-position file-id <file id> \
    ( set <position> ) | ( get [ define ] <position> ) \
    [ status [ define ] <status> ]

file-position will set or get position for a file opened with open-file, where <file id> is an open file identifier.

If "set" clause is used, file position is set to <position>.

If "get" clause is used, file position is obtained in <position>, which can be created with optional "define".

The optional <status> in "status" clause will be VV_OKAY if set/get succeeded, or VV_ERR_POSITION if not. <status> can be created with optional "define".

Note that setting position past the last byte of file is okay for writing - in this case the bytes between the end of file and the <position> are filled with null-bytes when the write operation occurs.

Examples

Open file "testwrite" and set file byte position to 100, then obtain it later:

open-file "testwrite" file-id define nf
file-position file-id nf set 100
...
file-position file-id nf get define pos

See also

File storage

Vely provides a file directory that you can use for any general purpose, including for storing temporary-files. This directory is also used for automatic upload of files from clients. It provides a two-tier directory system, with sub-directories automatically created to spread the files for faster access.

Files in Vely file directory are located in the sub-directories of:

/var/lib/vv/<app_name>/app/file

If you wish to place this directory in another physical location, see vv.

You can create files here by means of uniq-file, where a new unique file is created - with the file name made available to you. This directory is also used for uploading of files from clients (such as web browsers or mobile device cameras) - the uploading is handled automatically by Vely (see input-param).

In general, a number of sub-directories is created within the file directory that contain files, with (currently) the maximum of 40,000 directories with unlimited files per each directory (so if you had 50,000 files per each directory, then you could store a total of about 2 billion files). This scheme also allows for faster access to file nodes due to relatively low number of sub-directories and files randomly spread per each such sub-directory. The random spreading of files across subdirectories is done automatically.

Do not rename file names or sub-directory names stored under file directory.

See also

File uploading

Purpose: Upload a file to server.

input-param <name>

Files uploaded via client (such as a browser, curl etc.) are input-parameters.

Vely uploads files for you automatically, meaning you do not have to write any code for that specific purpose. An uploaded file will be stored in file-storage, with path and name of such a file generated by Vely to be unique. For example, a file uploaded might be named "/home/user/file/d0/f31881". When file is uploaded, the following input parameters can be obtained, assuming "name" attribute of "input" element is "myfile":

"myfile_filename" is the name of the file as provided by the client.
"myfile_location" is the full path on the server where file is stored (such as "/home/user/file/d0/f31881" for example)
"myfile_ext" is the extension of the file, for example ".jpg" or ".pdf". Note that ".jpeg" is always showing as ".jpg". The extension is always lower case.
"myfile_size" is the size of the file uploaded. See vv for setting the maximum file upload size.

For example, for an HTML form which is uploading a file named "myfile", such as

<input type='file' name='myfile'>

your code that handles this might be:

input-param myfile_filename
input-param myfile_location
input-param myfile_ext
input-param myfile_size

You have uploaded file <<p-web myfile_filename>> to a server file at <<p-web myfile_location>>

See also

Finish output

Purpose: Finishes the request output.

finish-output

finish-output will flush out and conclude all of the request's output (see output-statement). Any such output afterwards will silently fail to do so. As far as the client is concerned (or the user running a command-line program), all the output from the request is complete.

This statement is useful when you need to continue work after the request's output is complete. For example, if the request is a long-running one, you can inform the client that the job has started, and then take any amount of time to actually complete the job, without worrying about client timeouts. The client can inquire about the job status via a different request, or be informed via email etc.

Examples

finish-output

See also

Output
finish-output
flush-output
output-statement
p-dbl
pf-out
pf-url
pf-web
p-num
p-out
p-path
p-url
p-web
See all
documentation

Flush output

Purpose: Flush output.

flush-output

Use flush-output statement to flush any pending output to the client (such as FastCGI-command-line-client) or from the command-line application.

This can be useful if the complete output would take longer to produce and intermittent partial output would be needed.

Note that whether the client will actually receive the output in this fashion depends on several factors, including the client itself and any intermediaries such as proxy servers.

Examples

In this case the complete output may take at least 20 seconds. With flush-output, the message "This is partial output" will be flushed out immediately.

@This is partial output
flush-output
sleep(20);
@This is final output

See also

Output
finish-output
flush-output
output-statement
p-dbl
pf-out
pf-url
pf-web
p-num
p-out
p-path
p-url
p-web
See all
documentation

Get app

Purpose: Obtain data that describes the application.

get-app \
    name | directory | trace-directory | file-directory \
        | db-vendor <database configuration> | upload-size \
        | path | process-data \
    to [ define ] <variable>

Application-related variables can be obtained with get-app statement. The following application variables can be obtained (they are all strings except upload-size):

"name" returns the name of your application, as specified when created (see vf).
"directory" is the directory where your application resides, i.e. the application home directory (see how-vely-works).
"trace-directory" is the directory where your application trace files are written (if enabled, see how-vely-works).
"file-directory" is the directory where Vely file storage system is, i.e. file-storage.
"db-vendor" is the database vendor name of database given by <database configuration> (as used for example in run-query). You can use it to create database specific conditions. The database vendor name can be compared against predefined string constants VV_MARIADB (for MariaDB database), VV_POSTGRES (for Postgres database) and VV_SQLITE (for SQLite database).
"path" is the leading path of the URL request that can be used to build web forms and links leading back to your application. It returns the same value as p-path.
"process-data" obtains the pointer to global-process-data.
"upload-size" is the maximum allowed size of an uploaded file - this is a number.

Note that all the string values above (except "process-data") should be treated as constants - do not change them as that may cause program to malfunction. If you want to alter any of these values, make a copy first (see copy-string).

Examples

Get the name of Vely application:

get-app name to define appname

Get the vendor of database db:

get-app db-vendor db to define dbv
if (!strcmp (dbv, VV_POSTGRES)) {
    // do something Postgres specific
}

See also

Application information
get-app
set-app
See all
documentation

Get cookie

Purpose: Get cookie value.

get-cookie [ define ] <cookie value> = <cookie name>

get-cookie obtains the value of a cookie with the name given by <cookie name>. A cookie would be obtained from an incoming request from the client (such as web browser) or it would be set using set-cookie.

The value of cookie is stored in <cookie value>. If it does not exist, you can create it within the statement by using "define" clause. <cookie value> is allocated memory.

Cookies are often used to persist user data on the client, such as for maintaining session security or for convenience of identifying the user or its actions.

Examples

Get value of cookie named "my_cookie_name" - variable my_cookie_value will hold its value:

get-cookie define my_cookie_value="my_cookie_name"

See also

Cookies
delete-cookie
get-cookie
set-cookie
See all
documentation

Get hash

Purpose: Get usage specifics for a hash table.

get-hash  <hash > \
    ( length [ define ] <length> ) \
    | ( size [ define ] <size> ) \
    | ( average-reads [ define ] <reads> )

get-hash provides usage specifics of a hash table <hash> (created by new-hash).

Use "length" clause to obtain its <length> (i.e. the number of elements stored in it), "size" clause to obtain its <size> (i.e. the number of "buckets", or possible hash codes) and "average-reads" clause to obtains the average number of <reads> (i.e. how many string comparisons are needed on average to find a key).

Each of these number variables can be created with "define".

This information may be useful in determining the performance of a hash, and whether resize-hash is indicated.

Examples

get-hash h length define l size define s average-reads define r

See also

Hash table
get-hash
new-hash
purge-hash
read-hash
resize-hash
write-hash
See all
documentation

Get req

Purpose: Obtain data that describes the input request.

get-req \
    data | errno | error | cookie-count | cookie <cookie index> \
        | arg-count | arg-value <arg index> | input-count | \
        | input-value <input index> | input-name <input index> \
        | header <header> | referring-url | method \
        | content-type | trace-file | process-id | name \
    to [ define ] <variable>

Variables related to an input request can be obtained with get-req statement and the result stored into <variable>. The following request variables can be obtained (they are strings except where stated otherwise):

"data" obtains the pointer to global-request-data.
"errno" obtains the integer value of operating system "errno" tied to a last Vely statement that can return a status code. The "errno" value is saved internally; and restored here. It means that if "errno" changed for whatever reason since such Vely statement, you will still obtain the correct value. See error-code for an example. Note that errno value is undefined if there is no error, and can be 0 if the error is reported by Vely and not by operating system.
"error" returns the error message (a string) that correlates to "errno" value. The result of "error" clause is allocated memory.
"cookie-count" returns the number of cookies. This means any cookies received from the client plus any cookies added (with set-cookie) in the application minus any cookies deleted (with delete-cookie).
"cookie" returns the cookie value specified by <cookie index> (a number (0, 1, 2...) up to number of cookies), for instance in the block of C code:
```
num i;
get-req cookie-count to define cookie_c
for (i = 0; i < cookie_c; i++) {
    get-req cookie i to define cookie_val
    pf-web "cookie %s\n", cookie_val
    @<br/>
}
```
In this example, we get the number of input parameters, and then print out each name/value pair.
"arg-count" is the number of input arguments to your application (passed from a program caller, see vv).
"arg-value" is the string value of a single element from the array of input arguments, specified by <arg_index>. This array is indexed from 0 to one less than value obtained by "arg-count". Here is an example of using arg-count and arg-value:
```
get-req arg-count to define ac
pf-out "Total args [%lld]", ac
num i;
for (i=0; i<ac; i++) {
    get-req arg-value i to define av
    pf-out "%s\n", av
}
```
This code will display the number of input arguments (as passed to main() function of your program, excluding the first argument which is the name of the program), and then all the arguments. If there are no arguments, then variable 'ac' would be 0. See vv on passing arguments to your application.
"input-count" is the number of input parameters (from a request). Together with input-name and input-value, this lets you examine all input parameters in a request, especially in situations where the parameters are dynamic in nature.
"input-name" is the string value of a single element from the array of input parameter names (from a request), specified by <input_index>. This array is indexed from 0 to one less than value obtained by "input-count". See "input-value" for an example.
"input-value" is the string value of a single element from the array of input parameter values (from a request), specified by <input_index>. This array is indexed from 0 to one less than value obtained by "input-count". For example:
```
num i;
get-req input-count to define input_c
for (i = 0; i < input_c; i++) {
    get-req input-name i to define input_n
    get-req input-value i to define input_v
    pf-web "Input parameter #%lld, name is %s, value is %s\n", i, input_n, input_v
    @<br/>
}
```
In this example, we get the number of cookies, and then print out each. The cookie value is the full cookie string, including expiration date, strictness etc.
"header" is the value of HTTP request header <header> that is set by the client. For example, if the HTTP request contains header "My-Header:30", then hval would be "30":
```
get-req header "My-Header" to define hval
```
Note that not all HTTP request headers are set by the caller. For example, SERVER_NAME or QUERY_STRING are set by the web server, and to get such headers, use get-sys.
"method" is the request method. This is a number with values of VV_GET, VV_POST, VV_PUT, VV_PATCH or VV_DELETE for GET, POST, PUT, PATCH or DELETE requests, respectively. If it is not any of those commonly used ones, then the value is VV_OTHER and you can use get-sys with "web-environment" clause to obtain "REQUEST_METHOD" variable.

If you are writing a REST API, then you can use the method to implement Create, Read, Update, Delete (CRUD) operations, for instance:
```
get-req method to define request_method
if (request_method == VV_POST) {
    // create resource action
}
if (request_method == VV_GET) {
    // read resource action
}
if (request_method == VV_PUT) {
    // replace resource action
}
if (request_method == VV_PATCH) {
    // update resource action
}
if (request_method == VV_DELETE) {
    // delete resource action
}
```
"content-type" is the request content type. It is a string and generally denotes the content type of a request-body, if included in the request. Common examples are "application/x-www-form-urlencoded", "multipart/form-data" or "application/json".
"referring-url" is the referring URL (i.e. the page from which this request was called, for example by clicking a link).
"trace-file" is the full path of the trace file for this request (if enabled, see vv).
"process-id" is the "PID" (process ID) number of the currently executing process, as an integer.
"name" is the request name as specified in the request-URL.

Note that all the string values above (except "data") should be treated as constants - do not change them as that may cause program to malfunction. If you want to alter any of these values, make a copy first (see copy-string).

Examples

Get the name of current trace file:

get-req trace-file to define trace_file

See also

Request information
get-req
if-task
input-param
request-body
set-input
set-req
task-param
See all
documentation

Get sys

Purpose: Obtain data that describes the system.

get-sys \
    environment <var name> | web-environment <var name> \
        directory | os-name | os-version \
    to [ define ] <variable>

System-describing variables can be obtained with get-sys statement and the result stored into <variable>. The following system variables can be obtained:

"environment" returns the name of a given environment variable <var name> from the Operating System, such as in the following example, the HOME variable (i.e. the path to user's home directory):
```
get-sys environment "HOME" to define home_dir
```
Generally, environmental variable needs to be set before starting the vf program manager, or set during the program execution.
"web-environment" returns the name of a given environment variable <var name> from the web server, such as in the following example, the QUERY_STRING variable (i.e. the actual query string from URL):
```
get-sys web-environment "QUERY_STRING" to define home_dir
```
Note that the same name can be used for both Operating System environment variable (the "environment" clause) and for Web Server environment variable ("web-environment" clause), hence two separate clauses for them.
"directory" is the execution directory of the command-line program, i.e. the current working directory when the program was executed. Note that Vely will change the current working directory immediately afterwards to the application home directory (see how-vely-works). You can use this clause to work with files in the directory where the program was started. If your program runs as an application server (see FastCGI), then "directory" clause always returns application home directory, regardless of which directory vf program manager started your application.
"os-name" is the name of Operating System.
"os-version" is the version of Operating System.

Note that all the string values above should be treated as constants - do not change them as that may cause program to malfunction. If you want to alter any of these values, make a copy first (see copy-string).

Examples

Get the name of the Operating System

get-sys os-name to define os_name

See also

System information
get-sys
OS-conditional-compilation
See all
documentation

Get time

Purpose: Get time.

get-time to [ define ] <time var> \
    [ timezone <tz> ] \
    [ year <year> ] \
    [ month <month> ] \
    [ day <day> ] \
    [ hour <hour> ] \
    [ minute <minute> ] \
    [ second <second> ] \
    [ format <format> ]

get-time produces <time var> variable that contains string with time. <time var> is allocated memory.

If none of "year", "month", "day", "hour", "minute" or "second" clauses are used, then current time is produced. If variable <time var> does not exist, you can use define clause to create it within the statement.

Use timezone to specify that time produced will be in timezone <tz>. For example if <tz> is "EST", that means Eastern Standard Time, while "MST" means Mountain Standard Time. The exact way to get a full list of timezones recognized on your system may vary, but on many systems you can use:

timedatectl list-timezones

So for example to get the time in Phoenix, Arizona you could use "America/Phoenix" for <tz>. If timezone clause is omitted, then time is produced in "GMT" timezone by default. DST (Daylight Savings Time) is automatically adjusted.

Each variable specified with "year", "month", "day", "hour", "minute" or "second" is a time to be added or subtracted. For example "year 2" means add 2 years to the current data, and "year -4" means subtract 4 years, whereas "hour -4" means subtract 4 hours, etc. So for example, a moment in time that is 2 years into the future minus 5 days minus 1 hour is:

get-time to time_var year 2 day -5 hour -1

<format> allows you to get the time in any string format you like, using the specifiers available in C "strftime". For example, if <format> is "%A, %B %d %Y, %l:%M %p %Z", it will produce something like "Sunday, November 28 2021, 9:07 PM MST". The default format is "UTC/GMT" format, which for instance, is suitable for use with cookie timestamps, and looks something like "Mon, 16 Jul 2012 00:03:01 GMT".

Examples

To get current time in "GMT" timezone, in a format that is suitable for use with set-cookie (for example to set expiration date):

get-time to define mytime

To get the time in the same format, only 1 year and 2 months in the future:

get-time to define mytime year 1 month 2

An example of a future date (1 year, 3 months, 4 days, 7 hours, 15 minutes and 22 seconds into the future), in a specific format (see "strftime"):

get-time to time_var timezone "MST" year 1 month 3 day 4 hour 7 minute 15 second 22 format "%A, %B %d %Y, %l:%M %p %Z"

See also

Time
get-time
See all
documentation

Getting URL

To get the URL used to make a request, use get-sys statement with "web-environment" clause.

For a query string, get the "QUERY_STRING" environment variable. For a full URL path, combine "SCRIPT_NAME" and "PATH_INFO" variables.

To get a request body, use request-body statement.

Examples

To get the full URL path in variable "fpath", concatenate "SCRIPT_NAME" and "PATH_INFO":

get-sys web-environment "SCRIPT_NAME" to define sname
get-sys web-environment "PATH_INFO" to define pinfo
(( define fpath
    @<<p-out sname>><<p-out pinfo>>
))

See also

Get tree

Purpose: Get information about tree.

get-tree <tree> \
    ( count | hops ) \
    to [ define ] <result>

get-tree provides information about <tree> (created by new-tree), which is stored in <result> (in "to" clause):

"count" clause provides the number of keys (i.e. nodes) in the tree.
"hops" clause provides the number of nodes accessed to find a key in the last tree statement executed prior to get-tree. Note that "hops" is available only in debug Vely build (see installing from source); otherwise it is always zero.

<result> can be created with optional "define".

Examples
Get the number of nodes (keys) in a tree:
```
new-tree mytree
...
get-tree mytree count to define c
```
See also
Tree search
delete-tree
get-tree
new-tree
purge-tree
read-tree
use-cursor
write-tree
See all
documentation

Global process data
The purpose of global-process data is to help with use of global cross-request variable(s) anywhere in your code. The difference from global-request-data is in the scope: global request data is valid only for a single request, while global-process data is valid for all requests processed in sequence by a single process (see how-vely-works).

While you can create global variables in C (with C's "extern" to make them globally available), Vely's global-process data is an easier and more structured way to share data globally across requests. This way, you do not have to define and maintain global variables. It also makes it more maintainable as such variables are well-encapsulated and easy to track (for instance in code reviews).

What is global-process data
Global-process data is a generic pointer (void*) that points to any memory you wish to be shared among all your code across all the requests that any given process serves. This data cannot be shared between different processes. It is usually a structure containing information globally pertinent to your application that is shared between separate requests. The pointer's scope is a lifetime of the process and thus spans all the requests it serves. The global-process data pointer is initialized to NULL before startup-handler executes. It is stored in Vely's internal process context structure so it can be used anywhere.

Setting
```
set-app process-data <data>
```
where <data> is a pointer to any memory you wish to use anywhere in your code across all process' requests. This memory must be unmanaged; see memory-handling and the example below.

Getting
Global-process data can be obtained anywhere in your code with:
```
get-app process-data to <data>
```
where <data> is a pointer to any type.

Usage
Typically you would define a structure or a typedef that encapsulates any data that needs to be shared throughout your code in all requests in any given process. Different processes have separate process data that cannot be shared between them. For this reason, global-process data is not the same as global application data; rather it is global within any given process and not beyond that process.

In startup-handler, you would create a variable (or an array) of this type by using unmanaged new-mem that producess cross-request memory - this is done by using "off" clause in manage-memory. Use new-mem for any members that need allocating. Initialize anything that needs it. Finally, switch back to managed memory by using "on" clause in manage-memory statement.

The reason for using unmanaged memory is because otherwise the data would be automatically released at the end of the following request and would not stay allocated for the life of the process.

Next, save the pointer to the variable (or an array) you created by using set-app.

Finally, anywhere you need to set or get any data, use get-app with "process-data" clause to get the pointer and manipulate or read your global-process data. Don't forget that any manipulation must be in unmanaged mode as well.

Examples
Suppose your application has an include file my.h in which you define type "procdata":
```
#ifndef _MY
#define _MY

typedef struct s_procdata {
    bool some_flag;
    bool another_flag;
    char *ptr;
} procdata;

#endif
```
Your startup-handler (i.e. file _startup.vely) might look like this - note that my.h is included, providing your type definition "procdata":
```
#include "vely.h"
#include "my.h"

void _startup () {
    procdata *rd; // A pointer to global request data

    // Use unmanaged memory for process-wide data
    manage-memory off
    // Allocate global request data
    new-mem rd size sizeof(procdata)
    // Turn back on managed memory
    manage-memory on

    // Initialize values in it
    rd->some_flag = false;
    rd->another_flag = false;

    // Save the pointer so you can use it anywhere with get-req
    set-app process-data rd
}
```
In the above code, a new pointer "rd" of type "procdata" is created with new-mem. Data initialization takes place - anything that needs initialization should be initialized. Finally, pointer "rd" is saved to process' internal structure with set-app, so it can be used in any request for the life of the process.

In your code, wherever it's needed, you can obtain this data into a local pointer of the same type "procdata" (in this case pointer name is "mydata"). You can do that with get-app and then examine or set any global-process variable you wish:
```
#include "vely.h"
#include "my.h"

request-handler /mycode

    ...
    procdata *mydata; // declare local pointer 

    // get the actual value of a pointer, so now it points to global request data
    get-app process-data to mydata

    // do whatever you want with the data: examine, set etc.
    if (mydata->another_flag) {
        mydata->some_flag = true;
        // If you free, allocate or reallocate global-process data, use unmanaged memory
        manage-memory off
        resize-mem my_data->ptr size 1024
        manage-memory on
    }
end-request-handler
```
See also
Process
global-process-data
See all
documentation

Global request data
The purpose of global-request data is to facilitate using global variable(s) within a single request anywhere in your code. This is different from global-process-data which is used across many requests.

You can of course create global variables in C (and use C's "extern" to make them available anywhere). Vely's global-request data is an easier and more structured way to share data globally within a single request, without having to actually define and maintain global variables. It also makes it more maintainable because the usage of shared global information is well-encapsulated and easy to track (for instance in code reviews).

What is global-request data
Global-request data is a generic pointer (void*) that points to any memory you wish to be shared among all your code within a single request. This memory is usually a structure containing information globally pertinent to your application. The pointer's scope is a single request, and it is initialized to NULL before each request. It is stored in Vely's internal request structure so it can be used anywhere.

Setting
```
set-req data <data>
```
where <data> is a pointer to any memory you wish to use anywhere in your code.

Getting
Global-request data can be obtained anywhere in your code with:
```
get-req data to <data>
```
where <data> is a pointer to any type.

Usage
Typically you would define a structure or a typedef that encapsulates any data that needs to be shared throughout your code during a single request processing.

Then in before-request-handler, you would create the variable (or an array) of this type by using new-mem - this way the memory is automatically released at the end of the request. Use new-mem for any members that need allocating, thus eliminating the possibility of memory leaks. Initialize anything that needs it.

Next, save the pointer to the variable (or an array) you created by using set-req.

Finally, anywhere you need to set or get any data, use get-req to get the pointer and manipulate or read your global-request data.

Examples
Suppose your application has an include file my.h in which you define type "reqdata":
```
#ifndef _MY
#define _MY

typedef struct s_reqdata {
    bool some_flag;
    bool another_flag;
    char *ptr;
} reqdata;

#endif
```
Your before-request-handler might look like this - note that my.h is included, providing your type definition "reqdata":
```
#include "vely.h"
#include "my.h"

void _before () {
    reqdata *rd; // A pointer to global-request data

    // Allocate global-request data
    new-mem rd size sizeof(reqdata)

    // Initialize values in it
    rd->some_flag = false;
    rd->another_flag = false;

    // Save the pointer so you can use it anywhere with get-req
    set-req data rd
}
```
In the above code, a new pointer "rd" to type "reqdata" is created with new-mem. Data initialization takes place - anything that needs initialization should be initialized. Finally, pointer "rd" is saved to request's internal structure with set-req, so it can be used anywhere during request processing.

In your code, wherever it's needed, you can obtain this data into a local pointer of the same type "reqdata" (in this case pointer name is "mydata"). You can do that with get-req and then examine or set any global variable you wish:
```
#include "vely.h"
#include "my.h"

void mycode () {

    ...
    reqdata *mydata; // declare local pointer 

    // get the actual value of a pointer, so now it points to global-request data
    get-req data to mydata

    // do whatever you want with the data: examine, set etc.
    if (mydata->another_flag) {
        mydata->some_flag = true;
        my_data->ptr = "some data";
    }
}
```
See also
Requests
after-request-handler
before-request-handler
building-URL
getting-URL
global-request-data
non-request
normalized-URL
request
request-handler
request-URL
startup-handler
vely-dispatch-request
See all
documentation

Hash string
Purpose: Hash a string.
```
hash-string <string> to [ define ] <result> \
    [ binary [ <binary> ] [ output-length [ define ] <output length> ] \
    [ digest <digest algorithm> ]
```
hash-string produces by default a SHA256 hash of <string> (if "digest" clause is not used), and stores the result into <result> which can be created with optional "define". <result> is allocated memory. You can use a different <digest algorithm> in "digest" clause (for example "SHA3-256"). To see a list of available digests:
```
#get digests
openssl list -digest-algorithms
```
If "binary" clause is used without optional boolean expression <binary>, or if <binary> evaluates to true, then the <result> is a binary string that may contain null-characters. With the default SHA256, it is 32 bytes in length, while for instance with SHA3-384 it is 48 bytes in length, etc.

Without "binary" clause, or if <binary> evaluates to false, the <result> is null-terminated and each binary byte is converted to two hexadecimal characters ("0"-"9" and "a"-"f"), hence <result> is twice as long as with "binary" clause.

The actual length of <result> (regardless of whether "binary" clause is used or not) can be obtained in the optional "output-length" clause in <output length>, which can be created with optional "define".

Examples
String "hash" will have a hashed value of the given string, an example of which might look like "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855":
```
hash-string "hello world" to define hash
```
Using a different digest:
```
hash-string "hello world" to define hash digest "sha3-384"
```
Producing a binary value instead of a null-terminated hexadecimal string:
```
hash-string "hello world" to define hash digest "sha3-384" binary output-length define outlen
```
See also
Encryption
decrypt-data
derive-key
encrypt-data
hash-string
random-crypto
random-string
See all
documentation

How vely works
Creating an application
A Vely application <app name> is created by an application owner, i.e. an Operating System user who will own its processes and data, by using vf, such as:
```
sudo vf -i -u $(whoami) <app name>
```
This will create directory structure described here. Vely application can be used as a server (see FastCGI), a command-line program or a CGI program. Note that vf is the only Vely utility requiring sudo privileges, and only for application creation.

If you will build an application from source code, the above vf command must run in the directory that contains the application source code. Each application must be in its own separate directory which contains all of its source code.

How Vely works: code, requests, processes and main()
In a single step, Vely code (.vely files) is precompiled entirely into C, linked, and ready to run as a command-line program or as an application server, or both. Error reporting is based on Vely source code (.vely files), though you can specify generated code lines to be used (see diagnostic-messages).

vv will preprocess .vely source files and generate C code for supported statements (database, file, strings etc.). Then, vv builds both a command-line program and an application server executable for application <app name> (the name <app name> is specified when running vf with "-i" flag to create the application). Executables created are located in directory:
```
/var/lib/vv/bld/<app name>
```
A Vely program can be:
- A server that runs as a daemon, i.e. it runs in the background and does not exit, serving incoming requests. Typically, a number of server processes are started to serve incoming requests in parallel; each process serves a single request at a time and thus can serve great many requests during its lifetime; in adaptive mode, the number of processes running may vary according to the load - including zero running processes for near-zero memory usage.
- A command-line program that works exactly the same way as a server; it takes the same input and produces the same output. A command-line program runs as a single process and processes a single request.
- Both, meaning the same application can run as a server and as a command-line utility
Regardless, a request is processed by means of Vely dispatcher, which is extremely fast, because it uses a compile-time pre-built hash for near-instant dispatching of requests, specifically sized to the number of requests your application serves.

main() function (the entry point into your program) is generated so it can process requests fast, and in a uniform and consistent way. It calls the Vely dispatcher, sets up error handling, creates and releases program resources, sets up databases in use, etc. By doing so, it allows you to focus on application design and not the technical details, resource allocation and tuning performance.

.vely files and requests
Files with the .vely extension are Vely source files. You must have at least one Vely source file (.vely file). If a Vely source file starts with an underscore, it is a non-request file, which typically implements shared code used by more than one request; otherwise it is a request file. A request file implements a request handler, which handles a request-URL sent by the client (such as reverse proxies like Apache or Nginx or a FastCGI-API application written in any programming language) or provided to command-line program via environment variables.

.vely files for a program are always contained in a flat directory, meaning source code in any subdirectories is not used to make a program. This is by design for better readability and maintainability, as it allows for "at-a-glance" look at requests handled by the program and its overall functionality. Each request handler can be named as a hierarchical path, and have (sub)tasks within it (see below), for example if implementing a REST API.

Requests and request handlers
To call the request handler, state its name in the URL after the application path (see request-URL). A request handler must be implemented as a function in a .vely file named after it. For example, in a URL such as:
```
/trading/add-stock?ticker=ABC&price=130
```
the application path is "/trading" (by default it's the application name) and the request name is "add_stock" (hyphens are converted to underscore by Vely). The request name handler would be implemented in file "add_stock.vely" as:
```
request-handler /add-stock
    input-param ticker
    input-param price
    ...
end-request-handler
```
This correlation between requests and file names makes it easy to find the code you want and to maintain it. Vely will generate the code that will automatically route input requests to the appropriate code based on the request name, so in the above example Vely will automatically invoke function add_stock() implemented in file "add_stock.vely" with input parameters "ticker" and "price" (and their values) available via input-param statement, as shown above.

Note that you can also specify request name as a hierarchy path. For instance, if you wish to handle request "/stocks/add" (note that a hierarchical path starts and ends with an underscore):
```
/trading/_stocks/add_/?ticker=ABC&price=130
```
then you could implement a request handler:
```
request-handler /stocks/add
    input-param ticker
    input-param price
    ...
end-request-handler
```
In this case, forward slashes after the initial one would be substituted with double underscore for a file name, so the source Vely file name would be "stocks__add.vely". You can represent input parameters ("ticker" and "price" in this case) in many different ways, including as a hierarchical path:
```
/trading/_stocks/add_/ticker/ABC/price/130
```
See request-URL for more details.

Tasks
Each request handler can perform a number of tasks. A single invocation of a request may perform one or more tasks out of some number of choices. Tasks' functionality may or may not overlap, and they may have subtasks. To designate an input parameter as a task selector (i.e. as information that tells your code what task it's expected to do), use task-param. To use the task parameter as a selector and execute the appropriate code, see if-task. A request and optional tasks determine what kind of service your request should perform. Optional task-param and if-task statements provide semantic support for this, so that your code effectively self-documents its interface.

Non-request source code
Not every source code file in your application is meant to handle a request. You might have a number of non-request files that implement shared code, such as commonly used functions. Such files are non-request source code and their names must start with underscore ("_").

For example, file "_display_table.vely" might implement a common HTML table displaying code, which is used in various request handlers.

Some non-request source files have predefined meaning: "_startup.vely" (startup-handler), "_before.vely" (before-request-handler) and "_after.vely" (after-request-handler).

Building an application, Makefile
Vely builds an application by looking at .vely and .h files present in the current directory. Therefore you don't have to write a Makefile.

You don't have to specify what source code your application uses - all .vely files will be automatically picked up and compiled, and request handlers (.vely files that don't start with an underscore) are used to generate vely-dispatch-request.vely file (a request-dispatcher). Thus, it is easy to view at a glance what kind of requests an application serves and what is its general purpose. Files with .h extension are standard C include files and are automatically included.

You can see the default implementations for all auto-generated files in directory
```
/var/lib/vv/bld/<app name>
```
An example of building a Vely application is:
```
vv -q --db=mariadb:notes
```
"-q" options is the most important one - it specifies that you intend to build. In this instance, MariaDB database (named "notes") is used.

Vely builds an application using standard Linux "make" mechanism. Note that when you change the command line options (see vv) in a build statement like above, it has the effect of "make clean", recompiling source files.

Directories and files
While you can keep and compile Vely source files in any directory, the directories used by Vely are always under /var/lib/vv directory. A Vely application is always owned by a single Operating System user, while different applications can be owned by different users. This is the directory structure:
- /var/lib/vv/bld/<app name>
  Build directory is where your server application executable is built, with name <app name>.fcgi. The command-line executable is named <app name>. This is a scratch-pad directory, so do not alter files in it, or use for anything else.
- /var/lib/vv/<app name>
  Application directory. All application data, including internals such as sockets, are kept here. Each such directory is owned by its owner with 700 privileges, preventing other users from accessing its files.
- /var/lib/vv/<app name>/app
  Application home directory. This is the current working directory when your application runs. Copy/move here any files and directories your application needs.
- /var/lib/vv/<app name>/app/file
  file-storage used for uploads and new document creation. Do not write there directly; Vely does that for you, such as with uniq-file for instance, or when uploading files.
- /var/lib/vv/<app name>/app/trace
  Process tracing is written in per-process files if "--trace" option is used (see vv). Note that a special file "backtrace" is appended to when program crashes (or report-error statement is used), and this file shows a full source backtrace, allowing for easier debugging. In order to see Vely source code lines, you must use "--debug" option, and to see a backtrace, Vely must be built with debugging information included. You can use "--c-lines" in order to show the source code lines of the generated C code, instead of Vely code.
- /var/lib/vv/<app name>/app/db
  Database configuration directory contains the database-config-files used by Vely to connect to databases; this directory is maintained by Vely and you should not use it.
While Vely directories are fixed, you can effectively change their location by creating a soft link. This way, your directories and files can be elsewhere, even on a different disk. For example, to house your files on a different disk:
```
ln -s /home/disk0/file /var/lib/vv/<app name>/app/file
```
Remote calls
You can make remote requests, i.e. requests to other servers by using call-server statement, including unlimited number of parallel remote requests, meaning executing in separate threads at the same time without any thread programming on developer's part.

Because Vely processes that execute remote requests are kept alive, the cost of starting and terminating requests is zero. That's why this method of parallel distributed computing is very fast; it is also easier and safer because each request executed in parallel does not need to be MT-safe, since such requests execute in separate single-threaded processes. The cost of these processes vs threads is generally small on Linux and because Vely creates lightweight native executables.

To make remote calls across the Web, use call-web.

Memory allocation
Vely uses its own memory allocator based on standard Linux malloc family of functions. It is used internally and for allocating the results of any statements (each statement's documentation specifies which results, if any, are such). The memory will be automatically freed upon the completion of the request - you can also explicitly free them if you need to. This approach avoids costly errors, crashes and memory leaks. See memory-handling.

Data types
Vely statement-APIs use mostly strings (char* type only, i.e. no "const char*") and numbers ("num" type, which is "long long" for maximum range of uses, "num32" for a 32-bit integer, "dbl" for a double precision number), "bool" (with true and false value as provided by stdbool.h) and other data types as specified. When a "number" is specified anywhere without an additional qualification, it is assumed to be of "num" type. If a type is not specified, it is either a string or a number, depending on the context in which it is used - in those cases the type is not specified for simplicity.

Names of objects
Do not use object names (such as variables, functions, types) that start with "_vv", "_vely", "vv" or "vely" as those are reserved by Vely.

Libraries
You can use any external libraries with Vely (see "--lflag", "--cflag" with vv). For the implementation of its own functionality, Vely uses standard libraries wherever possible (such as cURL, OpenSSL, PostgreSQL, MariaDB, SQLite, pcre2 etc.). Those libraries are loaded when needed automatically; thus the run-time memory usage is minimized based on your program needs.

Containers (docker etc)
You can easily containerize-application.

Connect from other applications
Use FastCGI-API to connect to a Vely application server from any program, C or otherwise - as long as it has C linkage capability, which most languages do. This works with both single and multi-threaded applications.

Better error reporting and diagnostics
Vely provides enhanced diagnostic-messages which are color-coded for both Vely source code lines as well as generated C code, along with the messages from the C compiler; here is an example:

See also
General
about-Vely
application-architecture
deploying-application
how-vely-works
quality-control
rename-files
SELinux
vely-architecture
vely-removal
vely-version
vf
vv
See all
documentation

If task
Purpose: Select a request's task to process.
```
 if-task <task>
     <any code>
 [
 else-task <task>
     <any code>
 ] ...
 [
 else-task other
     <any code>
 ]
 end-task
```
if-task will compare a value from the last executed task-parameter with the list of string expressions, and execute code associated with the match.

If the value of task-param matches the string expression <task> in if-task, then <any code> below it is executed. If it is not a match, then the value of task-param is checked against values in "else-task" clauses, one by one until a match is found and code under that clause executes. If no match is found, then code under "else-task other" clause (if specified) is executed, otherwise program control passes outside of if-task statement.

In a given if-task statement, there can be only one "else-task other" clause, and it must be the last one in if-task statement.

For instance, a request named "customer" may be used to add a customer, update it, or delete it, and which one of these tasks is performed would be based on the value of a task-parameter, as in these URLs where task "act" can have values of "add", "update" and "delete":
```
http://web.site/app/customer/act/add/name/joe/email/joe@joe.com
http://web.site/app/customer/act/update/id/22/address/10+Main+Street
http://web.site/app/customer/act/delete/id/22
```
The request handler code in file "customer.vely" would have this code:
```
request-handler /customer
    out-header default

    // Specify input parameter "act" to be a task selector
    task-param act

    if-task "add"
        input-param name
        input-param email
        // Add customer
        ...
    else-task "update"
        input-param id
        input-param address
        // Update customer
        ...
    else-task "delete"
        input-param id
        // Delete customer
        ...
    else-task other
        @Unknown task.
    end-task
end-request-handler
```
Missing task parameters
Note that if there is no task-parameter (which is "act" in the above example), or if there is but such parameter is not present in request's input, the matched value in if-task will be an empty string (i.e. "").

Nesting and subtasks
if-task can be nested in case you want to change the task selector during the execution of a request handler, i.e. if you need to have sub-tasks. Such nesting can be up to 30 levels deep. In this example, a request "customer" has a task "act" with values of "add", "update" and "delete". A subtask "upd_type" (when task "act" has value of "update") has values of "address" and "phone":
```
request-handler /customer // request "customer"

    out-header default

    task-param act // task selector

    if-task "add" // task to add customer
        input-param name
        input-param email
        // Add customer
        ...
    else-task "update" // task to update customer
        input-param id
        task-param upd_type // subtask selector
        if-task "address" // subtask to update address
            input-param address
            // Update address
            ...
        else-task "phone" // subtask to update phone
            input-param phone
            // Update phone
            ...
        end-task
        ...
    else-task "delete" // task to delete customer
        input-param id
        // Delete customer
        ...
    else-task other
        @Unknown task.
    end-task
end-request-handler
```
See also
Request information
get-req
if-task
input-param
request-body
set-input
set-req
task-param
See all
documentation

Inline code
Purpose: Inline Vely code in string statements.
```
<<vely code>>
```
You can write Vely code within other Vely statements that build strings, whether for output (output-statement) or to build other strings (write-string), by using << and >> delimiters. There is no need for white space between the delimiters and Vely code, i.e. you could either write
```
<<p-web my_string>>
```
or
```
<<  p-web my_string  >>
```
to the same effect.

Examples
query-result statement displays the result of a query, and in the following code it's used to display results on the same line as other output (i.e. as inline):
```
run-query
   @<tr>
   @    <td>
   @        First name is << query-result  firstName >>
   @    </td>
   @    <td>
   @        Last name is << query-result# employees, lastName >>
   @    </td>
   @</tr>
end-query
```
In the code below, "some_function()" is a C function that uses Vely code to output some text, and it's used inline to output "Hello world":
```
@Hello <<.some_function();>>
```
(note the usage of dot statement to use any C expression, and finishing with semi-colon as a C statement). Function "some_function()" would simply output "world":
```
void some_function()
{
    @world
}
```
A write-string can be built with other Vely statements inlined, in this case we print the value of another string, resulting in "There is 42 minutes left!":
```
char * mins="42";
(( define my_string
There is <<p-out mins>> minutes left!
))
```
See also
Language
dot
inline-code
statement-APIs
syntax-highlighting
unused-var
See all
documentation

Input param
Purpose: Get input parameter for a request.
```
input-param <name>
```
To obtain input parameters from an incoming request, use input-param and/or task-param. Whether it's a web or command-line application, input parameters are specified as name/value pairs (see request-URL). The value of input parameter <name> will be in the string variable with the same name. Input parameter values are trimmed for whitespace (both on left and right).

Input parameter name can be made up of alphanumeric characters, hyphen or underscore only and cannot start with a digit. Note that a hyphen is automatically converted to underscore, so an input parameter "some-parameter" in the HTTP request will be known in your code as "some_parameter". For example in URL:
```
http://<your web server>/<app name>/some-request?my-par1=val1&my-par2=val2
```
the request name will be "some_request" (handled by some_request.vely source code), and input parameters will be "my_par1" and "my_par2":
```
input-param my_par1
input-param my_par2
```
File uploads
File uploads are handled as input parameters as well, see file-uploading.

Using input parameters
As an example, for HTML form input parameters named "param1" with value "value1" and "param2" with value "value2":
```
<input type='hidden' name='param1' value='value1'>
<input type='hidden' name='param2' value='value2'>
```
you can get these parameters and print out their values by using:
```
input-param param1
input-param param2
p-web param1
p-web param2
```
the output of which is:
```
value1
value2
```
A request may be in the form of a web link URL, and getting the parameter values is the same:
```
http://<your web server>/<app name>/<request name>&param1=value1&param2=value2
```
Duplicate parameter names
If there are multiple input parameters with the same name, such as
```
http://<web address>/<app name>/<request name>?par=val1&par=val2
```
the value of input parameter "par" is undefined. Do not specify multiple input parameters with the same name.

See also
Request information
get-req
if-task
input-param
request-body
set-input
set-req
task-param
See all
documentation

Json utf8
Purpose: Convert JSON text to UTF8 string.
```
json-utf8 <text> \
    [ status [ define ] <status> ] \
    [ error-text [ define ] <error text> ]
```
json-utf8 will convert JSON string value <text> to UTF8. <text> itself will hold the resulting UTF8 string. If you don't wish <text> to be modified, make a copy of it first (see copy-string). See utf8-json for the reverse conversion and data standards information.

You can obtain the optional <status> in "status" clause, which can be created with optional "define". <status> is VV_OKAY if successful, or VV_ERR_UTF8 if there was an error, in which case optional <error text> in "error-text" clause (which can be created with optional "define") will contain the error message.

Examples
```
// JSON Unicode value to encode to UTF8
char txt[] = "\u0459\\\"Doc\\\"\\n\\t\\b\\f\\r\\t\\u21d7\\u21d8\\t\\u25b7\\u25ee\\uD834\\uDD1E\\u13eb\\u2ca0\\u0448\\n\\/\\\"()\\t";

// Convert to UTF8 
json-utf8 txt status define txt_status error-text define txt_error

// Expected UTF8 result
char utf8[] = "љ\"Doc\"\n\t\b\f\r\t⇗⇘\t▷◮𝄞ᏫⲠш\n/\"()\t";

// Make sure conversion was successful
if (strcmp (utf8, txt) || txt_status != VV_OKAY || txt_error[0] != 0) {
    @Error in converting JSON string to UTF8
}
```
See also
UTF8
json-utf8
utf8-json
See all
documentation

License
Licensing and copyright
Vely is Free Open Source software licensed under Eclipse Public License 2.

Vely is copyright (c) 2019-now Dasoftver LLC.

Discussion
The following discussion is not legal advice.

Vely makes use of the following dynamically-linked libraries (with copyright by their respective owners), and only if your application actually uses them:
- LGPL-licensed MariaDB C connector (https://mariadb.com/kb/en/mariadb-connector-c/),
- Postgres-licensed Postgres C library (https://www.postgresql.org/docs/current/libpq.html),
- SQLite (public domain) SQLite C library (https://www.sqlite.org/docs.html),
- Curl-licensed Curl library (https://curl.se/),
- BSD-licensed PCRE2 (regex) library (https://www.pcre.org/),
- OpenSSL-licensed SSL/crypto library (https://www.openssl.org/).
- Open Market Licensed FastCGI library (https://fastcgi-archives.github.io/FastCGI_Developers_Kit_FastCGI.html).
These libraries are installed as Vely package dependencies by the Operating System packager (such as apt, dnf, pacman or zypper), but are not distributed with Vely. Vely does not link to any outside static libraries.

Vely uses FNV-1a hash function, which is released in the public domain (see wikipedia page) and is not patented (see Landon Noll's web page).

Vely source code uses SPDX, an open ISO standard for communicating sofware bill of material information (such as licenses), to improve on open source licensing compliance for companies and communities.

With regards to any application written using Vely, the prevailing (the most restrictive, i.e. final) license, from the Open Source License Compliance perspective, is the EPL-2 license.

If you change Vely code itself (i.e. create a derivative work) and distribute it, then the result of such changes must be made open source under the EPL-2 license. Note that code generated by Vely is not considered derivative work.

EPL-2 is know as "business friendly" license that makes it easy and straightforward to use Vely in commercial applications. If you are a software developer using Vely to write any applications, commercial or otherwise, you do not have to release your source code.

For a complete list of requirements, please refer to the EPL-2 license.

See also
License
license
See all
documentation

Lock file
Purpose: Locks file exclusively.
```
lock-file <file path> id [ define ] <lock id> status [ define ] <status>
```
lock-file attempts to create a file (deleting all contents of the previous file, if existed), and exclusively lock it. If successful, no other process can do the same unless current process ends or calls unlock-file. This statement is non-blocking, thus you can check in a sleepy loop for success.

<file path> should be either an existing or non-existing file with a valid file path.

<lock id> (in "id" clause) is a file descriptor associated with locked file.

<status> (in "status" clause) represents the status of file locking: VV_OKAY if successfully locked, VV_ERR_FAILED if cannot lock (likely because another process holds a lock), VV_ERR_INVALID if the path of the file is invalid (i.e. if it is empty), VV_ERR_CREATE if lock file cannot be created.

Generally, this statement is used for process control. A process would use lock-file to start working on a job that only one process can do at a time; o