How vely works

18.4.0 released Sep 25, 2023

How vely works

Creating an application

A Vely application <app name> is created by an application owner, i.e. an Operating System user who will own its processes and data, by using vf, such as:

sudo vf -i -u $(whoami) <app name>

This will create directory structure described here. Vely application can be used as a server (see FastCGI), a command-line program or a CGI program. Note that vf is the only Vely utility requiring sudo privileges, and only for application creation.

If you will build an application from source code, the above vf command must run in the directory that contains the application source code. Each application must be in its own separate directory which contains all of its source code.

How Vely works: code, requests, processes and main()

In a single step, Vely code (.vely files) is precompiled entirely into C, linked, and ready to run as a command-line program or as an application server, or both. Error reporting is based on Vely source code (.vely files), though you can specify generated code lines to be used (see diagnostic-messages).

vv will preprocess .vely source files and generate C code for supported statements (database, file, strings etc.). Then, vv builds both a command-line program and an application server executable for application <app name> (the name <app name> is specified when running vf with "-i" flag to create the application). Executables created are located in directory:

/var/lib/vv/bld/<app name>

A Vely program can be:

A server that runs as a daemon, i.e. it runs in the background and does not exit, serving incoming requests. Typically, a number of server processes are started to serve incoming requests in parallel; each process serves a single request at a time and thus can serve great many requests during its lifetime; in adaptive mode, the number of processes running may vary according to the load - including zero running processes for near-zero memory usage.
A command-line program that works exactly the same way as a server; it takes the same input and produces the same output. A command-line program runs as a single process and processes a single request.
Both, meaning the same application can run as a server and as a command-line utility

Regardless, a request is processed by means of Vely dispatcher, which is extremely fast, because it uses a compile-time pre-built hash for near-instant dispatching of requests, specifically sized to the number of requests your application serves.

main() function (the entry point into your program) is generated so it can process requests fast, and in a uniform and consistent way. It calls the Vely dispatcher, sets up error handling, creates and releases program resources, sets up databases in use, etc. By doing so, it allows you to focus on application design and not the technical details, resource allocation and tuning performance.

.vely files and requests

Files with the .vely extension are Vely source files. You must have at least one Vely source file (.vely file). If a Vely source file starts with an underscore, it is a non-request file, which typically implements shared code used by more than one request; otherwise it is a request file. A request file implements a request handler, which handles a request-URL sent by the client (such as reverse proxies like Apache or Nginx or a FastCGI-API application written in any programming language) or provided to command-line program via environment variables.

.vely files for a program are always contained in a flat directory, meaning source code in any subdirectories is not used to make a program. This is by design for better readability and maintainability, as it allows for "at-a-glance" look at requests handled by the program and its overall functionality. Each request handler can be named as a hierarchical path, and have (sub)tasks within it (see below), for example if implementing a REST API.

Requests and request handlers

To call the request handler, state its name in the URL after the application path (see request-URL). A request handler must be implemented as a function in a .vely file named after it. For example, in a URL such as:

/trading/add-stock?ticker=ABC&price=130

the application path is "/trading" (by default it's the application name) and the request name is "add_stock" (hyphens are converted to underscore by Vely). The request name handler would be implemented in file "add_stock.vely" as:

request-handler /add-stock
    input-param ticker
    input-param price
    ...
end-request-handler

This correlation between requests and file names makes it easy to find the code you want and to maintain it. Vely will generate the code that will automatically route input requests to the appropriate code based on the request name, so in the above example Vely will automatically invoke function add_stock() implemented in file "add_stock.vely" with input parameters "ticker" and "price" (and their values) available via input-param statement, as shown above.

Note that you can also specify request name as a hierarchy path. For instance, if you wish to handle request "/stocks/add" (note that a hierarchical path starts and ends with an underscore):

/trading/_stocks/add_/?ticker=ABC&price=130

then you could implement a request handler:

request-handler /stocks/add
    input-param ticker
    input-param price
    ...
end-request-handler

In this case, forward slashes after the initial one would be substituted with double underscore for a file name, so the source Vely file name would be "stocks__add.vely". You can represent input parameters ("ticker" and "price" in this case) in many different ways, including as a hierarchical path:

/trading/_stocks/add_/ticker/ABC/price/130

See request-URL for more details.

Tasks

Each request handler can perform a number of tasks. A single invocation of a request may perform one or more tasks out of some number of choices. Tasks' functionality may or may not overlap, and they may have subtasks. To designate an input parameter as a task selector (i.e. as information that tells your code what task it's expected to do), use task-param. To use the task parameter as a selector and execute the appropriate code, see if-task. A request and optional tasks determine what kind of service your request should perform. Optional task-param and if-task statements provide semantic support for this, so that your code effectively self-documents its interface.

Non-request source code

Not every source code file in your application is meant to handle a request. You might have a number of non-request files that implement shared code, such as commonly used functions. Such files are non-request source code and their names must start with underscore ("_").

For example, file "_display_table.vely" might implement a common HTML table displaying code, which is used in various request handlers.

Some non-request source files have predefined meaning: "_startup.vely" (startup-handler), "_before.vely" (before-request-handler) and "_after.vely" (after-request-handler).

Building an application, Makefile

Vely builds an application by looking at .vely and .h files present in the current directory. Therefore you don't have to write a Makefile.

You don't have to specify what source code your application uses - all .vely files will be automatically picked up and compiled, and request handlers (.vely files that don't start with an underscore) are used to generate vely-dispatch-request.vely file (a request-dispatcher). Thus, it is easy to view at a glance what kind of requests an application serves and what is its general purpose. Files with .h extension are standard C include files and are automatically included.

You can see the default implementations for all auto-generated files in directory

/var/lib/vv/bld/<app name>

An example of building a Vely application is:

vv -q --db=mariadb:notes

"-q" options is the most important one - it specifies that you intend to build. In this instance, MariaDB database (named "notes") is used.

Vely builds an application using standard Linux "make" mechanism. Note that when you change the command line options (see vv) in a build statement like above, it has the effect of "make clean", recompiling source files.

Directories and files

While you can keep and compile Vely source files in any directory, the directories used by Vely are always under /var/lib/vv directory. A Vely application is always owned by a single Operating System user, while different applications can be owned by different users. This is the directory structure:

/var/lib/vv/bld/<app name>
Build directory is where your server application executable is built, with name <app name>.fcgi. The command-line executable is named <app name>. This is a scratch-pad directory, so do not alter files in it, or use for anything else.
/var/lib/vv/<app name>
Application directory. All application data, including internals such as sockets, are kept here. Each such directory is owned by its owner with 700 privileges, preventing other users from accessing its files.
/var/lib/vv/<app name>/app
Application home directory. This is the current working directory when your application runs. Copy/move here any files and directories your application needs.
/var/lib/vv/<app name>/app/file
file-storage used for uploads and new document creation. Do not write there directly; Vely does that for you, such as with uniq-file for instance, or when uploading files.
/var/lib/vv/<app name>/app/trace
Process tracing is written in per-process files if "--trace" option is used (see vv). Note that a special file "backtrace" is appended to when program crashes (or report-error statement is used), and this file shows a full source backtrace, allowing for easier debugging. In order to see Vely source code lines, you must use "--debug" option, and to see a backtrace, Vely must be built with debugging information included. You can use "--c-lines" in order to show the source code lines of the generated C code, instead of Vely code.
/var/lib/vv/<app name>/app/db
Database configuration directory contains the database-config-files used by Vely to connect to databases; this directory is maintained by Vely and you should not use it.

While Vely directories are fixed, you can effectively change their location by creating a soft link. This way, your directories and files can be elsewhere, even on a different disk. For example, to house your files on a different disk:

ln -s /home/disk0/file /var/lib/vv/<app name>/app/file

Remote calls

You can make remote requests, i.e. requests to other servers by using call-server statement, including unlimited number of parallel remote requests, meaning executing in separate threads at the same time without any thread programming on developer's part.

Because Vely processes that execute remote requests are kept alive, the cost of starting and terminating requests is zero. That's why this method of parallel distributed computing is very fast; it is also easier and safer because each request executed in parallel does not need to be MT-safe, since such requests execute in separate single-threaded processes. The cost of these processes vs threads is generally small on Linux and because Vely creates lightweight native executables.

To make remote calls across the Web, use call-web.

Memory allocation

Vely uses its own memory allocator based on standard Linux malloc family of functions. It is used internally and for allocating the results of any statements (each statement's documentation specifies which results, if any, are such). The memory will be automatically freed upon the completion of the request - you can also explicitly free them if you need to. This approach avoids costly errors, crashes and memory leaks. See memory-handling.

Data types

Vely statement-APIs use mostly strings (char* type only, i.e. no "const char*") and numbers ("num" type, which is "long long" for maximum range of uses, "num32" for a 32-bit integer, "dbl" for a double precision number), "bool" (with true and false value as provided by stdbool.h) and other data types as specified. When a "number" is specified anywhere without an additional qualification, it is assumed to be of "num" type. If a type is not specified, it is either a string or a number, depending on the context in which it is used - in those cases the type is not specified for simplicity.

Names of objects

Do not use object names (such as variables, functions, types) that start with "_vv", "_vely", "vv" or "vely" as those are reserved by Vely.

Libraries

You can use any external libraries with Vely (see "--lflag", "--cflag" with vv). For the implementation of its own functionality, Vely uses standard libraries wherever possible (such as cURL, OpenSSL, PostgreSQL, MariaDB, SQLite, pcre2 etc.). Those libraries are loaded when needed automatically; thus the run-time memory usage is minimized based on your program needs.

Containers (docker etc)

You can easily containerize-application.

Connect from other applications

Use FastCGI-API to connect to a Vely application server from any program, C or otherwise - as long as it has C linkage capability, which most languages do. This works with both single and multi-threaded applications.

Better error reporting and diagnostics

Vely provides enhanced diagnostic-messages which are color-coded for both Vely source code lines as well as generated C code, along with the messages from the C compiler; here is an example: