Quality control

Quality control

Each candidate for a Vely release must build successfully for a given Linux distribution and on its native architecture, without simulators or cross-compilers. A suite of functional tests (including bug fixes to prevent regressions) must run without errors; currently, there are 1092 such tests.

Vely is tested on following distros: archlinux_rolling, debian_10, debian_11, fedora_35, fedora_36, mageia_8, opensuse_tumbleweed, redhat_8, redhat_9, ubuntu_18, ubuntu_20, ubuntu_22 however it may work on other distros derived from this list (see installation page). Both builds and tests are automated to prevent gaps in coverage.

Vely documentation is automated to a good degree, with internal/external link checking, HTML/troff make and reference/index building.

Test Vely

If you are building Vely from source, or wanting to get a degree of certainty about Vely functioning properly, you can run Vely tests. These are feature and regression tests located in func_test directory within Vely source code.

To get source code for functional tests:

tar xvf $(vv -o)/func_test/func_test.tar.gz
cd func_test

To run tests, you must have installed the web servers and databases used (Apache, Nginx, MariaDB, PostgreSQL, SQLite), lsof, as well as Docker (Ubuntu only). You must export the following environment variables for your system:

#MariaDB root database password, specify empty string for passwordless
export DBROOTPWD="..."

#Apache configuration file location, given here is a typical Debian location
export VV_APACHE_CONFIG="/etc/apache2/apache2.conf"

#Apache service name, given hee is a typical Debian name
export VV_APACHE_SERVICE="apache2"

#Nginx configuration directory
export VV_NGINX_CONFIG_DIR="/etc/nginx"

#Profile with valgrind (must be installed), 1 (profile), anything else (do not profile)
export VV_VALGRIND=""

#Include detailed valgrind leak info, 1 (include), anything else (do not include)
export VV_VALGRIND_LEAK=""

#Include verbose valgrind info, 1 (include), anything else (do not include)
export VV_VALGRIND_VERBOSE=""

#Architecture (use "aarch64" for an ARM 64-bit processor)
export VV_ARCH="x86_64"

After that you may run tests with:

./ctest 0

The output generally shows the tests completed, with "VELERROR" message prior to completion of a test if there was an issue.

Note that your testing environment should have Internet connection and be preferrably a clean-slate Operating System installation that is easily recyclible, such as a disposable cloned VM.  

If you are using VV_VALGRIND* variables, valgrind must be installed. Generally, valgrind run should be manually inspected as actual issues are sometimes not easy to recognize from false positives or other acceptable items. The result of profiling is in file "valrep".

VV_ARCH specifies the architecture (currently x86_64 or aarch64). While Vely runs the same on all architectures, some third party software may not, and this environment variable is generally used to control those aspects of testing.

General ( deploying_application   how_vely_works   quality_control   rename_files   SELinux   vely   vely_architecture   vely_removal   vf   vv   why_Vely  )  SEE ALL (documentation)