Example: file manager

This is a file manager application.

Files such as JPG, PDF or other can be uploaded, and each file can have a description tag.
A list of uploaded files can be displayed.
Each file can be viewed or downloaded, and each file can be deleted.

Files metadata is kept in the database (name, path, description, size, extension), while the files themselves are kept in Vely file_storage.

In a nutshell: 6 source files, 169 lines of code; PostgreSQL; web browser; Apache; TCP sockets;

Screenshots of application

This is the page where application user always goes first. Here, your application will display a web link that shows all files kept in the database so far, as well as HTML form to upload a new file where you can enter its description and choose a file from your computer or device. Here's a snapshot of what the page will look like - this is a basic layout and you can change the code to style it however you wish:

Files that have been saved will be shown in a list. Next to each file name will be its size, and links to show the file (display it or download it if can't be displayed), and a link to delete the file:

When a file is about to be deleted, a confirmation is asked of user:

Once confirmed, deletion of a file is carried out:

Setup prerequisites

Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.

Because they are used in this example, you will need to install Apache as a web server and PostgreSQL as a database.

After installing Vely, turn on syntax highlighting in vim if you're using it:

vv -m

Get the source code

The source code is a part of Vely installation. It is a good idea to create a separate source code directory for each application (and you can name it whatever you like). In this case, unpacking the source code will do that for you:

tar xvf $(vv -o)/examples/file_manager.tar.gz
cd file_manager

Setup application

The very first step is to create an application. The application will be named "file_manager", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:

sudo vf -i -u $(whoami) file_manager

This will create a new application home (which is "/var/lib/vv/file_manager") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.

Setup the database

Before any coding, you need some place to store the information used by the application. First, you will create PostgreSQL database "db_file_manager". You can change the database name, but remember to change it everywhere here. And then, you will create database objects in the database.

Note the example here uses peer-authentication, which is the default on all modern PostgreSQL installations - this means the database user name is the same as the Operating System user name.

Execute the following in PostgreSQL database as root (using psql utility):

echo "create user $(whoami);
create database db_file_manager with owner=$(whoami);
grant all on database db_file_manager to $(whoami);
\q
" | sudo -u postgres psql

Next, login to database db_file_manager and create the database objects for the application:

psql -d db_file_manager -f setup.sql

Connect Vely to a database

In order to let Vely know where your database is and how to log into it, you will create database_config_file named "db_file_manager". This name doesn't have to be "db_file_manager", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:

echo "user=$(whoami) dbname=db_file_manager"  > db_file_manager

The above is a standard postgres connection string that describes the login to the database you created. Since Vely uses native PostgreSQL connectivity, you can specify any connection string that your database lets you.

Build application

Use vv utility to make the application:

vv -q --db=postgres:db_file_manager

Note usage of --db option to specify PostgreSQL database and the database configuration file name.

Start your application server

To start the application server for your web application use vf FastCGI process manager. The application server will use a TCP socket 2310 to communicate with the web server (i.e. a reverse-proxy).

vf -p 2310 -w 3 file_manager

This will start 3 daemon processes to serve the incoming requests. You can also start an adaptive server that will increase the number of processes to serve more requests, and gradually reduce the number of processes when they're not needed:

vf -p 2310 file_manager

See vf for more options to help you achieve best performance.

To stop your application server:

vf -m quit file_manager

- SELinux and TCP ports
If you are using SELinux and have it enabled, then you must make the TCP socket accessible to the web server and Vely:

sudo semanage port -a -t vvport_t -p tcp 2310

Without this step, the application will appear to not work.

Setup web server

This shows how to connect your application listening at TCP port 2310 (started with "-p" option in vf) to Apache web server.

- Step 1:
To setup Apache as a reverse proxy and connect your application to it, you need to enable FastCGI proxy support, which generally means "proxy" and "proxy_fcgi" modules - this is done only once:

For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
```
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
```
For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
For OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```
Add this to the end of the file:
```
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
```

- Step 2:
Edit the Apache configuration file:

For Debian (such as Ubuntu):
```
sudo vi /etc/apache2/apache2.conf
```
for Fedora (such as RedHat), Archlinux:
```
sudo vi /etc/httpd/conf/httpd.conf
```
and for OpenSUSE:
```
sudo vi /etc/apache2/httpd.conf
```

Add this to the end of file ("/file_manager" is the application path, see request_URL):

ProxyPass "/file_manager" fcgi://127.0.0.1:2310/

- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:

sudo systemctl restart apache2

On Fedora systems (like RedHat) and Arch Linux:

sudo systemctl restart httpd

Note: you must not have any other URL resource that starts with "/file_manager" (such as for example "/file_manager.html" or "/file_manager_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/file_manager", see request_URL.

Access application server from the browser

Use the following URL(s) to access your application server from a client like browser (see request_URL). Use actual IP or web address instead of 127.0.0.1 if different.

#Start the file management application 
http://127.0.0.1/file_manager/start

Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.

Files

You are now done with the example! What follows are the source files in this project so you can examine how it works:

Storing file information in the database (setup.sql)

The information about files, such as file names, sizes, extensions, descriptions etc. is stored in the table "files". Where will the files themselves be stored?

Well, you could store them in the database, but that's not the most optimal way. You could store them anywhere on a file system, but Vely has a very simple (and generally transparent and automatic) way of storing files, which is Vely's file_storage. When a file is uploaded, it's automatically stored there; and you can also create files there as well.

create table if not exists files (fileName varchar(100), localPath varchar(300), extension varchar(10), description varchar(200), fileSize int, fileID bigserial primary key);

Application landing page (start.vely)

This code displays the page where application user always goes first. One thing you'll notice often in Vely code is the use of "@" to output text (in this case HTML page text) - "@" is an output_statement, and it makes it easy and readable. In general, Vely does not use traditional API for its functionality, rather it uses statement_APIs.

// SPDX-License-Identifier: EPL-2.0
// Copyright 2018 DaSoftver LLC.
#include "vely.h" // must always be here

// Upload and list/download files
void start ()
{

   out-header default

   @<h2>File Manager</h2>

   @To manage the uploaded files, <a href="<<p-path>>?req=list">click here.</a><br/>
   @<br/>

   // Form to upload a file

   @<form action="<<p-path>>" method="POST" enctype="multipart/form-data">
   @    <input type="hidden" name="req" value="upload">

   @    <label for="file_description">File description:</label><br>
   @    <textarea name="filedesc" rows="3" columns="50"></textarea><br/>
   @    <br/>
   @    <label for="filename">File:</label>
   @    <input type="file" name="file" value=""><br><br>

   @    <input type="submit" value="Submit">
   @</form>

}

Uploading files (upload.vely)

The next source code file is "upload.vely", which will upload a file when a user selects a file and clicks "Submit" button, which is in "start.vely" file. When you upload a file into Vely application, it's just an input-parameter, just like it would be with any GET or POST request parameter. In addition, Vely will automatically store the uploaded file into file directory (see file_storage) which is optimized for speed of access, and you can either keep it there, move it elsewhere or do whatever you want. All of this make is easy to work with files.

The above code will get the information about the file uploaded (such as file description, as well as file name, location, size, extension) - which is provided by Vely as input parameters. Then, use run-query statement to insert such information into the database table you created. From now on, the information about a file is stored in the database (and that includes the location where it was uploaded), and the actual file is in the file system. This allows fast access to files and easier management.

And finally, the message will be sent back to web client (such as a browser) that you've saved the file.

// SPDX-License-Identifier: EPL-2.0
// Copyright 2018 DaSoftver LLC.
#include "vely.h" // must always be here

// Upload the file
void upload ()
{
   out-header default

   // file description from the upload form
   input-param filedesc
   // file name
   input-param file_filename
   // the path to uploaded file
   input-param file_location
   // size in bytes
   input-param file_size
   // the file extension
   input-param file_ext
   VV_UNUSED (file_ext);

   @<h2>Uploading file</h2>

   // insert the information about the file into the database
   run-query @db_file_manager="insert into files (fileName, localPath, extension, description, fileSize) values ('%s', '%s', '%s', '%s', '%s')": file_filename, file_location, file_ext, filedesc, file_size
   end-query

   @File <<p-web file_filename>> of size <<p-web file_size>> is stored on server at <<p-web file_location>>. File description is <<p-web filedesc>>.<hr/>
}

Listing files (list.vely)

For files that have been saved, this code shows them in a list. Next to each file name will be its size, and links to show the file (display it or download it if can't be displayed), and a link to delete the file. To produce the list of files, the table "files" is queried, and the list of files displayed as an HTML table.

// SPDX-License-Identifier: EPL-2.0
// Copyright 2018 DaSoftver LLC.
#include "vely.h" // must always be here

// List files
void list ()
{
   // List current files in the database
   out-header default
   @<h2>List of files</h2>
   @To add a file, <a href="<<p-path>>?req=start">click here</a><br/><br/>
   @<table border="1">
   @<tr>
   @    <td>File</td><td>Description</td><td>Size</td><td>Show</td><td>Delete</td>
   @</tr>

   // get the list of files from the database
   run-query @db_file_manager="select fileName, description, fileSize, fileID from files order by fileSize desc" output fileName, description, fileSize, fileID
       query-result fileName to define file_name
       query-result fileID to define file_ID
       query-result description to define description
       query-result fileSize to define file_size
       // construct table output with links to Show and Delete files
       @<tr>
       @    <td><<p-web file_name>></td><td><<p-web description>><td><<p-web file_size>></td>
       @    <td><a href="<<p-path>>?req=download&amp;file_id=<<p-url file_ID>>">Show</a></td>
       @    <td><a href="<<p-path>>?req=delete&amp;action=confirm&amp;file_id=<<p-url file_ID>>">Delete</a></td>
       @</tr>
   end-query
   @</table>
}

Downloading files (download.vely)

When a link to download a file is clicked in the list of files, the following code is called. The ID of a file requested is queried in table "files", the file path on server is found, and the file is sent to the client using send-file statement.

// Copyright 2018 DaSoftver LLC.
#include "vely.h" // must always be here

// Download a file
void download ()
{
   // Show or download a file (its ID is in the database)
   input-param file_id
   char *local_path=NULL;
   char *ext = NULL;

   // get the local path and extension of the file
   run-query @db_file_manager="select localPath,extension from files where fileID='%s'" output localPath, extension : file_id row-count define num_files
       query-result  localPath to local_path
       query-result  extension to ext
   end-query

   // check we can find the file
   if (num_files != 1) {
       out-header default
       @Cannot find this file!<hr/>
       return;
   }

   // display JPG or PDF files in the browser, or download any other kind
   if (!strcmp (ext, ".jpg")) {
       send-file local_path headers content-type "image/jpg"
   } else if (!strcmp (ext, ".pdf")) {
       send-file local_path headers content-type "application/pdf"
   } else {
       send-file local_path headers content-type "application/octet-stream" download
   }
}

Deleting files (delete.vely)

Finally, deleting a file (by clicking on delete link in the list of files) is handled in the following code.

When a file is about to be deleted, a confirmation is asked of user - input parameter "action" is "confirm" in this case.

Once confirmed, deletion of a file is carried out; input parameter "action" is "delete" then (you can name parameter "action" anything you want).

Note that deletion is done as a transaction, only if both the record in table "files" is deleted and the actual file deleted. If the file cannot be deleted, transaction is rollback-ed.

// SPDX-License-Identifier: EPL-2.0
// Copyright 2018 DaSoftver LLC.
#include "vely.h" // must always be here

// Delete the file
void delete ()
{
   out-header default
   @<h2>Delete a file</h2>
   input-param action
   input-param file_id
   char *file_name = NULL;
   char *desc = NULL;
   char *local_path = NULL;

   // Get file information from the database
   run-query @db_file_manager="select fileName, localPath, description  from files where fileID='%s'" output fileName, localPath, description : file_id
       query-result fileName to file_name
       query-result description to desc
       query-result localPath to local_path
   end-query

   if (!strcmp (action, "confirm")) { // get file information to confirm what will be deleted
       @Are you sure you want to delete file <<p-web file_name>> (<<p-web desc>>)? Click <a href="<<p-path>>?req=delete&amp;action=delete&amp;file_id=<<p-url file_id>>">Delete</a> or click the browser's Back button to go back.<br/>

   } else if (!strcmp (action, "delete")) { // actual delete file, once confirmed
       begin-transaction @db_file_manager
       run-query @db_file_manager= "delete from files where fileID='%s'" : file_id error define err no-loop
       if (atol(err) != 0) {
           @Could not delete the file (error <<p-web err>>)
           rollback-transaction @db_file_manager
       } else {
           delete-file local_path status define st
           if (st == VV_OKAY) {
               commit-transaction @db_file_manager
               @File deleted. Go back to <a href="<<p-path>>?req=start">start page</a>
           } else {
               rollback-transaction @db_file_manager
               @File could not be deleted, error <<pf-web "%lld", st>>
           }
       }
   } else {
       @Unrecognized action <<p-web action>>
   }
}