18.4.0 released Sep 25, 2023
|
Web file manager in 160 lines of code
Uploading and downloading files is one of the most common tasks in web applications. This article shows how to build a file manager application in about 160 lines of code.
Files such as JPG, PDF or other can be uploaded, and each file can have a description tag. A list of uploaded files can be displayed. Each file can be viewed or downloaded, and each file can be deleted. Files metadata is kept in the database (name, path, description, size, extension), while the files themselves are kept in Vely file-storage.
In a nutshell: PostgreSQL; web browser; Apache; TCP sockets; 6 source files, 165 lines of code.
Screenshots of application
This is the page where application user always goes first. Here, your application will display a web link that shows all files kept in the database so far, as well as HTML form to upload a new file where you can enter its description and choose a file from your computer or device. Here's a snapshot of what the page will look like - this is a basic layout and you can change the code to style it however you wish:
Files that have been saved will be shown in a list. Next to each file name will be its size, and links to show the file (display it or download it if can't be displayed), and a link to delete the file:
When a file is about to be deleted, a confirmation is asked of user:
Once confirmed, deletion of a file is carried out:
Install Vely - you can use standard packaging tools such as apt, dnf, pacman or zypper.
Because they are used in this example, you will need to install Apache as a web server and PostgreSQL as a database.
After installing Vely, turn on syntax highlighting in vim if you're using it:
The source code is a part of Vely installation. It is a good idea to create a separate source code directory for each application (and you can name it whatever you like). In this case, unpacking the source code will do that for you:
tar xvf $(vv -o)/examples/file-manager.tar.gz
cd file-manager
The very first step is to create an application. The application will be named "file-manager", but you can name it anything (if you do that, change it everywhere). It's simple to do with vf:
sudo vf -i -u $(whoami) file-manager
This will create a new application home (which is "/var/lib/vv/file-manager") and do the application setup for you. Mostly that means create various subdirectories in the home folder, and assign them privileges. In this case only current user (or the result of "whoami" Linux command) will own those directories with 0700 privileges; it means a secure setup.
Before any coding, you need some place to store the information used by the application. First, you will create PostgreSQL database "db_file_manager". You can change the database name, but remember to change it everywhere here. And then, you will create database objects in the database.
Note the example here uses peer-authentication, which is the default on all modern PostgreSQL installations - this means the database user name is the same as the Operating System user name.
Execute the following in PostgreSQL database as root (using psql utility):
echo "create user $(whoami);
create database db_file_manager with owner=$(whoami);
grant all on database db_file_manager to $(whoami);
\q
" | sudo -u postgres psql
Next, login to database db_file_manager and create the database objects for the application:
psql -d db_file_manager -f setup.sql
Connect Vely to a database
In order to let Vely know where your database is and how to log into it, you will create database-config-file named "db_file_manager". This name doesn't have to be "db_file_manager", rather it can be anything - this is the name used in actual database statements in source code (like run-query), so if you change it, make sure you change it everywhere. Create it:
echo "user=$(whoami) dbname=db_file_manager" > db_file_manager
The above is a standard postgres connection string that describes the login to the database you created. Since Vely uses native PostgreSQL connectivity, you can specify any connection string that your database lets you.
Use vv utility to make the application:
vv -q --db=postgres:db_file_manager
Note usage of --db option to specify PostgreSQL database and the database configuration file name.
Start your application server
To start the application server for your web application use vf FastCGI process manager. The application server will use a TCP socket 2310 to communicate with the web server (i.e. a reverse-proxy).
vf -p 2310 -w 3 file-manager
This will start 3 daemon processes to serve the incoming requests. You can also start an adaptive server that will increase the number of processes to serve more requests, and gradually reduce the number of processes when they're not needed:
See vf for more options to help you achieve best performance.
If you want to stop your application server:
- SELinux and TCP ports
If you are using SELinux and have it enabled, then you must make the TCP socket accessible to the web server and Vely:
sudo semanage port -a -t vvport_t -p tcp 2310
Without this step, the application will appear to not work.
This shows how to connect your application listening at TCP port 2310 (started with "-p" option in vf) to Apache web server.
- Step 1:
To setup Apache as a reverse proxy and connect your application to it, you need to enable FastCGI proxy support, which generally means "proxy" and "proxy_fcgi" modules - this is done only once:
- For Debian (like Ubuntu) and OpenSUSE systems you need to enable proxy and proxy_fcgi modules:
sudo a2enmod proxy
sudo a2enmod proxy_fcgi
- For Fedora systems (or others like Archlinux) enable proxy and proxy_fcgi modules by adding (or uncommenting) LoadModule directives in the Apache configuration file - the default location of this file on Linux depends on the distribution. For Fedora (such as RedHat), Archlinux:
sudo vi /etc/httpd/conf/httpd.conf
For OpenSUSE:
sudo vi /etc/apache2/httpd.conf
Add this to the end of the file:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
- Step 2:
Edit the Apache configuration file:
- For Debian (such as Ubuntu):
sudo vi /etc/apache2/apache2.conf
- for Fedora (such as RedHat), Archlinux:
sudo vi /etc/httpd/conf/httpd.conf
- and for OpenSUSE:
sudo vi /etc/apache2/httpd.conf
Add this to the end of file ("/file-manager" is the application path, see request-URL):
ProxyPass "/file-manager" fcgi://127.0.0.1:2310/
- Step 3:
Finally, restart Apache. On Debian systems (like Ubuntu) or OpenSUSE:
sudo systemctl restart apache2
On Fedora systems (like RedHat) and Arch Linux:
sudo systemctl restart httpd
Note: you must not have any other URL resource that starts with "/file-manager" (such as for example "/file-manager.html" or "/file-manager_something" etc.) as the web server will attempt to pass them as a reverse proxy request, and they will likely not work. If you need to, you can change the application path to be different from "/file-manager", see request-URL.
Access application server from the browser
Use the following URL(s) to access your application server from a client like browser (see request-URL). Use actual IP or web address instead of 127.0.0.1 if different.
# Start the file management application
http://127.0.0.1/file-manager/start
Note: if your server is on the Internet and it has a firewall, you may need to allow HTTP traffic - see ufw, firewall-cmd etc.
The following are the source files in this application:
- Storing file information in the database (setup.sql)
The information about files, such as file names, sizes, extensions, descriptions etc. is stored in the table "files". Where will the files themselves be stored?
Well, you could store them in the database, but that's not the most optimal way. You could store them anywhere on a file system, but Vely has a very simple (and generally transparent and automatic) way of storing files, which is Vely's file-storage. When a file is uploaded, it's automatically stored there; and you can also create files there as well.
create table if not exists files (fileName varchar(100), localPath varchar(300), extension varchar(10), description varchar(200), fileSize int, fileID bigserial primary key);
- Application landing page (start.vely)
This code displays the page where application user always goes first. One thing you'll notice often in Vely code is the use of "@" to output text (in this case HTML page text) - "@" is an output-statement, and it makes it easy and readable. In general, Vely does not use traditional API for its functionality, rather it uses statement-APIs.
#include "vely.h"
request-handler /start
out-header default
@<h2>File Manager</h2>
@To manage the uploaded files, <a href="<<p-path>>?req=list">click here.</a><br/>
@<br/>
@<form action="<<p-path>>" method="POST" enctype="multipart/form-data">
@ <input type="hidden" name="req" value="upload">
@ <label for="file_description">File description:</label><br>
@ <textarea name="filedesc" rows="3" columns="50"></textarea><br/>
@ <br/>
@ <label for="filename">File:</label>
@ <input type="file" name="file" value=""><br><br>
@ <input type="submit" value="Submit">
@</form>
end-request-handler
- Uploading files (upload.vely)
The next source code file is "upload.vely", which will upload a file when a user selects a file and clicks "Submit" button, which is in "start.vely" file. When you upload a file into Vely application, it's just an input-parameter, just like it would be with any GET or POST request parameter. In addition, Vely will automatically store the uploaded file into file directory (see file-storage) which is optimized for speed of access, and you can either keep it there, move it elsewhere or do whatever you want. All of this make is easy to work with files.
The above code will get the information about the file uploaded (such as file description, as well as file name, location, size, extension) - which is provided by Vely as input parameters. Then, use run-query statement to insert such information into the database table you created. From now on, the information about a file is stored in the database (and that includes the location where it was uploaded), and the actual file is in the file system. This allows fast access to files and easier management.
And finally, the message will be sent back to web client (such as a browser) that you've saved the file.
#include "vely.h"
request-handler /upload
out-header default
input-param filedesc
input-param file_filename
input-param file_location
input-param file_size
input-param file_ext
@<h2>Uploading file</h2>
run-query @db_file_manager= \
"insert into files (fileName, localPath, extension, description, fileSize) \
values ('%s', '%s', '%s', '%s', '%s')" \
input file_filename, file_location, file_ext, filedesc, file_size
end-query
@File <<p-web file_filename>> of size <<p-web file_size>> \
is stored on server at <<p-web file_location>>. \
File description is <<p-web filedesc>>.<hr/>
end-request-handler
- Listing files (list.vely)
For files that have been saved, this code shows them in a list. Next to each file name will be its size, and links to show the file (display it or download it if can't be displayed), and a link to delete the file. To produce the list of files, the table "files" is queried, and the list of files displayed as an HTML table.
#include "vely.h"
request-handler list
out-header default
@<h2>List of files</h2>
@To add a file, <a href="<<p-path>>?req=start">click here</a><br/><br/>
@<table border="1">
@<tr>
@ <td>File</td><td>Description</td><td>Size</td><td>Show</td><td>Delete</td>
@</tr>
run-query @db_file_manager= \
"select fileName, description, fileSize, fileID from files order by fileSize desc" \
output define file_name, description, file_size, file_ID
@<tr>
@ <td><<p-web file_name>></td><td><<p-web description>><td><<p-web file_size>></td>
@ <td><a href="<<p-path>>?req=download&file_id=<<p-url file_ID>>">Show</a></td>
@ <td><a href="<<p-path>>?req=delete&action=confirm&file_id=<<p-url file_ID>>">Delete</a></td>
@</tr>
end-query
@</table>
end-request-handler
- Downloading files (download.vely)
When a link to download a file is clicked in the list of files, the following code is called. The ID of a file requested is queried in table "files", the file path on server is found, and the file is sent to the client using send-file statement.
#include "vely.h"
request-handler /download
input-param file_id
run-query @db_file_manager= \
"select localPath,extension from files where fileID='%s'" \
input file_id \
output define local_path, ext \
row-count define num_files
if (!strcmp (ext, ".jpg")) {
send-file local_path headers content-type "image/jpg"
} else if (!strcmp (ext, ".pdf")) {
send-file local_path headers content-type "application/pdf"
} else {
send-file local_path headers content-type "application/octet-stream" download
}
end-query
if (num_files != 1) {
out-header default
@Cannot find this file!<hr/>
return;
}
end-request-handler
- Deleting files (delete.vely)
Finally, deleting a file (by clicking on delete link in the list of files) is handled in the following code.
When a file is about to be deleted, a confirmation is asked of user - task parameter "action" is "confirm" in this case.
Once confirmed, deletion of a file is carried out; task parameter "action" is "delete" then (you can name parameter "action" anything you want).
Note that deletion is done as a transaction, only if both the record in table "files" is deleted and the actual file deleted. If the file cannot be deleted, transaction is rollback-ed.
#include "vely.h"
request-handler /delete
out-header default
@<h2>Delete a file</h2>
task-param action
input-param file_id
char *file_name = NULL;
char *desc = NULL;
char *local_path = NULL;
run-query @db_file_manager="select fileName, localPath, description from files where fileID='%s'" output fileName, localPath, description : file_id
query-result fileName to file_name
query-result description to desc
query-result localPath to local_path
end-query
if-task "confirm"
@Are you sure you want to delete file <<p-web file_name>> (<<p-web desc>>)? Click <a href="<<p-path>>?req=delete&action=delete&file_id=<<p-url file_id>>">Delete</a> or click the browser's Back button to go back.<br/>
else-task "delete"
begin-transaction @db_file_manager
run-query @db_file_manager= "delete from files where fileID='%s'" : file_id error define err no-loop
if (atol(err) != 0) {
@Could not delete the file (error <<p-web err>>)
rollback-transaction @db_file_manager
} else {
delete-file local_path status define st
if (st == VV_OKAY) {
commit-transaction @db_file_manager
@File deleted. Go back to <a href="<<p-path>>?req=start">start page</a>
} else {
rollback-transaction @db_file_manager
@File could not be deleted, error <<pf-web "%lld", st>>
}
}
else-task other
@Unrecognized action <<p-web action>>
end-task
end-request-handler
Examples
example-client-API
example-cookies
example-create-table
example-distributed-servers
example-docker
example-encryption
example-file-manager
example-form
example-hash-server
example-hello-world
example-how-to-design-application
example-json
example-multitenant-SaaS
example-postgres-transactions
examples
example-sendmail
example-shopping
example-stock
example-uploading-files
example-using-mariadb-mysql
example-utility
example-write-report
See all
documentation
You are free to copy, redistribute and adapt this web page (even commercially), as long as you give credit and provide a link back to this page (dofollow) - see full license at
CC-BY-4.0. Copyright (c) 2019-2023 Dasoftver LLC. Vely and elephant logo are trademarks of Dasoftver LLC. The software and information on this web site are provided "AS IS" and without any warranties or guarantees of any kind.