Let's be clear here, I'm French, so I'm complaining a lot! And some people are saying:

Loïc, you should not be so negative! Where are your solutions?

So let me tell you a nice story!

Static For the Win

As you may know, we are huge fans of static site generator, like Hugo, or Zola! You may also know that we are big fan of Angular (because React is a very bad technology ;-) ). But we are always facing the same issue: how do we deploy it?

Yes, you always have the Netlify or Vercel solution, but we are working a lot with companies which are self-hosting/are using a technology stack that they don't want to move away (Let's put everything in Kubernetes!).

As a senior sysadmin, I always rely on my amazing skills (keuf keuf Google/DuckDuckGo) to help me, so let's see the good practices about deploying Angular, for example.

The horror!

I've found some horrible stuff there, like people doing ng serve in a Docker container to go to production! Other people are doing something amazing, like doing some crazy stuff based on nginx:latest and pushing some configuration with some problems like no gzip, no cache, etc...

Besides, you can find what I call the hell of Docker:

on promethium /tmp on ☁️  (us-east-1) took 2s 
❯ docker pull nginx
Using default tag: latest
latest: Pulling from library/nginx
Digest: sha256:e1211ac17b29b585ed1aee166a17fad63d344bc973bc63849d74c6452d549b3e
Status: Image is up to date for nginx:latest
docker.io/library/nginx:latest

on promethium /tmp on ☁️  (us-east-1) took 2s 
❯ docker run -it nginx bash
root@a1b2ae08a3eb:/# nginx -V
nginx version: nginx/1.21.6
built by gcc 10.2.1 20210110 (Debian 10.2.1-6) 
built with OpenSSL 1.1.1k  25 Mar 2021
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -ffile-prefix-map=/data/builder/debuild/nginx-1.21.6/debian/debuild-base/nginx-1.21.6=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'
root@a1b2ae08a3eb:/# openssl version
OpenSSL 1.1.1k  25 Mar 2021
root@a1b2ae08a3eb:/# 
exit

on promethium /tmp on ☁️  (us-east-1) took 11s 
❯ docker pull nginx:alpine
alpine: Pulling from library/nginx
Digest: sha256:77cc350019d0188d3115084265483dcefdd8489ccf719ff4e4c956b48de8ff6a
Status: Image is up to date for nginx:alpine
docker.io/library/nginx:alpine

on promethium /tmp on ☁️  (us-east-1) took 2s 
❯ docker run -it nginx:alpine sh
/ # nginx -V
nginx version: nginx/1.21.6
built by gcc 10.3.1 20211027 (Alpine 10.3.1_git20211027) 
built with OpenSSL 1.1.1l  24 Aug 2021 (running with OpenSSL 1.1.1n  15 Mar 2022)
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --with-perl_modules_path=/usr/lib/perl5/vendor_perl --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-Os -fomit-frame-pointer -g' --with-ld-opt=-Wl,--as-needed,-O1,--sort-common
/ # 

on promethium /tmp on ☁️  (us-east-1) took 18s 
❯

As of today (2022/03/19), the latest NGiNX docker is compiled with an old version of OpenSSL (you know this library which is having some security issues), and the NGiNX alpine is compile with OpenSSL 1.1.1l, but executed with OpenSSL 1.1.n... and of course, in both cases, full options.

What do we want?

We want a simple, easy to deploy and performant HTTP server. Of course, I should talk about the amazing darkhttpd, which is working perfectly for this, but cannot host Angular, as it needs to have the infamous redirect to index.html.

Simple

What do I mean by simple? it should have 0 configuration files. How did we do this? We created a very weird way of loading data:

dhakira will check if the folder dhakira_html exists
if yes, it will check if the folder dhakira_html/websites or dhakira_html/spas exists.
if no, then it will quit

But Loïc, how do you manage the domain names?

Simple, the folder inside websites or spas is the domain name, for example, dhakira_html/websites/www.kalvad.com will consider that the website www.kalvad.com should be loaded.

Easy to deploy

As you know, we are big fan of 3 languages at Kalvad:

Unfortunately, even if Elixir could be performant enough, let's be clear, it's harder to deploy than Go, or Rust (single small binary).

We asked ourselves should we use Go for this project (and to be honest, it could be faster), but I cannot stand Go anymore, between the package manager and the changes in the language. Regarding Rust, I never had the energy to cross the first milestone.

If you read our blog, you should also know that Pierre is a big fan of Haskell, but I cannot stand the syntax, I prefer Erlang in that case.

We also had another language in mind: Zig (BTW, we are sponsoring Zig), but even if I think this language is incredible, it's not yet mature enough for what we wanted to do.

So we decided to go with Crystal! Crystal’s syntax is heavily inspired by Ruby’s, so it feels natural to read and easy to write, and has the added benefit of a lower learning curve for experienced Ruby devs. But it's compiled, and it's damn fast.

Furthermore, we did something nice: we have logs on stdout, so no need to cheat to get your logs in a container or a PaaS!

127.0.0.1:59917 [2022-03-19 18:03:38 UTC]  "GET localhost/js/main.cc248687f267df67ff747ca197e72f1ae051339a03ac2a78c8046913f2695c02.js" 200  11.37µs

Performant

Stupid question, but why, when we have a website doing 35MB, do we still load it by default from the disk? Nowadays, memory is cheap, and I'm pretty sure that we could have everything in memory, not as a cache, but as a running system.

That's what we did! When we load a website, we are doing the following tasks:

we check the file to load in memory, check it's MIME type, and save it
we load the file in the memory
we load the file in memory directly in GZip (brotli coming soon)

Which means that we don't process anything at delivery! It has a side effect, we cannot modify a file and change it's delivery without stopping the server, but this immutability, isn't it the base of our modern stacks and systems?

Let's be clear, Crystal is not a super trendy, and there is room for improvment, but the performance is really good: on big JS files, we are never going higher than 67 us!

Nants ingonyama bagithi baba

the reference here!

So here it is: Dhakira!

You have a nice repo, on Kalvad's Github, with some mermaidjs documentation!

Overall, the project is very small:

and is easily customizable!

I can also proudly announce, that a few websites, including Kalvad main website, Opaala's Angular Application and some of the UAE Governement websites are hosted with Dhakira!

The future

We still want to add more features, for example:

proper 404 management
Brotli
Etag

But we are going to maintain and help improve the core of Crystal for HTTP Server.

If you have a problem and no one else can help. Maybe you can hire the Kalvad-Team.