Chapter 1: Infrastructure

If you are a netizen, you should really have your own corner of the internet: your own personal website. When you google "how do I build a website?" you will usually get immediately redirected to website builders or CMS solutions like WordPress or Wix. For most this is fine, but if you are a software engineer (or just curious), it's important to understand how you can do all this yourself.

---

Hardware

To start, we need some sort of computer to turn into a server. Anything in between a Raspberry Pi, an old laptop, a gaming PC or a datacenter will do. Everything ultimately depends on this hardware, so we want to make sure we pick a stable host. Good servers are powered on 24/7 (ideally with a backup power supply) and have fast, stable internet connection (so ethernet, WiFi is not reliable).

To fit all these requirements I chose to buy a small VM from a cloud provider. Hetzner is a European one and I pay ~20 EUR per month with very low network costs.

Other choices include Azure, AWS, GCP or DigitalOcean. In some of these, can benefit from free trials or special offers, especially if you are a student or startup owner.

You might be tempted to use a solution with a generous free tier, like serverless computing. I would highly advise first starting with a VM and getting familiar with hosting services before using these. You might accidentally rack up a high cloud bill due to a misconfiguration or assuming that some settings are free.

Users

When you first buy a Linux VM, you can can usually connect to it with the root user. This is convenient in that you can run any command without having to prefix "sudo " before it, but it opens your box up to plenty of vulnerabilities. If anyone gains access to this user (potentially by making applications run arbitrary code, aka Remote Code Execution (RCE)), that attacker can then do anything they want. They could steal data, take down the server or lots of other not very nice things.

Therefore we are gonna set up multiple accounts:

• There will still be a root account, but we won't use it unless we really need to, and then we will just do "sudo ". We are never going to directly log into the root account again.
• We will have a personal account which we will actually SSH into. This account has access to sudo and is our daily driver.
• app user

It's a good idea to configure your SSH server such that clients can only connect using a private/public key, instead of a password. An upcoming article will go further in depth on how to configure this.

Reverse proxy

The machine will host multiple web servers, who are all listening for traffic on port 80. If you naively try to run 2 web servers like this, you will get an error that the port is already allocated. So instead, we will run each web service on their own port, and then use a reverse proxy listening on port 80 that will redirect traffic to the correct service. It will redirect traffic based on the hostname, so traffic going to stanrunge.dev will go to port 3000, and traffic to staging.stanrunge.dev will go to port 3001.

There are many reverse proxies we can use, but the most famous come down to NGINX, Traefik and Caddy. NGINX is the OG, but requires some boilerplate because you are configuring everything. Traefik solves this mostly by just attaching some labels to a Docker container, and it will automatically get picked up by Traefik. Caddy is a super simple one that needs minimal configuration and handles a lot of things automatically for you, such as SSL.

• Manage port allocations yourself ! (make a table)

Database

Most websites need a database to read and write data, and there are several options available.

Some people prefer to put their database in their Docker Compose files, meaning that an entire instance will be spun up for every project/environment. For small scale this is fine, but I prefer to run 1 central database instance with multiple databases created inside that instance. It makes things like exploring data and backups a lot simpler. As your individual projects start to grow in complexity and traffic, it might be worth it to divide into dedicated DB instances, or even dedicated VMs altogether.

Running a web server

• SvelteKit
• NodeJs
• Docker
• Docker Compose

Git server

We need a central place to store the source code of our projects, so that we can use that later for building and running the application. Most people use GitHub for this, and sure, it's nice that it's already hosted for you and they have a generous free tier. But it's not uncommon that GitHub has an outage, and depending on your project you might actually go past the free tier.

We therefore want to host our own git server, and there are 2 popular prebuilt options here: GitLab and Gitea. GitLab can take up about 4GB of RAM (our entire VM!!!), so I instead opted for the lightweight Gitea.

While it's nice to truly own my own data, pushing and pulling is snappier because my VM is closer to me and I'm not competing with millions of other developers trying to push and pull at the same time, it's true that I am setting myself up for a central point of failure. If my VM for whatever reason goes down, I am also losing all my source code, issues, pull requests and more. I therefore set up automatic code mirroring to GitHub, so that I still enjoy the benefit of a prehosted solution while mainly just using my own selfhosted solution.

CI/CD

As developers we want to spend our time writing code, not manually deploying new versions. We want to therefore automate the entire deployment process; whenever we make a new version of our website, the server should build that new version, take down the existing server and replace it with a new instance.

I set this up using Gitea runners, which are almost identical to GitHub Actions runners. The idea is to work with at least 3 git branches:

1. prod:
2. staging:
3. dev: You can also add more if you work in a team, or whatever other reason you might have.

• Cloudflare access

Networking

• Cloudflare
• DNS
• SSL
• IP (IPv4 vs IPv6?)
• Domain

Backups

While we obviously try our best to protect and not lose our data, catastrophies do happen, and you need to be prepared for them. Hardware is replaceable, but data isn't. So, we will back up all data in our database once a day in a completely different location.

We can install rclone with: sudo apt install -y rclone

I made a bucket in Cloudflare R2 which I will use to store all my backups. To prevent this bucket exploding in size, I will only store backups of the last year. This is built into rclone.

In a future article I will expand on this backup solution by also including all my client devices and other data.

Secrets

Monitoring

Logs

Analytics

Stress-testing

Expanding/migrating

• Vercel/Fly.io/Netlify managed hosting
• Cloud Docker runners
• Kubernetes
• VM is probably fine !!!

With this setup, I rely quite a bit on Cloudflare. DNS, SSL, Backup storage, Dev/staging access are all handled by them. Cloudflare has a very generous free, and even with moderate traffic I'd easily fit in that tier. Of course it's worth keeping an eye on the average uptime and changes to the free tier.