hillhome.uk
02 — The server room

A cupboard, mostly

How to run a few small self-hosted applications for a household without putting any of them on the open internet.

Self-hosting has an unhelpful reputation for being either trivial or a full-time job. In practice a home lab that runs a handful of small applications is neither: it is a short list of decisions made once, and a slightly longer list of failures that everybody meets in the same order.

What follows is both lists, written so they are useful to someone standing at the same fork. There is no inventory of what runs here — publishing what you run, where, and on which port is free reconnaissance for somebody else, and a list like that is the single most useful thing you can hand an attacker.

Architecture

Four decisions worth making once.

Private by default

If a service does not need to be reachable from the internet, do not make it reachable. A mesh VPN gives every device an address on a private network and every service a name on it, with no port forwarded, no login page exposed to scanning, and nothing to patch under time pressure. An unreachable service cannot be exploited by a passing bot.

One public thing, and keep it flat

Where something genuinely has to face the world, make it the least interesting target available: static files, a reverse proxy that obtains its own certificate, no database behind it, no session, no form to post to. There is no application logic to attack because there is no application.

Boring beats clever

A small Python application on a single-file database, run by a normal process manager, started at boot by the operating system. Each layer removed is a layer that cannot fail at eleven at night, and a stack you can hold in your head is one you can fix while tired.

A backup is a restore

An untested backup is a folder of good intentions. The test is not that the file exists or that the job exited zero — it is that you copied it somewhere harmless, opened it, and counted the rows.

Backing up SQLite without corrupting it Method → Why copying the database file is not a backup once write-ahead logging is on, what the online backup API does instead, and how to verify a restore in one command.
Failures

Four that everyone meets.

01

The scheduled job that succeeds while doing nothing

Symptom. A nightly backup ran on time every morning and reported success every morning. The local file copy was fine. The part that needed the network had not worked in weeks.

Cause. A machine that sleeps runs its scheduled task the moment it wakes, which is several seconds before the wireless interface has associated and been given an address. The script's network step failed instantly into a shell that was not checking exit codes, and the script carried on to its own success message.

Fix. Three separate things, all of which are needed. Make the script wait for connectivity rather than assume it — poll for a route with a bounded retry and give up loudly. Make every step's failure fail the script: check exit codes, or set the shell to stop on error. And alert on the absence of success, not just on failure, because a job that never runs at all sends no error either.

02

Two services, one port, one misleading error

Symptom. A new web server refuses to start: address already in use on port 443. Nothing else appears to be serving on 443. Stopping the obvious suspects does not help.

Cause. An existing process was listening on a specific address on that port. The new one asked for the wildcard — every address on the machine, port 443 — and the wildcard overlaps the specific one, so the kernel refuses it. Both bindings are legitimate; they simply cannot coexist, and the error message describes the collision without mentioning that it is only partial.

Fix. Bind each service to an explicit address rather than letting either take the wildcard. Once both are specific and different, they sit happily on the same port number, and you can check what actually holds a port with a socket listing that shows the bound address, not just the port.

03

The deploy script that only ever adds

Symptom. A server slowly accumulates every file that has ever existed in the project, including ones deleted locally months ago. Eventually an old route, template or config that should be long gone is still being served, or the disk fills.

Cause. Deploying by uploading changed files. Upload is additive: it has no way of expressing "this file should no longer exist", so the server's contents become sediment rather than a copy of anything.

Fix. Deploy by pulling a known-good revision, so the server's state is definitionally equal to a commit and deletions propagate for free. If you must sync files, sync with deletion enabled and an explicit exclude list for the few paths that legitimately live only on the server — data, secrets, logs.

04

Progress that only works on one worker

Symptom. A long-running job reports its progress perfectly in development and reports nonsense in production — jumping backwards, sticking at zero, or showing "not running" while the job is plainly running.

Cause. Progress kept in a module-level variable. With one worker process there is one copy, so it works. With several, each process has its own copy: the request that starts the job lands on one worker and the requests that poll for progress land on whichever the load balancer picks, which usually has never heard of the job.

Fix. Put shared state somewhere both processes can see — a row in the database, a file, a cache — regardless of how many workers you currently run. The worker count is a deployment detail that will change under you, and the variable will be wrong on the day it does.

Hardware

The cheapest machine that will do it.

Home lab advice tends to start with a rack. It does not have to. A second-hand thin client — the fanless, palm-sized kind that offices replace in batches and sell for less than a takeaway — is silent, draws a handful of watts and will comfortably run network-wide ad blocking plus one or two small applications.

The two constraints to know before buying one. Storage is usually soldered flash of eight or sixteen gigabytes, which is enough for the operating system and very little else, so anything that writes continuously — logs, a query database — wants redirecting to external storage or it will fill the disk and take the machine down with it. And that same soldered flash has a finite write life, which is the other reason to move the chatty writes off it.

What a thin client is not good for is anything that must never be off. A machine at home depends on your power, your router and your broadband; when the job becomes "this has to answer at three in the morning while we are away", the honest answer is a small always-on virtual machine somewhere else. Splitting the two — local things local, always-on things hosted — costs very little and removes a whole category of three-in-the-morning problem.