Running A Personal Cloud - Part 1
Overview
For my family and personal use, I have decided that I needed certain features and I wanted to run as much as possible as containers using Podman. Since I work a lot using Kubernetes, I wanted to define everything using Kubernetes Pod resources and then have Podman run the services from those Kubernetes Pod definitions. This means that at a later date I could deploy the same configuration with almost no changes to a Kubernetes cluster. I host all of this on an "auction" server at Hetzner where I get a AMD Ryzen 7 3700X (16 cores) @ 4.43 GHz, 64GB RAM, and 7.3T on 2 SATA SSD drives for about €55/month. I also pay for a 5TB StorageBox where I write my backups nightly, totalling about €70/month. I still have a lot of unused capacity on that host, so I am probably overpaying for what I am doing, but I am regularly considering adding new items to my configuration.
Components
These are the components that I have implemented:
- Keycloak - An SSO implementation which I integrated with each of the other services for identity and access management
- Nextcloud - Nextcloud can provide a place to share and sync your files, photos, and other features. I am using Mail, Files, Memories (for photo management), and Bookmarks
- OneDev - A Git "Forge" which allows you to host Git repositories, manage issues, do continuous integration, and more
- Vaultwarden - An Open Source BitWarden clone which is compatible with Bitwarden clients
- SearXNG - A privacy-focused meta-search engine which aggregates results from your personalized search engines which can be sanitized of tracking information and anonymize your searches
Podman & Quadlets
Podman integrates with the Linux systemd init process using a feature called Quadlets. Quadless allow you to manage container workloads easily with SystemD units. I used the kube and container quadlets format to run each of my services.
PostgreSQL Container or Containers?
I have gone back and forth on creating individual PostgreSQL containers for each application which requires a database or just using a single PostgreSQL container with separate databases. I decided to use a PostgreSQL instance for each service so that everything is a little more modular, but you could potentially save some resources if you use a single instance for multiple services.
HAProxy or Traefik?
Having a good reverse proxy to handle exposing your services via SNI virtual hosts makes a lot of this possible. I chose to use HAProxy because I am already familiar with it, but a recent post from Larvitz makes me think I may eventually switch to Traefik because it natively integrates LetsEncrypt in a much simpler way for Podman and Quadlets.
CertBot & LetsEncrypt
Having your services behind TLS encryption is so easy these days with LetsEncrypt. It provides a simple "Challenge" mechanism to verify that you own the domain/host you desire a certificate for. Using the certbot tool let's you easily request and receive a TLS certificate which is signed by the LetsEncrypt CA, which means that it is trusted by most major browsers used today.
Keycloak For Authentication
Keycloak requires both the Keycloak server and a database to function. I chose to use PostgreSQL because I am familiar with PostgreSQL and it is easily run inside of containers. Below you will see the Pod resource I use to launch Keycloak with it's associated PostgreSQL container.
Nextcloud
Nextcloud is a fork of OwnCloud which is Open Source, but also has options for commercial support. It can provide most of the services that people are used to from the major cloud providers: Documents, Photos, Task lists, webmail, etc... It is also extensible, so you can add any plugins from the Nextcloud App Store.
OneDev - Git, Issue Tracking, CI, CD, and Artifact Repository
If you use GitHub, GitLab, or Codeberg; you are probably familiar with the capabilities available in OneDev. OneDev is implemented in Java using Apache Wicket for the user interface. There is also an enterprise offering which provides support for the system.
Vaultwarden - A FLOSS Alternative To BitWarden
I LOVE BitWarden. It is very convenient, verifiably secure, and has excellent support for major browsers and operating systems. I actually pay for a family plan for BitWarden for my entire family, but I do not want to be tied to a cloud service over which I have no control, so I have chosen to start moving toward Vaultwarden for my personal needs.
SearXNG - Your Own Private Search Aggregator
SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled. It will even strip tracking information from search results if you choose. Overall, the most appealing part to me is that it strips out all of those AI and Sponsored results from the searches.
Next Steps
Each of these items is quite complex, so I will break each down into future blog posts which will focus on the specific service or capability. I will show how I automate everything with Ansible playbooks, and show how I can restore backups in a VM to verify that my backup strategy is reliable. All of this will be wrapped in Ansible playbooks which simplify everything and break it down into reusable roles which you can copy and use for your own cloud implementation if you like.
Final Thoughts
Am I crazy for hosting my own private cloud? Perhaps, but I have had too many experiences where a cloud service that I have become dependent on has been bought or closed or changed in such a way as to ruin it's value to me. I hope that by managing my own private cloud, I can maintain control and not have to worry about those sorts of experiences in the future. Feel free to reach out via Mastodon and share your journey or tell me how crazy I am for doing all of this.