Homelab 1 - Setting up K3s

| View comments on Hacker News

This series of posts will detail the setup of my homelab. It goes into the technical details of how I setup a single-node K3s Kubernetes cluster using cdk8s and Deno to generate all of the required YAML manifests. This is a practical deployment with:

  • Automated backups
  • Monitoring and alerting
  • Automated deployments
  • Automatic image/chart upgrades
  • Support for GPU acceleration
  • Secure secrets with 1Password
  • Secure remote access through Tailscale
  • Direct access for game servers and certain protocols like mDNS

Table of Contents

Background

I’ve had a homelab for around a decade. The hardware itself has gone from repurposed parts in college (a Core Duo served me very well from 2017-2022) to a very beefy server today:

(The full build is on PCPartPicker)

Over the years I’ve tried quite a few ways to manage it:

  • Manually installing everything without any automation
  • Artisinal, hand-written bash scripts
  • Ansible
  • Docker Compose

Those methods were mostly informed by what I was wanting to learn. That trend hasn’t changed with my move to Kubernetes. I had no experience with K8s back in December, and today I use it to manage my homelab quite successfully. Kubernetes is overkill for a homelab, but it does provide a great learning environment for me where the consequences are relatively low (as long as my backups keep working).

I name each iteration of my server so that I can disambiguate between references of older installations. Previously I named my servers after Greek/Roman gods, but now I’m using the names of famous computer scientists. The name of the latest iteration is “lamport”, named after Leslie Lamport who is known for his work in distributed systems.

Operating System

Because I was planning on running everything in Kubernetes, I considered using Talos rather than my usual choice of Ubuntu Server. Talos is an immutable Linux distribution with support for Kubernetes baked-in. I ultimately didn’t choose Talos because at the time required a dedicated drive for the operating system, meaning my entire 4TB SSD would be taken up by Talos.

I ultimately chose Ubuntu Server which has served me well for years despite the community seeming to be unhappy with some of the recent choices by Canonical. I didn’t do anything special for my installation other than setting up my RAID 5 array with mdadm and fstab.

Kubernetes

Similar to how there are multiple flavors of Linux distributions, there are many distributios of Kubernetes. I chose to use K3s since it has a reputation of being lightweight, easy-to-use, and stable.

You can configure K3s by creating a file at /etc/rancher/k3s/config.yaml. There are some options that cannot be changed after K3s is installed. The one most relevant to me is IPv6 support. Be sure to look through the configuration options.

Installation is straightforward:

$ curl -sfL https://get.k3s.io | sh -

And… that’s it! You now have a Kubernetes cluster running on your machine. You can use kubectl to interact with your cluster. I also use Aptakube on my MacBook when I want a GUI for monitoring my cluster.

I use Tailscale to securely access my homelab. Tailscale is a Wireguard-based VPN that’s, quite honestly, fun to use. You can use your favorite VPN to access your homelab, or if you’re brave you can expose it to the public internet.

Credentials for your cluster are stored at /etc/rancher/k3s/k3s.yaml. You can copy this file to ~/.kube/config on your local machine to use kubectl. Note: you’ll also need to change the server field to point to the address of your server.

Bootstrapping

This does require a small amount of manual bootstrapping, which I describe in my repository README. Whenever I setup a new cluster/node, I need to:

  • Install ArgoCD: kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
  • Create a secrets to access my 1Password vaults
  • Deploy the manifests in this repo: kubectl apply -f cdk8s/dist/apps.k8s.yaml

I’ve recreated my cluster a couple of times and I’ve been very pleased with how easy it is to get everything back up and running. It takes me just a few minutes from installing K3s to having all of my services back up and running.

Conclusion

I’ve shown how I setup my cluster with K3s. In future posts I’ll cover:

  • Using Deno and cdk8s to generate manifests
  • Automating deployments with ArgoCD
  • Ingress with HTTPS and Tailscale
  • Direct connections to pods
  • mDNS
  • Persistent volumes
  • Backups
  • Monitoring
  • Helm, Kustomize, and operators
  • Keeping things up-to-date

Recent posts from blogs that I like

Is there a built-in way in C++/WinRT to get the string name for a Windows Runtime enum?

No, and maybe you don't want to. The post Is there a built-in way in C++/WinRT to get the string name for a Windows Runtime enum? appeared first on The Old New Thing.

via The Old New Thing

Writing MIME, RSS, and existential suffering

I'm trying to get back into streaming regularly, and I decided to break the anti-streak by doing a good old fashioned blogpost writing stream. I broke down the problem, wrote out all the prose, and then published it.

After remembering that you actually have to push commits for them to be live.

via Xe Iaso

Synergy Greg

Synergy Greg would like to see you in His office, You will know Him when you see Him, Multitudinous is Synergy Greg! His handshake grinds the bones of the world, Beneficent is Synergy Greg! His lungs are mighty bellows, Go Team! Get! Shit! Done! He will wait by the sidelines, Any day now. His minion...

via Ludicity
Live in Seattle? Let's grab coffee!
github rss email