Mitigate 504s with keep-alive timeouts

One calm April day, our internal ops application started showing problematic responses in the form of 504s. A HTTP 504 error is a server side gateway timeout and can be hard to diagnose. However, this usually has to do with the load balancers sitting in front of your application and their communication upstream. Ultimately the fix for came in the form of ensuring the application keep-alive timeout is the same or greater than that of the load balancers sitting in front.

Continue reading ↦

Overriding ConfigMap variables in Kubernetes

Much like default values.yaml files for Helm, you can utilize ConfigMaps in Kubernetes for much the same capability. Our developers have started to do this to make it easier when managing Deployment objects across different environments. Give a ConfigMap: apiVersion: v1 data: allowed: '"true"' enemies: aliens lives: "3" kind: ConfigMap metadata: name: engage namespace: default and a Deployment: spec: containers: - env: - name: enemies value: badguy envFrom: - configMapRef: name: engage Taking a look inside the container:

Continue reading ↦

Execute a Cronjob manually in k8s

We often find ourselves needing to test our Cronjobs more than we’d like out of band with their standard scheduled runtime. Kubernetes makes it super easy to do this by spawning a Job from the CronJob itself. kubectl create job --from=cronjob/<cronjob-name> <job-name> Thanks again stackoverflow!

AWS Key Fingerprints on your laptop

The AWS Fingerprints listed on the Key Pairs page (in the ec2 web console) are not derived in a simple ssh-keygen fashion. Instead of installing their custom tools (ec2-api-tools, which is java) to compare fingerprints with keys locally, you can use the following command to generate the fingerprint on either the private or public key. openssl pkcs8 -in ~/Downloads/stockx.pem -nocrypt -topk8 -outform DER | openssl sha1 -c

Halo on PC: Spartans Rejoice

This past weekend, I went down the rabbit hole on a project I recently discovered called Halo Online. From a past project between 343 and Saber Interactive, Halo Online was meant to be a free-to-play multiplayer experience based on Halo 3 available on PC. Sadly, outside of a closed beta in Russia, the game was cancelled. Luckily, those game files became available and modders created ElDewrito, enabling Forge mode and online servers for players everywhere plus other gameplay enhancements.

Continue reading ↦

Bitperfect direct audio output to a DAC in Windows

Since I have my own DAC/Amp (the coveted Schitt Stack), I very much like to play FLAC files directly to my DAC, ensuring no operating system mixer is filtering the output. This happens to be especially tricky when utilizing your optical audio output. One way you know you aren’t getting bitperfect playback is when you hear other system sounds or playback from other applications through your headphones. While this is pretty easy to achieve in Linux (VLC has a direct option as do other applications like deadbeef), it’s a bit harder in Windows.

Continue reading ↦

Use nofail for your mounts!

Nowadays, systemd will halt your boot and dump you in emergency mode if any designated device mounts fail. While incredibly annoying, this is a safety feature. I encountered this whilst trying to boot my Raspberry Pi headlessly and wondering why I couldn’t ssh! To solve this, ensure you specify the nofail option for your in /etc/fstab for each of your mounts which shouldn’t interrupt the boot process. This is useful for external hard drives, NAS devices, and other headless systems.

Continue reading ↦

Increasing My Phone's Longevity with LineageOS

In early 2016, I bought a Nexus 6p. It has by far and wide been my favorite device with a comfortable size, great screen, and a headphone port. While I’ve strongly considered picking up a fresh shiny Pixel, with both the proliferation of “fix it” stores and the sustained development of custom operating environments, I think its time for a Nexus Redux! First off, the battery. While Nexus phones are notorious for having an unrecoverable reboot looping problem, they also have batteries that can dye unexpectedly fast (i.

Continue reading ↦

Effective Content Filtering

While my friends have been raving about Pi-hole for a long time, I’ve never actually started using it. This is for a variety of reasons mainly relating to how setup for my (semi-classic) router seems relatively sub-par. While I may not adopt it for my entire network, there’s a new player on the scene which covers all your devices, including mobile (without root), and it looks quite impressive: Adguard. From their main site, you can get a lifetime license for your device, which helps guarantee I’m not wasting my precious data (Project Fi ftw!

Continue reading ↦

Thoughts and Recordings from DevOpsDays Detroit 2018

Today we officially announced availability of the recordings for DevOpsDays Detroit 2018 are live. Among them are some of the best talks I’ve heard in a while covering everything from DevSecOps to Chaos Engineering to an audience of almost 300! I’m still high from the new friends I made and DevOps learnings I soaked up. When’s the next one??? I want to take a moment to thank everyone who co-organized the conference with me, especially Joe Arnet, Dave Barnett, and Archie Cowan, all of whom I worked immensely close with.

Continue reading ↦