28 May 2021, 20:59

Make Linux Fast Again

https://make-linux-fast-again.com/ has big promises, boy oh boy. I was wondering how much I should blame the recent spate of hardware vulnerability mitigations for my laptop being pokey, so I decided to turn it off. Slamming all of those options blindly into my kernel command line worked OK- but nothing perceptible happened.

/proc/cmdline is the place to go to check your changes worked end-to-end, by the way.

I was, however, oddly locked out of Flatpak- but was seeing this in the kernel logs-

flatpak-portal[1967]: segfault at c ip 000055665f85f8d0 sp 00007ffce0ffa800 error 4 in flatpak-portal[55665f83f000+d4000]

seems bad, right? Cutting back the kernel flags to just mitigations=off fixed Flatpak, and I’m still Living Dangerously-

$ grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/itlb_multihit:KVM: Vulnerable
/sys/devices/system/cpu/vulnerabilities/l1tf:Mitigation: PTE Inversion; VMX: vulnerable
/sys/devices/system/cpu/vulnerabilities/mds:Vulnerable; SMT vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Vulnerable, IBPB: disabled, STIBP: disabled
/sys/devices/system/cpu/vulnerabilities/tsx_async_abort:Not affected

So, I’m an old web dork, and I don’t have any go-to benchmarks to go by, but I vaguely remember Octane being important like ten years ago, and I don’t have to install anything- and they’re CPU benchmarks, which is sort of relevant. So here’s a baseline, with mitigations on-

Octane Score: 10622 (mitigations=on, firefox)
Octane Score: 10814 (mitigations=on, firefox)
Octane Score: 24399 (mitigations=on, chrome)
Octane Score: 24858 (mitigations=on, chrome)

Wow! Chrome is a lot better at this than Firefox. I mean, it’s their benchmark, but wow!

With all of the Spectre, Meltdown, etc mitigations off-

Octane Score: 12341 (mitigations=off, firefox)
Octane Score: 12602 (mitigations=off, firefox)
Octane Score: 28102 (mitigations=off, chrome)
Octane Score: 28221 (mitigations=off, chrome)

🥳 Running at 110% of normal capacity sounds pretty good.

What about something closer to what I care about, like GUI interactivity? I kind of flailed around a bit on this, trying to find something that measured jank specifically, and came up short. There is, however, good ‘ol GtkPerf, which at least measures something close to what I care about. Fullscreen runs stabilized at around 8.9 seconds with mitigations on and 8.7 seconds with mitigations off. Is that an improvement? Is it good we can draw rubber ducks slightly faster?

22 Apr 2021, 08:59

Developing inside of a Dark Forest

I have these- and other- news items on my mind-



For decades now, we as developers have all really benefited from a strong culture of sharing. From experience reports to polished open source projects, from stack overflow answers to professionals livecoding, it’s been almost all love and rainbows1.

Unfortunately, the kind of people who break things for a living look at all of the rainbows and say “lol there are no gates here”, and are very busily and aggressively poisoning all of the wells they can find.

Instead of a freewheeling community zone, we’re entering something that’s going to look more like a Dark Forest. This is one of the less appealing answers to the question of “why, if we can see so much of the universe, can we not see any evidence of other life?” In the Dark Forest model, it’s because anything that’s noisy gets eaten.

The next steps are grim and unavoidable. We’re going to need to lock down our repos, and be suspicious of newcomers. We are going to have a rise of new, restrictive corporate policies at work, and we are going to have to share less.

Incompetent proprietary companies like SolarWinds will be exempt from the new, more highly scrutinized routine- it will only negatively effect people trying to be generous with each other. Everything will get worse, but it will definitely feel like we are being more responsible, especially at first.

Even worse, the logical next step is probably more anti-immigrant paranoia and team firewalling. It’s going to be a rough few years.

I’m feeling pretty down about it.

  1. I already know, don’t bug me about it [return]

12 Feb 2021, 20:00

How to do basic shell completion

So you wrote some dumb tool for your coworkers. It works, but no one can remember the dang arguments. Half of them use zsh, the other half use fish, the grumpy loudmouth uses bash, and the new hire uses something that sounds like a tolkein reference. The documentation for everything is garbage, stackoverflow is filled with lies, and most of the available examples take up six pages. Can’t you do this simply?


Here are some minimum-viable-shell-completions for popular shells- just enough to make the <major command> <minor command> pattern work, you know, like git $command or aws $product $verb.


The big kahuna, the default on Linux distributions, and the old default on OS X, before the GPL v3 and Apple’s Great Terror. There are, as you might expect, Many Ways To Do It, but my favorite, because it is simple & bloody minded, is good ‘ol complete-

:~$ complete -W "fee fi fo fum" giant
:~$ giant <tab>
fee  fi   fo   fum  


The New Hotness in 2005 (I know it dates to 1990), and the default on MacOS for the past few years. Here there are also Many Ways To Do it, but look, I’m not going to lie- I like the old, deprecated compctl, because I am old and deprecated. Also, you can bang this out in two lines instead of 2 paragraphs, and brevity is brilliance.

% declare -a _giant_commands=(fee fi fo fum)
% compctl -k _giant_commands giant
% giant <tab><tab>
fee  fi   fo   fum


If you were too cool in 2005 to use something that worked, fish was the shell for you. That is still sort of true, because dealing with cross-shell compatibility is still slightly more of a hassle than you really want to put up with. It also uses the complete command for completion, but the documentation is better, and it’s totally different from bash’s.

> complete -c giant -a "fee fi fo fum"
> giant <tab>
fee  fi  fo  fum


If you like to get in fights on message boards lamenting the lack of structured streams in the shell, Elvish is the shell for you. It is very neat, and has a truly wild feature set, including a built-in multi-pane file manager. Definitely worth a look if you’re feeling like changing your workflow up.

> edit:completion:arg-completer[giant] = [@args]{
    possibilities = [fee fi fo fum]
    put $@possibilities

> giant <tab>
 COMPLETING argument
fee  fi  fo  fum

I’m really feeling the namespacing, if I’m being honest. It makes introspection of the shell configuration a lot less like dipping your face into a firehose.

04 Feb 2021, 22:00

Podman Notes

more niche container system usage notes

Podman is a great alternative to docker for your laptop- it’s daemonless, so when you aren’t using it, it’s not wrecking your damn battery. It also doesn’t require sudo, which feels pretty nice.

wait why doesn’t this work?

Hurl in --log-level=debug anywhere to figure out why things go bad. The logs are good!

pull from dockerhub by default

Update your registries.conf-

$ cat ~/.config/containers/registries.conf
registries = ['docker.io']

registries = ['docker.io']

Insecure, as far as I can tell, is about the Notary vs GPG schism- Docker went with an experimental new system called Notary, Red Hat has always used GPG and continues to use GPG, and DockerHub images are not GPG signed.

It’s all kind of dumb, to be honest- the real protection is from TLS. I am not going to try and work through random third party’s adventures with key management.

rejected by policy error on pull

Error: Source image rejected: Running image docker://ubuntu:latest is rejected by policy.

or you know, whatever

My default policy.json-

$ cat /etc/containers/policy.json
    "default": [
            "type": "reject"

That’s quite a policy!

You can fix this in a local override, easy peasy-

$ cat ~/.config/containers/policy.json
    "default": [
            "type": "insecureAcceptAnything"
            "": [{"type":"insecureAcceptAnything"}]

I feel very… secure now… I think.

29 Nov 2020, 11:33


Speaking of CLI mail programs, this is one I’ve been meaning to tour. It’s actively developed, seems a bit more ambitious about reaching for the future, and the original author is Drew DeVault, whose outspoken software freedom stuff speaks to me, as an old dork who cares about that sort of thing.

Anyway, there are no binary releases, so you gotta go get it the old fashioned way-

git clone https://git.sr.ht/~sircmpwn/aerc
cd aerc
PREFIX=$HOME/.local make install

There is a first-time-setup wizard, which was pretty great. The tutorial help that pops up immediately is very useful, and you can get it back at any time with :help tutorial or by running man aerc-tutorial in some other terminal window.

jk or ↑↓ go up and down through messages, JK go up and down through folders in the sidebar. A archives messages, D deletes them.

Anyway- aerc is fast, stays fast with large imap mailboxes, integrates well with vim, which makes me happy, and generally has safe & sane defaults that you don’t need to configure. I did have to set this in my ~/.config/aerc/accounts.conf-

copy-to       = [Gmail]/Sent Mail

but uh, that was it.

HTML emails

For person-to-person emails, plain text is fine, but a lot of automated emails, newsletters, etc., are… very html heavy. That also seems like the general trend of things, which makes it ever-harder to do mail outside & unrelated to a web browser. There’s a knob to turn on w3m for text/html mails, but I don’t know- it isn’t amazing. It’s not going to be an experience that sparks joy.

tiny software soapbox

It’s kind of a grim time to be a computer user. Lightweight UIs are all TUIs now, because the terminal is documented, stable, well understood, and portable. Heavyweight UIs are really heavyeight, and bundle a whole damn browser. Gtk+ now wants to be the toolkit of choice for a tiny network of friends, and I haven’t seen a Qt project in years. Apple doesn’t even document their desktop APIs now- and you couldn’t port any of that if you wanted to. Microsoft’s UI things get abandoned every 2 years.

So anyway… if you want a portable program, not a website, you have two options: * Write Electron apps. People who buy a new $3,000 dollar macbook every other year will love it. * Write terminal apps. People with slower computers will love it. It will not be accessible to non-turbonerds, but you know, what are you gonna do.

Trapped inside of this paradigm, aerc is pretty great.

26 Nov 2020, 16:44


So Mutt 2.0 got released, so I figured it was time to take it for another spin. The gmail web client has gotten much slower and less pleasant over the years, and hopefully the rough edges on Mutt + imap have been sanded down. There are a ton of instructions floating around, and I got a little scared about “well what if this wasn’t 2.0 compatible”, so I figured I’d write down the steps again with ‘Mutt 2.0’ at the top for SEO purposes and maybe help the next poor soul.

Anway, first, go get a mutt-specific password here- https://security.google.com/settings/security/apppasswords

Then go plug it into

set realname = "$USERNAME"
set from = "$EMAIL"
set use_from = yes
set envelope_from = yes

set smtp_url = "smtps://${EMAIL}@smtp.gmail.com:465/"
set smtp_pass = "${APP_PASSWORD}"
set imap_user = "${EMAIL}"
set imap_pass = "${APP_PASSWORD}"
set folder = "imaps://imap.gmail.com:993"
set spoolfile = "+INBOX"
set ssl_force_tls = yes

From here, you should be able to type mutt, and see your inbox.

Luckily, Mutt is not breaking-change happy, so all of the 2 year old instructions in the wiki still work-

set spoolfile = "+INBOX"
set postponed = "+[Gmail]/Drafts"
set record = "+[Gmail]/Sent Mail"
set trash = "+[Gmail]/Trash"
# You need the "noop" bind so that the line editor accepts IMAP
# folders with spaces in their names. The gi, ga, gs and gd shortcuts help
# get around the "navigation quirks" mentioned above too.
bind editor <space> noop
# a steps over the default alias management 'a', but I am too lazy to use aliases,
# so I don't mind
macro index,pager a "<save-message>=[Gmail]/All Mail<enter><enter>" "Archive"
macro index gi "<change-folder>=INBOX<enter>" "Go to inbox"
macro index ga "<change-folder>=[Gmail]/All Mail<enter>" "Go to all mail"

Anyway… it still has the problems it used to. If you open up a large folder, like ‘all mail’, it blocks until it downloads headers for everything. Over 90k emails in my case, at ‘maybe go get a cup of coffee and come back’ speeds. Now, maybe I should have a smaller inbox, and get rid of mailing list archives from 2004, but I’m not sure I want to do a bunch of maybe-scary data deletion & organization just for a mail client that doesn’t do windowing. I also don’t want to sign up for syncing emails to a maildir again, it really hasn’t worked out in the past. So it’s not going to be a mutt year for me, still :/

12 Sep 2020, 15:35

which where what?

Finding a binary in your $PATH can sometimes be confusing. Especially when which mybin and whereis mybin don’t find it, but command -v mybin does, and worse, your shell finds it- so what is wrong with which?

It has to do with how you define your path.

export PATH=~/bin/:$PATH

will work with bash and command -v, but which and whereis aren’t hip to shell metacharacters, and won’t pick up anything in ~/bin/. Solving this is pretty easy, too-

export PATH=$HOME/bin:$PATH

19 Apr 2020, 09:53

Caddy and Cloudflare

self signed certs for running Caddy behind Cloudflare

I saw some goofy logs this morning-

acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Cannot negotiate ALPN protocol "acme-tls/1" for tls-alpn-01 challenge, url:
[ERROR] Renewing: acme: Error -> One or more domains had a problem:
[INFO] Unable to deactivated authorizations: https://acme-v02.api.letsencrypt.org/acme/authz-v3/4017030008
[INFO] acme: Trying to solve TLS-ALPN-01
[INFO] acme: use tls-alpn-01 solver
[INFO] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/4017030008
[INFO] acme: Obtaining bundled SAN certificate
[INFO] acme: Trying renewal with -3768 hours remaining

I had a Caddy server doing ACME challenges behind Cloudflare, it turned out. That wasn’t really working. I restarted Caddy, and then it just sat there trying to do its ACME challenge and not serving any pages.

Anyway, in case it ever helps anyone else, the magic Caddyfile incantation is

mydomain.com:443 {
    proxy  / localhost:1234 {
    tls self_signed

That :443 is the real trick, because without it, Caddy wants to run self-signed domains on port 2015, which doesn’t do anyone any good.

Systems administration. The gift that keeps on giving.

10 Dec 2019, 19:10

Really Canonical
 * Overheard at KubeCon: "microk8s.status just blew my mind".

Last login: Thu Nov 21 08:36:04 2019 from
hank@tinyserver:~$ microk8s.status
microk8s is running
cilium: disabled
dashboard: enabled
dns: enabled
fluentd: disabled
gpu: disabled
helm: disabled
ingress: enabled
istio: disabled
jaeger: disabled
juju: disabled
knative: disabled
kubeflow: disabled
linkerd: disabled
metallb: disabled
metrics-server: disabled
prometheus: disabled
rbac: disabled
registry: enabled
storage: enabled

I mean, really? That blew your mind?

I resent folksy advertising. Just plop a static ‘upgrade here’ link in the MOTD and leave me alone.

05 Oct 2019, 09:53

Weekend Update

I don’t know what I’m doing with my life but this doesn’t really seem like it should be it. Crouched with my head aching over a podcast and a mug of cold coffee staring at the sun outside dreading the resumption of duties and obligations. Happy Saturday everybody.