The thing I really need to do this week is take care of myself. Watch my sleep schedule, get my exercise, conduct my business in a fulfilling and sustainable way. This is something I have told myself a lot. It’s my own little Mount Everest. Lets give it another shot.
I’m a big believer in code reviews, possibly more than is warranted. It’s also true that it is hard work, and it is often hard to get started. So here are a list of the things I try to do, maybe they will help someone else.
Say nice things about nice code. We all have our ups and downs. Code reviews are often about preventing things from going wrong in the future- and it can be hard on the people in the present. I value people doing the work, so I want to communicate that. If a function looks clean, say so. If there is a workaround for an ugly wart in the language or framework, commiserate. Think of it like NBA players high-fiving their teammates after free throw attempts- everyone is a professional, no one technically needs it, it’s a little rote- but even perfunctory social gestures help. Even if everyone knows they’re a little forced.
File tickets for tech debt. As a reviewer, it’s more polite to create real tickets for focused followup work than to unload a dump truck of scope creep. I’m sort of in “treat the new hires well” mode in my personal life, but the same applies to long-time developers. The exception is when someone is doing a ton of work in the same section of code and just kind of creating a mess. If your org is too dysfunctional to allow you to work on tech debt tickets occasionally… I don’t know how to help, to be honest. That sounds like its own problem.
Double check that people aren’t re-implementing existing code. New developers on a project are particularly susceptible to this- they don’t know the codebase yet. It’s fine, it’s just about communicating.
Double check the names for naming conventions. This is the right time to make sure that everything isn’t called FooBar except for the one new feature where everything is BarFoo.
Run through the security checklist. Don’t build strings and send them to the
shell, especially if they have user input in them. Use your language’s execve(2).
Don’t build strings and send them to a SQL database, use parameterized queries.
User supplied data has got to get escaped in the template. Etc.
Run through the test checklist. Is it tested? Where is it tested? Do the tests exercise the usual boring edge cases- too much, too little, garbage input.
Run through the integration input. Does this require new monitoring? Is there a companion change to the monitoring configuration somewhere?
Anyway, once you’ve run through this list, you’ve read the code a couple times, you’ve thought about it a little, and you’re probably in a better place to think about it as a holistic thing. And all of that needs to get done anyway.
I don’t really do frontend work any more, so I never got around to actually using it until now. It’s amazing though-
<form>
<label name="title">Title:</label> <input name="title"></input>
<label name="question">Question:</label> <input name="question"></input>
<label name="answer">Answer:</label> <input name="answer"></input>
</form>form {
display: grid;
grid-template-columns: 100px 1fr;
grid-gap: 10px;
padding: 10px;
background-color: #eee
}
form label {
grid-column: 1; /* put the labels on the left */
text-align: right;
}
form input {
grid-column: 2; /* put the inputs on the right */
}And bam, a totally passable form:
That would have been… either a ton of extra markup to put it inside a table, or a fiddly float hellscape ten years ago.
The web is nice, I’ve missed it in my long night of operations plumbing.
I whipped this together to try and work on my blog less in vim. It wasn’t that bad, I should write more of these.
Since I collect abandonware container systems:
from quay, the coreos dockerhub competitor-
# sudo rkt fetch quay.io/coreos/alpine-sh
# sudo rkt run --interactive quay.io/coreos/alpine-sh --exec=/bin/sh
from dockerhub-
# sudo rkt --insecure-options=image fetch docker://alpine
# sudo rkt run --interactive docker://alpine --exec=/bin/sh
the dockerhub stuff also creates a fake rkt registry for docker-
# sudo rkt run --interactive registry-1.docker.io/library/alpine --exec=/bin/sh
will also work.
Finally, Quay mirrors the default Docker library under quay.io/dockerlibrary, so
# sudo rkt fetch quay.io/dockerlibrary/debian:9
Gets you debian.
syntax got me for a while, key is to realize the inside/outside distinction-
# rkt run --volume logs,kind=host,source=/srv/logs \
example.com/app1 --mount volume=logs,target=/var/log \
example.com/app2 --mount volume=logs,target=/opt/log
“volume” is outside, “mount” is inside, easy-peasy.
Clean everything up real quick-
rkt gc --grace-period=0s
One of the nice things about Concourse is that everything gets a normal, read-write directory tree to work in, but changes made aren’t persisted, so you don’t have to worry about temporary files, scratch work, mistakes, etc., interfering with other jobs down the line. It turns out you can do this yourself, and it’s not super hard.
overlayfs is a newer Linux filesystem, the new default Concourse filesystem driver, and pretty cool.
To use it, you don’t need anything fancier than good old mount and mkdir. There is, however, some setup, so let’s walk through the steps.
First, let’s get a git repo-
$ git clone https://github.com/hfinucane/jsonproc.git
Then, we’ll make some directories for overlayfs to do its work in-
$ mkdir Lower Upper Work
and finally, lets make the overlayed directory we’re going to be using-
$ mkdir ScratchBuildDir
And we’ll put it together-
$ sudo mount -t overlay overlay -o lowerdir=jsonproc:Lower -o upperdir=Upper -o workdir=Work ScratchBuildDir
The error message situation isn’t great, make sure to run dmesg | tail if
something goes wrong. That said, lets look at what we can do now-
$ cd ScratchBuildDir
$ go build -o jproc
$ ls
jproc LICENSE main.go main_test.go README.md
Now lets go look at our original directory-
$ cd ..
$ ls jsonproc
LICENSE main.go main_test.go README.md
It’s not there, all our work is isolated in the scratch build directory. Other directories, however, have been affected-
$ ls Upper
jproc
When you tear down the overlay, they will remain-
$ sudo umount ScratchBuildDir
$ ls Upper
jproc
so if you’re building your own system with safe, ephemeral working directories, you’ll need unique Upper directories, or you’ll need to clear them out between uses.
I haven’t really touched on the Lower directory. overlayfs doesn’t want to
let you run without doing any overlaying, so you have to overlay something on
something, even if it’s just an empty directory for this example. If you were
writing a container-based build system, Lower might be the OS tree, and you’d
want the git checkout to be nested inside of var/tmp/build or something.
You’re not limited to two directories either- jsonproc:Lower:Lowest will stack
jsonproc on top of Lower on top of Lowest.
Software development is hard, working with other people is hard. Making sure you never skip any steps is hard, reminding other people to not skip any steps is harder. If you are really strict about it, you are a jerk, if you are not, you are a vindictive jerk. The airline industry solves this with checklists, but something about office workers really resists a checklist. It is an admission that you do not know everything, or that your contribution is fungible. Gloomy assertions aside, I have had good results with computer-run checklists.
Continuous integration, or ‘making a computer run the tests’, is table stakes for responsible professional software development. This will work, and be reliable, and is a reasonable place to stop, but there are a bunch of things around the edges that can overwhelm. The most common problems are testing too late, unspecified or underspecified build environments, and management of the build system.
The problem dearest to my heart is that testing the ‘master’ branch is testing too late. That branch is where other developers start when they go to fix a bug, or add a feature. Starting from a broken place can waste an enormous amount of time, and is the sort of thing that leads to hurt feelings. Instead, we should test every branch, as soon as possible, and publish those results.
At a the most basic level, this gives developers a responsible robot, never forgetting to run the tests. There are social benefits too- it means that when another developer goes to review a pending change, they don’t have to go and double check that the checklist has been followed. This makes it easier to concentrate on higher level concerns, and prevents embarrassing “that can’t work” moments that erode a team’s trust.
The management and evolution of the build system should not be an afterthought. It is a crucial part of your workflow, it can be an enormous force-multiplier, and it should be treated with the same care and process as a production environment. Many build systems were built with manual administration through a web ui as a first class citizen, and automation as a second class citizen, and it shows when you try to be rigorous with them. I can be a little dogmatic on this point, but all of your configuration should come from source control. The build system is no exception. Your team knows- or really should know- how to use source control to look at why things changed, your team can use source control to revert changes precisely, your team should use source control to review changes to the build code, and your team should be aware of and reviewing changes to the build code.
Now that I have staked out a vague, philosophical stance, what about concrete recommendations? You should look at Concourse. It is not an out of the box solution for all build problems, but it is quite serious about solving the build management problem, and the rest falls out of that.
After self-actualization, a little known fact is that the next, tiniest part of the pyramid is the ability find your cell phone before you leave. The normal solution is to have someone call it for you- sound being a part of the hierarchy of senses- but if you’re alone, trying to leave, and you can’t find the thing, what then?
I bought this little board for almost nothing, got a trial account at Twilio, and hacked up a little program.
Calling Twilio was most of the work-
type TwilioPlay struct {
Name xml.Name `xml:"Response"`
Say string `xml:",omitempty"`
Play string `xml:",omitempty"`
}
type TwilioResponse struct {
Sid *string `json:"sid"`
DateCreated *string `json:"date_created"`
DateUpdated *string `json:"date_updated"`
DateSent *interface{} `json:"date_sent"`
AccountSid *string `json:"account_sid"`
To *string `json:"to"`
From *string `json:"from"`
Body *string `json:"body"`
Status string `json:"status"`
Flags *[]string `json:"flags"`
APIVersion *string `json:"api_version"`
Price *interface{} `json:"price"`
URI *string `json:"uri"`
// Optional exception params
Message *string `json:"message"`
MoreInfo *string `json:"more_info"`
Code *int `json:"code"`
}
func Call(toNumber, fromNumber, sid, token, twimlURL string) {
u := url.URL{
Scheme: "https",
Host: "api.twilio.com",
Path: path.Join("2010-04-01/Accounts/", sid, "/Calls.json"),
}
q := u.Query()
q.Set("To", toNumber)
q.Set("From", fromNumber)
q.Set("Url", twimlURL)
r, err := http.NewRequest("POST", u.String(), strings.NewReader(q.Encode()))
if err != nil {
panic(err)
}
r.SetBasicAuth(sid, token)
r.Header.Add("Accept", "application/json")
r.Header.Add("Content-Type", "application/x-www-form-urlencoded")
resp, err := (&http.Client{}).Do(r)
if err != nil {
panic(err)
}
var tresp TwilioResponse
body, err := ioutil.ReadAll(resp.Body)
err = json.Unmarshal(body, &tresp)
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
// fine
} else {
panic(fmt.Sprintf(*tresp.Message))
}
}
Possibly not best practice, but my theory here is to just crash the program
when things go wrong, and let systemd restart it. So there’s no real error
handling anywhere, just unconditional panics.
To trigger the thing, I hooked up a button to the board’s GPIO pin 18. linux-sunxi.org has good documentation, but as someone without much experience with the conventions of little single board computers, it took me a while to figure out how to map their GPIO table to the pins on the board. The trick is there is a little ▶ character next to pin 1- count from there left to right, top to bottom.
The GPIO filesystem
is very unixy, and made it really easy to test that the wiring was working.
Hold the button down, cat /sys/class/gpio/gpio18/value. Let go of the button,
cat /sys/class/gpio/gpio18/value. I spent some time wandering around looking
for a Go library, and had some bad luck,
so I went ahead and wrote my own. The documentation mentions that
If the pin can be configured as interrupt-generating interrupt
and if it has been configured to generate interrupts (see the
description of "edge"), you can poll(2) on that file and
poll(2) will return whenever the interrupt was triggered. If
you use poll(2), set the events POLLPRI and POLLERR. If you
use select(2), set the file descriptor in exceptfds. After
poll(2) returns, either lseek(2) to the beginning of the sysfs
file and read the new value or close the file and re-open it
to read the value.
So I went ahead and did that-
func poller(pin string, values chan []byte) {
var fds []unix.PollFd = make([]unix.PollFd, 1)
fd, err := os.Open(pin)
fds[0] = unix.PollFd{Fd: int32(fd.Fd()), Events: unix.POLLPRI | unix.POLLERR}
for {
if err != nil {
panic(err)
} // everything is busted
n, err := unix.Poll(fds, 10000)
if err != nil {
panic(err)
} // everything is definitely busted
if n == 1 {
contents, err := ioutil.ReadAll(fd)
if err != nil {
panic(err)
} else {
values <- contents
}
_, err = fd.Seek(0, 0)
if err != nil {
panic(err)
}
}
}
}
Throwing it all together-
func main() {
var pin string = "/sys/class/gpio/gpio18/value"
throttle := time.Tick(time.Second * 30)
values := make(chan []byte)
go poller(pin, values)
for {
edge := <-values
// we're doing pulldown
if bytes.Compare(edge, []byte{'0', '\n'}) == 0 {
Call(call_number, from_number, os.Getenv("SID"), os.Getenv("TOKEN"), twimlURL)
<-throttle
}
}
}
I needed the goroutine and channel for the initial implementation, but since I
switched to poll(2), I guess I could just have one loop.
systemd, as always, makes the whole “I would like a supervised process” problem trivial-
[Unit]
Description=Monitor a GPIO port, and call a phone when it goes low
[Service]
Environment=SID='{{sid}}' TOKEN='{{token}}'
# Turn on the GPIO port
ExecStartPre=/bin/echo "18" > /sys/class/gpio/export
# Set high/low behavior
ExecStartPre=/bin/echo both > /sys/class/gpio/gpio18/edge
Exec=/usr/local/bin/findphone
Restart=always
[Install]
WantedBy=multi-user.target
Hopefully this is helpful to someone somehow.
I like Gnome Builder, and I had the copious free time required to be the change I wanted to see in the world. So I looked into writing a Go plugin.
A directory, in /home/hank/.local/share/gnome-builder/plugins/${plugin}, an
empty python file, called ${plugin}.py, and a .plugin file, maybe
${plugin}.plugin. Maybe something like this:
[Plugin]
Name=Go Plugin
Module=go
Loader=python3
X-Project-File-Filter-Pattern=*.go
X-Project-File-Filter-Name=Go Project
This is enough of a skeleton to convince Gnome Builder that folders with Go files constitute a Go project, and will make it easier to navigate the ‘Open Project’ dialog.
For this, we’ll fill out ${plugin}.py:
#!/usr/bin/env python3
import gi
from gi.repository import Ide
class BobBuildPipelineAddin(Ide.Object, Ide.BuildPipelineAddin):
def do_load(self, pipeline):
context = pipeline.get_context()
srcdir = pipeline.get_srcdir()
# Register a BUILD action, which will get run when the user hits 'build'
get_launcher = pipeline.create_launcher()
get_launcher.set_cwd(srcdir)
get_launcher.push_argv("go")
get_launcher.push_argv("build")
get_stage = Ide.BuildStageLauncher.new(context, get_launcher)
self.track(pipeline.connect(Ide.BuildPhase.BUILD, 0, get_stage))
def do_unload(self, application):
pass
def _query(self, stage, pipeline, cancellable):
stage.set_completed(False)
And that’s it- click ‘build’ in Builder, and get a build.
There are a bunch of different build phases, I haven’t done much with them,
although DOWNLOAD seems to map well to go get, INSTALL with go install,
and go generate seems like it would map well to AUTOGEN. The Build Phase
announcement
is sort of tantalizing.
I pieced this together from the official docs, which are good, if incomplete. The upstream plugins are also a great resource.