little ci/cd tweaks

August 14, 2022

Did you know that the best way to avoid car accidents is not driving a car?

what is the problem?

You and your team may be wasting computing cycles and discussion time on things that shouldn’t really matter, like linting, code style, and formatting. Here’s a way to save CI/CD time and shorten the feedback loops in your team.

suggested solution

If you run your linters and unit tests on the runner, try moving them to the developer’s machines.

You may not only save precious dollars of cloud costs that your department head has to beg the finance for, but also reduce developer bickering about just a few simple things around code style and other functionally inconsequential aspects of your codebase.

This may not matter if your team is small. But if your organization is growing and costs and CI time slowly is bubbling up, read on.

It’s an example of the implementation of the shifting left idea - moving certain activities earlier in the development lifecycle. If you imagine the flow of time as a line from left to right, with your developers on the left and production release on the right, you’ll see it in your mind the way I do.

what to shift left?

There are heaps of things you could be shifting left to help you deliver those sweet performance gains and shorten development lifecycle, but today I want to focus on something that as an engineer I honestly never want to be dealing with: formatting, style, and linter discussions in the PRs. And if you are not keen on moving your codebase to go and using gofmt (hail gopher), maybe this piece of advice you’ll find handy.

my take on it

In most projects I work on I implement wrappers around git push that enable me and my teammates to deliver PRs to each other that have already been groomed. An example wrapper could be as simple as this:

import subprocess

# take note this is a simplified example
# please be more judicious and careful
# `your_custom_assertion`is a placeholder for well, you know, your custom assertion, whatever it may be
def push_wrapper():
	command = "your_linting_command"
	result = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
	if result.stdout == your_custom_assertion:
		subprocess.call("git push origin main")
	else:
		exit(1)

I toss this into a ./deploy.py script and call it a day. What assertions could you exercise? For example in my python projects I keep using those:

  • black to format code
  • flake8 to check style against PEP 8 and other problems
  • isort to sort import statements

my motivation for the solution

Why do I choose these particular ones? Cause developer conversations in PRs about problems that the above tools solve are an absolute waste of time (and money). People paid $150k+ a year spending hours a month talking about code style isn’t the best use of an engineering team’s effort. Cost savings wise, it is a two-birds-with-one-stone kind of situation, since you’re literally cutting the CI runner time as well as developer efforts.

You can also just blame it on the bot. It redirects the emotional efforts that would be aimed at each other, at the machine. Well, unless someone changes the config of those tools too often that is… But that’s a story for another time.

And there you have it.

addendum

I’ve received a lot of replies to this post saying how those tools should be a part of every developer’s IDE/workflow and not be a concern of the PR or CI steps. I agree. In a perfect world that’s what would happen, but implicit team agreements don’t ever work, and no one wants to be told what tools to use, especially engineers. Hence why I like the PR script approach. It’s a simple, low cost solution. And developers who will be irked by the fact that they’re stopped at the git push level will employ those practices and use this tooling in their environments eventually anyway.


Written by Daniel Kaczmarczyk, a software engineer and educator. you can find me on twitter or email me at daniel.kacz@protonmail.com

a pale blue and yellow circle