Map/Reduce at an Election
As is my habit, I was serving as a polling clerk during the last elections for the EU parliament and city council. This was back in 2024.
The city council elections in the German state of Brandenburg are interesting. The ballots are huge because the votes are cast for persons, not parties, and each party can nominate several people. This time, some of the parties had no less than 14 candidates listed. Also, voters can distribute three votes over all of the candidates on one single ballot. In other words, there’s a lot of circles (three per candidate) on the ballot, each of which can be marked, and no more than three marks on the entire ballot are allowed (less are OK, of course).
The city had prescribed a peculiar way of counting for these ballots. Of the eight polling clerks, two were supposed to analyse (four-eyes principle is good) each single ballot and then announce the result for that ballot to the remaining six clerks. Each of these would have a list in front of them, which would count the votes for a slice of the candidates. Basically, the city expected us to play bingo.
Now, at just over 700 ballots, and a rough estimate of 20 seconds to “get” one ballot, we’d have ended up at something like four hours to count everything. During the process, two of us would have had to look closely at 700 pieces of paper, Argus-eyed, while the other six would mostly have sat around waiting for their candidates to be announced.
I felt that that was neither a good (even) distribution of work, nor a good utilisation of available resources (eyes and brains).
I recalled the map/reduce pattern, and suggested we apply it like this. Forming groups of two, pairs of people could analyse ballots and maintain a vote count for all of the candidates (“map”). We would then simply add up the vote counts for the candidates afterwards on the official lists (“reduce”).
Initially, folks were skeptical, but when I ran them through the math (700 ballots, 20 seconds each, four hours, divide by four thanks to parallelising the work, end up with one hour), they were convinced. We started counting, finished almost exactly an hour later, and put the results together.
I love it when a plan works, and applying computer science principles to other fields can be so much fun.
Tags: the-nerdy-bit
Agile Onion
A Scrum Master I was working with once pointed me to the concept of the Agile Onion. In a nutshell (pun intended), there are five layers to the onion. From outermost to innermost, they’re mindset, values, principles, practices, and tools and processes.
It may look counterintuitive that mindset isn’t at the core and why tools and processes aren’t the outermost layer. It does make sense though: everything begins with the mindset, once you’ve adopted that, you can go deeper, until you reach the concrete implementation.
This reminded me of seniority (as in: maturity, not: age). Seniority, too, comes with a mindset. Let’s take engineering as an example domain. A truly senior engineer will not engage in programming language or editor wars and tabs-vs.-spaces conflict unless in jest. Knowing a broad range of tools and applying the one that’s most fit for the job is what makes a senior. Of course there are preferences, but they’re not fussed about.
In that vein, it would be fair to say that a solid agile mindset denotes a certain seniority when it comes to processes. Not bad.
Tags: work
Mini Raytracer
This, right here, shows off a little raytracing animation implemented in 256 bytes of HTML and JavaScript. I’m baffled. The programmer has pulled some very nifty tricks, all of which the web page describes in detail. The ingenuity reminds me of things I’ve seen in the C64 demo scene.
Tags: hacking, the-nerdy-bit
Fix Bugs
I’ve long held the opinion that a bug’s life should be short. (Sorry, Flik.) My former manager Holger Hammel here describes the idea of a “Zero Bug Backlog Policy”. In brief, the idea is to either fix a bug at once, or to discard it as “won’t fix”. Holger also lists a range of typical questions and responses. I feel that one - frequent in case of legacy or not-so-well-working software - isn’t prominent enough:
Our system is so buggy that we’ll take considerable time to properly address everything. Should we do that?
Honest answer: heck yes.
Any bug (that’s really a bug) is a debt owed to customers and internal stakeholders. It should be obvious why I mention customers. Internal stakeholders suffer from the bug because if it keeps rearing its ugly little head, it creates a constant distraction for multiple parties. Engineers have to address the immediate effects, distracting them from feature work (which stakeholders are usually interested in). Stakeholders, e.g., in customer service, have to keep the customers calm. Everybody has to make excuses because of that bug. So, rather fix it already.
If there is a bunch of bugs, the distractions pile up. That doesn’t scale. So, again, heck yes, fix them already. It will take some time now but ensure much smoother delivery in the future. Everyone involved should appreciate that.
Tags: work
Mlem
A board game featuring cats in space? Of course! Here’s Mlem.
Each player represents a cat, who together embark on a trip to outer space to take control. Because that’s what cats do. Along the way, a lot can go wrong - and when the fuel runs out, the spaceship crashes. As cats have multiple lives, that’s fine for a while, just embark the next spaceship and start over. While on the spaceship, the cats are mostly a crew - but they have their own interests to inhabit planets and make it to the farthest possible outpost.
Mlem is a dice roller. The dice control how far the spaceship can go in a move. Beware: dice get depleted, and not all dice can be used at every step along the trip. If the final dice are used up, or if no usable dice are rolled, the spaceship crashes.
During each move, the cats have the option to perform one of several actions. Disembark, boost the spaceship, kick someone off, let a die disappear, ensure a soft landing (avoid a crash), and so forth. Picking the right actions at the right time is key to success. The more planets and moons are inhabited, the more points a cat can get - and planets farther out in space yield more points. Reaching outer space, obviously, yields the most points.
The game is very easy and quickly to learn, fast-paced, and fun. Mlem gets bonus points for the printed neoprene playmat. Cardboard wears out quickly and always is a bit awkward to fold back in place. The playmat is simply unrolled for the game, and afterwards, just as simply rolled up again. Also, the mat has a better grip on the table surface.
So, all in all, I warmly recommend this as a fun game for families or friends.
Tags: games
To Microserve or Not to Microserve
“You probably don’t need microservices” says this article. It’s easy to dismiss that as flame bait. The article is quite sensible though, looking at typical scenarios for employing microservices and how that can go wrong depending on the surroundings.
In a nutshell, here are the article’s examples for how microservices can go wrong:
Some companies end up having more microservices than developers. That’s a nightmare to maintain.
Low coupling and high cohesion are harder to get right in microservices. Lots of little services require lots of collaboration across teams if something needs to change.
Services are islands of knowledge, and the bus factor really hits hard that way. Fewer services make for more simplicity, not more services.
When a startup starts with microservices right away, then bam! we have an instant maintenance nightmare.
The article doesn’t really nail where the hidden costs of microservices are, but the examples give an idea. Its sensible bottom line is that if microservices limit a company’s ability to innovate, a strategy to reduce operational overhead should be introduced. This can be as simple as merging services. After all, they don’t magically turn into big ugly monoliths that way.
GAM FTW
I want to sing some praise for GAM. This is a command line power tool for messing around in Google Drive (and then some).
A while back, I had been facing a bit of a tough time when all of a sudden files I had inherited appeared in my Google Drive root instead of in a dedicated folder with the original owner’s e-mail address as its name. Several thousands of folders and files made my Google Drive practically unusable. Moving everything away manually was not an option. Cream on top, a subset of these files (still thousands strong) were shared externally, which needed to be revoked.
GAM helped considerably, and saved me a lot of time. All in all, I spent about a day’s worth of time figuring it all out, but doing this manually would still have taken me longer.
Setting up GAM is a bit involved, and requires a lot of dancing around in the Google Admin Console, creating projects and assigning permissions. My wonderful internal IT team helped me with that, as proper super-admin rights are required for the setup. That makes GAM less than perfectly convenient: you can’t just download and start using it.
After that, I was working with a list of all the IDs of the files I had inherited, neatly assembled in a CSV file. I first had to reduce this list to the IDs of those in my Drive root. I used GAM to get a list of all file and folder IDs in the Drive root, and then some Unix command line magic to filter those that were inherited into another CSV file. Moving these to a dedicated folder was a one-liner using GAM:
gam \
csv files_in_root.csv \
fields "fileID" \
showcmds \
gam user <my_user_id> \
update drivefile "~fileID" parentid <my_foleder_id>That’s a mouthful. The gam csv command processes a CSV
file line by line, and applies a degree of parallelism to save time.
This is all automated, and works flawlessly. The
fields "fileID" bit is because the CSV file is just a
sequence of IDs, without a header row. The showcmds part
makes the individual commands show up on the console.
The second part, starting with gam user, is the function
applied to each line in the CSV file;
update drivefile ... parentid is GAM’s way of saying “move
this file to that folder”.
Running this took about 20 minutes.
Next, I had to unshare the inherited files, but not all of them: files shared with the mothership company should remain in that state. This I did in two steps. First, I obtained the ACLs of all inherited files, and stored them in a CSV file:
gam \
redirect csv ./acl-collection.csv \
multiprocess csv all_inherited_files.csv \
fields "fileID" \
showcmds \
gam user <my_user_id> \
print drivefileacls "~fileID" oneitemperrowAgain, it’s a mouthful, but in the end,
acl-collection.csv contains all inherited files’ ACLs, one
entry per row. Unsharing everything not shared inside my company or with
the mothership goes like this:
gam \
config csv_input_row_drop_filter
"permission.domain:regex:(mycompany.com)|(mothership.com)" \
csv ./acl-collection.csv \
gam user <my_user_id> \
delete drivefileacl "~id" "id:~~permission.id~~"The first part instructs GAM to drop all rows from the ACL collection that indicate sharing with domains belonging to my company or the mothership. The remaining lines are then processed by deleting the respective ACL entries.
It took about 120 minutes to process all 23,000 inherited files like that.
Only one file was erroneously unshared, and was quickly added back.
I celebrate the power of the command line.
Tags: hacking
Culture is a Commitment
Here’s a controversial one. It is often said that companies should not hire for culture fit, because diversity is favourable and hiring should rather enrich the culture. I think companies should hire precisely for culture fit, and avoid hiring people that aren’t cultural matches.
At first, that sounds like a complete contradiction. I don’t think it is, especially when it comes to the diversity bit. Diversity of thought, when brought to a foundation that encourages healthy conflict (as in: conflict of ideas), can only be good. Diversity of culture, however, is detrimental.
A company’s culture is the sum of its actions and behaviours. If nurtured right, it is the very outcome of a well-understood and carefully picked set of cultural values. Hiring someone who disagrees with those can’t be right, because it would lead to a deterioration of the culture, rather than enriching it.
Disagreement with the culture, with the cultural values, is a no-go. Otherwise, what are those values worth to begin with? Consequently, when a company hires, it should hire people it expects to strengthen the culture, not weaken it.
That is independent of the aforementioned diversity of thought - the “foundation” bit above is key. This foundation is the culture. As long as there’s agreement with the culture, a broad variety of views and opinions can only grow the cultural strengths.
Tags: work