Links - 7^th January 2024

Climate instability means more uncertainty in the larger context of infrastructural systems. As the environment in which they’re embedded becomes more variable, that means de-optimizing these systems. It means investing in making them reliable under a wider and less predictable range of conditions and putting in the safeguards necessary to prevent outlier events from becoming natural disasters, where the highest human costs are paid by the most vulnerable. If the twentieth century was about building infrastructural systems that could manage or even just displace risks, the twenty-first century is going to be about building out systems — human as well as technological — that are resilient enough to sustain our communities during conditions of ongoing uncertainty.

What else do you expect your code to smell like

Presumably, a kernel language is a minimal language on top of which a larger language can be built. Since its primary function is being an implementation language for function libraries and a target language for macros, a Kernel language need not be particularly “user-friendly” or provide a lot of flexibility in its interface. Although it is desirable, a Kernel language does not have to be a strict subset of the full language, since some of its features may not be visible in the full language. A Kernel language should be simpler and more efficient to implement than the full language, however, to avoid an abstraction inversion,in which a simpler notion is defined in terms of more complex notions.

I see a similar dichotomy with software projects. Certain technology decisions lead to projects that are warm-blooded: everything is great when there’s constant motion on the project, generating heat. But put warm-blooded software in the freezer, and you’ll pull out a corpse six months later.

Maybe your CI isn’t working because one of the services you depend on got bought or ran out of money. You add a new dependency and find yourself needing to upgrade your compiler. Another package you depend on is deprecated, and doesn’t work with the latest version of the compiler.

Some projects are different. You work alone, make some changes when you’re inspired, and then don’t touch it again for another year, or two, or three. You can’t run something like that as a warm-blooded project. There’s not enough activity to keep the temperature up.

A cold-blooded project is like the baby painted turtle. You can freeze it for a year and then pick it back up right where you left off.

A cold-blooded project uses boring technology. The build and test scripts don’t depend on external services that might change, break, or disappear entirely. It uses vendored dependencies.

Meanwhile, as information travels longer distances, you get all the normal information loss problems coupled with the biases at each hop. “I don’t want it to seem like we’re raising too much of a fuss”, says the middle manager. “I think we need to be more pragmatic”, says the VP (who doesn’t need to do any of that pragmatic work). When leaders finally get frustrated enough to dig into the details, they are horrified to see how hard it is to get actual work done and quickly close up that box (“Come on, we need to simplify! It can’t be this hard!”).

The folks on the front lines will do anything at this point to spend less time in soul-sucking meetings. So they often just throw up their hands and say “Yes” without knowing what they’ve said yes to and knowing that the details will almost certainly change, and timelines outside their control will slip so that they will be off the hook anyway.

Big enterprises pondering a “transformation” are fine doing SaFE because they have nothing to lose. The rapidly growing company wants to hold on to the illusion that it is nimble and people are independent, sentient beings instead of cogs.

[…]

In high-growth companies, you tend to observe far more dependencies than you expect when you get into the details, but the “cost” of those dependencies tends to be muted and opaque.

When the number of dependencies in your company increases, you will need:

1. An increasingly formalized mechanism for making “tie-breaking” decisions. Should team A help team B or team C? This mechanism can be a system — an apples-to-apples prioritization heuristic, for example — or it can be a person or group of people (or both).

2. To start thinking about limiting work in progress across an increasingly larger boundary of teams, groups, and departments. An independent team can focus locally on its work-in-progress limits and utilization rates. When things go wrong, the impact is local. As teams become increasingly dependent, you must think about flow and utilization at the team-of-teams or organization-wide level; otherwise, things can get out of control very quickly. In effect, you have one big team.

3. To start thinking about capacity allocation and return on investment on an org-wide level. Imagine you have 100 independent teams. With 100 independent teams, you can ask questions like "How much do we pay team X, and how much money did they make us?" Even with a moderate number of dependencies, limits on flow, throughput, and impact start to take on a more global flavor. An uncomfortable high % of all money spent on salaries is going to context switching, handoffs, teams being blocked, and meetings about surfacing, advocating for, coordinating, and re-coordination dependencies.

There’s a lot of nuance in this example. Still, the big takeaway is that many of the pains related to dependencies again have less to do with dependencies and more with the surrounding situation. If an org incentivizes extremely high WIP, it doesn’t matter how you try to approach dependencies. If things are going slow and trust is low, you’ll just propagate that into any work you attempt. If teams are already burning 50% of their time around debt, adding dependencies will crush their productivity. If teams can’t say "no, " they’ll keep starting without finishing their dependency work. 

Changing the standard a code base compiles against is a new agreement between the programmer and the implementer; it is akin to changing your code (as is changing your compiler), and there should be no expectation that everything remains identical to how it was before. Older functionality will be deprecated, newer functionality will be added, bugs will be fixed, etc. New standard == new treaty.

I think we should be careful about blind adherence to this principle. It’s obviously true that there’s a lot of economic value tied up in existing C code bases and that rewriting that code is both expensive and error-prone. However, C has undergone multiple revisions in its long history. A good idea in 1978 is not necessarily even acceptable practice in 2024. I would love to see this principle updated to set a time limit, along the lines of: existing code that has been maintained to not use features marked deprecated, obsolescent, or removed in the past ten years is important; unmaintained code and existing implementations are not. If you cannot update your code to stop relying on deprecated functionality, your code is not actually economically important — people spend time and money maintaining things that are economically important.

There’s an interesting inversion here. The job of this hinting program is to keep the visual appeal of the font, but it accomplishes this by intentionally distorting the proportions of each glyph.

And then there is, finally, McKenzie’s claim that extremists must not be censored, but defeated in the marketplace of ideas. Not only does this completely ignore that there is no “marketplace” in which ideas are being judged purely by their intrinsic quality, but there is power, and status, and access. It also implies that there is something worth considering, still worth debating about Nazis, something that has not been defeated yet — as an idea, as a way to order politics and society. But, look, no ideology has been studied more, and the first half of the twentieth century actually happened: We know what these ideas lead to in practice. They have been defeated in every way you can imagine, including on the literal real-world battlefield. There really is no need to take anyone seriously who believes Nazis need to be defeated in debate clubs, or that their ideas will organically go away once exposed to the public.

A surprising set of wrong decisions generally happens after the investigation following major incidents

Those accounts used by individual users for interactive logins need to be differentiated from the accounts used by the application — the service accounts. In terms of user/role-level settings.PostgreSQL allows us to have parameter settings at different levels and scope. More stringent restrictions are to be placed on the interactive login accounts. It is very much OK to have settings like idle_session_timeout for those accounts, preferably not exceeding 5 minutes. Most importantly, the idle_in_transaction_session_timeout also does not exceed a few seconds. On the other hand, I would recommend NOT to use idle_session_timeout for service accounts. But using idle_in_transaction_session_timeout for a couple of minutes is acceptable as it helps us to find problematic application logic.

A data quality issue can be sorted out later. Somebody dying because a doctor was arguing with a computer instead of doing a life-saving surgery, can’t.

And that’s even before I learnt the Biological Period, that strange punctuation mark that appears in medical texts all the time. It looks just like a period, but it’s pronounced But Biology Is Messy, And Sometimes There’s Exceptions. Some people’s hearts are on the right. Some people have six toes. Some people might well have two appendixes, so just because one was taken out three years ago is no good reason for a computer to argue a doctor wanting to schedule a new appendectomy. There will be time to count the appendixes later. Saving a life can’t wait.

And that’s why, in 2023, my computer represents timezone offset with a field large enough to exceed the lifetime of the universe by a factor of 40.