Archiloque weekly

Links - 8th July 2019

Hypothetical Indexes in PostgreSQL

The execution time of an SQL in most of the relational databases depends on the cost of the execution. An optimizer chooses the plan that has the lowest cost, and thus considers further phases such as execute and fetch using that plan. One of the easiest ways of optimizing a well-written SQL is through appropriate indexes that are suitable for that query. An index may reduce the number of pages fetched from disk and may live in the cache due to its size (<<< table size). So, indexing is always a low-hanging fruit for admins and developers wishing to tune an SQL.

But often developers wish to see if an index can be really helpful. The only option available is to create the index and check to see if the queries are able to use it with a reduced cost. Creating the index may be fine if it is in a test environment, or a table that is not huge in size, but if for some reason you wish to see if an index can be helpful for an SQL without actually creating it, then you may test and try this extension to create hypothetical indexes.

Information Maintenance as a Practice of Care: An Invitation to Reflect and Share

Acts of maintenance sustain and repair people and things, and include the many actions, large and small, that keep our sociotechnical world going, as well as the interfaces we design to function between and among information systems. Maintenance is not the opposite of change, however, and its primary aim and value is not to uphold stasis. We view acts of repurposing and revision or reuse as part of maintenance, and observe that, traditionally, organizations overseeing complex technologies have used moments of maintenance and repair not just to sustain, but to upgrade and re-imagine their systems. Similarly, moments of upgrade and re-imagining can also resource and provide justification for maintenance processes.

Where maintenance-minded approaches do promote continuity, they should not be uncritically conservative of systems of historical or contemporary oppression; nor do we wish to valorize the maintenance of systems that exacerbate other harms, such as those resulting from or contributing to anthropogenically-driven climate change, surveillance and exploitation of people through monetized data capture, or discrimination based on race, class, gender, or other perceived differences. Generally, however, we argue that the value and vitality of practices of maintenance are not adequately recognized in cultures of the new, which prize, reward, and orient themselves toward “disruptive” innovation. The devaluation of maintenance in such contexts holds true across fields and can encompass a lack of recognition of traditional or Indigenous knowledge-keeping as well as of the everyday ingenuity of people’s adaptations to altered circumstances and infrastructure.

Mistakes we made adopting event sourcing (and how we recovered)

Confusion between event-driven and event-sourced architecture

In an event-driven architecture, components perform activity in response to receiving events and emit events to trigger activities in other components. In an event-sourced architecture, components record a history of events that occurred to the entities they manage, and calculate the state of an entity from the sequence of events that relate to it.

We got confused between the two, and had events recorded in the history by one component triggering activity in others.

We realised we’d made a mistake when we had to make entities distinguish between reading an event in order to react to it, and reading an event in order to know what happened in the past.

This also led to us…

Using the event store as a message bus

We added notifications to our event store so services could subscribe to updates and keep their projection up to date. Bad idea! Our event store started being used as an event bus for transient communication between components, and our history included technical events that had no clear relationship to the business process. We noticed that we had to filter technical events out of the history displayed to users. For example, we had events in the history about technical things like “attempt to send email failed with an IOException”, which users didn’t care about. They wanted to see the history of the business process, not technical jibber-jabber.

The literature describes event-sourced and event-driven architectures as orthogonal, and that tripped us up. We came to realise that clearly distinguishing between commands that trigger activity and events that represent what happened in the past is even more important than Command/Query Responsibility Segregation, especially at the modest scale and strict consistency requirements of our system. The word “event” is such an overused term we had many discussions about how to name different kinds of event to distinguish between those that are part of the event-sourcing history, those that are emitted by our active application monitoring, those that are notifications that should trigger activity, and so on. In our new applications we use the term Business Process Event for events recorded in our event-sourcing history.

ConstExprPreter — clang constexpr interpreter

TL;DR: Fast interpreter for C++ constexpr to replace the existing tree evaluator

I am a PhD student at the University of Cambridge currently interning at Apple, working with JF Bastien and Michael Spencer on improving constexpr performance. Constexpr evaluation is presently slow and difficult since it relies on a monolithic AST walker. We propose replacing the tree evaluator with a bytecode interpreter to improve compilation times. The tree evaluator also poses significant maintenance and scalability problems, which we intend to ameliorate using the interpreter. While being generally faster, the interpreter does present two critical issues: a slightly increased memory footprint and added complexity to the compiler. This tradeoff is justified, as efficient constexpr evaluation could prove to be valuable as the language evolves.

We would like to integrate this interpreter into clang. This RFC details the benefits of an interpreter and describes an initial implementation, along with a roadmap for replacing almost all of the tree evaluator with the interpreter. Even without optimizations, the performance of the interpreter matches that of the tree evaluator, thus the short-term focus is on feature completeness, not evaluation speed, as reflected by known inefficiencies in the current implementation. We would highly appreciate comments related to integration into clang and our roadmap towards replacing the evaluator, as well as feedback on the initial patch. This project serves mostly as a prototype in order to determine what kind of bytecode and compiler is required to efficiently evaluate code and emit useful diagnostic messages, paving the way for a future fast, potentially JIT-ed, interpreter.

Distributed Tracing — we’ve been doing it wrong

The foundational premise of a microservices architecture is that as the business requirements grow more complex, so will the organization structure. Proponents of microservices argue that decomposing different business functionality into standalone services will enable small, autonomous teams to own the end-to-end lifecycle of such services, unlocking the ability to build, test and deploy these services independently. However, since any such decomposition comes at the cost of a loss of visibility into how each service interacts with the others, distributed tracing is purported to be an indispensable tool for debugging the complex interactions between the services in unison.

If you truly have a mind-bogglingly complex distributed system, then no one single person can have a complete understanding of the system in their head at any given time. In fact, building tooling under the assumption that this is even possible or desirable seems a bit of an anti-pattern. What’s ideally required at the time of debugging is a tool that’ll help reduce the search space, so that engineers can zero in on a subset of dimensions (services/users/hosts etc.) that are of interest to the debugging scenario at hand. Requiring engineers to understand what happened across the entire service graph at the time of debugging an incident seems counter to the ethos of microservices architectures in the first place.

Designing for Coziness

Dear designer whom I care for,

I wish for you that game-making be a refuge from the storm. I take joy from the games you make, and I hope you feel fulfilled when you make them. As a colleague, I want you to feel safe to express your inner self, to take creative risks in your craft. As a friend, I wish that you can escape the ever-present hurry and pressure of our industry and world, into a restful, healthy practice.

If you feel comfortable, I invite you to make a game that reflects those moments in your life that were meaningful, where you were content and cared for. I invite you to make a game that offers moments for players to reflect and be at ease. You don’t have to show it to me; you don’t have to share it with anyone. But I would like to be a companion in the journey towards cozier games, and I think others would, too, if you would have us.

It’s difficult and slow and I’m probably asking a lot from you. But if you try and fall short of your expectations, please know that I will still support and celebrate you. I care about you, and your work is but a small part of what makes you wonderful.

Good luck, if and when you’re ready, (your name)