<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Jin Lo — Engineering Notes</title><description>Long-form essays on architecture, performance, and the craft of building systems that last. Written by a working software engineer for readers everywhere.</description><link>https://blog.waystoneai.online</link><language>en</language><copyright>© 2026 Jin Lo</copyright><lastBuildDate>Tue, 27 Oct 2026 00:00:00 GMT</lastBuildDate><item><title>Writing English as a Second-Language Engineer</title><link>https://blog.waystoneai.online/blog/english-as-a-second-language</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/english-as-a-second-language</guid><description>The grammar is the easy part. The hard part is writing with authority when every sentence sounds borrowed, and learning to stop translating your first language.</description><pubDate>Tue, 27 Oct 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;The first design document I wrote in English was grammatically correct and completely without authority. I had translated the structure of a Chinese memo, kept every polite hedge, and produced three pages that ended by asking whether the direction might be acceptable. The feedback was that the document was clear but sounded like it was asking permission. I was. That was the problem.&lt;/p&gt;
&lt;h2&gt;The authority problem&lt;/h2&gt;
&lt;p&gt;Non-native speakers often write with stacked hedges because politeness is encoded differently in their first language. English technical writing rewards a claim and its evidence. If you learned English in a context where directness was rude, the result is a sentence like &amp;quot;I think we might possibly consider whether...&amp;quot; That is not polite. It is unclear.&lt;/p&gt;
&lt;p&gt;Authority is not volume. It is stating what you know, stating what you do not, and not making the reader do the work of deciding which is which. A recommendation with a reason reads as more credible than a paragraph of qualifications.&lt;/p&gt;
&lt;h2&gt;Translation versus thinking in English&lt;/h2&gt;
&lt;p&gt;There are two modes, and they produce different prose. In the first, I compose the argument in my first language and translate it sentence by sentence. The grammar is usually fine. The rhythm is borrowed, and the sentences are longer than English wants them to be. In the second, I outline in whatever language is fastest, then write the sentences directly in English.&lt;/p&gt;
&lt;p&gt;The second mode is slower at first because my vocabulary is smaller. It is faster overall because I stop translating. The test I use is simple: read the sentence out loud. If I would not say it to a colleague, I rewrite it. This catches more problems than a grammar checker, which is a low bar.&lt;/p&gt;
&lt;h2&gt;Sentence-level habits&lt;/h2&gt;
&lt;p&gt;Most of the improvement is mechanical. These are the habits that moved my writing furthest.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Short sentences. One idea each. If a sentence has two &amp;quot;and&amp;quot;s, it is probably two sentences.&lt;/li&gt;
&lt;li&gt;Strong verbs. &amp;quot;Decide,&amp;quot; not &amp;quot;make a decision.&amp;quot; &amp;quot;Use,&amp;quot; not &amp;quot;utilize.&amp;quot; &amp;quot;Build,&amp;quot; not &amp;quot;perform the construction of.&amp;quot;&lt;/li&gt;
&lt;li&gt;Cut hedges. One &amp;quot;I think&amp;quot; is a position; three is a fog. Keep the one that means something.&lt;/li&gt;
&lt;li&gt;Subject, verb, object. Long noun phrases are where meaning goes to hide.&lt;/li&gt;
&lt;li&gt;Active voice unless the actor is unknown or irrelevant.&lt;/li&gt;
&lt;li&gt;Delete throat-clearing. &amp;quot;It is important to note that&amp;quot; can almost always become nothing.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Instead of&lt;/th&gt;
&lt;th&gt;Problem&lt;/th&gt;
&lt;th&gt;Write&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;utilize&lt;/td&gt;
&lt;td&gt;Latinate inflation&lt;/td&gt;
&lt;td&gt;use&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;in order to&lt;/td&gt;
&lt;td&gt;extra words&lt;/td&gt;
&lt;td&gt;to&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;It is important to note that&lt;/td&gt;
&lt;td&gt;throat-clearing&lt;/td&gt;
&lt;td&gt;(delete)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;We might want to consider&lt;/td&gt;
&lt;td&gt;stacked hedge&lt;/td&gt;
&lt;td&gt;We should consider&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;leverage&lt;/td&gt;
&lt;td&gt;jargon&lt;/td&gt;
&lt;td&gt;use&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;as per my previous message&lt;/td&gt;
&lt;td&gt;passive-aggressive&lt;/td&gt;
&lt;td&gt;following up on&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;I am not sure but maybe&lt;/td&gt;
&lt;td&gt;authority leak&lt;/td&gt;
&lt;td&gt;I recommend, or I do not know yet&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;h3&gt;Idiom traps&lt;/h3&gt;
&lt;p&gt;Idioms are where a second-language writer becomes visibly borrowed. &amp;quot;Move the needle,&amp;quot; &amp;quot;low-hanging fruit,&amp;quot; and &amp;quot;cut corners&amp;quot; are fine in speech and often slightly wrong in a design document. If I am not certain of the register, I use the literal phrase. &amp;quot;Reduce the number of steps&amp;quot; beats &amp;quot;move the needle&amp;quot; when the reader needs to know what to do.&lt;/p&gt;
&lt;p&gt;Phrasal verbs are a smaller trap. &amp;quot;Spin up&amp;quot; and &amp;quot;roll out&amp;quot; are natural in engineering English. &amp;quot;Spin down&amp;quot; is less universal than it feels. When in doubt, use the concrete verb: start, deploy, stop.&lt;/p&gt;
&lt;p&gt;Articles are the thing I still get wrong. I fix them by reading a lot and by asking someone to edit me. A good editor is worth more than a grammar book for this.&lt;/p&gt;
&lt;h3&gt;Sounding like yourself&lt;/h3&gt;
&lt;p&gt;Do not adopt a persona. The goal is not to sound native; the goal is to sound like a specific engineer. Keep the words you actually use. If you say &amp;quot;this is a mess&amp;quot; in speech, write &amp;quot;this is a mess&amp;quot; instead of &amp;quot;this situation presents certain challenges.&amp;quot;&lt;/p&gt;
&lt;p&gt;Your accent in writing is your rhythm and your examples. Use stories from your own work. Nobody else has your incidents, your migration, or your outage. That is the part readers remember, and it is the part no style guide can teach.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;You do not need to sound native. You need to sound like you, in a language the reader can follow.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Before and after&lt;/h2&gt;
&lt;p&gt;Here is a paragraph I would have written five years ago, followed by the version I would send now.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Before:
It is important to note that there are several considerations which
might possibly affect our decision regarding whether we should
migrate the service to the new platform, and it seems that we may
need to take into account the performance implications, although
this is not entirely clear at this stage.

After:
We should migrate the service. The new platform removes two manual
deployment steps, and the performance cost is within our budget.
The open question is the database migration, which we have not
tested at production size.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The second version is shorter, but the real change is that it makes a claim. The first version asks the reader to assemble one. That is the authority problem in miniature.&lt;/p&gt;
&lt;h2&gt;An editing pass checklist&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Read it out loud and mark every sentence I would not say.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Delete the first sentence of each paragraph and see if it still works.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Search for &amp;quot;I think,&amp;quot; &amp;quot;maybe,&amp;quot; &amp;quot;perhaps,&amp;quot; and &amp;quot;just&amp;quot;; cut two of three.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Replace nominalizations with verbs.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Check that every &amp;quot;this&amp;quot; and &amp;quot;it&amp;quot; has a clear referent.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Read the opening as the second reader, not the author.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Ask a colleague to summarize the decision in one sentence.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Keep one sentence that sounds like me, even if it is imperfect.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;Why publishing in English is worth it&lt;/h2&gt;
&lt;p&gt;The audience is larger, the feedback is more useful, and the act of writing in a second language forces clarity. You cannot hide behind fluency. You have to say the thing.&lt;/p&gt;
&lt;p&gt;It is also a career skill. Most code review and incident write-ups I have seen happen in English, even when the office language is something else. Writing it well is part of the job.&lt;/p&gt;
&lt;p&gt;The first drafts will be bad. They are bad for native speakers too. The difference is that native speakers often do not notice, which is not the same as being good.&lt;/p&gt;
&lt;p&gt;I still write more slowly in English than in my first language. I also write more clearly, because every sentence has to earn its place. The hard part was deciding what I meant, and then being willing to say it without three hedges and an apology. If you want the reader&amp;#39;s side of that contract, &lt;a href=&quot;/blog/the-second-reader&quot;&gt;Designing for the Second Reader&lt;/a&gt; is the essay I would read next. If you want a way to make the editing time actually exist in your week, that is what &lt;a href=&quot;/blog/deep-work-system&quot;&gt;A Small System for Deep Work&lt;/a&gt; is about.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://developers.google.com/tech-writing&quot;&gt;Google — Technical Writing courses&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.plainlanguage.gov/guidelines/&quot;&gt;Plain Language Guidelines&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>5 min</readingTime><category>career</category><category>writing</category></item><item><title>Your Build Pipeline Is a Product</title><link>https://blog.waystoneai.online/blog/build-pipeline-as-product</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/build-pipeline-as-product</guid><description>If your pipeline has users, it deserves a product manager. Treat build time as a feature, flakiness as churn, and every red build as a support ticket.</description><pubDate>Tue, 06 Oct 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Our CI took roughly twenty-two minutes from push to green. Nobody had decided that. It had grown one job at a time, the way a garage fills up. The day a reviewer asked whether a red build meant the code was broken or the runner was tired, I realized we were not running a pipeline. We were running a rumor with a status badge.&lt;/p&gt;
&lt;h2&gt;Who the users are&lt;/h2&gt;
&lt;p&gt;A pipeline has three user groups, and they want different things.&lt;/p&gt;
&lt;p&gt;Developers want a fast, trustworthy answer to &amp;quot;did I break it?&amp;quot; They pay for latency in context switches. Reviewers want a check they can believe before they spend time reading. On-call wants a build they can reproduce during an incident, when the first question is whether the last deploy actually matches the commit.&lt;/p&gt;
&lt;p&gt;None of these users file tickets. That is why the pipeline ends up with a twenty-minute runtime and a rerun button nobody trusts.&lt;/p&gt;
&lt;h2&gt;The two metrics that matter&lt;/h2&gt;
&lt;p&gt;If I can only track two numbers, they are time to feedback and false failure rate. Time to feedback is the gap between a push and the first useful signal — not necessarily full green, just the first thing that tells the author whether to keep working. False failure rate is the share of failures that were not caused by the change.&lt;/p&gt;
&lt;p&gt;I track them as percentiles, not averages, for the same reason latency work uses percentiles: a pipeline that is fast for small changes and slow for the ones that touch the build config is slow where it hurts. The table below is the dashboard I would want on a wall.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Definition&lt;/th&gt;
&lt;th&gt;Target I have found workable&lt;/th&gt;
&lt;th&gt;What it hides&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Time to first signal&lt;/td&gt;
&lt;td&gt;Push to first pass or fail&lt;/td&gt;
&lt;td&gt;Under 5 minutes&lt;/td&gt;
&lt;td&gt;Whether the signal is meaningful&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Time to full green&lt;/td&gt;
&lt;td&gt;Push to all required checks&lt;/td&gt;
&lt;td&gt;Under 15 minutes&lt;/td&gt;
&lt;td&gt;Queue time split across jobs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;False failure rate&lt;/td&gt;
&lt;td&gt;Failures not caused by the diff&lt;/td&gt;
&lt;td&gt;Under 2%&lt;/td&gt;
&lt;td&gt;Tests that fail for environmental reasons&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Rerun rate&lt;/td&gt;
&lt;td&gt;Jobs rerun without a new commit&lt;/td&gt;
&lt;td&gt;Under 5%&lt;/td&gt;
&lt;td&gt;Flakiness that happens to pass&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Queue time&lt;/td&gt;
&lt;td&gt;Waiting for a runner&lt;/td&gt;
&lt;td&gt;Under 1 minute&lt;/td&gt;
&lt;td&gt;Slow setup inside the job&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Those targets come from my experience, not a standard. The point is to pick numbers and watch them, not to defend the numbers.&lt;/p&gt;
&lt;h2&gt;The cost of a twenty-minute pipeline&lt;/h2&gt;
&lt;p&gt;A twenty-minute pipeline is not twenty minutes of waiting. It is twenty minutes of waiting plus the cost of whatever the developer started while waiting. If the next task takes ten minutes, the developer has now context-switched twice and holds two problems in their head.&lt;/p&gt;
&lt;p&gt;Roughly: on a team of eight that merges ten times a day, twenty minutes per run is more than three hours of collective waiting, before counting the second task everyone started. The arithmetic is illustrative, but the shape is real. A pipeline longer than the work you switch to trains people to multitask.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A slow pipeline does not just cost time. It teaches the team to stop paying attention.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Caching and parallelization&lt;/h2&gt;
&lt;p&gt;The cheapest wins are usually caching and ordering. Cache dependencies by lockfile hash. Cache build outputs by source hash. Run the fast checks first so a typo fails in ninety seconds instead of twenty minutes. Then parallelize the slowest stage, and shard tests by measured time rather than by file count.&lt;/p&gt;
&lt;p&gt;Here is a generic workflow sketch. The details matter less than the order: install, cache, fail fast, then fan out.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-yaml&quot;&gt;name: ci
on: [pull_request]

jobs:
  verify:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run lint
      - run: npm run typecheck
      - run: npm test -- --shard=${{ matrix.shard }}/4
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Two cautions. Cache keys are easy to get subtly wrong; a cache that never hits is just a slower pipeline with more YAML. And do not parallelize a pipeline that is failing for reasons nobody understands. Fix the signal before you multiply it.&lt;/p&gt;
&lt;h2&gt;What to do about flaky tests&lt;/h2&gt;
&lt;p&gt;A flaky test fails for reasons unrelated to the code under test: time, ordering, shared state, a network call, a random seed. The rerun button is a painkiller. It makes the symptom go away without telling you anything about the disease.&lt;/p&gt;
&lt;p&gt;My rule is quarantine, owner, deadline. Move the test out of the required set immediately, link the failure, assign one person, and set a date. If it is not fixed in a week, delete it or skip it with a ticket. A skipped test is honest. A flaky test that passes on the second try is a lie the team has agreed to believe.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Can I reproduce it locally with the same seed and environment?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Does it depend on time, order, or shared state?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Is it making a real network call?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Who owns it, and when will they look?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Is it quarantined, or is it still blocking other people?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; If we deleted it, what coverage would we actually lose?&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;A pipeline upgrade sequence&lt;/h2&gt;
&lt;p&gt;The order matters. Doing these out of order produces a faster pipeline that nobody trusts.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Measure time to first signal and false failure rate for two weeks.&lt;/li&gt;
&lt;li&gt;Move lint and typecheck to the first job and fail fast.&lt;/li&gt;
&lt;li&gt;Cache dependencies and build outputs; verify the keys actually hit.&lt;/li&gt;
&lt;li&gt;Parallelize or shard the slowest stage, measured rather than guessed.&lt;/li&gt;
&lt;li&gt;Quarantine flaky tests, assign owners, and set deadlines.&lt;/li&gt;
&lt;li&gt;Add a merge queue only after the pipeline is fast enough to deserve one.&lt;/li&gt;
&lt;li&gt;Put the dashboard where developers already look, not in a wiki.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;A pipeline is a product because it has users, a failure mode, and a budget. Build time is a feature; flakiness is churn. The best compliment a pipeline can get is that nobody talks about it. The same is true of a monolith you have finished &lt;a href=&quot;/blog/rewriting-the-monolith&quot;&gt;rewriting&lt;/a&gt;: the goal is not to make it interesting, it is to make it boring enough that the team can forget it and go build the product. The pages it ships deserve the same treatment — &lt;a href=&quot;/blog/cls-in-practice&quot;&gt;a layout that never shifts&lt;/a&gt; is one the reader never has to think about either. And when the pipeline does get slow, measure it the way you would measure an API — by percentile and population, as I described in &lt;a href=&quot;/blog/slow-apis&quot;&gt;A Field Guide to Slow APIs&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;This essay is part of the &lt;a href=&quot;/performance&quot;&gt;Performance guide&lt;/a&gt;, which connects tail latency, layout stability, and build time.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://dora.dev/&quot;&gt;DORA — DevOps Research and Assessment&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://docs.github.com/en/actions&quot;&gt;GitHub Docs — Actions&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>5 min</readingTime><category>devops</category><category>engineering</category></item><item><title>A Small System for Deep Work</title><link>https://blog.waystoneai.online/blog/deep-work-system</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/deep-work-system</guid><description>Deep work does not need another app. It needs a defensible block of time, a paper notebook, and a shutdown ritual that closes the loops before the day ends.</description><pubDate>Wed, 16 Sep 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;I do not have a productivity system. I have a calendar block, a paper notebook, and a shutdown ritual I follow maybe four days out of five. That has been enough. The weeks when I abandon all three are also the weeks I end with the vague feeling of having been busy without having moved anything.&lt;/p&gt;
&lt;h2&gt;The two-hour block&lt;/h2&gt;
&lt;p&gt;Two hours is the unit I protect. Four would be better and is usually a fantasy in a job with meetings. Two is long enough to load the problem into my head and short enough that I can defend it on a calendar.&lt;/p&gt;
&lt;p&gt;The block has one rule: I write the outcome before I start. Not a task, an outcome. &amp;quot;Fix the upload bug&amp;quot; is a task. &amp;quot;A failing upload produces a useful error in the UI, and I have watched it happen&amp;quot; is an outcome. If I cannot name one, the block is for reading or thinking, not coding.&lt;/p&gt;
&lt;p&gt;The block is not sacred because of its length. It is sacred because it ends. Knowing when it stops makes it easier to start.&lt;/p&gt;
&lt;h2&gt;Why task lists fail&lt;/h2&gt;
&lt;p&gt;A task list is an inventory, not a decision. It grows because adding is cheap and finishing is expensive. Forty open items is not a plan; it is a mood.&lt;/p&gt;
&lt;p&gt;The failure mode is predictable. I pick the easiest item to feel progress, or I spend the first twenty minutes reorganizing the list, which feels like work and produces nothing. The list is also where anxiety accumulates, because every item is a reminder of something I am not doing right now.&lt;/p&gt;
&lt;p&gt;What works better is one outcome per block, plus a separate list for everything else that I do not open during the block. The big list is for the weekly review. The block gets one thing.&lt;/p&gt;
&lt;h2&gt;The shutdown ritual&lt;/h2&gt;
&lt;p&gt;The end of the day matters more than the start. If I close the laptop with six open loops, my brain keeps running them at dinner. The ritual is ten minutes and it is mostly writing.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Write tomorrow&amp;#39;s first action in one sentence.&lt;/li&gt;
&lt;li&gt;Move everything unfinished to a dated note, not back to the top of the big list.&lt;/li&gt;
&lt;li&gt;Close the editor and the issue tracker.&lt;/li&gt;
&lt;li&gt;Write one thing that went wrong and one thing I will do differently.&lt;/li&gt;
&lt;li&gt;Say, in a note or out loud, that the day is over.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The phrase sounds ridiculous until you try it. Open loops are what keep you awake; the note is a contract with tomorrow-you.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Deep work ends at a time, not when the problem is solved.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h3&gt;A daily checklist&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; One outcome written before the block.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The block is on the calendar and I have not scheduled over it.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Notifications off, phone out of reach rather than face down.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; One tab for the task; everything else closed.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Shutdown note with tomorrow&amp;#39;s first action.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; One sentence about what I learned.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;The weekly review&lt;/h2&gt;
&lt;p&gt;Thirty minutes on Friday. I do not use an app for this; a text file is enough and it opens faster than anything with a login screen.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Week of:

Shipped:
- ...

Still open:
- ... (next action, owner)

Learned:
- ...

Next week:
- Protected blocks: Tue 09:00-11:00, Thu 09:00-11:00
- Meetings I can decline:
- One thing to stop doing:
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The review is not a report. Nobody reads it. Its job is to move the open loops out of my head and into a place where they still make sense on Monday. If I skip it for two weeks, the third week is recognizable by the number of things I have started and not finished.&lt;/p&gt;
&lt;h2&gt;Protecting time on a team&lt;/h2&gt;
&lt;p&gt;Protecting a block is a social problem before it is a calendar problem. A few things have worked for me.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Put the block on the calendar with a plain title. &amp;quot;Focus&amp;quot; is fine; &amp;quot;deep work&amp;quot; invites comments.&lt;/li&gt;
&lt;li&gt;Publish office hours for questions and direct people there. Being always available is not a virtue; it is a routing policy.&lt;/li&gt;
&lt;li&gt;Agree on response expectations. Most messages do not need an answer in five minutes, and saying so out loud makes it true.&lt;/li&gt;
&lt;li&gt;Make the trade explicit: &amp;quot;I can do that today or the review, not both.&amp;quot; People respect a choice; they do not respect a vague yes.&lt;/li&gt;
&lt;li&gt;Protect other people&amp;#39;s blocks too. Do not schedule over a focus block because your meeting is urgent.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;A calendar with no blank space is not a sign of importance. It is a sign that someone else is doing your prioritization.&lt;/p&gt;
&lt;h2&gt;When the day is fragmented&lt;/h2&gt;
&lt;p&gt;Some days are twenty-minute gaps between meetings, and pretending otherwise makes them worse. I keep a short list for those days: review a pull request, write a decision note, update an issue, read one section of documentation. None of it requires loading a large problem into my head.&lt;/p&gt;
&lt;p&gt;What I try not to do is start a deep task I cannot finish. Leaving a half-loaded problem costs more than it saves, because tomorrow I have to load it again and re-derive the context I dropped. If I get forty-five uninterrupted minutes, I take them and call it a win. If I get twenty, I do correspondence and stop pretending.&lt;/p&gt;
&lt;p&gt;The system is small on purpose. A system you can follow on a bad day is worth more than one that works only on a good one. The tools promising to organize all of this are their own kind of &lt;a href=&quot;/blog/cost-of-convenience&quot;&gt;convenience&lt;/a&gt;, and the same instinct applies to the feedback loops you build at work: a pipeline nobody has to think about is a pipeline that respects your attention, which is the argument in &lt;a href=&quot;/blog/build-pipeline-as-product&quot;&gt;Your Build Pipeline Is a Product&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://calnewport.com/books/deep-work/&quot;&gt;Cal Newport — Deep Work&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.apa.org/topics/multitasking&quot;&gt;American Psychological Association — Multitasking&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>4 min</readingTime><category>career</category></item><item><title>Cumulative Layout Shift, in Practice</title><link>https://blog.waystoneai.online/blog/cls-in-practice</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/cls-in-practice</guid><description>Layout shift is rarely a CSS bug. It is a missing decision about who reserves space, made late in the rendering pipeline.</description><pubDate>Wed, 02 Sep 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;The layout shift that finally made me care was a consent banner. The article had started rendering, the first paragraph was on screen, and then the banner loaded and pushed everything down by the height of a small paragraph. Readers who had started reading lost their place. The fix took two lines. Finding it took a week of not believing the score.&lt;/p&gt;
&lt;h2&gt;What CLS actually measures&lt;/h2&gt;
&lt;p&gt;CLS is a score, not a stopwatch. It measures how much visible content moves, and how far, when the user did not ask for the movement. A late image without dimensions counts. A font swap that changes line height counts. A user opening a panel does not. The score is roughly the affected area multiplied by the distance moved, accumulated across the page&amp;#39;s life, with a windowing rule that keeps one bad moment from dominating forever.&lt;/p&gt;
&lt;p&gt;The practical version: if content moves after it has been painted, and the user did not cause it, CLS is watching. Under 0.1 is the usual &amp;quot;good&amp;quot; threshold; in my experience, anything above 0.2 is usually one missing dimension and one late banner. The number matters less than the location. A shift above the fold is a different problem from a shift in a footer nobody has reached.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The score is a symptom. The node that moved is the bug.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;The usual suspects&lt;/h2&gt;
&lt;p&gt;Most shifts come from a short list. I check these before I open a profiler.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Symptom&lt;/th&gt;
&lt;th&gt;Likely cause&lt;/th&gt;
&lt;th&gt;Fix&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Article jumps when the font loads&lt;/td&gt;
&lt;td&gt;Fallback metrics differ from the web font&lt;/td&gt;
&lt;td&gt;Match metrics with &lt;code&gt;size-adjust&lt;/code&gt;, or choose a closer fallback&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Image pushes text down&lt;/td&gt;
&lt;td&gt;No intrinsic size&lt;/td&gt;
&lt;td&gt;&lt;code&gt;width&lt;/code&gt; and &lt;code&gt;height&lt;/code&gt; attributes or &lt;code&gt;aspect-ratio&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Banner appears late&lt;/td&gt;
&lt;td&gt;Injected DOM above content&lt;/td&gt;
&lt;td&gt;Reserve a slot or overlay it&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Comments jump in&lt;/td&gt;
&lt;td&gt;Async content with no placeholder&lt;/td&gt;
&lt;td&gt;&lt;code&gt;min-height&lt;/code&gt; on the container&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Header grows on scroll&lt;/td&gt;
&lt;td&gt;Sticky element changes layout&lt;/td&gt;
&lt;td&gt;Animate &lt;code&gt;transform&lt;/code&gt;, not &lt;code&gt;height&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Two of these are decisions about space. The rest are decisions about when content arrives. Both are made before the browser paints, which is why the fix is rarely a CSS tweak at the end.&lt;/p&gt;
&lt;h2&gt;Reproducing it with throttling&lt;/h2&gt;
&lt;p&gt;Local dev is a bad place to look. Assets are warm, the network is fast, and the banner might not load at all. I test the production build with the cache disabled, CPU throttled roughly four times, and the network set to Slow 4G. Then I record a performance trace and read the Layout Shifts track, not just the final score.&lt;/p&gt;
&lt;p&gt;A console snippet is faster when I want to know what moved:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-js&quot;&gt;new PerformanceObserver((list) =&amp;gt; {
  for (const entry of list.getEntries()) {
    if (entry.hadRecentInput) continue;
    const nodes = entry.sources?.map((source) =&amp;gt; source.node) ?? [];
    console.log(entry.value.toFixed(4), nodes);
  }
}).observe({ type: &amp;#39;layout-shift&amp;#39;, buffered: true });
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The output names the nodes. That is usually enough to stop the argument about whether the page &amp;quot;feels&amp;quot; jumpy.&lt;/p&gt;
&lt;h2&gt;Fixes that actually hold&lt;/h2&gt;
&lt;p&gt;The reliable fix is to give the browser the size before the content arrives. For images, that means dimensions or an aspect ratio.&lt;/p&gt;
&lt;p&gt;Before:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-html&quot;&gt;&amp;lt;img src=&amp;quot;/images/cover.jpg&amp;quot; alt=&amp;quot;A desk with a notebook&amp;quot;&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;After:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-html&quot;&gt;&amp;lt;img src=&amp;quot;/images/cover.jpg&amp;quot; alt=&amp;quot;A desk with a notebook&amp;quot;
     width=&amp;quot;1200&amp;quot; height=&amp;quot;630&amp;quot;&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The attributes let the browser reserve the right box and keep the text where it is. CSS can do the same for containers:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-css&quot;&gt;.cover {
  aspect-ratio: 1200 / 630;
  width: 100%;
  height: auto;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For fonts, the shift comes from the fallback being narrower or taller than the webfont. &lt;code&gt;font-display: swap&lt;/code&gt; makes the text visible, but it also makes the swap visible. A fallback face with adjusted metrics reduces the jump:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-css&quot;&gt;@font-face {
  font-family: &amp;quot;Source Serif Fallback&amp;quot;;
  src: local(&amp;quot;Georgia&amp;quot;);
  size-adjust: 105%;
  ascent-override: 92%;
  descent-override: 23%;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Those numbers are specific to the font pair. Measure them; do not copy them. And do not patch the layout with JavaScript after load. That does not remove the shift; it moves it a few hundred milliseconds later, where it is harder to attribute.&lt;/p&gt;
&lt;h2&gt;CLS as an editorial requirement&lt;/h2&gt;
&lt;p&gt;This site treats CLS as an editorial requirement. A post does not merge if an embed shifts the column at 360 pixels. That is a policy, not a performance budget: the reader came for the text, and moving the text is a way of losing their place. It is the same courtesy I argued for in &lt;a href=&quot;/blog/the-second-reader&quot;&gt;Designing for the Second Reader&lt;/a&gt;. The measurement discipline is the same one I use for latency in &lt;a href=&quot;/blog/slow-apis&quot;&gt;A Field Guide to Slow APIs&lt;/a&gt;: segment by page, device, and network before believing the average. We enforce the rule in CI rather than in review, because a policy that lives in a checklist gets skipped on a Friday; &lt;a href=&quot;/blog/build-pipeline-as-product&quot;&gt;the build pipeline is where a rule actually holds&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;A testing checklist&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Test the production build, not the dev server.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Disable the cache and throttle the network to Slow 4G.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Throttle the CPU; shifts are worse when layout is slow.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Reload at 360 px and at desktop width.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Read the trace&amp;#39;s Layout Shifts track, not only the final score.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Block the font request and see how much moves.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Scroll the whole page; shifts below the fold still count while visible.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Check every injected element: consent, promo, related posts.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Re-run after the image CDN changes sizes.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;What I check after a fix&lt;/h2&gt;
&lt;p&gt;A lower score is not the same as a fixed page. After shipping a layout change, I reload three times with the cache disabled and watch the trace instead of the number. If the shift is gone on the first load but returns on the third, the cause is usually a cached font or a lazy component arriving in a different order.&lt;/p&gt;
&lt;p&gt;I also check the page at 200% zoom. Fixed heights that look generous at default size start clipping once the text grows, and a clipped heading is a worse outcome than the shift it was meant to prevent. The goal is space that adapts to its content, not space that is guessed once and never revisited.&lt;/p&gt;
&lt;p&gt;Layout shift is rarely a CSS bug. It is a missing decision about who reserves space, made late in the rendering pipeline. The browser is not being difficult. It is telling you that you forgot to make the decision, and it is telling you in the only language it has.&lt;/p&gt;
&lt;p&gt;This essay is part of the &lt;a href=&quot;/performance&quot;&gt;Performance guide&lt;/a&gt;, which connects tail latency, layout stability, and build time.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://web.dev/articles/cls&quot;&gt;web.dev — Cumulative Layout Shift&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/API/LayoutShift&quot;&gt;MDN — LayoutShift&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>5 min</readingTime><category>performance</category><category>engineering</category></item><item><title>The Hidden Cost of Convenience</title><link>https://blog.waystoneai.online/blog/cost-of-convenience</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/cost-of-convenience</guid><description>Convenience is a loan. The interest stays invisible until the abstraction you did not understand needs to change — and you own a dependency you never chose.</description><pubDate>Tue, 18 Aug 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;I added a library to format relative timestamps. It saved me eleven lines of code and brought a plugin system, a locale registry, and a dependency tree I never looked at. Six months later a daylight-saving bug in that tree broke every &amp;quot;edited 3 minutes ago&amp;quot; label on the site. I fixed it in an afternoon. The library was fine. I had just not noticed I was paying twice.&lt;/p&gt;
&lt;h2&gt;The convenience debt ledger&lt;/h2&gt;
&lt;p&gt;Convenience is a loan. The purchase price is visible in the pull request; the interest is paid later, by whoever has to change the behavior. I keep a mental ledger with three columns: what I saved, what I borrowed, and when the bill comes due.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Convenience&lt;/th&gt;
&lt;th&gt;What it saves now&lt;/th&gt;
&lt;th&gt;What it costs later&lt;/th&gt;
&lt;th&gt;Who pays&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;A UI component library&lt;/td&gt;
&lt;td&gt;Weeks of styling and accessibility work&lt;/td&gt;
&lt;td&gt;A theming system you do not control, plus an upgrade cliff&lt;/td&gt;
&lt;td&gt;Whoever touches design next&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;An ORM with migrations&lt;/td&gt;
&lt;td&gt;Handwritten SQL for CRUD&lt;/td&gt;
&lt;td&gt;Query behavior you cannot see and a migration tool with opinions&lt;/td&gt;
&lt;td&gt;Whoever debugs the slow endpoint&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A state management library&lt;/td&gt;
&lt;td&gt;Prop drilling today&lt;/td&gt;
&lt;td&gt;A second place where truth lives&lt;/td&gt;
&lt;td&gt;Whoever debugs the impossible render&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A hosted queue&lt;/td&gt;
&lt;td&gt;Retry logic this sprint&lt;/td&gt;
&lt;td&gt;A failure mode behind someone else&amp;#39;s status page&lt;/td&gt;
&lt;td&gt;Whoever is on call&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A date/time library&lt;/td&gt;
&lt;td&gt;Eleven lines&lt;/td&gt;
&lt;td&gt;Locale, timezone, and DST edge cases in code you do not read&lt;/td&gt;
&lt;td&gt;Whoever ships near a holiday&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The ledger is not an argument against any of these. It is an argument for writing the second column down before the first column becomes a habit.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A cost you have named is a decision. A cost you have not named is a surprise with a calendar entry.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Saving typing versus saving thinking&lt;/h2&gt;
&lt;p&gt;Some dependencies save typing. Some save thinking. The distinction matters more than the license.&lt;/p&gt;
&lt;p&gt;A date formatter saves typing. You know what the output should be, you could write it, and the package removes keystrokes. A state management library saves thinking. It gives you a model for where truth lives and how changes propagate. The first kind of convenience is cheap to reverse. The second kind becomes architecture, whether or not you called it that.&lt;/p&gt;
&lt;p&gt;I am suspicious of tools that save thinking in small doses. If the problem is small, the mental model is overhead. If the problem is large, the model is the product, and you should choose it deliberately. The dangerous case is the small problem that will grow.&lt;/p&gt;
&lt;h2&gt;The dependency I still regret&lt;/h2&gt;
&lt;p&gt;I once added a rich text editor to an internal tool because the alternative was a textarea. The editor gave us bold, links, and paste-from-Word in an afternoon. It also gave us a document model, a plugin API, a serialization format, and a browser compatibility matrix.&lt;/p&gt;
&lt;p&gt;The regret was not the initial cost. It was the second feature. We needed comments inside the document, and the editor&amp;#39;s model did not represent them the way we needed. So we built a parallel model beside it and synchronized the two. We had adopted a product and then built a second product to translate between it and ours.&lt;/p&gt;
&lt;p&gt;The question I should have asked is not &amp;quot;does this library do what I need today&amp;quot; but &amp;quot;what will I need to change first, and does this library let me change it?&amp;quot; A library that solves the current feature and blocks the next one is not convenience. It is a down payment on a rewrite.&lt;/p&gt;
&lt;h2&gt;The &amp;quot;one more library&amp;quot; habit&lt;/h2&gt;
&lt;p&gt;The &amp;quot;one more library&amp;quot; habit is rarely one decision. It is a series of small ones, each defensible, that add up to a stack nobody chose. I have watched a project acquire a date library, a validation library, a fetch wrapper, a form library, and two utility collections, each to avoid writing something the team already knew how to write.&lt;/p&gt;
&lt;p&gt;The cost is not bundle size, although that is real. The cost is the surface area of behavior you did not author. Every dependency is a set of decisions someone else made, and some of those decisions will be wrong for you at the worst possible moment. I removed a convenience wrapper from this site&amp;#39;s build and wrote the twenty lines it replaced, and the build got easier to reason about. That is &lt;a href=&quot;/blog/boring-architecture&quot;&gt;the case for boring architecture&lt;/a&gt; applied at the package level.&lt;/p&gt;
&lt;p&gt;There is a useful test. If the library disappeared tomorrow, could one person on the team reimplement the part you use in a day? If yes, it is a convenience. If no, it is infrastructure, and it deserves the scrutiny you give infrastructure.&lt;/p&gt;
&lt;h2&gt;A decision checklist&lt;/h2&gt;
&lt;p&gt;I run this before adding a package. It is not a ban; it is a pause.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; What exactly am I avoiding writing, and how many lines is it?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Is this saving typing or saving thinking?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; What is the exit cost if the library is abandoned or changes direction?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; How many transitive dependencies does it bring, and can I name them?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Does it solve the next feature too, or only this one?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Would I still choose it if it were twice the size and half as maintained?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Who on the team can debug it without reading the source?&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// The utility package, replaced.
function relativeTime(date: Date, now = new Date()): string {
  const seconds = Math.round((now.getTime() - date.getTime()) / 1000);
  if (seconds &amp;lt; 60) return &amp;#39;just now&amp;#39;;
  const minutes = Math.round(seconds / 60);
  if (minutes &amp;lt; 60) return `${minutes} minute${minutes === 1 ? &amp;#39;&amp;#39; : &amp;#39;s&amp;#39;} ago`;
  const hours = Math.round(minutes / 60);
  if (hours &amp;lt; 24) return `${hours} hour${hours === 1 ? &amp;#39;&amp;#39; : &amp;#39;s&amp;#39;} ago`;
  return date.toISOString().slice(0, 10);
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The type checker keeps this honest without a runtime schema, which is its own convenience. &lt;a href=&quot;/blog/typescript-narrowing&quot;&gt;TypeScript narrowing&lt;/a&gt; gets you most of the safety people buy a validation package for, as long as the data is already typed at the boundary.&lt;/p&gt;
&lt;h2&gt;The balanced rule&lt;/h2&gt;
&lt;p&gt;I am not anti-library. I use them, I recommend them, and I have been grateful for a good one on a deadline. The rule I have settled on is this: spend convenience on the parts of the system that are not your product, and pay for it in the parts that are.&lt;/p&gt;
&lt;p&gt;A date formatter is not your product. A rich text editor might be. A build tool is not your product; the build time is. When a dependency touches the thing users came for, own the behavior. When it touches the plumbing, rent.&lt;/p&gt;
&lt;p&gt;And write down the rent. A sentence in the pull request — &amp;quot;this saves us two days now; we will revisit if we need comments in the editor&amp;quot; — turns an invisible loan into a decision. The interest is still there. At least you know when it is due. The same ledger tells you when a dependency has stopped paying: the question stops being whether to remove it and becomes how small a piece you can replace at a time, which is the case I make in &lt;a href=&quot;/blog/rewriting-the-monolith&quot;&gt;What I Learned Rewriting a Monolith&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;This essay is part of the &lt;a href=&quot;/architecture&quot;&gt;Architecture guide&lt;/a&gt;, which maps the decisions from foundations to migration seams.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/&quot;&gt;Joel Spolsky — The Law of Leaky Abstractions&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.hyrumslaw.com/&quot;&gt;Hyrum&amp;#39;s Law&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>5 min</readingTime><category>career</category><category>architecture</category></item><item><title>TypeScript Narrowing, Without the Guesswork</title><link>https://blog.waystoneai.online/blog/typescript-narrowing</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/typescript-narrowing</guid><description>TypeScript narrowing is not magic. It follows a small set of rules, and most confusion comes from the places it gives up.</description><pubDate>Tue, 04 Aug 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Most TypeScript frustration I hear is not about types being wrong. It is about types being right in a way the developer did not expect. Someone checks a value, calls a function, and gets &amp;quot;possibly undefined&amp;quot; anyway. That is not a bug in the compiler. It is narrowing working exactly as designed, in a place where the design stops.&lt;/p&gt;
&lt;h2&gt;What narrowing actually is&lt;/h2&gt;
&lt;p&gt;Narrowing is control-flow analysis over a single reference. At every point in a scope, TypeScript computes the set of types a variable could have, given the checks and assignments that came before. &lt;code&gt;typeof x === &amp;#39;string&amp;#39;&lt;/code&gt; removes every non-string member from that set for the rest of the block.&lt;/p&gt;
&lt;p&gt;Two consequences matter. First, narrowing is lexical and local. It applies to a specific reference in a specific scope, not to the value everywhere it travels. Second, narrowings are erased at runtime. They are guesses the compiler can prove from syntax, not validations you performed.&lt;/p&gt;
&lt;p&gt;Once you internalize that, most surprises stop being surprising.&lt;/p&gt;
&lt;h2&gt;The reliable tools&lt;/h2&gt;
&lt;p&gt;The everyday checks are reliable within their limits:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;typeof&lt;/code&gt; covers primitives, and it is the only way to separate &lt;code&gt;string&lt;/code&gt; from &lt;code&gt;number&lt;/code&gt;. Remember that &lt;code&gt;typeof null === &amp;#39;object&amp;#39;&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;instanceof&lt;/code&gt; works for class instances. It fails for values from another realm and for plain objects that merely resemble a class.&lt;/li&gt;
&lt;li&gt;The &lt;code&gt;in&lt;/code&gt; operator narrows a union by property presence. It tells you the key exists, not that its value has the type you hope.&lt;/li&gt;
&lt;li&gt;Equality against a literal narrows unions, which is the foundation of the pattern in the next section.&lt;/li&gt;
&lt;li&gt;Truthiness narrows out &lt;code&gt;null&lt;/code&gt;, &lt;code&gt;undefined&lt;/code&gt;, &lt;code&gt;0&lt;/code&gt;, &lt;code&gt;&amp;#39;&amp;#39;&lt;/code&gt;, and &lt;code&gt;false&lt;/code&gt; together. If zero is a valid value, a truthiness check is a bug with good syntax.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Construct&lt;/th&gt;
&lt;th&gt;Narrows when&lt;/th&gt;
&lt;th&gt;Stops working when&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;&lt;code&gt;typeof x === &amp;#39;string&amp;#39;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;x is a primitive union&lt;/td&gt;
&lt;td&gt;x is &lt;code&gt;null&lt;/code&gt; or a boxed &lt;code&gt;String&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;x instanceof Foo&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;x is a class instance&lt;/td&gt;
&lt;td&gt;The value crossed a realm or came from JSON&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;&amp;#39;key&amp;#39; in x&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The union differs by keys&lt;/td&gt;
&lt;td&gt;Key exists with the wrong value type&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;x.kind === &amp;#39;circle&amp;#39;&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;The union has a literal discriminant&lt;/td&gt;
&lt;td&gt;A variant is missing the discriminant&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;if (x)&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;x may be null or undefined&lt;/td&gt;
&lt;td&gt;&lt;code&gt;0&lt;/code&gt;, &lt;code&gt;&amp;#39;&amp;#39;&lt;/code&gt;, or &lt;code&gt;false&lt;/code&gt; are valid values&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;isFoo(x)&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;You wrote a compiler-verified guard&lt;/td&gt;
&lt;td&gt;The guard body lies&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The discipline is the same one that works for performance: eliminate possibilities with evidence instead of asserting the answer, which is how I approach latency in &lt;a href=&quot;/blog/slow-apis&quot;&gt;A Field Guide to Slow APIs&lt;/a&gt;. The compiler is doing the eliminating for you; your job is to give it facts it can read.&lt;/p&gt;
&lt;h2&gt;Discriminated unions are the workhorse&lt;/h2&gt;
&lt;p&gt;If you standardize one pattern, make it this. Give every variant of a union a literal field, then switch on it. The compiler narrows each branch precisely, and the &lt;code&gt;never&lt;/code&gt; check makes future changes loud.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;type Shape =
  | { kind: &amp;#39;circle&amp;#39;; radius: number }
  | { kind: &amp;#39;square&amp;#39;; side: number }
  | { kind: &amp;#39;rectangle&amp;#39;; width: number; height: number };

function assertNever(value: never): never {
  throw new Error(`Unhandled case: ${JSON.stringify(value)}`);
}

function area(shape: Shape): number {
  switch (shape.kind) {
    case &amp;#39;circle&amp;#39;:
      return Math.PI * shape.radius ** 2;
    case &amp;#39;square&amp;#39;:
      return shape.side ** 2;
    case &amp;#39;rectangle&amp;#39;:
      return shape.width * shape.height;
    default:
      return assertNever(shape);
  }
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Add a &lt;code&gt;triangle&lt;/code&gt; variant and &lt;code&gt;assertNever&lt;/code&gt; fails to compile, because &lt;code&gt;shape&lt;/code&gt; is no longer &lt;code&gt;never&lt;/code&gt; in the default branch. That is the entire benefit: the compiler enumerates the places that need updating. Exhaustiveness is not a style preference; it is a maintenance tool.&lt;/p&gt;
&lt;p&gt;The same pattern works for API responses, reducer actions, and form states. A &lt;code&gt;status: &amp;#39;idle&amp;#39; | &amp;#39;loading&amp;#39; | &amp;#39;success&amp;#39; | &amp;#39;error&amp;#39;&lt;/code&gt; field with payloads attached to the relevant variants removes an entire category of impossible states.&lt;/p&gt;
&lt;h2&gt;Type guards and their honesty problem&lt;/h2&gt;
&lt;p&gt;A user-defined guard looks like this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;function isUser(value: unknown): value is User {
  return typeof value === &amp;#39;object&amp;#39; &amp;amp;&amp;amp; value !== null &amp;amp;&amp;amp; &amp;#39;id&amp;#39; in value;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The compiler trusts the return type. It does not verify that &lt;code&gt;id&lt;/code&gt; is a string, that the other required fields exist, or that the object is not an array. The body can be wrong, and every call site inherits the lie. Guards are promises, and TypeScript has no way to audit them.&lt;/p&gt;
&lt;p&gt;This is why I keep guards small and close to the boundary where data enters the system. A guard that checks three fields is reviewable. A guard that attempts to validate a nested payload by hand is a schema validator with worse error messages. If the data is external, parse it once with a real validator and hand typed values inward.&lt;/p&gt;
&lt;p&gt;Assertion functions (&lt;code&gt;asserts value is User&lt;/code&gt;) have the same honesty problem plus a syntax constraint: the function must have an explicit type annotation, and arrow functions assigned to variables cannot be assertion functions in the way you might expect. They are useful for invariants inside a module. They are a poor substitute for validation.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A type guard is a promise the compiler cannot verify. Write small promises.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Where narrowing gives up&lt;/h2&gt;
&lt;p&gt;Narrowing is lost in a few predictable places. Knowing them saves hours.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Aliases.&lt;/strong&gt; Narrowing applies to the reference you checked. Check &lt;code&gt;obj.value&lt;/code&gt;, then call a function that might mutate &lt;code&gt;obj&lt;/code&gt;, and the compiler resets what it knows about &lt;code&gt;obj.value&lt;/code&gt; afterward. Copy the value into a &lt;code&gt;const&lt;/code&gt; first when the narrow needs to survive a function call.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Closures and callbacks.&lt;/strong&gt; A check outside a callback does not narrow the value inside it, because the callback may run later, after the variable changed. This is the classic:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;let current: string | null = getValue();

if (current !== null) {
  items.forEach(() =&amp;gt; {
    console.log(current.toUpperCase()); // current may be null again
  });
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Copy the narrowed value into a &lt;code&gt;const&lt;/code&gt; before the callback, or narrow inside it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Indexed access.&lt;/strong&gt; &lt;code&gt;obj[key]&lt;/code&gt; where &lt;code&gt;key&lt;/code&gt; is a union of strings produces a union of all possible value types, and a check on one branch does not narrow the others. Destructure first or use a mapped lookup.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Non-null assertions and &lt;code&gt;as&lt;/code&gt;.&lt;/strong&gt; These do not narrow. &lt;code&gt;value!&lt;/code&gt; silences the error and leaves the runtime unchanged. &lt;code&gt;as User&lt;/code&gt; tells the compiler to stop asking questions. Both are sometimes necessary at a boundary; neither is a substitute for a check.&lt;/p&gt;
&lt;h2&gt;A checklist before you reach for &lt;code&gt;as&lt;/code&gt;&lt;/h2&gt;
&lt;p&gt;I use this list whenever a cast feels tempting.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Is there a discriminant field I could add to the union instead?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Does the guard body actually validate every field it claims?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Can this value be &lt;code&gt;const&lt;/code&gt; so the narrowing survives?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Is the check at the boundary, or ten frames deep in business logic?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Did I handle the &lt;code&gt;never&lt;/code&gt; case, or did I write a default that returns &lt;code&gt;undefined&lt;/code&gt;?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Would parsing the value once with a schema validator be more honest?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; If I remove the assertion, what does the compiler actually know?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Narrowing is one of the few parts of TypeScript that rewards reading the error instead of fighting it. The error usually names the exact reference and the exact line where the compiler lost confidence. Treat it as a question about scope, not an insult about your code.&lt;/p&gt;
&lt;p&gt;And write the types as if the next person will read them without you, because they will; that is the same argument I make in &lt;a href=&quot;/blog/the-second-reader&quot;&gt;Designing for the Second Reader&lt;/a&gt;. A discriminated union with a clear discriminant is documentation that the compiler checks. An &lt;code&gt;as&lt;/code&gt; is a comment that the compiler cannot.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://www.typescriptlang.org/docs/handbook/2/narrowing.html&quot;&gt;TypeScript Handbook — Narrowing&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/typeof&quot;&gt;MDN — typeof&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>5 min</readingTime><category>engineering</category></item><item><title>What I Learned Rewriting a Monolith</title><link>https://blog.waystoneai.online/blog/rewriting-the-monolith</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/rewriting-the-monolith</guid><description>A rewrite is a bet that you understand the old system better than its builders. The safer path is a migration with a clean seam and a rehearsed rollback.</description><pubDate>Mon, 20 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;We called it a rewrite. It was a migration, and the difference cost us about six weeks. The old application was a monolith with a single Postgres database, a pile of Express routes, and a cron directory that had accumulated eleven years of good intentions. The plan was to rebuild the worst third of it behind a cleaner service boundary. The plan did not survive contact with the admin scripts.&lt;/p&gt;
&lt;h2&gt;A rewrite is usually a migration&lt;/h2&gt;
&lt;p&gt;If data survives, it is a migration. A true rewrite kills the old system and starts over with an empty database, which almost nobody can afford. What most teams mean is: build the new thing, move the data, keep the business running during the move. That is a migration, and migrations are won or lost on coexistence, not on code quality.&lt;/p&gt;
&lt;p&gt;The bet at the center of every rewrite is that you understand the old system better than the people who built it. Sometimes that is true because the builders are gone and the code is genuinely incomprehensible. Often it is only partially true: you understand the parts you have debugged, and the rest is a rumor with a schema.&lt;/p&gt;
&lt;p&gt;Before writing code, we listed every route and asked: do we understand its behavior well enough to reproduce it, including the weird parts? Three routes got a &amp;quot;no.&amp;quot; Those three took longer than the other thirty combined, which is the normal distribution of migrations.&lt;/p&gt;
&lt;h2&gt;The strangler pattern, in practice&lt;/h2&gt;
&lt;p&gt;The strangler pattern works if you pick the seam carefully: the smallest place where you can route traffic without changing either side. For us it was the HTTP route, proxied through a tiny switch keyed by tenant and route name.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;const handler = flags.useNewOrders(tenantId)
  ? newOrdersHandler
  : legacyOrdersHandler;

return handler(req, res);
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That twenty-line switch did more for the migration than the new service design did. It let us send one internal tenant to the new code, watch it, and send it back without a deploy. It also forced us to keep the old handler runnable for the whole migration, which was the most valuable constraint we had.&lt;/p&gt;
&lt;p&gt;Start with a read endpoint. Reads are reversible; writes are not. A wrong read is a bad page view. A wrong write is a reconciliation project, a customer email, and a very long Tuesday.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The old system has to keep working longer than you want it to. Plan for that, not around it.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Dual writes and the reconciliation problem&lt;/h2&gt;
&lt;p&gt;Once reads are split, writes become the hard part. We wrote to both systems for three months. Dual writes sound simple; there is no transaction spanning the two stores.&lt;/p&gt;
&lt;p&gt;The rules that saved us:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Write to the new system first, then the old, so the old remains the source of truth for anything not yet migrated.&lt;/li&gt;
&lt;li&gt;Make every write idempotent with a stable key, because retries will happen.&lt;/li&gt;
&lt;li&gt;Wrap each write in an outbox and a retry worker; never let a silent exception decide which store wins.&lt;/li&gt;
&lt;li&gt;Reconcile on more than row counts. Compare sums, null counts, and a hash of a sampled set of columns.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Our mistake was treating reconciliation as a nightly report instead of a gate. For a month we compared daily counts and felt good. Soft-deleted rows were never copied, because the backfill query filtered on &lt;code&gt;deleted_at IS NULL&lt;/code&gt; and the old system&amp;#39;s &amp;quot;delete&amp;quot; was actually a status column. We found out when a report that aggregated historical orders came in low by a few percent.&lt;/p&gt;
&lt;h2&gt;The contracts nobody documented&lt;/h2&gt;
&lt;p&gt;The real interface of a monolith is not its API. It is everything that reads its database.&lt;/p&gt;
&lt;p&gt;We inventoried the obvious callers and still missed things. The list that eventually mattered:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Cron jobs living in three different crontabs, two of which nobody owned.&lt;/li&gt;
&lt;li&gt;CSV exports that finance imported manually every Monday.&lt;/li&gt;
&lt;li&gt;A read replica serving a BI tool with hand-written SQL.&lt;/li&gt;
&lt;li&gt;Admin scripts with hardcoded table names and no tests.&lt;/li&gt;
&lt;li&gt;Old mobile clients pinned to response shapes we had already stopped generating.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The one that caught us was an internal admin page everyone believed was read-only. It had a single button that reset an account flag by direct SQL. Someone used it monthly. When we moved writes to the new schema, the button kept writing to the old one, and the two systems disagreed for eleven days before a support ticket surfaced it.&lt;/p&gt;
&lt;p&gt;That is what I would change: inventory readers and writers by watching actual database traffic for a week, not by asking people what they use. People describe the system they intend to use.&lt;/p&gt;
&lt;h2&gt;What not to rewrite&lt;/h2&gt;
&lt;p&gt;The most useful decision was narrowing the scope. A partial rewrite is not a compromise; it is the correct default. The instinct to replace everything at once is usually &lt;a href=&quot;/blog/cost-of-convenience&quot;&gt;a convenience purchase with a deferred bill&lt;/a&gt;.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Phase&lt;/th&gt;
&lt;th&gt;Reads to new&lt;/th&gt;
&lt;th&gt;Writes&lt;/th&gt;
&lt;th&gt;Data state&lt;/th&gt;
&lt;th&gt;Exit condition&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Inventory&lt;/td&gt;
&lt;td&gt;0%&lt;/td&gt;
&lt;td&gt;Old&lt;/td&gt;
&lt;td&gt;Unchanged&lt;/td&gt;
&lt;td&gt;Every reader and writer named&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Shadow&lt;/td&gt;
&lt;td&gt;0% (compare only)&lt;/td&gt;
&lt;td&gt;Old&lt;/td&gt;
&lt;td&gt;Backfill running&lt;/td&gt;
&lt;td&gt;Mismatch rate within tolerance for a week&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Canary&lt;/td&gt;
&lt;td&gt;5%, internal tenants&lt;/td&gt;
&lt;td&gt;Dual&lt;/td&gt;
&lt;td&gt;Backfill complete&lt;/td&gt;
&lt;td&gt;Error and latency parity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Expand&lt;/td&gt;
&lt;td&gt;5% to 50%&lt;/td&gt;
&lt;td&gt;Dual&lt;/td&gt;
&lt;td&gt;Nightly reconciliation&lt;/td&gt;
&lt;td&gt;No unexplained drift for two weeks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cut over&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;td&gt;New, old mirrored&lt;/td&gt;
&lt;td&gt;Old read-only&lt;/td&gt;
&lt;td&gt;Rollback window elapsed&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Retire&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;td&gt;New&lt;/td&gt;
&lt;td&gt;Old kept for audit&lt;/td&gt;
&lt;td&gt;No callers for thirty days&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;We left four things on the old system permanently: a reporting query with regulatory meaning, a batch job that ran twice a year, a legacy import path, and the admin button. Rewriting them would have added risk without adding value. This is the same argument I made in &lt;a href=&quot;/blog/boring-architecture&quot;&gt;The Case for Boring Architecture&lt;/a&gt;: the goal is not a clean diagram, it is a system you can operate.&lt;/p&gt;
&lt;h2&gt;Rollout and rollback&lt;/h2&gt;
&lt;p&gt;A rollout plan is a product, with users, failure modes, and a support story. Ours was a checklist we reviewed before every traffic change. I have come to think of it the same way I think about &lt;a href=&quot;/blog/build-pipeline-as-product&quot;&gt;the build pipeline as a product&lt;/a&gt;: it is infrastructure someone has to own, not a document you write once.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Each route has a flag scoped by tenant, defaulting off.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The old path is still deployed, warmed, and reachable.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Rollback has been rehearsed in staging within the last week.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Backfill jobs are idempotent and can be re-run safely.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Dual-write lag and mismatch alerts page a named owner.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Stop criteria are written down before the change window.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Someone with authority is present for the cutover.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The rollout sequence matters less than the ability to stop. We cut over writes on a Tuesday morning and reverted one route by lunch. Nobody noticed, which is the highest praise a migration can receive.&lt;/p&gt;
&lt;p&gt;The rewrite did not end. The old system still serves that twice-yearly batch job, and I suspect it will outlive the new service. That is fine. The win was not replacing the monolith; it was learning which parts of it were worth keeping, and building a seam that let us decide one route at a time. If you take one thing from this, take the seam. The new code will be rewritten again. A good seam survives.&lt;/p&gt;
&lt;p&gt;This essay is part of the &lt;a href=&quot;/architecture&quot;&gt;Architecture guide&lt;/a&gt;, which maps the decisions from foundations to migration seams.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://martinfowler.com/bliki/StranglerFigApplication.html&quot;&gt;Martin Fowler — StranglerFigApplication&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://docs.aws.amazon.com/prescriptive-guidance/latest/cloud-design-patterns/strangler-fig.html&quot;&gt;AWS Prescriptive Guidance — Strangler fig pattern&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>6 min</readingTime><category>architecture</category><category>devops</category></item><item><title>Designing for the Second Reader</title><link>https://blog.waystoneai.online/blog/the-second-reader</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/the-second-reader</guid><description>You are not writing for the person who asked. You are writing for whoever inherits the question six months later — with no context, no thread, and a deadline.</description><pubDate>Wed, 01 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;The person who needs your design document is not in the room. They have not read the thread, they do not know why option B was rejected, and they are on a deadline that started before they found your file. If the document only makes sense to the people who were there when it was written, it is not documentation. It is a souvenir.&lt;/p&gt;
&lt;h2&gt;The second-reader test&lt;/h2&gt;
&lt;p&gt;I have a crude test for any design note or decision record. Imagine handing it to a competent engineer who joined after the decision, knows the codebase, and has twenty minutes. Can they answer four questions without asking anyone?&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;What problem were we solving?&lt;/li&gt;
&lt;li&gt;What did we choose, and what did we reject?&lt;/li&gt;
&lt;li&gt;What does this choice cost us?&lt;/li&gt;
&lt;li&gt;What would make us change our minds?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If the answer to any of those is &amp;quot;it&amp;#39;s in the thread,&amp;quot; the document has failed. The thread will be buried under six months of notifications, and the person who wrote it may have left.&lt;/p&gt;
&lt;p&gt;This is the same reason I care about the second reader in prose generally. The first reader asked a question and is emotionally invested in the answer. The second reader inherited the question and owes it nothing. You get their attention once, in the first paragraph.&lt;/p&gt;
&lt;p&gt;There is a practical version of this test I use with new documents: ask a teammate who did not attend the discussions to summarize the decision in two sentences. If they have to scroll, the document is organized for the author, not the reader.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The first reader asks a question. The second reader has to live with the answer.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Why ages better than how&lt;/h2&gt;
&lt;p&gt;Most technical documents are written as instructions: here is how the system works, here is how to change it. That format decays quickly. Files get renamed, functions move, diagrams drift by one release, and the &amp;quot;how&amp;quot; section quietly becomes wrong. Nobody updates it because nobody knows it is wrong.&lt;/p&gt;
&lt;p&gt;The &amp;quot;why&amp;quot; ages differently. Constraints outlive implementations. &amp;quot;We cannot use long-running background jobs because the platform kills a request at sixty seconds&amp;quot; stays true until it doesn&amp;#39;t, and when it stops being true, that sentence tells you exactly what to revisit. A decision record is less a tutorial and more a set of assumptions with dates attached.&lt;/p&gt;
&lt;p&gt;Writing down rejected options sounds like bureaucratic throat-clearing. It is the opposite. The most expensive meeting in software is the one where a new person proposes the thing you evaluated and discarded two years ago, and nobody can remember why. A short list of &amp;quot;we considered X and rejected it because Y&amp;quot; prevents that meeting. It also protects you from yourself: if you cannot articulate why the obvious option is wrong, maybe it is not wrong.&lt;/p&gt;
&lt;h2&gt;What belongs in a decision record&lt;/h2&gt;
&lt;p&gt;Context, options, decision, consequences. That is the whole format. Everything else is decoration.&lt;/p&gt;
&lt;p&gt;The consequences section is the one people skip, and it is the one the second reader needs most. A decision record with no downsides is a press release. Every real choice has a cost: an operational burden, a migration you still owe, a feature you will not build, a dependency you now maintain. Write it down while you remember it.&lt;/p&gt;
&lt;p&gt;Here is the template I use. It fits on one screen, which is the point.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-md&quot;&gt;# Decision: &amp;lt;one sentence, present tense&amp;gt;

- Date:
- Status: accepted | superseded by &amp;lt;link&amp;gt;
- Owner:
- Review date:

## Context
What is true right now that forces a choice. Constraints, numbers,
links to the ticket. No history lesson.

## Options considered
1. &amp;lt;option&amp;gt; — why not.
2. &amp;lt;option&amp;gt; — why not.
3. Do nothing — what that costs.

## Decision
What we chose, and the reason in one sentence.

## Consequences
What gets better. What gets worse. What we still owe.
What would make us reopen this.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Two fields do most of the work. &amp;quot;Status&amp;quot; means an old record can point at its replacement instead of becoming a lie. &amp;quot;What would make us reopen this&amp;quot; turns the document into a trigger for future work rather than a snapshot.&lt;/p&gt;
&lt;h2&gt;Openings that waste the first minute&lt;/h2&gt;
&lt;p&gt;Weak openings describe the document. Strong openings describe the problem and the decision.&lt;/p&gt;
&lt;p&gt;Bad: &amp;quot;This document describes the authentication refactor and its various components.&amp;quot;&lt;/p&gt;
&lt;p&gt;Bad: &amp;quot;As discussed in the meeting on Tuesday, we have been considering several approaches to session handling over the past few weeks.&amp;quot;&lt;/p&gt;
&lt;p&gt;Both make the reader wait for the point. The second reader does not care what you discussed. They care what is true now.&lt;/p&gt;
&lt;p&gt;Good: &amp;quot;Sessions currently expire after fifteen minutes, and mobile clients re-authenticate silently every ten. We are moving to 24-hour refresh tokens because the current behavior causes roughly one support ticket per hundred daily users. This note records the tradeoffs.&amp;quot;&lt;/p&gt;
&lt;p&gt;Three sentences: current state, decision, reason. A reader who stops there still knows what happened. That is the standard. Every additional paragraph should be optional for someone who only needs the decision.&lt;/p&gt;
&lt;p&gt;This matters even more when the document will be read by people outside your native dialect. Clarity is not a native-speaker virtue; plain sentences are a courtesy to every reader, including the one who inherited your file. I made a longer version of that argument in &lt;a href=&quot;/blog/english-as-a-second-language&quot;&gt;English as a Second Language&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;A checklist before you publish&lt;/h2&gt;
&lt;p&gt;I run this before sending a design note to anyone who was not in the original conversation.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The first paragraph states the problem and the decision, not the document&amp;#39;s purpose.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The date, status, and owner are at the top.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Rejected options are listed with a reason each.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Consequences include at least one cost, not just benefits.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; There is an explicit condition for revisiting the decision.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Internal names and acronyms are defined on first use.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Links point to durable artifacts, not a chat thread.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Someone outside the project has read it and could explain it back.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;One more habit: write the decision record when you make the decision, not after the project ships. The version written afterward is shaped by the outcome, and the second reader can tell. The version written before is honest about uncertainty, which is the most useful thing you can hand them.&lt;/p&gt;
&lt;p&gt;The person who inherits this will probably never meet you. They will open the file at a bad moment, looking for one answer, and they will either find it in the first minute or spend an hour reconstructing your context from fragments. Writing for them is not extra work on top of engineering. It is the part of engineering that survives the sprint. It is also, not coincidentally, why the most durable systems tend to be the ones with the most boring decisions written down in plain language — the argument I made in &lt;a href=&quot;/blog/boring-architecture&quot;&gt;The Case for Boring Architecture&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://developers.google.com/tech-writing/one/audience&quot;&gt;Google — Technical Writing: Audience&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://diataxis.fr/&quot;&gt;Diátaxis — A systematic framework for technical documentation&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>5 min</readingTime><category>writing</category><category>engineering</category></item><item><title>A Field Guide to Slow APIs</title><link>https://blog.waystoneai.online/blog/slow-apis</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/slow-apis</guid><description>Your API is not slow everywhere. It is slow for one query, one tenant, one hour. Find that hour, read the p95, and fix the tail instead of the average.</description><pubDate>Tue, 16 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;The first thing I do when someone tells me the API is slow is ask them to make it slow for me. The second thing I do is stop them from opening the query log until we agree on which request, for which account, at which hour. Almost every latency investigation I have been part of went wrong at the same place: someone optimized the average while the actual complaint was about a tail.&lt;/p&gt;
&lt;h2&gt;The average is a rumor&lt;/h2&gt;
&lt;p&gt;Averages are the most comfortable number in performance work, which is why they are the least useful. If I serve nine requests in 20 ms and one in two seconds, the average is about 220 ms. That number describes no request that ever happened.&lt;/p&gt;
&lt;p&gt;Percentiles describe request populations. p50 is the typical request. p95 is your busy or heavier requests. p99 is the tail that produces timeouts, retries, and angry messages. For an endpoint that fans out to ten downstream calls, if each call has a one percent chance of hitting its p99, the chance that at least one does is roughly ten percent. In my experience that arithmetic is why the p99 becomes the p50 of your users&amp;#39; complaints.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;What it answers&lt;/th&gt;
&lt;th&gt;What it hides&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;p50&lt;/td&gt;
&lt;td&gt;What a typical request feels like&lt;/td&gt;
&lt;td&gt;Every bad experience, by definition&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;p95&lt;/td&gt;
&lt;td&gt;The worst case most users still see&lt;/td&gt;
&lt;td&gt;Rare tenants with pathological data&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;p99&lt;/td&gt;
&lt;td&gt;What triggers retries and alerts&lt;/td&gt;
&lt;td&gt;Whether the tail is one query or many&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Max&lt;/td&gt;
&lt;td&gt;The single worst request&lt;/td&gt;
&lt;td&gt;How often it happens&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The first useful question is not &amp;quot;what is our average latency&amp;quot; but &amp;quot;which percentile is failing, and for whom.&amp;quot;&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A slow endpoint is a measurement problem before it is an optimization problem.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;CPU-bound or queue-bound&lt;/h2&gt;
&lt;p&gt;Once you have a percentile and a population, the next split is whether the time is being computed or waited on. The fix is completely different, and guessing here is how teams spend a week adding indexes to a connection pool problem.&lt;/p&gt;
&lt;p&gt;CPU-bound slowness looks like this: p50 is already elevated, CPU sits near its ceiling under load, and a profiler shows time inside your code — serialization, a tight loop, a regex, JSON encoding a large object graph. Adding capacity helps until it doesn&amp;#39;t; the real fix is doing less work per request.&lt;/p&gt;
&lt;p&gt;Queue-bound slowness looks the opposite: p50 is fine, p99 is ugly, CPU is bored, and the time is spent waiting. The wait has an address: a connection pool, a row lock, a thread pool, an event loop backlog, or a downstream service. You can often see it before you understand it, because latency rises with concurrency while utilization stays flat.&lt;/p&gt;
&lt;p&gt;If you cannot tell which one you have, you are not ready to change code.&lt;/p&gt;
&lt;h2&gt;Triage order: measure, segment, trace, reproduce&lt;/h2&gt;
&lt;p&gt;I follow the same order every time. It is deliberately boring.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Measure.&lt;/strong&gt; Get server-side timings for the route, not browser timings and not a colleague&amp;#39;s recollection. Record start, each downstream call, and response write. If the number does not match the complaint, the complaint is about something else.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Segment.&lt;/strong&gt; Break the distribution down by route, tenant, query shape, and hour. A global p99 can be perfectly healthy while one tenant&amp;#39;s export job poisons every request that touches the same table.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Trace.&lt;/strong&gt; Take one slow request and one fast request for the same endpoint, and compare the spans. The difference is usually a single call whose duration changes by an order of magnitude, not a uniform slowdown.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Reproduce.&lt;/strong&gt; Replay the slow request with the real payload against a dataset with the real row counts. Synthetic payloads and empty staging databases lie in your favor, which is the wrong direction.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Only after step four do I open a profiler. Skipping ahead is how you end up &amp;quot;optimizing&amp;quot; something that was never on the hot path.&lt;/p&gt;
&lt;h2&gt;The usual suspects&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;N+1 queries.&lt;/strong&gt; The classic. It survives because each individual query looks cheap in the log and the total is invisible unless you count them.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ts&quot;&gt;// Bad: one query for the list, then one per row.
const orders = await db.order.findMany({ where: { status: &amp;#39;open&amp;#39; } });
for (const order of orders) {
  order.customer = await db.customer.findUnique({
    where: { id: order.customerId },
  });
}

// Fixed: two queries, one per table.
const openOrders = await db.order.findMany({ where: { status: &amp;#39;open&amp;#39; } });
const customerIds = [...new Set(openOrders.map((o) =&amp;gt; o.customerId))];
const customers = await db.customer.findMany({
  where: { id: { in: customerIds } },
});
const byId = new Map(customers.map((c) =&amp;gt; [c.id, c]));
for (const order of openOrders) {
  order.customer = byId.get(order.customerId);
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The fix is not always a join. A batched lookup is often easier to cache and easier to reason about, and it keeps the query count constant as the page size grows.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Connection pools.&lt;/strong&gt; A pool that is too small turns concurrency into a queue; a pool that is too large turns your database into a swapping contest. Watch pool wait time, not just pool size. And set a timeout, because a request waiting forever for a connection is indistinguishable from a hung service.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Lock contention.&lt;/strong&gt; Long transactions holding row locks will make unrelated endpoints slow in a way that looks random. &lt;code&gt;pg_stat_activity&lt;/code&gt; and wait events usually settle the argument in minutes. Migrations that rewrite a table are a recurring cause; they are also the easiest to schedule away from peak, which is why they keep happening at peak.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Retry storms.&lt;/strong&gt; A timeout shorter than your p99 is a retry generator. Retries add load, load pushes p99 higher, more retries fire. Add jitter, cap attempts, and use a budget. If the downstream is already struggling, the kindest thing your client can do is give up earlier.&lt;/p&gt;
&lt;h2&gt;A diagnostic checklist&lt;/h2&gt;
&lt;p&gt;Before changing anything, I want answers to these. If half are blank, the investigation is not done.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Which exact route, tenant, and hour are slow?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Which percentile moved, and when did it start?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Is the timing server-side, and does it match the user report?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Is CPU busy or idle during the slow window?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Does p99 correlate with pool wait time, lock waits, or GC pauses?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; How many queries per request, and does that number grow with page size?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; What is the timeout, and how many retries does one failure produce?&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Can I reproduce it with a real payload and realistic row counts?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;None of this requires a new observability vendor. It requires one afternoon, a dashboard you already have, and the discipline to segment before you speculate. If you want the front-end counterpart — the metric that measures how stable a page feels while it loads — that is a different distribution with the same lesson, which I wrote about in &lt;a href=&quot;/blog/cls-in-practice&quot;&gt;CLS in Practice&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;The last thing I do is write down what the slow request was, because in six months someone will report the same symptom and the fastest path is the note. That is also why I treat reproducible traces as a deliverable rather than a byproduct; &lt;a href=&quot;/blog/build-pipeline-as-product&quot;&gt;the build pipeline as a product&lt;/a&gt; is the same idea applied to CI. A slow API is rarely a mystery. It is usually one unexamined assumption, repeated a few thousand times an hour.&lt;/p&gt;
&lt;p&gt;This essay is part of the &lt;a href=&quot;/performance&quot;&gt;Performance guide&lt;/a&gt;, which connects tail latency, layout stability, and build time.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://sre.google/sre-book/handling-overload/&quot;&gt;Google SRE Book — Handling Overload&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://www.brendangregg.com/usemethod.html&quot;&gt;Brendan Gregg — The USE Method&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>5 min</readingTime><category>performance</category><category>engineering</category></item><item><title>The Case for Boring Architecture</title><link>https://blog.waystoneai.online/blog/boring-architecture</link><guid isPermaLink="true">https://blog.waystoneai.online/blog/boring-architecture</guid><description>Every team wants the interesting choice. The interesting choice is usually the one you pay for at 3 a.m., with interest.</description><pubDate>Tue, 02 Jun 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;The most interesting system I ever shipped was also the one that woke me up the most. That correlation took me embarrassingly long to notice. I used to think &amp;quot;boring&amp;quot; was an insult you paid a team when you could not think of anything nicer to say about their stack. Now I think it is the highest compliment I can pay an architecture, and I will explain why with a migration that went sideways for exactly the reason you would guess.&lt;/p&gt;
&lt;h2&gt;The migration that made the argument for me&lt;/h2&gt;
&lt;p&gt;We had a job scheduler that was, by every measure I cared about, boring. It was a table in Postgres, a worker process, and a cron entry. It ran reminders, nightly exports, and a few cleanup tasks.&lt;/p&gt;
&lt;p&gt;Then we decided to replace it with a workflow engine. The pitch was reasonable: retries, visibility, backoff, a UI where support could see what happened. We ran a migration over a long weekend. On Monday, a subset of reminder emails went out twice, and a small number of invoices were generated with a stale amount. Nothing catastrophic, but enough that I spent two days reconciling rows by hand while explaining to people that the new system was &amp;quot;mostly working.&amp;quot;&lt;/p&gt;
&lt;p&gt;The root cause was not a bug in the engine. It was a behavior nobody documented: the engine retried a step on timeout even when the underlying handler had already committed. Our old table-based worker had the same class of problem, but we knew it, because we had written the idempotency key ourselves and everyone on the team had read that twenty-line function. With the new engine, the behavior lived in someone else&amp;#39;s source tree and someone else&amp;#39;s changelog.&lt;/p&gt;
&lt;p&gt;We ended up running the engine in a mode that effectively reproduced the old queue-and-cron design, plus a wrapper. We paid for a new dependency and kept the old mental model.&lt;/p&gt;
&lt;h2&gt;The upgrade tax&lt;/h2&gt;
&lt;p&gt;Every dependency charges rent. The question is whether the rent is predictable.&lt;/p&gt;
&lt;p&gt;Boring technology charges rent you can budget for. Postgres gets a minor release; you patch it. A queue table gets a new column; you write a migration. The failure modes are documented in a manual that many people have read, so when something breaks, the search results are from 2016 and they are still correct.&lt;/p&gt;
&lt;p&gt;Interesting technology charges a different kind of rent. The changelog is a marketing page. A minor version rewrites the internals and calls it performance. The docs describe the happy path and leave the failure modes to your imagination and your on-call rotation. You are not just adopting a tool; you are adopting a second job maintaining your understanding of it.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Novelty is a loan. The interest is paid by whoever is on call.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;That is &lt;a href=&quot;/blog/cost-of-convenience&quot;&gt;the cost of convenience&lt;/a&gt; in one sentence: the purchase price shows up in the design review, and the carrying cost shows up later.&lt;/p&gt;
&lt;h2&gt;Three questions before you adopt anything&lt;/h2&gt;
&lt;p&gt;I now ask three questions whenever someone proposes new infrastructure. They are not clever, and that is the point.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Who debugs this at 3 a.m.?&lt;/strong&gt; Not &amp;quot;who owns it&amp;quot; in the RACI sense. I mean: can one of us open a terminal, read the logs, and form a hypothesis without opening a vendor support ticket? If the answer is &amp;quot;the vendor,&amp;quot; you have outsourced your worst hour to someone with a queue.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;What does removal cost?&lt;/strong&gt; Every addition should have an exit price in engineer-days. If migrating off the thing is a quarter of work, you have not adopted a tool; you have adopted a decade.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;What does it replace, and does it actually beat that?&lt;/strong&gt; Compare it to the boring status quo, not to nothing. A new search engine beats &amp;quot;no search.&amp;quot; Does it beat &lt;code&gt;ILIKE&lt;/code&gt; plus a trigram index for the two thousand rows you actually have?&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;If those answers are uncomfortable, the interesting choice is probably not ready.&lt;/p&gt;
&lt;h2&gt;Boring versus interesting, side by side&lt;/h2&gt;
&lt;p&gt;The table below is how I frame the trade in practice. The &amp;quot;boring&amp;quot; column is not always right, but it is the default that has to be argued against.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Need&lt;/th&gt;
&lt;th&gt;Boring choice&lt;/th&gt;
&lt;th&gt;Interesting choice&lt;/th&gt;
&lt;th&gt;What the interesting choice usually costs&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Background jobs&lt;/td&gt;
&lt;td&gt;A jobs table plus a cron worker&lt;/td&gt;
&lt;td&gt;Managed workflow engine&lt;/td&gt;
&lt;td&gt;New retry semantics, new dashboard, same idempotency work&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Caching&lt;/td&gt;
&lt;td&gt;In-process LRU with a TTL&lt;/td&gt;
&lt;td&gt;Dedicated cache cluster&lt;/td&gt;
&lt;td&gt;A second consistency model and a new outage mode&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Search&lt;/td&gt;
&lt;td&gt;Postgres full-text and trigram indexes&lt;/td&gt;
&lt;td&gt;Separate search service&lt;/td&gt;
&lt;td&gt;Dual writes and an indexing lag nobody budgeted for&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Event delivery&lt;/td&gt;
&lt;td&gt;Outbox table with a poller&lt;/td&gt;
&lt;td&gt;Message broker&lt;/td&gt;
&lt;td&gt;Ordering guarantees you now have to reason about&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Notice that the boring column is not &amp;quot;do nothing.&amp;quot; It is &amp;quot;do the small version of the thing, in a place you already operate.&amp;quot;&lt;/p&gt;
&lt;h2&gt;What boring actually buys you&lt;/h2&gt;
&lt;p&gt;Boring buys debuggability. One database, one process, one log stream means a failure has exactly one place to hide. When the system is interesting, the first hour of every incident is spent figuring out which of six moving parts is lying to you.&lt;/p&gt;
&lt;p&gt;It buys onboarding. A new engineer can read the schema and the cron file and understand the whole system in an afternoon. Teams that can absorb new people quickly can also absorb new incidents quickly.&lt;/p&gt;
&lt;p&gt;It buys reversibility. Swapping a cron script takes an afternoon. Swapping an event platform takes a quarter. Reversibility is the thing you want most when you are wrong, and you will be wrong sometimes.&lt;/p&gt;
&lt;p&gt;And it buys sleep. Not metaphorically. The boring system fails in ways you have seen before, which means you fix it in twenty minutes and go back to bed. The failure modes of a monolith are the ones I described in &lt;a href=&quot;/blog/rewriting-the-monolith&quot;&gt;What I Learned Rewriting a Monolith&lt;/a&gt;: visible, local, and usually caused by something you can read.&lt;/p&gt;
&lt;p&gt;A boring stack is not a small stack. It is a stack whose failures you have already met.&lt;/p&gt;
&lt;h2&gt;An adoption checklist&lt;/h2&gt;
&lt;p&gt;Before adding anything to the stack, I run through this list. If more than one item is unchecked, the answer is &amp;quot;not yet.&amp;quot;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; One engineer can explain the failure modes without opening the vendor dashboard.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The rollback path is documented and does not require a support contract.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The new tool removes a category of code rather than relocating it.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; We have run it in staging under production-like load for at least a week.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The exit cost is written down in engineer-days.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; We would still choose it if the price doubled next year.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; At least one person on the team has operated it before, not just read about it.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The goal is not to be boring forever. The goal is to spend your novelty budget on the one or two places where it changes the product, and to make everywhere else as unremarkable as possible. Most systems do not need a more interesting architecture. They need a smaller one, with fewer ways to fail, run by people who can see all of it at once.&lt;/p&gt;
&lt;p&gt;This essay is part of the &lt;a href=&quot;/architecture&quot;&gt;Architecture guide&lt;/a&gt;, which maps the decisions from foundations to migration seams.&lt;/p&gt;
&lt;h2&gt;Further reading&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://martinfowler.com/bliki/MonolithFirst.html&quot;&gt;Martin Fowler — MonolithFirst&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://mcfunley.com/choose-boring-technology&quot;&gt;Dan McKinley — Choose Boring Technology&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</content:encoded><language>en</language><readingTime>6 min</readingTime><category>architecture</category></item></channel></rss>