Blog | Tristan Kernan

Good List Week 16

2026-07-20T00:00:00-04:00

West Essex Trail

I signed up for another 5k (and am leaning towards a 10k as well) in order to keep myself running - I am very external validation oriented. Feeling a bit tired of Brookdale Park I explored other options in my area and recalled the West Essex Trail - an old railroad line that's been converted to a walking, running and biking trail. It's well maintained, flat, and soft dirt which makes it ideal for long runs. The scenery is lovely, especially on a warm summer day, as the trail slices through the backyards of suburban houses, small businesses, and even crosses a small highway over an old railroad bridge.

The total length is about 3 miles, which means for a 10k I'd run a full loop one way and then back again. I like that this provides a physical gauge of training progress, as I slowly add distance to my runs.

Karaoke

I organized a karaoke outing with friends last week, which I was quite nervous about - I have long held insecurities over my voice, especially singing. Going with trusted friends gave me the psychological safety necessary to open up, and that I did, belting out Mr Brightside, Ice Ice Baby, and All the Small Things. It may be that I am reading a book on evolutionary biology/psychology (Moral Evolutions), because I began to see the communal emotional outlet of group making music and singing together. Certainly seeing my friends (and myself) express themselves so openly and honestly was deeply nurturing for me.

Climbing

Rounding out the week's adventures I went to a local climbing gym with my girlfriend. I had climbed once before, so had some basic idea of what I was doing, but clearly I had forgotten just how darn challenging it is to do even basic climbs. We started with bouldering, floundering around v0s and v1s, until Maria succeeded in finishing a climb. Before, I was hesitant about climbing too high and falling, but after a switch in me flipped (Maria said she could see my eyes changed) and I immediately scrambled up the wall to finish the same climb. The competitive spirit within me runs deep, clearly.

We tried our hands at several of the easy bouldering climbs, and then moved onto rock climbing. I managed to climb about a third up the wall before jumping down, which I practiced repeatedly (perfect opportunity to face/accept/float a strong fear!). Maria went way up the wall, which my competitiveness was uninterested in besting, overridden by the height.

After seeing how much we both enjoyed the experience (for similar and different reasons), we are definitely going back. Climbing meets some real psychological and physical needs: healthy risk and excitement, team-work, and goal accomplishment.

Good List

And that leads me to the Good List itself. I can't believe I've kept at it for 16 weeks, or 4 months. It's been motivating in a positive way, as I have found I both look forward to posting what I've been up to, as well as challenging myself to try new experiences to post about. Maybe influencer is a potential career change for me, haha.

I have decided to take a break from the posts, to give myself some time and space to reflect on the process (maybe irregular one offs instead of strict weeklies), and to focus my time and attention on writing my new recovery series, which is currently near and dear to my heart.

Side note: the Good List Visualizer has been updated to include these new entries.

Recovery: Burnout - Origins

2026-07-14T00:00:00-04:00

I've been blocked writing about my experience with burnout for a full year now. I feel so much fear and shame: I'm a failure and I shouldn't talk about it. It's even been hard for me to open up in safe spaces, with close friends - the message inside my head of "here he goes again, talking about his brain fog for the hundredth time, why doesn't he stop feeling so sorry for himself and just push through?" That burnout is dominated by inner symptoms and relatively invisible on the outside means I am limited in receiving external validation, which has kept me locked up inside myself, perpetuating the very conditions that fuel burnout. Blah. It's a vicious cycle.

As I described in my article on anxiety, I think of the anxiety condition as a system of stored stress and fear. Part of the healing journey is in releasing these pent up emotions, and a key method for me is expressing my experiences in writing. It's my intent to share openly and honestly about my experiences, first to help myself relieve the fear and shame, and second to help others who are stuck in these emotions, bewildered as to their condition, its causes and its healing.

One year ago

Last July I caught the flu. It was a terrible case of the flu, I was knocked off my ass for a good week. Symptoms persisted for another week, then another week, and I dragged myself to the doctor to get tested, presuming that I had caught COVID again and could I get on some of those antivirals? I tested negative for COVID, positive for the flu, and going back a couple weeks later once more tested positive for the flu a full month out. My physical prowess recovered, and I was able to move through life, but my brain kept the score: I experienced persistent brain fog. It's taken many aspects over the past year, it swings up and down and back and forth in intensity. Some of the ways I've described my brain fog are mental dullness, lack of sharpness, sluggishness, and like a match that just won't light.

This "long flu" terrified and perplexed me. When will it fade so I can get back to my old self? Why do I have good days and bad days, good weeks and bad weeks? Most frustrating - doesn't my body understand that in order to correct my environment to reduce stress I need my full cognitive capacity? Where's the fairness in a condition that prevents me from addressing it?

In hindsight, now, I can see the progression of the condition. I can see the alarms flashing. Back at the time, though, my world shrunk to getting through each day, hyperfixated on managing my various crises. I can't say what could've been if I had had someone to sit me down, slap me, and thoroughly explain my condition and how to act skillfully. Perhaps I would've dutifully nodded, and immediately returned to crisis management. It's a really hard mental habit to break.

Crisis management

In the month preceding the flu, I ran the stress gauntlet.

I had started applying to new jobs, playing the market to see if any opportunities interested me. I was immensely fortunate, out of a dozen or so applications I found myself in several active interview cycles. This quickly proved to be too much for me. I have intense test anxiety, owing to growing up with perfectionistic expectations around grades. Interviews that take the form of proctored tests trigger this anxiety quite strongly. I noticed that I was overdoing it and withdrew my applications, but that stress had been seeded.

I started dating a new woman. I was instantly obsessed. It's really incredible how my brain picks up on subtle cues that hearken back to childhood and past relationships, how I latch onto someone I don't know rationalizing the overwhelming feelings as "passion." This new relationship had all the hallmarks of unhealthy codependence: intense initial attraction, over-sharing, boundary violations, love-bombing. I recognized these patterns in real time but I struggled to escape; I was hooked. After a fight a month in I gathered my courage and ended the relationship. I was left reeling, deeply vulnerable after this experience.

Most significant, one of my reports at work went through his own breakdown. This tested me on so many levels: as a recovering codependent, I had to maintain strict boundaries with myself to not try to fix or caretake my employee. I also had to set boundaries with my own boss: as a brand new people manager with no training under my belt, I decided to turn over the situation to my boss and HR, and follow their instructions. Unbeknownst to me at the time, my boss has his own undiagnosed abandonment issues. While my employee spent multiple months contributing nothing or next to nothing, my boss purposefully misled, gaslit, and abused me over the situation.

Burning out at work

This latter situation was especially distressing because it mirrored what I experienced growing up. When my father and stepmother divorced, my father placed me in the care of my now ex-stepmother, despite full awareness that she was mentally unwell and could not provide a safe environment for a child. As the abuse mounted by my ex stepmother, my father turned the tables and told me that it was my fault, my choice, and that really I was the one abandoning him. At ten years old, I had no power to challenge him, and I internalized this guilt. Over the years, my ex stepmother spiraled down into depression and drug addiction, and at every juncture my father repeated the same lies: that I was responsible for my ex stepmother, that he was the true victim.

When my employee broke down, it took me back to when my ex stepmother broke down. I was at the prestigious governor's school summer program for gifted high schoolers. At a time when I should have been fully engrossed in the experience, making friends, learning, and playing, I was on the phone begging my father to help my ex stepmother whose drug addiction had escalated to the degree of erratic and dangerous behavior. At work last year, I had to beg my boss to help me with my employee, to provide minimal guidance and support. My boss not only refused to help, but actively misled and manipulated me, making me out to be responsible for my employee's breakdown and lack of recovery.

I kept my head down and tried to manage the work situation for several months before I realized that the toll on me was too great, that there was no path to resolution: my boss said that firing is off the table, so short of a spontaneous and miraculous recovery, the status quo would persist endlessly. I withdrew from managing this employee, offering my full support to help with the transition but making it clear that I had to step back for my own sake.

The other half of narcissistic abuse comes after you set a boundary and step back. I had naively hoped that by playing it nice, not naming the abuse, I could placate my boss. Just as when I was a child dealing with narcissistic parents, it didn't matter how small I made myself, how little I spoke up for myself, how much abuse I accepted by turning the other cheek. After stepping back from managing this employee in crisis, my boss began a targeted campaign to force me out. Every week there'd be some new abuse. I thought I was handling it well by continuing to not engage, not react, keeping my profile as small as possible.

When a teammate suddenly departed, I saw how my boss was crushed, actually devastated. He honestly believed he could convince this person to stay, when they had already accepted an offer elsewhere. I had a lot of empathy for my boss at that moment, seeing how his narcissism was sourced from the same abandonment wounds that scarred me. My boss for his part felt shocked and laid off the abuse for a couple weeks - but the narcissistic drive to dominate or exile me quickly returned. The "fix was in" when I got my performance review that made no mention of the months I spent managing my employee in crisis. A week later my boss had a meltdown and threw a temper tantrum alternating between begging me to stay and threatening to fire me.

I had been holding on for months, rationalizing it as accumulating savings for a period of time off to rest and recover. After the gaslighting performance review, the part of me that was holding on let go. After the temper tantrum, I knew it was time to go. I walked away from a cool ten to fifteen grand by quitting before the end of the year.

Warning signs

From June to December, I spent a solid six months in acute and intense stress and anxiety. Looking back I don't know how I managed to make it work for so long. I suppose it's because it felt familiar, that I was trained for this in my childhood. While my mind was reeling from the stress and brain fog, my body was marking the score. Long flu was the first physical ailment, rare and usually responded to with "don't you mean long COVID?" Next was jaw tension - my mouth and jaw were practically locked tight, with night time grinding, leading to aches and pains during the day. When I went to the dentist, they also found a reinfection of a tooth that had previously been treated with a root canal - a root canal that the endodontist said was perfectly done. I was told that reinfection is exceedingly rare. But I wasn't surprised. I knew that months of stress was causing my body to deteriorate, to be prone to infection and reinfection and reducing my body's ability to fight off infection. Finally my gastric system started acting up, with bowel distress daily.

More than the cognitive symptoms, the physical symptoms really spooked me into leaving the job and taking time for recovery. As uncomfortable as an emergency root canal is, it was the wake up call I needed that I had to make a change to right my trajectory. My body was screaming out to me.

It took me a long time to listen, but I'm thankful I did because not everyone can stop the process of self sabotage before their life is in shambles, or worse. I left my job with my head held high and finally started on the long journey of recovery.

Good List Week 15

2026-07-13T00:00:00-04:00

Anderson Park

One of my favorite ways to spend a spare hour is to bring a book and coffee to Anderson Park in Montclair. The park itself is lovely, a sprawling bowl of green grass where frisbee, volleyball and soccer is played by athletic youths. Around the central grass the walking path has several shaded benches, ideal for plopping down and nestling into a book (last week's read was A Brief History of the Earth by Andrew Knoll, a fascinating tour of the planet's and our own evolution in time and space). Sometimes I'll bring my picnic blanket and spread out on the grass, lying and taking in the warm summer air.

Nature Walk

For my artist's date this week (it's hard to believe I'm already on week 11! time flies!), I went to Eagle Rock Reservation for a nature walk. I had purchased a loupe (which happened to be a recent crossword answer, serendipitous!), for peering into the depths of moss and leaves and such. Turns out that with a loupe the target must be basically pressed against my face, which was beyond my comfort level to do against the ground or a tree. Loupes aside, I brought a notebook refashioned as a field notebook and pretended to be a wildlife biologist (inspired by a recent read of Annihilation), recording observations on plants and animals. As luck would have it, I got a great look at a fox who I surprised in the woods, a couple deer, and even a large frog swimming in a small puddle left over from recent rains. My inner artist was very pleased.

The Invite

On Friday I had been feeling some kind of way, so I decided to cancel plans and take myself to the movies. I saw The Invite, on the basis of some youtube trailer reviews, and was pleasantly surprised to find myself laughing throughout the movie. The plot revolves around a dinner invite from a couple to their upstairs neighbors. From the get-go, the dialogue and acting were superb, I especially enjoyed the dynamic between Seth Rogen and Olivia Wilde's characters.

Side note: the Good List Visualizer has been updated to include these new entries.

Katamari

2026-07-10T00:00:00-04:00

I've been playing my nintendo switch again, grabbing games from the library. First, what a great deal, it reminds me of going to blockbuster as a kid, except there's no fee to rent, and generous return turnarounds. Second, it's been fun to play random indie games alongside more popular hits. I grabbed Once Upon a Katamari a couple weeks back, and what a blast, it took me right back to playing Katamari on my playstation 2 as a teenager.

For the uninitiated, Katamari is a game with a sacred mission: rolling up the world into a giant ball. You start small, maybe a few centimeters in size, rolling up paper clips and peanuts. Then, as you grow, you start rolling sushi rolls and playing cards, then small animals, then big animals, then people, bushes, trees, houses, and eventually islands. It's as silly as it sounds.

Reading the Artist's Way, and getting into a daily schedule, I wanted to get back into what it feels like to play at work. So I started where I started as a teen: making games in the terminal. Back then, it was battleship printed to the console; that evolved into curses games, followed by graphical games with sdl2 (C) and libgdx (java) and pygame (python).

I made a simple Katamari-inspired game in the terminal this week (code-named Battle, you can see my creative energy went into the game not the name!) - check it out here. The object is the same: roll over other players of lower levels to accumulate them and grow in size, while avoiding higher level and larger players.

It was fun, even doing 2d math in the convoluted dimensions of curses where (0,0) is the top left corner, and positive is down and right. I set a daily goal of 1 hr and regularly exceeded this goal to the point I had to hold myself back (pacing works in both directions, doing not too much and not too little). I am pleased with how it came out, though not perfectly pleased - it has some rough edges and bugs remaining, but I'm intentionally practicing marking it done now so as to also get in the habit of shipping imperfect code over endlessly tinkering (how many half-done projects lie in my github private-repo graveyard!).

As Julia Cameron said, when work is fun, it's a lot easier than when work is forced and perfectionistic. Thank you, Julia!

Recovery: Anxiety

2026-07-06T00:00:00-04:00

I have been thinking a lot about anxiety recently. I recently read the wonderful Claire Weekes' book Hope and Help for Your Nerves. Ironically I was anxiously blocked to read the book; I had it on my shelf for almost two full months, I picked it up and started it two times before putting it back down again. Third time's the charm, I read it cover to cover.

How could I be anxious about reading a book on anxiety? I believe that anxiety, as a coping mechanism, is strongly defended and rationalized by the mind: at a certain point, I'd prefer to remain anxious and blocked than face my fears and move forward. Claire Weekes, with simple and penetrating insight, fully reveals anxiety as the emperor without clothes; my psychological defenses ramped up to prevent this realization from bubbling up, sending the very fear response I'm seeking to face to overwhelm me and cause an avoidance behavior.

Anxiety as bewilderment

Claire Weekes does a great job at listing the various ailments that anxiety sufferers face. I made several new connections myself; while I had been aware of the more obvious symptoms like body tension, shortness of breath, and susceptibility to illness, I saw how episodes of low blood sugar leads to weakness and panic, my bowels are thoroughly permeated with anxiety, and most poignantly that avoidance is an anxiety response. The anxiety sufferer begins with bewilderment of such symptoms, worrying if they've become the victim of some terrible disease like cancer. Health anxiety and internet doomscrolling has been my bread and butter for ages; ordering random supplements and convincing myself I must have some terrible condition lurking within me.

That so many of my health symptoms can be plausibly traced to anxiety seems obvious in hindsight. The Myth of Normal by Gabor Mate is relevant here, with its thorough accounting of the various ailments affecting trauma and anxiety sufferers. The story that sticks with me is of the doctor's office that noted its patients tended to be the most pleasant people, in other words people pleasers. The behavior tendency to suppress one's emotions, especially the most challenging like guilt, shame, fear, and pain, is directly linked to rare disorders of the body. As a wake up call, I took home the critical importance of continuing on my healing journey and seeking out safe loving communities.

Anxiety as avoidance

Avoidance is maybe the most subtle symptom of anxiety for me. Through rationalization and denial I am typically not even aware I am avoiding some task, person or feeling of which I am afraid. One clear example is addressing health issues promptly. I've had my share of bodily injuries, but what sets me apart is waiting weeks, months, or even years to seek treatment, all the while convincing myself that I am definitely taking care of myself! Once I do seek treatment, any bump in the road sets me back on the path of avoidance. When I injured my foot, a podiatrist told me that they had no idea what was wrong and good luck. Instead of seeking a second opinion, I took this to heart and continued suffering in silence for many more months before taking another positive step forward. I now make a conscientious effort to address health issues promptly, and encourage others to do the same. So much suffering is avoidable just by taking the first few positive actions.

Another major area for avoidance for me is relationships. Growing up in a dysfunctional family, I learned that speaking up for myself was dangerous and liable to be responded to with shame, guilt, neglect and abandonment. As an adult, I've swallowed my feelings, my wants and needs, my interests, my complaints, out of fear. But it was so second nature that I wasn't even aware of it, I rationalized my behavior as normal, if not outright heroism: here I am taking the high road by putting someone else first! I'm not going to turn out like my parents, explosive and selfish and rageful. Lo and behold, as the ancient Greeks said, the very steps I took to avoid my fate led me directly to it. I was thoroughly dysfunctional myself, just in the inverted, codependent-dominant way.

It took many painful friendships, relationships, and jobs to see my role in my unhappiness. It took recovery for codependence for me to see how I was sabotaging myself by not speaking up for myself, setting boundaries, and ending unhealthy relationships promptly rather than sticking around and trying to fix them. It took years after starting to put new behaviors in practice for me to begin to see how deep-rooted the fear is within me, how I've been running from that ancient, primordial fear of abandonment my entire life.

Other avoidance strategies that I employ are not getting mail from my mailbox for weeks (and then leaving the mail unopened on my table); using "safe" chores like dishes, laundry, cleaning to fill in empty time; running errands to businesses unnecessarily far away to add extra driving time; picking up my phone or alt tabbing to the browser to doomscroll social media and the news. Like I said, it's pervasive in my life.

I rank avoidance as one of the harder symptoms of anxiety to resolve, because the mind is so capable of hiding it. I'm putting into practice new mindful approaches to how I spend my time, in order to seek out, explore and understand my avoidant behaviors. One of my favorite avoidant strategies is constant busy-ness, so I'm optimistic that casting a compassionate eye on my time management will reveal underlying fears and let me face them with love and compassion.

Anxiety in the body

When I started anxiety recovery, I was completely numb to my body. I was aware of intense sensations, like sharp pain or orgasm, but numb to the every day average sensations that comprise the vast bulk. What helped me to start to get in touch with my body was emotional recovery through program, consistent meditation practice, and finally yoga. The timeline was incredibly slow, more like evolutionary punctuated equilibrium than steady progressive gains over time. The fits and starts have been meaningful, and revelatory over time, but I'm far from fully in tune with myself, I only just became aware of my consistent chest tension (thanks Claire Weekes!).

The first bodily sensations I noticed were tendonitis, as they were chronic and loud. I developed some tendonitis in my forearm (tennis elbow) after over-doing it at the gym, but I rank the real underlying cause to be my unhealthy relationship to work. Showing up at the computer with my hands and forearms taut with stress for hours every day for work, followed by extra hours of personal use, with breaks to use my phone.. yeah, repetitive strain is all but guaranteed.

Next was my back and shoulders. I began to notice my back when I started going for massages - the masseuses exclaimed shock at how knotted my back was. I was aware without their surprise, as their hands punished me and I called it quits when it was too much (quick comparison to today when a back massage is deeply relaxing and soothing). Yoga has helped me tremendously at unlocking the tension stored in my back and shoulders, and helping to keep that stress moving in and through me rather than accumulating.

Next was my breathing. This has come in fits and starts, at first I became aware during some meditations that I'd go through cycles of not breathing and then gasping for air. Learning and practicing paced breathing methods like resonance breathing (5 seconds in no hold 5 seconds out no hold repeat), 4-7-8 (4 seconds in hold 7 seconds 8 seconds out no hold repeat) and box breathing (equal 4 seconds each in, hold, out, hold) has helped me tremendously in this area. A breakthrough for me was that a good meditation does not require sitting with intense anxiety for long stretches; instead, I learned that calming is part of the practice, and sitting with guided resonance breathing for twenty to thirty minutes is superior to the same duration gasping for air.

One marker of my growth in this area came during my recent 5k training. I started training when on a no-tech week, so I ran without phone or music. I enjoyed the time with myself and my body enough that I kept at it through the entire training and race. It was great practice to check in with myself, to observe how my calves felt at the start, middle, and end of a race; to be fully present with my breathing; to notice how my body started to unwind itself and stretch out. Writing this now I am blushing as I recognize my perfectionism showing up; this is what it's like to be present with my body, to be aware of myself.

Anxiety as a positive feedback loop

The last topic I want to share on today is the stressor-fear-stressor cycle of anxiety and panic:

graph LR A[Stressor] --> B[Fear] B --> A

otherwise known as the first and second fears, or the two arrows in Buddhism. I find it meaningful that many different cultures came up with the same understanding of how fear works.

The basic process is as follows: some stressor triggers an automatic fear response within me, which causes some set of body and mind sensations, which generates more fear, cycling adding more fear until I'm in a state of panic and collapse. The wise or skillful approach is to catch myself, at any point in this cycle, and respond without fear. In the best case, this leads the cycle to fizzle out, as without more fear added the stress will naturally fade away (how hard it is to remember this in the moment!). Typically, the fear lingers or even continues to grow, but this does not reflect poorly on me, as the panic cycle is well worn and deeply grooved within me; it takes significant repetition and practice to train the mind to respond differently.

graph LR A[Stressor] --> B[Fear] B --> C[Acceptance] C --> D[Resolution]

This is an example of a positive feedback loop, which despite the name is not so positive when experienced as a panic attack. The classic example of this process is two telephones on call together held next to each other, where the sound from one is amplified by the other, then received back and amplified further, cycling and amplifying more and more. It may also be thought of as an interest bearing account, where the interest is reinvested as capital, leading to more interest, leading to more capital, and so on.

Two pillars of recovery

Understanding the feedback loop dynamic completes the picture of my systems thinking of anxiety recovery, which I define with two main features. The first is the stored anxiety, pain, grief, trauma, etc., that inflames reactions towards the extreme. Emotional recovery, in the form of 12 step, trauma informed therapy, etc., thaws and processes these stored emotions to reduce the kindling for future emotional eruptions. The second feature is reducing the amount of new emotions bottled up. This means regular emotional processing of current events through journaling, intimate conversations, therapy, somatic work, etc. It also means interjecting new behaviors into the fear cycle by reacting differently in the moment, for example through Claire Weekes' face/accept/float/let time pass method.

Put together, this leads to a state of baseline calm and less reactivity. In other words, nerves that have adapted to be calm and return to calm more readily. It takes enormous effort to do this work and make these changes, but I can't see any other way to live at this point.

Good List Week 14

2026-07-05T00:00:00-04:00

New York Historical

For my artist date this week, I decided to go to a museum. I first went to the library to pick up a museum pass; from their available selection, I chose the New York Historical. I had never been, and it was a good excuse to get into the city, plus I knew that the museum has the epic Course of Empire series by Thomas Cole.

I had an uneventful time getting into the city, even with the heat wave. I met my sister for coffee. Then I ventured off to the museum, located just beside the Natural History Museum. I figured that the Historical would be small, judging from the outside, but it felt large once inside: three floors of exhibits, plus a mini movie theater with repeating documentaries on the hour. I noticed they host free tours, and I was just a little early so I explored on my own for a bit, chancing upon the new democracy exhibit which includes Course of Empire.

Okay maybe I am a little obsessed with this series. I also do admit to having prints of Voyage of Life hanging on my walls. What can I say, I love epics, and anything that gets to the cyclical nature of existence. Face to face with the series at last, I was impressed by the scale. It's one feature that computer screens can't replicate: the big-ness of the paintings, the relative scales between them (Consummation being larger than the other four). The level of detail is just incredible, with small stories told in the grander scheme: in the Arcadian, a solitary mathematician draws in the dirt, while a shepherd tends his sheep.

I got onto the tour, which even with more than an hour skipped over multiple exhibits, focusing on some of the key items, mostly of primary historical and local interest: Washington's inauguration chair, Washington's army cot, Thomas Nast's snake vase. I was impressed to hear of the latter that Thomas Nast was offered a half million dollar bribe by Boss Tweed to quit making cartoons of him; Nast turned the money down and saw Tweed locked up in jail. That kind of moral virtue towards corruption in politics and business is sorely lacking today. By chance I just read Sarah Wynn Williams' Careless People, and I am reminded of her own courage in sharing about Facebook's evil-doings at significant financial risk to herself (borne out in the punishments since doled out!).

My inner teen is excited by stories of justice and courage like these. It's long been an inner drive of mine to call out injustice, and it has been a frequent source of conflict for myself at work, albeit the retaliation on a significantly smaller scale than Nast or Williams experienced. I'm also reminded of the line from The Republic: justice is determined by those with the power to enforce it. Stories of courageous truth tellers remind me that the powerful are not invulnerable.

5k

On July 4^th, I woke up at 5am to run a 5k race! It's not my first 5k, I ran one in college a decade ago. But I have not run since. I injured my foot some years back and stayed away from high-impact cardio. Recently, I've been feeling much better physically, and with a year of yoga under my belt, I felt ready to take on a new physical challenge. I trained on a modified couch-to-5k schedule, slowly ramping up mileage over the course of five weeks. I ran in rain, cold, heat, and wind; some days were quiet and peaceful, other days the wood chippers blared in my ears as if I was running from some mechanical nightmare.

All in all I kept at it with my training schedule, and with the generous time allotted I ran 3 practice 5ks before the race. Talking with a friend, I set myself the final goal of finishing, healthy. My secondary goal was a 45 minute finish; my training times were in the 46-48 minute range so this felt do-able.

Day of the race, my girlfriend asked me how I was feeling, and I was surprised to find myself saying that I felt totally indifferent. How could that be?! I had spent a month training 3x per week. I realized this was a fear response; avoidance. Growing up, I was expected to be perfect. This started with school grades: report cards were practically judgments from God, as anything less than straight A's meant intense shaming and guilting ("Mr A Minus" was my father's nickname for me after one imperfect report card). Beyond school, I ran track for a semester in high school, where I was mostly motivated by chasing girls and copying my older brother. My father showed up for one track meet before telling me, "Why would I come to watch you lose?" and never showing up again (including the regional track meet where, in the middle of the race, my shoes came off and I finished in just my socks - my coach was quite proud that I didn't give up).

Needless to say, I had my share of baggage around running, so shutting down before the race was a defense mechanism to avoid past pains. I comforted my inner teenager who felt so hurt, and I embraced the race as best I could, knowing that like most teens, he'll come around eventually. That's how I found myself Saturday morning, eating my hardboiled eggs and heading to the race. The weather reports were iffy ahead of time; that morning it was a nice and dry 80 degrees at 8am. I was joined on the race by a friend, and we paced each other the entire time. I felt great running, the energy from the other runners and supporters was infectious. It showed in my performance: I averaged 12 minute miles and finished in 37 minutes flat! I was shocked, I beat my wildest expectations.

My automatically-moving-goalposts ramped up that evening, and I found myself looking up 10k races and training schedules. After some calm, I decided to do some exploratory training towards a 10k, with the goal of running a Halloween race (there's a race in Sleepy Hollow - how spooky!). If I like the training, and find it fits well into my structure, then I'll keep at it. If not, I'm thrilled that I set and achieved this personal goal.

Side note: the Good List Visualizer has been updated to include these new entries.

Recovery: Codependence

2026-06-30T00:00:00-04:00

I wasn't sure what to write about this week.. meditation? supplements? attachment? But then I was reminded, the best topic to share on at any given time is the one dearest to my heart at that moment. Today that is most definitely codependence.

What is codependence? Many people have their own definitions, mine is inspired by Melody Beattie, my work in 12 step, and my own experiences. The main component is dependence: when I am no longer able to operate independently of another person. In the past this has ranged from complete obsession to more subtle, covert addiction. Within this structure, typical patterns play out: the drama triangle from victim to victimizer, the date-breakup-date cycle, explosive highs and destructive lows.

I was made aware of my codependent behaviors when I read Codependent No More by Melody Beattie. This book was a mirror into my spirit, it was as if Melody knew me better than I knew myself. Most painful and challenging for me was the realization that I was not a victim in my past relationships, as I had believed. After all, I wasn't the one exploding, picking fights, initiating breakups, threatening, testing, etc., I was the victim of such behaviors by my exes. While I wasn't engaged in such overt behavior, I was consumed by covert behavior: passive aggressiveness, withdrawal, withholding affection, conflict avoidance, lack of boundaries, lack of authenticity, etc.

Seeing my part in the drama was painful. Even more painful was the realization that my behavior was not disinterested: I got payoff by staying in codependent relationships. By dating emotionally unavailable women, I could focus totally on chasing and fixing and rescuing, and remain completely numb to my own needs and emotions. In recovery, I've learned to break this pattern, and focus on myself, which has been an incredibly painful and challenging procedure as I have decades of grief and shame to process. That was my payoff: keeping my pain exiled, out of sight and out of mind.

The other payoff I got was the explosive highs. Being so numb to myself, so disconnected and dissociated, it took extreme activation to pierce through and impact me. The whiplash from breaking up the day before to expressing mad love the next gave me the high I desperately craved, the high strong enough to temporarily abate the pain and overwhelm me with affection.

Thankfully it's no mystery where this behavior stems from. As Claire Weekes' puts it, I was trained for it. My mother, an alcoholic, was obsessed with me as a small child. When she lost custody of me, I experienced profound abandonment: overnight I lost my mother, brother and sister. I was then raised by my stepmother, also obsessed with me, but prone to violent and random outbursts of rage. My father, for his part, abandoned me to the care of my stepmother, while himself ranging emotionally from mockery to rage to icy indifference.

The template for unhealthy attachment style was programmed into me. Anxious-avoidant, craving the safe and loving affection I never received, while terrified of it; like Tantalus, doomed to reach for something always just out of range. So I spent my teenage and adult years going from one codependent relationship to another. If my partner wasn't buying it, if my partner brought a healthier attachment style to the table, I ran away.

Now, with the help of recovery, I feel I am in a much healthier attachment style, though far, far, far from perfect. Just recently I've been feeling the codependent urges: if my girlfriend won't pick a fight, then dammit I'll pick one! I still crave the highs and lows, the overwhelming excitement and terrifying abandonment. When the urges arise, I have tools available to me now: call a fellow, go to a meeting, journal, check in with myself. Usually it's a sign that I'm overextending myself for others, and not taking care of myself.

For instance, I've been helping my girlfriend with adulting recently, and I noticed a feeling of resentment creeping up. Where's my payday? When do I get to be the center of attention? Who's taking care of me? As hard as it is to share these thoughts and feelings, it's critical for me to do so, because it's when I keep them locked up inside that they build and build and then explode. The thoughts are also intimately revealing: I am feeling like I'm doing too much for others.

Today I can validate that feeling for myself, and recognize that no one has been forcing me to help them, I've been taking this action of my own accord, and I can re-evaluate my involvement with others at any time. Right now, that looks like re-orienting toward myself: focusing on my wants and needs this week, and when someone asks for help (or I imagine they're asking for help, how often I confuse the two), I can turn them over to God and trust that they can take care of themself, just as I am doing for myself.

Good List Week 13

2026-06-29T00:00:00-04:00

It's the end of June; summer is in full swing (I've got to get to the beach still!). There's a heatwave this week, I'm hunkering down with the A/C on, hoping that there's a break in the heat for my 5k run this coming weekend. The past week was slower, but felt faster - I notice that my inner energizer bunny is ramping up. I'm planning to be intentional this week about slowing down in volume and in individual efforts.

What about last week?

Obsession

I had a premonition that I'd relate to this movie, and I sure did. I thankfully went with some friends, and was blessed with a good crowd in the theater: there were as many laughs as gasps and moans. To say that I relate to Bear, well, it's painful to admit to myself and the world but it's oh so true. My relationship with my stepmother growing up, as well as my romantic relationships, closely modeled Bear and Nikki: I was unable, unwilling, or did not want to break off unhealthy relationships. When I hear others criticize Bear as the bad guy in the movie, well, they're not wrong. It was selfish of me to stay in unhealthy relationships, and I played a major part in the dysfunction.

The movie itself was fantastic, it seems that we're in an era of quality low budget horror movies. That hints to me that as a society we're seeking our own terror and pain reflected back to us on the screen.

Cold Plunge

I had heard about cold plunges for years; I had my first exposure to Wim Hof some years ago. I dutifully tried cold showers, but once winter rolled around I couldn't stand the intense chill. I've recently been getting back into the breathing exercise and cold showers after reading Breath. I wanted to fulfill the full journey with a true cold plunge, though, so I recruited my girlfriend and we booked a session at a contrast studio. It was way too expensive, given that what's provided is a hot tub and a cold tub, but options were limited and we were excited to do it so we figured let's splurge.

The studio is located inside an upscale gym, so we got to shower and hit the sauna before dipping in. As my focus was on anxiety recovery through activation and relaxation, we spent the majority of our time in the hot tub, soaking in the warmth. The basic structure was a few minutes in the hot tub, followed by a brief cold dip, and repeat. The tub was 50 degrees fahrenheit, which I way underestimated as "not that cold" - my bones were aching within a few seconds in the water. I faced my fear of the cold and submerged myself over and over, noticing that I became anticipatory towards it, my heart pounding. I managed to stay in the cold about ten to twenty seconds at a time, gradually extending the time, and ultimately submerging my head helped.

I noticed pretty clearly my thought patterns and automatic/body reactions. As I said, my heart was pounding in anticipation of entering the cold. Once in the water, the fear signals were overwhelming, which led to a panicked mind. While I was exercising my nervous system, I became obsessed with elongating each cold dip: I had to stay in longer than my girlfriend, certainly, and I should really be staying in the recommended full three minutes! I wasn't able to override this thought pattern, but I was able to be aware of it and not judge myself, as perfectionism is how I was raised and continues to haunt me.

Empty Cloud

I have been attending meditation sessions at Empty Cloud for over three years. I feel so grateful to have access to an incredible space so close to me in West Orange. The grounds are an old Christian monastery, evident in the architecture, crosses and old wall paintings. Empty Cloud is run by monks who live there full time (though they travel quite a lot). The monastery offers retreats for laypeople, and weekly programs. I usually attend on Saturdays for the mindful evening program, typically structured as 30 minutes of meditation, 30 minutes of dharma talk/lecture, and 30 minutes of q&a with the monks. Each monk who leads the program has their own twist, and it's been my pleasure to learn from so many different monks over the years.

In particular I've enjoyed the different meditation practices: body scan, loving kindness, elements, emptiness, instruction-less. I always get something out of the talks, and my favorites are always on the basics, which I feel are practical for my every day life: witnessing, making small changes, accepting, letting go.

Side note: the Good List Visualizer has been updated to include these new entries.

Recovery: Meetings

2026-06-23T00:00:00-04:00

This is the first post in a new weekly series on recovery. Growing up in a dysfunctional family, I was ill prepared to meet life on life's terms. I've experienced depression and anxiety most of my life. I have blown up my life multiple times, dropped out of college, lost all my friends, abused drugs, burned out, struggled with my health, went no contact with my parents, quit jobs. I've been addicted to video games, social media, pornography, work, drugs, dating.

I've been through a lot. Putting it mildly. Speaking at rehabs for the past year has shown me that I have a story to tell, that my experience, strength and hope can be of service to others who are struggling, many who struggle without knowing what it is that ails them. I spent most of my life in that place, blind to the patterns I was repeating, deaf to the origins and causes.

I want to make this series to honor my journey. To relieve myself of the shame and guilt I carry. I want to explore my past, including foundational and key events. I want to talk about how I am healing. I want to be able to look back at myself in a decade and smile in gratitude for having put in the work to realize a more authentic life for myself.

I'll structure each week on a specific topic or aspect of my recovery. I thought about organizing this chronologically, or thematically, but I think keeping it flexible and letting my story unfold naturally is the right course. To start, then, I have to talk about meetings.

Meetings

I didn't walk into my first meeting in 28. As a child I grew up attending Alcoholics Anonymous meetings with my father. The impressions that remain are styrofoam cups, hospital meeting rooms, and the serenity prayer. My connection to 12 step pre-dates even this. My parents met at AA, had a brief affair, and produced me. It feels full circle to now attend meetings myself as an adult.

I first heard of Adult Children of Alcoholics as a teenager. My stepmother ¹ suggested to me that I attend meetings, considering that my birth mother was an alcoholic, my father a recovering alcoholic, and she herself a recovering alcoholic. The irony at the time was that I had no means of independent transportation, seeing as we lived in a rural area without public transit. So, as she did not take me to a meeting, I did not go.

Fast forward a dozen years, and I am dating someone with whom the relationship proved to be wildly unstable. We had three breakups in the span of six months. After a breakup, I decided to order a copy of a book that I'd seen on her bookshelf: Codependent No More by Melody Beattie. This book blew me away, it described my exact patterns of behavior in such clear terms that it pierced my denial. I had actually picked up the book in order to understand why my ex was so crazy. Instead it showed how crazy I was, more a mirror into myself than a microscope into another.

In the book, Melody suggests attending ACoA meetings. I was once again faced with the dilemma of trying to argue myself out of attending. As Groucho Marx said, I don't want to be part of any group that would have me as a member. The name of the group was unarguable, though; I had three alcoholic parents, I am the adult child of them, ergo, I might as well see what it's all about.

I looked up the meeting list on adultchildren.org, found a nearby meeting, and reached out to the contact person: the meeting says closed, does that mean it's closed to new attendees? Nope, open/closed refers to whether non-identifying people are allowed to attend, i.e. friends, partners, counselors, etc. With much anxiety, I went to my first meeting. I opened my mouth and shared. I came back the next week. And I've been attending every week since, for more than three years now.

Fellowship

Attending meetings is the bedrock of my recovery. It is the medicine that heals the root of my disease. Weekly attendance, barring travel or sickness, is non-negotiable. Not that I'd want to negotiate anyway, I just love going to meetings. I've met some of the kindest people in the rooms. I've made so many close, trusted friends, that my life feels full of connection, warmth and love now. Contrast that with the past, when I could count my friends on one hand, and couldn't tell you what emotional availability meant.

Gabor Mate says that authenticity and attachment are the key to recovery. I believe that meetings provide both. In shares, I can express myself, however I like, whatever I like, as much myself as I am able to muster in the moment. With stable meetings, and friendships with fellows, I settle into healthy attachment patterns. I'm able to be myself and be accepted for that. This is a special opportunity, and I treasure it.

I've worked the steps, I've worked with sponsors, I've chaired meetings and held service positions and carry the message. Recovery is a process, not a finished product, something that Julia Cameron is reminding me this week. So as much as this sounds like a lot to share, it took a long time to shape. At first, I ran out of the door when the meeting ended, not talking to anyone. After six months, I got my first sponsor, and started the steps. My sponsor encouraged me to reach out to other fellows, text and call. I started facing my fear and connecting with others. This slowly blossomed into friendships that lasted. I challenged myself to contribute, to take on responsibilities at meetings, to show up to events. It wasn't comfortable, and it occasionally still isn't, but I know the program works on me as I work on it.

After a year in ACoA, I started attending an additional fellowship based on relationships and sexuality. This was a missing puzzle piece for me. I had been trying to date again, but my old patterns kept emerging: chasing the wrong people, staying in bad situations, and ignoring red flags. I was able to take a deep look at my dating and relationship behaviors in this fellowship, and it's paid off tremendously: I've been dating my current girlfriend for about ten months. It's been the most sober relationship to date, without the highs and the lows, the fights and break-ups and make-ups, the walking on eggshells, the lack of authenticity or healthy attachment. That's not to say it's been easy, in fact sober dating is even more challenging than unhealthy dating, in part because it feels so uncomfortable, as I'm doing something that was never modeled for me by my parents.

Recovery

My life is far from perfect today. I am taking a break from work as I heal from burnout. I lack the markers that I associate with success at this age: a house, a family, a career. What I do have is the capability of tapping into inner peace. And I see now that if I had chased those markers, I'd never have found this peace. I'd have been chronically dissatisfied, just like my parents.

This peace has been hard won. It's taken years of meetings, fellowship, therapy, meditation, journaling, yoga, etc., etc. At one point I estimated my investment in recovery at more than 15 hours per week. The payoff is real, and beyond what I could have imagined - when I first joined, I really only wanted to get back with my ex girlfriend. Now I am opening to my inner world, to myself, rediscovering old dreams and discovering new ones.

12 Step is the bedrock of my recovery.

Terminology note: I was raised jointly (but separately) by my father and his ex wife, who we continued to refer to as my stepmother after their divorce. ↩

Good List Week 12

2026-06-22T00:00:00-04:00

Wow, it's been 3 months of good lists, and I am feeling accomplished for maintaining such consistency in this project. I also feel that the level of adventure is increasing over time, no small thanks to working the Artist's Way throughout.

Mount Tammany

Having read about the Delaware Water Gap in A Walk in the Woods, I beat myself up for having never visited. This week I remedied the matter by hiking Mount Tammany on the red and blue trails. The day started with panic: I became convinced that going in the middle of the week, I'd be the only person on the trail, and therefore guaranteed to be eaten by a vicious bear. I accepted the fear, I reached out to a friend who hikes a lot, and with reassurance in hand, I ventured out west.

I had been warned the hike was steep, but I had to experience it to understand: the climb up is tough, it's about 1200 feet gain over a mile. I practiced taking breaks, extra breaks, breaks every couple hundred feet. I felt a little like the tortoise in the old story, as I leap-frogged other hikers who pushed through, ultimately beating them to the top.

The views were spectacular, and I faced even more fear by winding down the rock ledge at the overlook. I had packed a sandwich and coconut water, which tasted all the more delicious with mountain climbing accomplishment for dressing. After some rest, I took the trail back down, which was in some ways more challenging, as the going was rocky the whole way down - I rolled my ankle once, thankfully no more damage than a little soreness the rest of the day. I paused at the stream and falls near the end of the trail, and plopped back into my car and happily drove home - hitting a rainstorm no more than five minutes away! The day really lucked out: clear weather in the 70s, plenty of folks on the trail, no bears (the worst was grasshoppers jumping on me, haha).

With Mount Tammany hiked, I can mark the Appalachians / Water Gap on my map.

Drumming

My girlfriend had suggested we visit a recording studio with drum rentals - I like the percussion of playing drums, and had never in my life sat down to play a drum set! She's an experienced drummer, so with her as instructor and me as mostly attentive student, we went to Angry Papa studios in Wallington, NJ. For $20/hr, we got to jam out to our heart's contents, me trying to get the rhythm of syncing the kick drum with drum sticks, and Maria showing me up with her skills.

I had a blast, the two hours flew by, I don't think I have any natural talent on the drums, but I am learning that I do like playing music, and can follow a basic beat on an instrument. I next want to get an intro guitar lesson from a friend, as part of a musical tour, before settling down and investing more serious effort into learning one instrument well.

Catskills

My friend moved to Sullivan County in the Catskills about a year ago, and I dragged my feet on visiting until this past weekend - and how silly do I feel! The area is stunningly beautiful, covered in trees and filled with exciting wildlife. It took about 90 minutes to drive up, a peaceful and easy drive, punctuated by the loss of NPR on 93.9 and poor reception for the Catskills public radio on 90.5. We started at Mamakating Environmental Education Center, or MEEC for short, which is a charming building nestled into Basha Kill Wildlife Refuge. There were binoculars positioned to view woodpeckers nesting in a nearby tree, I was lucky enough to see the baby poking its head out. A local artist is working on a residency project, stitching a mandala with colored layers from the birds observed in the area.

We then went to Basha Kill proper, an incredible sight, the valley carved out by glaciers some millennia ago. Onward to lunch at Brew in Rock Hill, where I feasted on delicious bbq cauliflower "wings" and an impressive veggie burger. Then to the next adventure, hiking at the Upper Delaware Scenic and Recreational River park, on the lookout for bald eagles. My friend is spectacularly lucky to have bald eagles nesting outside his apartment! We spotted the baby in the nest, but hoped to find an adult on the Delaware. We spotted a juvenile, maybe one or two years old, feathers not fully developed but white underbelly clear. We soaked our shoes in the water tramping about trying to get a better view.

There was a troupe of kayakers on the river, and it made me dream of a weekend long kayak and camping trip down the river. Then I thought, how would my back feel sitting in a kayak all day? Well, maybe it'll feel just fine :)

Side note: the Good List Visualizer has been updated to include these new entries.

Good List Week 11

2026-06-14T00:00:00-04:00

It was my birthday this week, I celebrated with loving friends and family. I turned 32, thankfully still in my early thirties for the time being (ie still in the "plenty of time to figure it all out!" period of my life, at least I tell myself..). This was my best birthday in so many years, as I was able to open myself up and feel the good in my life.

I Love Boosters

I chanced hearing about this movie on the radio, Boots Riley gave an interview about it, helpfully explaining that boosters is a term for shoplifters who resell to the community at discounted prices (in other words, the facebook marketplace resellers of laundry detergent). I had previously seen Sorry to Bother You, so I was excited, but didn't have much in the way of expectations going in. And I am glad I didn't, because Boosters blew me away: it is such a fun, whacky, wild adventure through class, fashion, and labor organizing. I just adored the dialectical materialism raygun, heighten the contradictions indeed, it took me way back to my days as a youthful organizer.

A Walk in the Woods

Another media hit this week: A Walk in the Woods by Bill Bryson. I grabbed this and The Wild Parrots of Telegraph Hill at Montclair Book Center, the premiere used book store in Montclair. I was disappointed to see that they're moving away from used books to higher price/margin new books at an eye watering $18 minimum price, but I scored these two books for less than $10 each.

It's a trend for me now that I'm reading so many outdoors oriented books: How to Do Nothing, Into Thin Air, and now A Walk in the Woods. It also coincides with my withdrawal from technology and into slowing down and presence. I can't say I don't still get easily spooked and startled outside, whether from a bee or a deer. But I'm in good company with Bryson, who journeyed the Appalachian Trail (AT for short) in the late 90s. I appreciate his sense of humor, as he describes the typical American as someone more likely to drive than walk to their own mailbox, as someone who's experience with nature amounts to little more than driving into the parking lot of a national park, taking a picture, and getting back in the car. I appreciate the irony that as much as Bryson disdains bland, monotonous consumer culture replicating across the country, nothing pleases him more than a hamburger and cheap motel room after just a few days on the trail.

His walk through history and geography was of particular interest to me, as a New Jersey denizen. I did not realize that the AT is less than an hour drive from me; I also cop to having never visited the Delaware Water Gap despite living so close for so long (I do plan to address this promptly). Bryson's description of nature on the edge, of the total destruction of so many species at record speed (e.g. the American chestnut tree just about eradicated in a few decades, having a population in the billions prior), left an impact on me, in the vein of How to Do Nothing: I feel disconnected from the earth, from my environment; driving in a car through New Jersey's sprawl feels distinctly uncomfortable now, as do the kept, clean and polished spaces where I spend my time.

Boonton Coffee Open Mic Night

For my artist date this past week, I went to Boonton Coffee's open mic night, where local musicians and performers play a couple songs or practice their art for the crowd. Boonton Coffee is one of my favorite cafes in the area, a short hop for me on 280/287. Their coffee is high quality, their pastries and toasts tasty, and the space is big and comfortable, with plenty of seating.

I've been attending open mic nights here for the past couple years, as I'm consistently impressed with the quality of the performers, ranging from hippie dads with an acoustic guitar to up and coming young singers. I also like that the music tends to be more folksy and original, with most musicians opting to play their own music. The star of this most recent show was an eclectic performance by a young artist who danced and sang to what I can only describe as five minutes of scrolling through tiktok, short segments of random songs and audio clips. That she was painted in clown make up and dressed like Chappelle Roan fit the bill.

After attending so many sessions, I do start to wonder about performing myself, though I have nothing even close to presentable (nor any talent developed enough anyway). Still, with a missed karaoke opportunity this past weekend, my inner teen is expressing the desire to get on stage, to be seen and heard and join in on the fun.

Old Photos

I ventured back into the forbidden lands and reactivated my facebook this week. In working with my inner teen, I realized that I don't have any photos of myself as a teenager. But of course facebook does, and I get enough login code emails to be reminded of that. It took a lot of courage, and a couple drinks, but I logged on and revisited all the old memories (curated selection above).

So many things stood out to me. Was photo quality really so bad back then? I wonder if HD (now 4k)-by-default plays a part in technology vampirizing our attention. The way we used to post full photo albums, I remember, with a digital camera and just bulk uploading fifty or a hundred photos at once. No taking the perfect picture, cropping, editing, etc etc, just dumping what you had and moving on. And that we'd look at albums of other people! Now it's a chore to look at one photo sent from a friend. I was also amazed at how innocent the internet was back just ten to fifteen years ago: adding random people as friends, posting on public walls, status updates ranging from banal to banal.

What hit me most was seeing myself as a teenager again. I think that was my block these past years against doing so. I've long carried so much guilt and shame about my teenage years, feeling like a failure. In my healing, I started with my the basics with my inner teen, then worked with my inner child's wounds, and now am back working with the deep wounds of my inner teen. Some pictures are shocking: I can't believe how much pain is written into my face, my posture. Sure, there's gawkiness and angst, but then there's wounds, layered and layered.

In my teenage self, I see so much longing, longing for relief, for acceptance, for guidance. As my teenage years progressed, unfortunately, my home life deteriorated until I was homeless, sleeping on a friend's couch. I had to shut down so much of myself to survive. I see now my bravery, ingenuity, creativity, resourcefulness in my inner teen. Rebuilding my relationship with my inner family will take time, but I recognize the immense progress in this action, and feel the release.

Side note: the Good List Visualizer has been updated to include these new entries.

Good List Week 10

2026-06-08T00:00:00-04:00

Birthday hike and swim

Last year, I had planned to host friends at a nearby park for an outdoor picnic on my birthday. Due to rain, the party moved indoors to my apartment, where we crowded around the living room and played pictionary. This year, I wanted to up the ante; I jokingly said I wanted a birthday that folks would remember. I think I accomplished that with a hike and swim at Harriman State Park. We roughly followed this Pine Meadow Lake trail, making an extra stop at Lake Wanoksink first, before making camp at Pine Meadow Lake. When we arrived, we were warned of a giant snapping turtle in the water; some folks (me included) stayed at the water's edge, while others merrily swam out. Some folks napped and sunned, while others munched away at their trail mix and snacks.

By the time we got back, and made it to the restaurant (Characters in Sloatsburg - great stopover for traditional American fare), we were all exhausted, thirsty and starving. Massive plates of food were consumed, or attempted to be consumed. The drive home was peaceful and quiet on the Jersey Turnpike, as we let the accomplishment sink in and our bodies rest.

I personally had a positive but challenging time. Last year as this year, I found myself focused on everyone but myself, are they having a good time, do they have enough food and water, is everyone comfortable and socializing. All the same, progress is progress, and my shoulders were elevated but not nearly as high this time as last. I found myself also once again overwhelmed by how many people showed up, there were 14 of us all together, just an incredible display of love and connection that I cherish.

While some of us doubted, the turtle turned out to be very real, and very large.

Comedy show

I love live performances and theater, and I love comedy, so when my stand-up comic friend invited me to her show I jumped at the chance. The crowd reminded me of college: loud, boisterous friends leaning over to greet each other; cackling laughs from select audience; my seat neighbor finishing off five cans of beer. The comics were fantastic, all local, really hyperlocal, as the headliner grew up nearby and shared his story of buying condoms for the first time in the same place that he now performed (Lackawanna Plaza used to host a Pathmark supermarket).

Ramapo artist date

For my artist's date this week, I listened to my inner teen and leaned into "expansiveness" in time and space. I realized that I orient my time around responsibilities, and try to compartmentalize play. My inner teen can't settle into a fixed hour of time set aside; he craves long drives to far off places, whole afternoons and evenings without a care in sight. I recalled that as a teenager I liked to write sappy poetry - as cringe as it is now, I'm going to link to my teenage poetry blog. I had a desire to revisit Ramapo Valley County Reservation, particularly the waterfall. I stopped at the store and bought myself a little black notebook (yes, an homage to Pink Floyd).

I then took the long drive to Mahwah, listening to The Wall. I hiked up to MacMillan Reservoir, a significantly steeper venture than I had recalled. I was surprised by a frog, and watched a dog playfully splash about in the water. I then hiked down, and stopped where I could sit on a rock in the rushing brook. I breathed in the fresh air, dancing light, and running water. I wrote poetry. Then I hiked down to the waterfall, where I sat in meditative quiet, admiring all the ways the water effortlessly snakes down the rocks.

Side note: the Good List Visualizer has been updated to include these new entries.

Artist's Way Week 4

2026-06-02T00:00:00-04:00

I started the Artist's Way a month ago, seeking to get in touch with my inner child/inner artist, to learn to play and be more spontaneous. So far, so good, as I've started to open myself up to more of the good in the world. I don't buy into the "benevolent god" idea of a Higher Power, and that's long been a point of some contention for me in 12 Step. This book is helping me to better understand my own idea about good and Higher Power: there is good and bad in the world, in my closed off / victimized / solipsistic / narcissistic / ego-driven state I can't see nor access the good, but as I practice opening up / embracing the discomfort I see there is good, or at least opportunity for good in the world. As Julia says (we are on a first name basis, I don't think she'd mind), I still have to do the footwork, it's just that, as a monk shared last weekend, rather than walking in a circle blind to repeating the same patterns, I'm starting to notice and have the courage to walk new, unknown, uncertain paths.

Speaking of discomfort, I did not know to expect the week 4 media deprivation challenge. However, it came about with some serendipity: all of my tv shows ended their seasons the week before (SNL, 60 Minutes, The Boys). I therefore entered the week with courage and slightly less FOMO.

As the book was written in the 90s, I sought out others' perspectives on a modern take on reading deprivation. It seemed the idea remained the same, just with updated tactics: stop taking in others' ideas, giving my own inner voice room to speak. Practically, that meant:

No:

TV, movies, video games (I unplugged my TV for the week)
Reading books, checking the news, browsing social media (I put a 0 minute app timer on most apps so they'd be inaccessible by default)
Music, radio, podcasts
AI tools

Minimal:

Computer, acceptable usage were checking emails twice a day, medical information, paying bills
Texting, instead preferring calls and in person meetups
Phone, acceptable usage including maps, phone calls, planning events

Yes:

Solitaire
Cleaning my apartment
Exercising
Writing
Meditation
Walks
Coffee with friends
Quiet time

I am proud to say that I survived the full week, seven full days (Sunday midnight to Sunday midnight) following these rules. From a small sample size, I got the vibe that others generally did not take this as seriously as I did, whether skipping, or limiting instead of eliminating media consumption. That's not to say that some don't take it even more seriously, one person's experience included no non-critical texting nor reading (not even restaurant menus!). I am satisfied with my ruleset, and the proof for me in my experience is the widened perspective I gained.

I was most excited about dropping distractions when eating, on the toilet, before bed, and when tired. I've long tried to embrace the discomfort of just doing the activity itself, with myself, but could not make it more than a couple days before the old distractions returned. And I will readily admit, I experienced a lot of discomfort last week. I felt it most apparent in my body, specifically in my back, as it remained tense for most of the week. I took the opportunity to accept the discomfort, sit with it, help it to move with yoga, breathing exercises, cold exposure, running. Nothing eliminated the discomfort, and I was surprised to note after the first few days that part of me started to lean into the slower pace of stimulation, even as another part felt stressed.

It was helpful working with my therapist to see that my inner teen felt the tech deprivation specifically to be a punishment. Growing up, I was stuck at home in a rural area, so I leaned on technology (video games, social media) to escape a chaotic home life. Pulling back on tech last week, my inner teen felt that he was losing something, or having something taken away; this helped me to understand some of the tension, and start to think about how I could integrate technology in a way that affirms this part of my life, without becoming a compulsive distraction that curtails my current life.

A key insight of my experience was that chronic distraction doesn't alleviate anxiety, it actually reproduces it. That's to say, as I reduced stimulation, I also craved it less. Time started to flow differently, suddenly slower paced activities didn't seem like such a drag - for example, running without headphones (I noticed I was the only person doing so!) I felt the run go by significantly quicker than in the past when I suffered through each footfall music blaring in my ears. Long drives without radio or music went by quickly. My meditations were much better, I experienced dropping into a quiet and calm state much quicker and easier than usual. I also found myself craving more time on the cushion, longer sessions, trying out seldom used approaches like progressive muscle relaxation at night.

I also noticed my cognition and memory improved, quite drastically: suddenly I could remember all of the events of the previous day. Over time, I felt my inner voice start to speak up, and it was like greeting an old friend who had slowly faded away some long time ago. I am gifted with a powerful and creative mind, but for some time it's been relegated to passive observer instead of active participant. It's special to get back in touch with myself in this way.

My journey to a full tech detox did not start last week, but goes back years. I knew that I had an unhealthy relationship, and started tapering off maybe three or so years ago, in successive year-long rounds. The first round of tapering was eliminating the "worst of the worst" of social media, video games and pornography usage. The next year, the "less bad" social media. And then I was reduced to mostly browsing the news and wikipedia. So the background for a full week detox paints a comprehensive picture of a multi year journey to radically reshape my relationship with technology.

Moving forward, my plan is to continue with tech-free mornings, meals, nights. I'll be flexible, I'm not going to deny myself a Saturday morning movie if I am really excited about it (that's the fun and play, as opposed to self-deprivation mindset). Even two days after ending my fast, I haven't watched tv nor turned on the radio in the car. I feel the slower pace working through me, fiber by fiber, and I'm curious to see what more time will reveal.

Good List Week 9

2026-06-01T00:00:00-04:00

Memorial Day

For Memorial Day, I co-organized a hike and barbeque with friends: about twenty people showed up, the grill was manned, food was shared, wiffle ball was played (as team captain I drafted to win, my competitive spirit got the best of my sportsmanship). With the weather uncertain, we were able to grab multiple tables and a grill without competition; we returned to Watchung Reservation for the second year.

Blood Drive

I invited a new friend to meet for coffee, and he mentioned that we'd have to plan around a blood drive he was participating in. Undeterred, I suggested I join him for the blood drive; it's been years since I donated blood. It felt good to be of service, I was thinking of the phrase, "to feel esteem, do esteemable acts." I did end up almost fainting, having forgotten that I get shaky with the blood loss; the kind attendants resurrected me and I left with a whole box of girl scout cookies (samoas!).

Pickle Ball

One symptom of my progress with the Artist's Way is trying new activities. My friend had been trying to plan a pickleball match with a larger group, so I suggested we get some cheap paddles and play ourselves in the meantime. A trip to Target and $20 later, I was in possession of a set of paddles. As we arrived at the court, the group playing walked off, serendipitous indeed. We ended up playing with another pair that showed up, and got our butts kicked - all in good fun, considering we had twenty minutes of experience prior to our first match!

Woodstock, NY

As I've been experimenting (playing?) with music and instruments, I've talked about getting some basic drums, either a drum pad or bongo drums. My girlfriend suggested we take a day trip to Woodstock, NY (not to be confused with the festival, though you wouldn't know by visiting the town), where there's a drum circle on Sunday evenings from 4-6pm. The town is really cute, there were so many clever store names in the area (e.g. Candlestock). We ate at the Garden Cafe, an all vegan restaurant, which was just delectable.

Finally, we joined the drum circle, they provide all the instruments and chairs, and I'd guess the crowd was about thirty or forty people total, all drumming along together. I liked it, but I realized that I struggled to really sink into the music, as the drum circle was more chaotic than ordered: there was no one beat that everyone played to, so it was more a joining of many different rhythms to form one song. In contrast, I feel I can sink into a single, steady rhythm that all are tuned to.

Side note: the Good List Visualizer has been updated to include these new entries.

Good List Week 8

2026-05-24T00:00:00-04:00

Kite flying

For my artist date this week, I bought a kite at the toy store and went to the local park to fly it. It is funny to me that I just read Julia Cameron share on synchronicity and serendipity. I first went to Target, assured by Claude that they'd have kites in stock. The store worker looked at me funny and suggested I order one on Amazon. Not to be deterred, I called a nearby toy store and they were stocked! The weather that day (Tuesday) chanced to be the last beautiful day for some time (it's rained every day since), at a gorgeous mid nineties with just the right breeze.

Flying the kite, I was taken back to memories from childhood, flying kites at the beach, and at a nearby park from my home. I could recall the string spinny handler thing rolling in my hands; the kite diving into the water, to be resurrected back into the sky.

Presby Iris Gardens

Driving through Montclair, I passed the Presby Iris Gardens, and noticed cars parked along the side and people walking among the flowers. With a tinge of spontaneity, I went back later that day, and stopped to smell the flowers, a mix of sweet, earthy, candy, bitter. I must've driven by this park dozens of times, plenty during in spring season, and yet have never visited this delight before.

Running

After reading Into Thin Air by Jon Krakauer, I (presumably in good company) became very interested in mountaineering. In order to manage my expectations and build up to Everest, I set a modest goal of running a 5k. With serendipity, there is a local race six weeks away, plenty of time to train. I went to my local thrift shop to pick up some running shorts, and they had the exact kind I was looking for, at a steep discount.

I have hit the track at Brookdale Park twice so far: 1 mile, 1.25 miles. I'm following a modified couch to 5k schedule, reformulated for a track (eg 1 lap run, 1 lap walk) rather than time (30s run, 60s walk).

It's taken me back to the year I ran track in high school. I also ran a 5k in college. I remember running a mile or so maybe six years back or so. I've long had a difficult relationship with running: I didn't feel comfortable in my body, the internal pressure to achieve was more harm than help. This time around, with about a year of yoga under my belt, and a much better relationship with myself, these training sessions have been, well, surprisingly easy.

Veggie points

This was a fun one, my girlfriend suggested that we log each unique vegetable that we eat for one week. The catch is that the veggies should not be processed in order to count (we made exceptions for minimally processed foods like peanut butter and hummus). The target was 30 unique veggies, which we both hit! As vegetarians it wasn't particularly challenging, and I'd say I didn't make any special effort to consume new veggies for the list.

Side note: the Good List Visualizer has been updated to include these new entries.

Meditation Misconceptions Part 2

2026-05-22T00:00:00-04:00

After two and a half years, I wanted to share some new insights into meditation.

Meditation is just sitting

Most of my past meditation practice was simple: just sit on the cushion. A Buddhist monk shared that it's common to practice gentle movement before, however, maybe walking meditation, or yoga. Walking was my first foray into pre-sitting movement, trying the monk's suggestion of balancing the time at 10 minutes each of walking, then sitting. The results were limited for me, I found it hard to concentrate during the walking portion, and not particularly relaxed once on the cushion.

This year, having been practicing yoga, I experimented with 5-10 minutes of yoga instead, and have found it effective. The basic idea is that I'm carrying a lot of stress and tension in my body, and without an outlet, sitting on the cushion will bring that stress and tension with me. For me, movements involving my hips, back, and shoulders are highly effective at easing that tension, such that once on the mat, I'm able to ease into the meditation much easier.

I think of this as calming before meditation.

Meditation is just watching the breath

This one caught me by surprise, after spending hours on the cushion watching myself breathe shallowly: it's not against the practice to guide my breath, especially when I'm in an elevated state of stress. This misconception relates to the above, in that meditation is allowed to be a calming experience. Not that sitting with stress and discomfort isn't part of the practice, but I think of it as bringing myself out of the overwhelm zone and back into the challenge zone.

Some easy techniques include 4-7-8 breathing (for calming) and box breathing (for regulation and focus).

I think of this as calming during meditation.

Meditation is torture

Related to the previous, I spent hours trying to meditate in a state of heightened anxiety bordering on panic. I'd get through to the end, up to 45 minutes, and not feel any better for it; sometimes I'd feel exhausted afterward. The simple truth of the matter is that meditation does not need to be torture. If I'm in a panicked state, then silent meditation may not be the right call; that's not failure, that's meeting myself where I'm at.

The story the monks like to tell is of the king's chef. The first chef makes the same meal every day, despite the king's protests, inclinations, and moods. This chef does not last long in the king's court. The next chef, however, has a keen eye, and closely monitors the king before, during, and after each meal. This chef cooks the right meal for the right moment, and the king is delighted.

I think of this as choosing the right tool. It may look like swapping meditation for another activity: yoga, walking, socializing. I notice that after these, I feel lighter and calmer.

Good List Week 7

2026-05-18T00:00:00-04:00

Turtleback Zoo

My inner child loves animals, but I've long been resistant to visiting the zoo: animals in pens and cages, isolated and away from their natural habitat. I recognized those thoughts as adult discomforts, and challenged myself to take my inner child to the zoo anyway. It went well, standing by the cows my inner child wanted to hug their furry bodies and befriend them. This zoo has a wide array of animals, though not all were exhibiting themselves: alligator, komodo dragon, snakes, penguins, birds of prey, hyena, flamingos, gibbons, giraffes (unseen included the lion and bear).

I remembered that I grew up near the Cohanzick Zoo, going there as a small child, and passing by often over the years, walking, biking, driving. Zoos have a deeper meaning for my inner child, and I am thankful for the courage to explore that meaning.

Mills Reservation

Another nearby spot that I've managed to miss in the several years I've lived in Essex County - Mills Reservation is a delightful wooded park with trails and an overlook looking over New Jersey and New York City. We had perfect spring weather, and when a friend invited me, I jumped at the chance to get outdoors in company. She's big into outdoors meditation, sitting in the park, listening to the breeze rustle the trees, the people, the dogs, the birds, I realized that outdoor meditation had been a goal of mine and by opening myself to saying "yes" I found myself exactly where I wanted to be.

Ridgewood

I'm a fan of the library system here in New Jersey, and the dozens of libraries part of the BCCLS system. I pasted the list of libraries into claude and was recommended to visit Ridgewood library, so I ventured out on the parkway north to the unvisited town of Ridgewood. It is charming, very similar to Montclair (in fact, many of the same restaurants haha), except with my main gripe resolved: a quaint park in the center of town, with benches and space to just sit and be. I opted for Korean for dinner, and was pleasantly surprised by the quality.

I spent more time in town than expected, so by the time I got to the library I was in a bit of a hurry. Still, I took time to absorb it, and found it to be a classic library: history displays, art displays, a seed sharing program, a reading room, and floors of rows and rows of books. I found the book I was seeking, Into Thin Air by Jon Krakauer - I had heard him on NPR last week - and said til next time to Ridgewood.

Side note: the Good List Visualizer has been updated to include these new entries.

Red Lentil Curry

2026-05-13T00:00:00-04:00

I've wanted to document my recipes somewhere for the longest time and I am finally ready to sit down and do so. I am a vegetarian and tend towards simpler one pot meals that I can batch prep (curries, chilis, etc). I'll spare the long intro and jump into the recipe.

Ingredients

Makes 4 servings

•1 russet potato, cubed
•1 small onion, diced
•1 red bell pepper, diced
•1 cup red lentils
•1 cup spinach (sub frozen kale)
•3 garlic cloves
•2 cups vegetable stock
•1 cup water
•1 medium carrot
•1 can coconut milk
•1 tbsp red chili powder
•1 tbsp curry powder
•1/2 tsp ginger
•Oil (olive or coconut)
•Salt and pepper to taste
•Yoghurt
•Naan, pita or sourdough bread

Steps

1.Heat oil in pot. Add garlic, onion, carrot. Cook for a few minutes.
2.Stir in spices. Cook for one minute.
3.Add lentils, potato, vegetable stock, and water. Bring to boil, let simmer for 10 minutes.
4.Stir in coconut milk and spinach. Generously add salt and pepper. Return to boil and simmer for 5 minutes.
5.Serve in bowl with heap of yoghurt and bread.

Notes

This is a flexible recipe: potato can be dropped, onion and peppers can be frozen (when cooking frozen veggies, the trick is to heat just the veggies with no oil yet, letting the water boil off before adding oil), spices can be pretty much any mix of traditional curry spices. Swapping the lentils for chickpeas would also work, maybe with less liquid volume.

Good List Visualized

2026-05-12T00:00:00-04:00

As I've been writing the good list series of posts, I've been interested in visualizing the activities on a map. I could've created a shared google maps list, but I wanted to do something a little more bespoke, specifically around the design. I also find the google maps lists to be cluttered; my default maps view is filled with pins.

So I spent an hour with claude code this morning converting the lists into usable shape and rendering on a beautiful watercolor inspired map from Stadia / Stamen:

I think it came out quite well. I find the visualization to add an extra dimension, well, quite literally of space.

Check out the live site here: Good List Visualizer.

Good List Week 6

2026-05-11T00:00:00-04:00

Montclair Art Museum

The Montclair Art Museum hosts a free first Thursdays event monthly, open in the evening. There's activities and crafts, live performances, and free guided tours.

There's currently an exhibit on the American landscape, including classic works by Hudson River School painters Thomas Cole and Thomas Moran.

Gathering Hay in the Salt Marshes by Martin Johnson Heade

Enter the Dragon

Montclair film occasionally puts on special screenings, this past week showing martial arts classic Enter the Dragon. There was a pre-show demonstration by a local martial arts group, providing some context to the movie; though the movie really stands on its own, with epic fight scenes and dialog.

Cloudflare Email Workers

2026-05-05T00:00:00-04:00

I have tried various todo / personal task management apps over the years, and I get all excited setting them up with categories and add-ons and integrations only to find that I don't even look at the app a month later. Blargh.

One of the methods that has stuck with me is emailing myself. It's not too surprising - an email inbox is pretty much a flexible todo list. So I wanted to try meeting myself where I'm at, by hooking into the email habit programmatically to build a personalized todo app.

Building a todo app is a fairly tedious crud exercise, but programmatic email access was new to me. For my domain, tmk.name, I use cloudflare to forward emails to my personal account. I was excited to discover that cloudflare supports connecting their worker product with an email trigger:

In my use case, I wanted to forward the email via webhook to my backend. Editing the worker code in the browser was suitable for such a small snippet of code:

JavaScript

export default {
  async email(message, env, ctx) {
    // message.raw is a ReadableStream, needs to be converted to a string
    // see https://developers.cloudflare.com/email-routing/email-workers/runtime-api/#forwardableemailmessage-definition
    const body = await new Response(message.raw).text();

    await fetch(env.WEBHOOK_URL, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "X-Webhook-Secret": env.WEBHOOK_SECRET
      },
      body: JSON.stringify({ from: message.from, to: message.to, raw: body})
    });
  }
}

The webhook url and secret are environment variables, configured in the worker settings. Cloudflare handily exposes the results of their security checks in the message.headers['authentication-results'], which makes verifying dkim and dmarc for the originating email easy ¹.

Update (May 8)

Cloudflare stopped sending the authentication-results header, breaking this approach. Instead, parse the headers from the email.

Et voilà - free programmatic email access via webhooks in a few short steps.

In my application, I chose to forward the headers to my backend to keep the worker code as simple as possible. ↩

Good List Week 5

2026-05-04T00:00:00-04:00

Contra dancing

Oh, what fun - wholesome smiling sweating moving dancing - I had at my first contra dance this weekend. I was invited by a friend, and because my girlfriend is a dancer, I suggested we go - nervous as I am, as someone with little dancing experience. I expected something stiff and ballroom like, something like I've seen in old English country dances in the movies.. all my expectations were cast aside, as I was thrown into the spinning and twirling.

Contra is a social event, incredibly welcoming to newcomers. Dancers partner up each dance, asking or being asked, in a friendly way. Each dance is composed of a series of moves, e.g. do-si-do and balance and swing, that repeat in a fluid motion up and down the dance line, changing matching partners every set. The structure exposes the dancer to interact with just about everyone else at the event.

For me, I really dig the structured dance. For one, it's a forcing function that exposed me to interact with just about everyone else, hands-on and face-to-face, quite the difference from the interpersonal distance I'm accustomed to, with strangers nonetheless. For another, the structure isn't total, with dancers improvising and playing, adding their own style to the mix. Finally the energy of a room of friendly people dancing to live music was just infectious; I didn't realize how exhausted I was until I walked out the door.

I've never seen so many smiles on so many faces for something so wholesome.

Good List Week 4

2026-04-27T00:00:00-04:00

Jersey Shore

Oh, the weather has just been unpredictable these past few weeks. From that brutal, brutal winter to a false summer of a few days, to now chilly, dreary spring. Well, I was fortunate enough to get to the beach on a warm day, and bask in the ocean.

I reminisced - I went to the beach as a child, but my memories are so fragmented, I couldn't recall any one trip clearly. Even without the recollection, I felt the waves crash deep within me, and found some small peace to take home.

Improv

Montclair film, aka the Clairidge, puts on a few shows just about every month, including an improv show. This was my third time going, and it was nice, clean fun for an evening. I've been exhausted lately, watching way more television than I'd like, so the opportunity to watch live community theatre was hard to pass up on.

It's also fun to engage, shout out suggestions for the performers. The audience volunteer this time was a hoot, an older lady who clammed up and refused to share any definitive details about her life: how many kids does she have? maybe 3, maybe 4; what was her career? better not to say! It was hilarious.

Card Games

Nothing like a deck of cards to bring people together. I'm fairly activity oriented (as opposed to lounge oriented), so the chance to play some card games this weekend with friends was welcome. I learned the game slap-jack, which is similar to spit, but purely focused on slapping the pile as fast as possible. I get so animated with games like this, it's a fun rush to win some hands.

Good List Week 3

2026-04-13T00:00:00-04:00

Another week, another list.

Public libraries

I've been utilizing public libraries a lot the past few months, especially for books: the interlibrary loan system is a life hack, the network that includes my local library meets about 90% of my requests. The experience couldn't be easier: search for the book in the library catalog, request it to my local branch, and pick it up in a few days.

Last week I met up with a friend to work at the Maplewood public library. It's recently renovated, and situated right on a beautiful park with a turtle pond.

Grounds for Sculpture

I visited the Grounds for Sculpture in Hamilton - what a delight! I had recently learned the word kitsch, loosely meaning tacky imitative art, and just in time, because oh boy are some of these artworks kitsch! All the same, the grounds and gardens are beautiful to walk around, and the cherry blossoms were in bloom, making for a perfect spring day.

Saravanaa Bhavan

Dark horse pick here, but I was happy to find and introduce someone to my favorite global chain restaurant. I may be one of their most well traveled customers, having visited locations in the US, UK, Europe, India and Asia. A well crafted dosa just hits the spot sometimes. For decent Indian vegetarian food almost anywhere in the world, it can't be beat.

Good List Week 2

2026-04-06T00:00:00-04:00

Continuing on the gratitude practice of recognizing the good in the world - here's my good list for the past week.

Blossoms

It's spring in New Jersey, and that means the flowers are starting to bloom. Essex county is home to one of the largest cherry blossom populations outside of Japan, if I remember the Branch Brook Park signs correctly. While that park gets absolutely mobbed during cherry blossom season, you don't have to work hard to see the blossoms: they are just about everywhere in the area. Here's a lone cherry blossom tree I spotted at Hilltop Reservation:

After that brutal winter, it's nice to feel the warmth of springtime.

Volunteering

I volunteered at two drug and alcohol rehabs last week, speaking to over a hundred people about family and generational trauma. The response is always moving, with people regularly opening up about experiences that they've never told anyone before. I was reminded of the power of the work when a tough-looking guy stopped speaking mid-sentence and left the room to cry. The level of identification is humbling: it shows that beyond race, class, background, so many of us shared similar experiences. It's said that service to others is a pillar of a good life, and I'm grateful for the opportunity to be of service.

He that plants trees loves others besides himself

Thomas Fuller

House plants

I picked up a few new house plants last week at a local nursery. I added money tree and tradescantia to my collection, which also includes: snake plant, monsteria, zz plant, pothos, and pepperomia. Having life in my apartment, and creatures to care for, adds light to my life.

Good List Week 1

2026-03-30T00:00:00-04:00

I heard on the radio today that the New York Times started publishing a "good list" of fun activities as experienced by reporters. The idea being to share the positive, on-the-ground experiences; get off-line and remember that life is to be lived, not watched from the sideline (or screen). With spring in the air, I feel inspired to do the same, starting with a backlog of the past month or so:

High school musical

Last year, I discovered Morristown high school's theatrer department when they put on Les Mis. It was amazing! The talent from the students really impressed me, from the performance, to the stage and costumes. I've since been back for the fall presentation of Little Women, and their recent spring musical Fiddler on the Roof.

Great way to spend $15 for a quality performance in the local community.

Opera

I attended the opera at the Met for the first time, just last Friday. I had some fear that, with the language barrier, and Timothée Chalamet's recent comments, I'd not enjoy the show - but I was pleasantly surprised. Language-wise, each seat has its own subtitles, making following the story easy. I put on my khakis, expecting folks to be dressed up - I might as well have worn blue jeans, compared to the ballroom gowns and dapper suits I was surrounded with. The performance itself, Madame Butterfly, was exquisite - I was mesmerized by the stage design, the use of simple lights and mirrors to incredible effect:

Birding

Birding was recommended to me as a way to be more present and engaged. And it certainly has that effect. Typically when taking a walk or hike, I'm unconsciously trying to set my personal best. With birding, I am significantly slower, walking slower and pausing occasionally so as to listen and look. I'm encouraged to explore new parks in the region, to see what different species are present. To that end my recent hotspots are Eagle Rock Reservation, Verona Park, and Hilltop Reservation.

Three Years of Fellowship

2026-03-27T00:00:00-04:00

This is a follow-up to my two year reflection post last year.

Driving a car off a cliff

A metaphor I've often used for my life is that of driving a car off a cliff. Growing up in chaos and dysfunction, I lived my adult life in chaos and dysfunction. I had been headed towards a certain path, the particular details of which impossible to predict, but the result the same: catastrophe.

I view my recovery and healing as turning the car wheel and applying the brakes. Given the momentum, these are not instant fixes; I often ask myself if I started braking early enough, or if I'm doomed to vault over the cliff. Perhaps given the past year, I can say that I stopped the car as it's teetering off the edge of the cliff, with little room to spare.

These are just stories I tell myself. As I aptly heard someone share this week, there's a difference between honesty and truth. Honesty is what I believe to be true, as opposed to the objective truth. I am quite skilled at deceiving myself.

What's been going on

I am coming out of one of the most challenging years of my life. And I'm no stranger to challenging years. I'm not one for omens or precognition, but events were seemingly choreographed: I distinctly recall when multiple people shared about their physical debilitation, shortly before my own. It was a summer flu which struck me down: I had been red-lining my stress, and paid the price. It's been nigh on nine months of burnout and recovery.

Work went from stable to ongoing crisis as I was forced into withdrawal from workaholism (not much of a surprise there, it runs in the family). In re-evaluating my relationship to my job, and setting healthier boundaries for myself, I discovered my boss' covert narcissism. Discover sounds like a detached scientific process; this was trial by fire, incurring my boss' narcissist wrath.

It's incredible how long I can survive in chaos. There's a part of me that, like the protagonists of my beloved Culture series, believes I'm invincible. What a painful but healing revelation that I am not. It took my health deteriorating, and a coup de grâce from my boss, for that warrior part of me to step aside, which enabled me to give myself the gift of letting go.

In the past few months, I've stepped back from work to focus on my health. It's been a miserable period, contrary to the "with time off, enjoy yourself!" mentality. The cognitive incapacitation from burnout has been painful in many ways: one, as an intellectual, losing my core strength; two, the uncertainty, not knowing when or if I'll recover. I am feeling better, week after week, faster than I fear, slower than I hope.

What's new

Somehow in the midst of all that, I managed to take some significant positive steps. I started volunteering, bringing a message of hope and healing to drug and alcohol rehab centers in my area. This experience has been incredible for me, coming from a family of alcoholics and addicts. Growing up, when passing the local rehab, my father would say to me, "I went there, your mother went there, and one day you'll go there." To fulfill this prophecy upside down feels like taking control of my own story.

One of my goals for the last year, physical fitness, I dove into with yoga, making a consistent habit of it. After nine months of practice, my body, and my relationship to my body, feels great: decreased tension, decreased aches and pains. It's said that emotions are stored in the body, with yoga as a great means to move the emotions: this has been my experience.

Another goal, travel; I visited England, Scotland, Ireland and Greece. Each trip I take changes me, or unlocks accumulated changes within me; either way, coming back I feel refreshed, invigorated, excited for life. In Greece, I solo traveled abroad for the first time, walking along the ancient marble of the Acropolis, where my ancient heroes once walked. Standing amidst history, I realized (for the nth time) the obviousness of the truth: what I'm looking for so desperately is within, not without. Spiritual insight aside, I can't wait to travel again soon.

What's next

One of the Buddhist insights I relate to strongest is that peace is available within myself, anytime. It reminds me of the recovery saying, that serenity is not freedom from the storm, but safe harbor within the storm. As the world continues to dive deeper into chaos, I am grateful for my serenity. It is my foundation stone as I once again dust myself off and venture back into the fray.

Study Guide

2026-03-23T00:00:00-04:00

Recently, I've started to rouse from my hibernation. As I tend to be slightly eclectic and eccentric, I want to organize (or at least document) my approach to preparing to re-join the workforce as a software engineer.

I'll aim to update this page over time.

Interview practice

Leetcode/Neetcode, leveraging llms for interactive problem solving and hints.

Books

Done:

Machine Learning (MIT Essential Knowledge): finished; relatively recent, solid high level intro to ML for someone without the background.
7 Concurrency Models in 7 Weeks: finished; interesting read, my first intro to CSP.

In-progress:

System Design Interview: halfway through.

Queued:

7 Databases in 7 Weeks
Architecture of Open Source Applications

Projects

Before you ask, yes, I do like to name things "simple xyz" - as a reaction to the ballooning complexity of most software products.

Etc

Studying outside the home: coming from years of remote work, I am re-familiarizing myself with in-person work (cheers to Panera sip club!)

Resources

Wat

2026-03-09T00:00:00-04:00

I use emacs, and within emacs I use the pytest plugin to run tests. When I need to debug tests, I turn to ipdb with a quick yasnippet, dropping in import ipdb; ipdb.set_trace(). When debugging with ipdb, I just love this little tool called wat. It makes inspecting objects a breeze, and is a must-have in all my projects now.

As an example, I was debugging the difference between Django's Request class and Django Rest Framework's Request class, and their interop. Dropping into a debugger in the view, I could quickly inspect the request object:

Python

ipdb> wat / request
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
value: <rest_framework.request.Request: GET '/osrm/route/v1/driving/-74.172400,40.735700%3B-74.027600,40.744000'>
type: rest_framework.request.Request

Public attributes:
  FILES: django.utils.datastructures.MultiValueDict = {}
  POST: django.http.request.QueryDict = {}
  accepted_media_type: str = 'application/json'
  data: dict = {}
  ... reduced for brevity

  def force_plaintext_errors(value)

Private attributes:
  _auth: NoneType = None
  _data: dict = {}
  _files: django.utils.datastructures.MultiValueDict = {}
  _request: django.core.handlers.wsgi.WSGIRequest = <WSGIRequest: GET '/osrm/route/v1/driving/-74.172400,40.735700%3B-74.027600,40.744000'>

  def _authenticate() # Attempt to authenticate the request using each authentication instance…
  class _content_type() # Placeholder for unset attributes.…
  ... reduced for brevity

Public and private attributes and methods are listed. Where available, types, classes, values and docstrings are displayed inline.

Auto import

It's tedious to have to import wat in every shell session, so I looked online and found that it can be automatically imported. There's two different configurations to apply.

For automatic import into ipython shells (i.e. manage.py shell), modify the ipython startup profile¹, by creating a new file at ~/.ipython/profile_default/startup/00-first.py, with the contents from wat import wat.

For automatic import into debugger ipdb shells (i.e. ipdb.set_trace()), add from wat import wat to .pdbrc in your main repository folder.

And - don't forget to pip install the wat package in your repository 🙂

Thanks to https://switowski.com/blog/ipython-startup-files/ ↩

Django: dj-stripe

2026-03-04T00:00:00-05:00

I spent the last week integrating Stripe into a side project. Given the popularity of the library, I expected dj-stripe to simplify the implementation and save me time. Internally, I bucketed a few hours to a complete implementation. Boy, was I wrong.

Background

For context, my side project is a metered api gateway. I offer a free tier and a monthly subscription tier. My basic use cases are:

User signs up for subscription (via stripe checkout)
User manages (cancel, edit payment) their subscription (via stripe portal)

When a user signs up for a subscription, they'll be upgraded to the higher tier internally. Likewise on cancellation, they'll be downgraded.

What does dj-stripe do, anyway?

I expected dj-stripe to integrate cleanly with a Django application, through webhook handling and customer object relations. While the library does provide these functions, what I didn't realize is that dj-stripe does so much more: it actively syncs your stripe account with your database. Every product, price, transaction, subscription, tax zone, etc., etc. is sync'd into your database. This is purportedly for efficiency: rather than fetching data from the stripe api, it can be pulled in via a db query. I recognize that this is a real performance issue, at a previous company some endpoints serially queried the stripe api multiple times, leading to multi-second response times on critical paths.

For a side project of my scale, though, in hindsight it was definitely overkill. I have one product, and one price; customers can be created via stripe api; subscription status can be managed through webhooks.

Documentation

The dj-stripe documentation is, bluntly, awful. It somehow fails to describe how to do much of anything that I was interested in doing; I imagine that SaaS subscription billing is common among Django developers! Where the documentation does exist, it's misleading if not outright wrong. Even popular blog posts, like How to Create a Subscription SaaS Application with Django and Stripe are confusing and lack details.

As an example, let's look at the demo code for stripe checkout. It's more than a hundred lines, and manually associates Customer and User objects. Compare to my own implementation (courtesy of claude):

Python

class SubscribeCheckoutRedirectView(LoginRequiredMixin, View):
    def post(self, request, **kwargs):
        customer, _ = Customer.get_or_create(subscriber=request.user)

        success_url = (
            self.request.build_absolute_uri(reverse("dashboard")) + "?subscribed=1"
        )
        cancel_url = self.request.build_absolute_uri(reverse("dashboard")) + "?bailed=1"

        session = stripe.checkout.Session.create(
            customer=customer.id,
            success_url=success_url,
            cancel_url=cancel_url,
            mode="subscription",
            line_items=[
                {
                    "price": settings.STRIPE_HOBBY_PRICE_ID,
                    "quantity": 1,
                }
            ],
        )

        return redirect(session.url)

A fraction of the code - and the only usage of dj-stripe is in Customer.get_or_create(), the resulting customer id automatically associating the session/transactions with the internal user.

Configuration

Which leads me to api key management in dj-stripe. This singlehandedly cost me at least an hour of time - unheard of for configuring an api key! The project is apparently migrating towards supporting multiple stripe accounts in a single application (despite nobody asking for it, as far as I can tell). So the documentation says don't put your stripe api keys in django settings, put the keys in the database - but guess what, some parts of dj-stripe still do expect the api key in django settings! So the api key has to be configured there, too. Under what name? Well, there's STRIPE_SECRET_KEY, STRIPE_LIVE_SECRET_KEY, STRIPE_TEST_SECRET_KEY... seriously, why not have one page to describe all configuration settings, with deprecations and recommendations clearly marked?

And why on earth am I storing my stripe api keys in my database!? That feels gross from a security perspective.

Webhooks

Finally, after all of the above, I'm able to actually sit down and build the key piece of functionality: webhook integration. Ah, this has also been rewritten, much to the community's chagrin; it's now exposed via django signals. Thankfully this was relatively simple to build, though the docs for most tutorials and guides are outdated since the rewrite.

Dj-stripe actually manages the webhook for you, creating it via django admin. The use of random urls for the webhook confused me; is that all the security provided? I looked into the code and the database and found that dj-stripe does verify the webhook signature, with the webhook secret stored in the database. Why even bother with the random urls then? Tsk, tsk.

Wrap up

My take is that dj-stripe is trying to do too much, with too little resources. The lack of documentation, the botched api key configuration migration, the multi-account use case.. I think there's a market for a significantly streamlined django + stripe integration for SaaS products, more opinionated in the sense of keeping the feature set small (with an escape hatch when outgrown), and less opinionated in the sense of standard practices like api key management.

BirdNet-Pi

2026-02-27T00:00:00-05:00

After upgrading my homelab from a raspberry pi 4B to a beelink SER5 MAX, my rpi collected dust in the corner.. until I discovered BirdNet Pi! Now, my rpi is working hard to catalog passing avian visitors:

As the community is in flux, I wanted to share my setup.

Audio

I found several long discussions about microphones and acoustics; as a newbie I chose a sub-$20 USB microphone on amazon: CMTECK USB Computer Microphone G009. It's only been a day, but I'm already getting solid results with multiple 80-90% confidence ratings. The key points for me were the price, and the compatibility: USB-A means no dongle or adapter necessary.

For more advanced suggestions, check out this discussion on github.

Software

The original BirdNet-Pi repo has been archived; there is an active fork, which I decided to use. The setup guide is straightforward; plus, the new raspberry pi imager software supports configuring the wifi connection, meaning the rpi can connect via wifi on first boot. The longest part of the process was the BirdNet-Pi installer script, which seemed to install hundreds of packages.. hair-raising, for sure, but it installed without issue.

The software ecosystem is growing: there's also Birdnet-Go, a complete re-write that's under active development. Under the hood, it's all powered by the Cornell Lab of Ornithology, via their trained models: see the birdnet team github.

That's it

Everything worked out of the box! Loading up BirdNet in my browser streamed my microphone audio; after an hour or so the first bird ids started coming in. Next up, notifications when visitors stop by :)

One-month update

Ah, but my rpi ran into some issues and I took birdnet down for the time being. Here's a screenshot of the data collected over the month:

From Cloudflare to Umami Analytics

2026-02-22T00:00:00-05:00

For a recent project, I integrated Cloudflare web analytics, as I had been using it for this blog for some time. Out of curiosity, I looked into alternative, self hosted analytics options: plausible and umami. Imagine my surprise when I see in the plausible docs that cloudflare data is inaccurate, as stats are based on a 10% sample. I had been suspicious that something like this was occurring, owing to the round number bias: the phenomena whereby round numbers seem like estimates rather than exact calculations.

As an example, for this blog, as of writing, Cloudflare reports 340 total visits in the past 30 days. Visits by country breaks down to: 90 USA, 70 Singapore, 20 Mexico, and so on. Browser breakdown is: 270 Chrome, 20 Firefox, 20 Safari, and so on. Every number in the dashboard is rounded to the nearest 10. Take a look:

In the Cloudflare dashboard, there's a tooltip that states:

Based on a 10% sample of page load events.

... confirming the suspicion. Given the relatively low traffic to my blog, and even more so my new project, I wanted to do better. At the scale of a few dozen requests per day, I should be able to get complete information. Hence I turned to self hosting umami. I chose umami over plausible after reading on reddit that self hosted plausible is limited, and folks reported positive experiences with umami. So far, so good: umami provides everything I am looking for in an analytics tool, plus it's simple to host and easy to use. Here's how it looks, since switching a few days ago:

Since upgrading my homelab from an rpi to a proper mini pc, I have been feeling more confident in self hosting, and I am pleased with umami for delivering where cloudflare fell short.

Django Rest Framework Throttle Race Condition

2026-02-18T00:00:00-05:00

In a recent project, I used drf's SimpleRateThrottle to implement rate limiting on some views. Curious about the implementation, I reviewed the source, and noticed that the throttler uses the cache backend with a sliding window log algorithm. The high level approach is:

Generate key for request
Lookup window in cache by key
Purge old entries (outside current window)
Check if window is full
If so, fail
Otherwise, insert current timestamp into window
Set window in cache
Pass

Notice anything? There's a race condition between steps 2 and 7: multiple requests may retrieve the window from the cache in parallel, and later when setting the updated window, the last request will overwrite the other entries. To demonstrate this behavior, I created a small tool to visualize the success rates of requests against a rate limited url:

rlui tests requests against a url for success, over time at a given rate. In the above, it's clear that SimpleRateThrottle is vulnerable to the above race condition, as requests repeatedly exceed the configured limit of 10 req/s: at an input of 40 req/s, the peak success was 38 req/s, with an average success of 24 req/s.

Now, I'm not the first to point this out - it's been reported before. I agree with the maintainers that it's out of scope of drf to provide a perfect solution, as rate limiting algorithms are diverse in their costs and benefits. Out of curiosity, I decided to implement my own rate limiting algorithm, using redis: the same sliding window log algorithm, without the race condition.

Redis provides a sorted set primitive, which maps each set entry to a score, which is used for ranking or sorting. To implement a sliding window log, map each request key to its numeric timestamp. Redis provides a command to cull the sorted set within a given range, perfect for purging entries outside the current window. At a high level:

Generate key for request
Purge old entries (outside current window)
Add current timestamp into window
Check if window is full
If so, remove current timestamp from window, and fail
Pass

Here's how it runs:

The rate limiter never allows requests above 10 req/s. Voila!

The key difference is that the window updates do not overwrite each other, i.e. entries may be safely added and removed in parallel. The window is therefore not prone to data loss, as in SimpleRateThrottle. In code, it looks like this:

Python

now = self.timer()
req_key = str(now)

pipeline = redisclient.pipeline()

pipeline.zremrangebyscore(self.key, 0, now - self.duration)
pipeline.zadd(self.key, {req_key: now})
pipeline.expire(self.key, self.duration)
pipeline.zcard(self.key)

_, _, _, num_requests_in_window = pipeline.execute()

if num_requests_in_window > self.num_requests:
    # remove key, request was not processed and doesnt count
    redisclient.zrem(self.key, req_key)
    return False

return True

The relevant redis commands are:

zadd: add an element (and its score) to a sorted set. Used to add requests to the sliding window.
zcard: measure the cardinality, or size, of the set. Used to count the number of requests in the sliding window.
zremrangebyscore: removes entries from a set by score. Used to purge old entries in the sliding window, from 0 (beginning of time) to the start of the current window (now - duration).
expire: expire a key. Used to cleanup the database; after duration seconds, the sliding window may be purged.

Note the use of a pipeline, which is similar to a database transaction: this sends all commands together to be executed atomically. This ensures that the window size is counted correctly, and also prevents crashing midway from preventing expiring the key.

A savvy reader will note that this code has its own race condition: there's a time gap between adding a request to the window and checking the window's size, and later removing the request from the window. If a request comes in and exceeds the window limit, then before that request is removed, a request could come in that should pass, but doesn't, because the window has extra requests. In short, this implementation will reject more requests under heavy pressure, as opposed to allowing them. This could actually be resolved too, but would require a lua script implementing the same logic. Following the drf authors, I wanted a solution that held up, not one that's perfect.

For the full code of the throttler, see this gist.

Split Tunneling with Network Interfaces

2026-02-17T00:00:00-05:00

In setting up my new homelab server, I ran across a networking issue: my cloudflare tunnels were regularly failing when connected through my vpn. The failing network architecture:

architecture-beta group homelab(server)[Homelab] service qbt(server)[Qbittorrent] in homelab service cft(server)[Cloudflare Tunnel] in homelab service app1(server)[Application 1] in homelab service app2(server)[Application 2] in homelab service vpn(internet)[VPN] service pub(internet)[Public internet] junction jleft in homelab junction jcenter in homelab junction jright in homelab junction cfj in homelab qbt:T -- B:jleft jleft:R -- L:jcenter jcenter:T -- B:vpn cft:T -- B:jright jright:L -- R:jcenter vpn:T -- B:pub cfj:T -- B:cft app1:R -- L:cfj app2:L -- R:cfj

I experimented with different vpn endpoints, but each new endpoint would reliably fail after a short time. Clearly cloudflare is blocking connections through the vpn. Disabling the vpn was out of the question, so I researched different approaches. Split tunneling is afforded by vpns in wireguard via AllowedIPs, but that only works if the destination IPs are known and stable - which in p2p they are not. I needed to split tunnel based on the application itself, as in the following network diagram:

architecture-beta group homelab(server)[Homelab] service qbt(server)[Qbittorrent] in homelab service cft(server)[Cloudflare Tunnel] in homelab service app1(server)[Application 1] in homelab service app2(server)[Application 2] in homelab service vpn(internet)[VPN] service pub(internet)[Public internet] junction cfj in homelab qbt:T -- B:vpn cft:T -- B:pub cfj:T -- B:cft app1:R -- L:cfj app2:L -- R:cfj

Claude suggested a fairly complicated solution, involving isolated networks and bridges. I demurred and searched for something sleek and simple, and found it in the qbittorrent settings: binding the application to a specific network interface, in this case the vpn network interface. I had actually enabled this before, in order to cut off network access when the vpn is down. The missing piece of the puzzle was in the wireguard configuration: setting Table = off disables wireguard cli from configuring automatic network route binding, in other words from automatically routing all traffic through the vpn interface. Applications may still route traffic through the vpn network interface by explicitly specifying it via the SO_BINDTODEVICE socket option.

Not all applications support binding to a particular network interface, so this won't always work; when it does, as in this case with qbittorrent, it makes for trivial split tunneling.

Migrating a Static Site from S3 to R2

2026-02-13T00:00:00-05:00

I host a few static sites (including this blog!), and have journeyed from self hosting on a raspberry pi through to cloud hosting. I migrated to S3 static site hosting, which besides the arcane setup steps is otherwise cheap and easy to maintain. Using cloudflare for dns, though, I had to downgrade the SSL/TSL encryption level, as S3 and cloudflare do not "play nice" together when fronting a bucket with a domain. I decided to move my static sites to R2, cloudflare's object storage offering to restore the encryption level.

Creating a bucket in R2 is simple enough (once one finds the feature in cloudflare's confusing dashboards). From there, head to settings and add a custom domain. Cloudflare will automatically create or convert the relevant DNS record.

For upload, the aws cli supports an --endpoint-url <url> param to interface with other providers (e.g. minio, garage, backblaze, R2, etc.). The makefile command to publish this blog is:

Bash

aws s3 sync "$(OUTPUTDIR)"/ s3://$(R2_BUCKET) --endpoint-url $(R2_URL) --delete

where R2_BUCKET is the bucket name and R2_URL is your S3 compatible URL (it'll look like https://...long string....r2.cloudflarestorage.com). In the R2 dashboard, I generated S3 API keys for use with aws cli; I use direnv to dynamically populate my shell environment variables, there I configured AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

With the bucket created, dns setup, and content uploaded, your public site should be accessible. The first thing you may notice is that root traffic errors out, e.g. blog.tmk.name/ does not serve the blog. It turns out that R2 does not support a "bucket root" item like S3 static site hosting, in other words there's no built-in feature to use index.html as the root page.

Not to worry, cloudflare has a rules feature that solves this pain point. Create a new rule, specifying a custom filter expression:

URI Path -> equals -> /
AND
Hostname -> equals -> blog.tmk.name

Then...

Rewrite to -> Static -> index.html

... and save. This will implicitly redirect traffic to blog.tmk.name/ to serve index.html. For nested static sites, it's possible to configure the page rule to dynamically serve each folder's index.html - see this helpful community post for details.

All in all, it took me about 30 minutes to migrate from S3 to R2, and upgrade my SSL/TSL encryption.

OSM Ecosystem

2026-02-12T00:00:00-05:00

I've been working on a few projects leveraging the incredible open ecosystem provided by the Open Street Maps network - I want to document a few, because the landscape is quite confusing to a newcomer.

Geocoding

Geocoding is the process of converting raw, text addresses (e.g. 1600 Pennsylvania Ave.) to lat/lng geo-coordinates, which are necessary for geospatial applications like viewing on a map, calculating distances, and so on. Nominatim is built on top of OSM data, and provides a free API (at a low rps limit).

To call the public API from JS:

JavaScript

let nominatimUrl = 'https://nominatim.openstreetmap.org/search?addressdetails=1&format=jsonv2&limit=1';
nominatimUrl += '&q=' + encodeURIComponent('1600 Pennsylvania Ave');

const addressSearchResults = await fetch(nominatimUrl).then(res => res.json());

This will return a payload containing geo-coordinates, and other interesting data points sourced from OSM.

Point of interest search

Points of interest include parks, businesses, museums, libraries, etc. - all the various land uses one finds around the world. OSM contains a rich dataset of crowdsourced data on points of interest. That dataset can be queried by Overpass, via its API. Overpass is interfaced with a complex query language (QL), described in the wiki as a "procedural, imperative programming language written with a C style syntax."

If you are still with me, then take solace that simple queries in Overpass QL are thankfully simple to write. As an example, Finding all cafes in a given bounding box:

Text Only

[out:json][timeout:25];
// gather results
node({{bbox}})["amenity"="cafe"];
// print results
out geom;

Where {{bbox}} is a comma separated list of the form: southernmost latitude, westernmost longitude, northernmost latitude, and easternmost longitude. OSM tags include multiple amenity types, including cinemas, bars, libraries, etc.

Making a request against the API took some figuring out; it seems the approach is to make a POST request against the interpreter endpoint, having encoded the query into the payload with a body= prefix, as so:

JavaScript

const overpassUrl = 'https://overpass-api.de/api/interpreter';

const overpassQuery = `
[out:json][timeout:25];
// gather results
node(${overpassBoundingBox})["amenity"="cafe"];
// print results
out geom;
`;

const overpassResults = await fetch(overpassUrl, {
    method: 'POST',
    body: 'data=' + encodeURIComponent(overpassQuery),
}).then(res => res.json());

This will result in potentially a lot of elements returned; each element includes geo-coordinates and a set of tags, which usually (but is not guaranteed to) includes name, address components, etc. Note that the overpass-api.de service is intermittently flaky; there are other free public hosts listed in the project wiki.

Routing

The traveling salesman problem asks the question, given N cities to visit, what's the optimal order to visit them in, so as to minimize drive time or distance? Generalized, the vehicle routing problem posits multiple vehicles driving multiple routes, looking to visit all locations in the optimal order. To solve these problems, it's necessary to provide driving distance or duration matrices. OSRM and Valhalla are two (of many) routing-based engines that operate on top of OSM route data (think streets, highways, etc.), providing the distance matrices necessary to solve the vehicle routing problem.

To generate the distance matrix via OSRM:

Bash

curl "https://router.project-osrm.org/table/v1/driving/-73.9855,40.7580;-74.0134,40.7127;-73.9718,40.7678;-73.9614,40.7143"

Which can then be provided to VRP solvers, e.g. vroom, or-tools, pyvrp, etc.

Hurl

2026-02-10T00:00:00-05:00

Today, in a project, I wanted to set up a few smoke tests to verify that my http api worked in production. hurl came to the rescue, a simple and straightforward text-based tool (and text does run the world!).

Here's a sample of my tests file (generated handily by claude):

Bash

# --- Auth ---

# Reject missing auth
GET {{api_url}}/osrm/route/v1/driving/-74.172400,40.735700;-74.027600,40.744000
HTTP 401

# Reject invalid key
GET {{api_url}}/osrm/route/v1/driving/-74.172400,40.735700;-74.027600,40.744000
Authorization: Bearer 00000000-0000-0000-0000-000000000000
HTTP 401

# --- OSRM Route ---

# Basic route request
GET {{api_url}}/osrm/route/v1/driving/-74.172400,40.735700;-74.027600,40.744000
Authorization: Bearer {{api_key}}
HTTP 200
[Asserts]
header "Content-Type" contains "application/json"
jsonpath "$.code" == "Ok"

Hurl supports variables, which makes running against different servers (local / prod) a breeze. I configured the variables in my .env which get auto-populated in my shell by direnv:

Bash

## dev
HURL_VARIABLE_api_key=asdf
HURL_VARIABLE_api_url=http://localhost:8000

Note that environment variables are prefixed with HURL_VARIABLE_<name>, and interpolated in the hurl file with just <name>.

To run the tests:

Bash

$ hurl tests/smoke.hurl --test --error-format long

This will print a summary of the test results, bailing on error with the failing response.

Browsing the docs, hurl can be used as just an http automation tool, or in test mode as above, and also as a load/stress test tool with the --repeat <count> option.

I found hurl to fit the niche in between manual tests and unit tests.

Hide Category From Index in Pelican

2026-02-09T00:00:00-05:00

Fitting enough for my first TIL post, I had to figure out how exclude posts in the TIL category from my pelican blog, without breaking pagination. The feature has been requested in the pelican github, and thankfully the code snippet at the end has the solution:

Python

class ExcludeWriter(Writer):

    def write_file(
        self,
        name,
        template,
        context,
        relative_urls=False,
        paginated=None,
        template_name=None,
        override_output=False,
        url=None,
        **kwargs
    ):
        articles = kwargs.get("articles")
        page_name = kwargs.get("page_name")
        if page_name == 'index':
            ### set categories to exclude here
            articles = [a for a in articles if not a.category == 'TIL']
            kwargs["articles"] = articles

        super().write_file(
            name=name,
            template=template,
            context=context,
            relative_urls=relative_urls,
            paginated=paginated,
            template_name=template_name,
            override_output=override_output,
            url=url,
            **kwargs
        )


def get_writer(sender):
    return ExcludeWriter


def register():
    """Register the new plugin"""
    signals.get_writer.connect(get_writer)

register()  # <-- this line added to the code in the snippet

Place this in pelicanconf.py and restart the devserver.

Async Django

2026-02-01T00:00:00-05:00

For a recent project, I decided to go all-in on async Django in order to get a feel for its advances in recent years. I've long been inspired by the infamous "Erlang replaces your entire stack" slide:

And for this project, I wanted to see what I could replicate with Django and async Python.

Project

How do I share text snippets (long links, code snippets, log lines, etc.) between phone, laptop, and desktop? To this day, I rely on email and whatsapp, messaging myself. To improve this, I created SimplePasteService (source code) - a website for quickly sharing text content between devices without long random ids nor the email login bootstrapping problem. For fun, I added live updates: two devices on the same paste will see new text updates stream in.

The project requirements demanded two technical solutions. First, as pastes expire, they should be deleted in a recurring background task. Second, cross-client communication is necessary for live updates.

For a typical sync Django project, I'd solve the former with celery and celery beat. This adds two additional processes to run and manage, in addition to the message broker and result store. For the latter, I'd lean on a pubsub server, possibly a SaaS solution. The complexity jumps for each feature are out of proportion, in my view.

What I'd like is to keep complexity low. Could I replace my stack with asyncio?

Architecture

The short answer is yes, but with caveats. The core limitation is that async Python is a single process, and single threaded, as opposed to the Erlang/BEAM, which runs across all cores, and can be extended to run across multiple machines. As this is a hobby project, I made the decision early to accept this limitation: I'd deploy a single process with a single thread to run the webserver, pubsub, cron, and background tasks. This is not a scalable solution, clearly, but for a low-traffic hobby project, the reduced complexity is a compelling benefit.

Before diving into these components, one more architectural decision had to be made. For live streaming updates, I decided on server sent events, a simple push-only protocol that works mostly out of the box. To implement, I leveraged htmx's sse extension for the frontend, and Django's StreamingHttpResponse on the backend.

Webserver

Django comes out of the box with asgi support; all that's needed is a compatible webserver. I chose the popular uvicorn. For development, I used the following invocation:

Bash

uvicorn simplepasteservice.asgi:application --port 8000 \
  --reload \
  --reload-include "*.html" \
  --timeout-graceful-shutdown 1

Note --timeout-graceful-shutdown: I noticed when developing with long lived sse connections that the server would not automatically restart until the connections closed. This parameter will close those connections automatically after the grace period. I also added html files to the reloader, so that template updates would automatically refresh.

Background tasks and cron

To replace celery and beat, I decided on apscheduler, which has an asyncio-friendly scheduler, and is easy to set up. I updated asgi.py to add:

Python

scheduler = AsyncIOScheduler()

# also possible to use string "simplepasteservice.tasks:delete_expired_pastes"
scheduler.add_job(delete_expired_pastes, trigger=IntervalTrigger(hours=1))

scheduler.start()

And with just these few lines, I had my desired recurring background task. One-off immediate-run tasks need no special setup when queued from an async view, look at asyncio.create_task() - but I didn't have a need for such tasks in this project.

Pubsub

The basic feature here was: when a paste is updated with new content, ping the clients subscribed. As the application is just a single event loop, I decided to build a trivial pubsub server within the application itself. The core data structure is a dictionary mapping pastes to subscriber queues:

Python

paste_update_listeners = defaultdict(list)

...

async def view_paste_sse(request, key):
    """Creates and listens on an update queue, pinging client on updates"""
    global paste_update_listeners

    # ensure paste exists
    paste = await aget_object_or_404(Paste.objects.unexpired(), key=key)

    q = asyncio.Queue()
    paste_update_listeners[key].append(q)

    async def event_stream():
        try:
            while True:
                await q.get()

                # update occurred!
                yield "event: update\ndata:\n\n"

        except asyncio.CancelledError:
            # connection closed, clean up
            paste_update_listeners[key].remove(q)
            q.shutdown()

    return StreamingHttpResponse(
        event_stream(),
        content_type="text/event-stream",
    )

When pastes are updated, all clients are notified:

Python

async def create_paste_item(request, key):
    global paste_update_listeners

    paste = await aget_object_or_404(Paste.objects.unexpired(), key=key)

    form = CreatePasteItemForm(request.POST, paste=paste)

    if form.is_valid():
        await PasteItem.objects.acreate(paste=paste, text=form.cleaned_data["text"])

        # TODO: serial processing here could be made async
        for listener_q in paste_update_listeners[key]:
            await listener_q.put("bump")

        return redirect("view_paste", paste.key)

    messages.error(request, "Error adding paste item")

    return redirect("index")

Nice. In a few dozen lines, live streaming cross client updates, all in async Django!

Pitfalls

I am overall quite pleased with the final result, having replaced several elements of my typical web application stack with async Django. That said, asyncio feels off, suffering from the function coloring problem that took several headaches to imprint ¹. Django suffers from the same, and it feels like async support is still some distance away from prime time.

Async Django

I set up my project entirely along async lines: async views, async-compatible middleware. This wasn't strictly necessary, but I wanted to get a feel for the state of an all async project. For views, what bit me was that template rendering is not async friendly. It will work, up until async code is called, or in an async view, if any "synchronous only" code is called. The recommended solution is to wrap template rendering in sync_to_async and call it a day; as an async zealot for the day, I refused this approach, and instead accessed my async database lookups in the view in painstaking fashion.

In my research, I discovered Jinja2 does support async rendering, even leveraging incremental html generation to stream the results to the client. I'd take async-compatible Django template rendering without the bells and whistles, though the prospect of colored templates (either async or non-async) makes me nervous.

The middleware story is also sub-par: if any middleware is non-async friendly, then the entire request processing flow becomes non-async, which means there goes my async processing! For this project, I had few third party middleware, but sadly whitenoise tripped the async unfriendly wire, and had to be replaced with blacknoise. This alternative is set up on top of the asgi application, rather than as a django middleware:

Python

application = BlackNoise(get_asgi_application())
application.add(BASE_DIR / "staticfiles", "/static")

I don't know if this is a technical limitation (perhaps due to file streaming in async?), but I'd have much preferred the standard Django middleware integration.

SSE

The server sent events protocol is not perfect, but suffices for a project of this scope. As described above, I leveraged Django's StreamingHttpResponse with an async streaming generator to send events down the wire. In my homelab deployment with Caprover, I noticed that updates were not coming through; this is due to nginx reverse proxying not streaming sse appropriately by default, the following header instructed nginx to play nice:

Python

return StreamingHttpResponse(
    event_stream(),
    content_type="text/event-stream",
    headers={
        # for nginx proxying sse support
        "X-Accel-Buffering": "no",
    },
)

The other major issue I ran into involved stuffing html into sse event payloads. I first attempted to configure htmx to stream the html from sse events directly into the dom, but ran into a problem with the sse protocol: newline sensitivity. As Django renders templates with many newlines, I attempted to simply remove them:

Python

 # rendered html has to be stripped of newlines, sse format limitation
 # see https://github.com/bigskysoftware/htmx/issues/2292
 # ah.. but then the copy-paste wont work with newlines... :(
 html = render_to_string(
     "paste_detail.html#paste-items-list",
     {
         "pasteitems": pasteitems,
     },
 )

 html = html.replace("\n", "").replace("\r", "")

 yield "event: update\n"
 yield f"data: {html}\n\n"

This would have worked, except that I had newline sensitive code in the templates. Suppose a user inputs a text with newlines (eg paragraphs or log lines). To implement a "copy on click" feature to the page, I added a data-copyable="{{ content }}" attribute to relevant elements, and an event listener that copied the attribute content into the clipboard. That inner content could include newlines that could not simply be stripped out without breaking the copy feature ²

Finding myself limited by the protocol, I pursued a simpler approach: on sse message, make a request to a view to refresh the template partial. While an extra round trip to the server, it fit in more cleanly and saved the headache. I'd advocate that htmx add support for a friendlier encoding scheme, to support html over sse.

Final thoughts

While async in Python and Django has come a long way, I would not choose this stack for a production project. For a hobby project, it's impressive how much of a typical web stack can be eliminated; I think this could be a boon for solo and aspiring devs on side projects. I plan to check out some of the newer async-first kids on the Python-web-framework block, but I know I'll miss Django's batteries.

For instance, Python coroutines do not run unless await'd or explicitly scheduled, unlike Javascript promises, which are automatically scheduled. ↩
Perhaps the content could have been JSON encoded in the template, and parsed in the Javascript. ↩

Security Stories Part 4

2026-01-19T00:00:00-05:00

I am back once again with another entry from my archive of security stories. Once more, the purpose is education: as an engineer, it pays dividends to be curious and think critically about the software one is writing and maintaining.

Token interchange

One company I worked for leveraged magic links: the "click this link to log in" in an email that signs you in without needing to provide your password. The security basis for these links is the presumption that access to email justifies access to external accounts using the email. After all, if I control the email for victim@example.com, to access their account on foo.com I need only reset their password via email.

Let's set up a scenario so that the exploit path may be described: our imaginary company develops a job board, where organizations post job listings, and candidates apply, with their applications managed within the job board itself. The company decided to improve the candidate experience through magic links. To log in, an applicant is sent an email with a magic login link. After applying, an applicant is sent a magic link to access, update, or withdraw the application.

I had been suspicious of these magic links for some time, as any security minded engineer should be wary of non-standard authentication schemes. It was in reviewing a new feature related to magic links that I realized a core insecurity. First, some basic background about the implementation of magic links. They are typically of the form:

Bash

https://foo.com/login/magic/<token>

where <token> may be a randomly generated string mapping to a stored object (think cache or db), or a hash of an id (think hmac of user id), or an encoding of an id (think encrypting the id), or a combination (think jwt). This company followed the encoded id approach, that is, the <token> translated to an object id.

In studying the new feature, I made the connection that tokens could be used interchangeably between the two magic link systems, because the ids were not namespaced, i.e. encoded alongside the resource type. That meant a token to login user 1 could be used to access job application 1, and vice versa: an application magic link token for application 72 could be used to login as user 72.

The exploit lended itself to recursive scraping:

create a new account, user N
use user id N token to access application N
access application N owner account, user M
repeat step 2 with M

Given that an attacker may create many new user accounts, this means mass account takeover and data scraping. Ouch!

The key to securing these magic links was, as mentioned, namespacing the ids to prevent re-use. This looked like encoding the resource type alongside the object, in effect encoding the tuple (object id, resource type). For the user authentication tokens, that meant (<user id>, "user::login"), and for the application tokens, (<application id>, "application::crud").

Each consumer of tokens would check to ensure that the input token was of the appropriate resource to access the underlying functionality; e.g. the TokenLoginView would check that the token was of type "user::auth" before authenticating the given user.

As the magic links were part of a live system, with existing magic links in play, the migration from the old to the new token schema was a key challenge. In effect, the new token schema was rolled out alongside support for the previous, itself removed after an agreed-upon expiration timeline.

This vulnerability underlined the importance of the common phrase, "don't roll your own crypto." Here I'd amend that phrase to "don't roll your own authentication." As with all such phrases, it's not a commandment, but intended to share the common wisdom of warnings from the generations past that "here be dragons" when going down such paths.

Security Stories Part 3

2026-01-16T00:00:00-05:00

I am back to share another security story from my archive. In addition to the pleasure of sharing my experiences, I write so that security may be taken more seriously by software organizations. It's all too easy to __dangerouslySetInnerHtml and call it a day; it takes active intent and investment to secure software systems.

Rogue templates

At one company I worked for, there was a vast array of legacy internal software. One particular piece of internal glue code had caught my eye: using the django templating engine over user provided inputs. To avoid particulars, let's say, for example, that the relevant feature was built to send an email each time a user registers. A user registers for the platform with a name and profile blurb, which should be included in the email to admins.

Now, the secure way to do this would be to create an email template:

Text Only

{% include "email_header.txt" %}

User {{ username }} registered! Their profile blurb is:

{{ blurb }}

{% include "email_footer.txt" %}

and render it, using context to safely interpolate user input:

Python

from django.template import engines

user = User(username="tmoney", blurb="Happy to be here")

django_engine = engines["django"]
template = django_engine.get_template("user_registration_email.txt")
rendered = template.render({"username": user.username, "blurb": user.blurb})

Suppose that a user provided a malicious blurb. At first, thinking about the template engine, I figured the worst would be a denial of service attack via resource exhaustion, e.g.

Text Only

What I hear on the news:

{% for i in "x"|rjust:"10" %}
  .. repeat for loops ..
  blah
{% endfor %}

In the safe rendering approach, the template code in this blurb would not be executed, just safely output as-is into the email. This wouldn't be a security story if the company had used the safe approach, of course! Instead, the email templating code was written to include the blurb as part of the raw template, as in:

Python

from django.template import engines

user = User(username="tmoney", blurb="Happy to be here")

django_engine = engines["django"]
template = django_engine.from_string(f"""
    {{% include "email_header.txt" %}}

    User {user.username} registered! Their profile blurb is:

    {user.blurb}

    {{% include "email_footer.txt" %}}
""")
rendered = template.render()

With this approach, the malicious blurb template code is executed by the template engine. Assuming that the worst that could be achieved is resource exhaustion, I filed this away as a low priority vulnerability. But it kept creeping up in my mind... knowing that django templates can call methods, and thinking about the django orm, if I could get a reference to a django model then I could easily chain that model to related models or possibly the entire table via model querysets.

After more digging, I finally found it: get_admin_log. This is automatically available in django templates when the admin site is enabled and vulnerable when the admin site is used - and you are using it, right, that's part of the django batteries included deal! ¹

This template tag queries recent admin actions (specifically, the LogEntry model), think creations, edits, deletions. Handily, it links to the user who made the change, and that user is presumably a superuser, and presumably has links to most other objects in the system... now we are cooking!

Let's think through some of the possibilities for exploitation. With access to any User object, it's easy to access all user objects, via django meta model magic: user._meta.model.objects.all(). From there, one may head straight for deleting objects, including following the model meta magic train to related models and wiping much of the database.

Fortunately, django is ahead of us on this front. First, "private" properties beginning with underscores are not accessible in template lookups. Second, common side-effect methods like queryset.delete() or model.delete() are marked with a special flag that prevent execution in template rendering (via alters_data, see the docs).

That excludes the most obvious exploit of wiping tables via the standard django orm. It doesn't exclude rendering data, though, in the case that attackers are able to view the results of the injected template code. And, given the relative obscurity of template injection protection (blacklist unsafe methods, rather than whitelist safe methods), it doesn't exclude developer defined model methods. It's common to define helper methods that overlay the very same side-effect methods like model.delete(), and such helper methods would be executable within the template.

Let's continue with the example, supposing that there's a custom method User.deactivate(), that is typically run when a user deactivates their account. Not thinking about the potential attack vector of template injection, this method is not marked unsafe for templates. Now, an attacker may provide a malicious blurb that deactivates admin users, locking them out of the system:

Text Only

Nothing to see here :D

{% load log %}
{% get_admin_log 100 as log %}
{% for entry in log %}
  {{ entry.user.deactivate }}
{% endfor %}

This is just one example. The true attack surface depends on the context provided to the rendering call (e.g. the current user), and the methods available on objects available within the context (whether directly provided or provided via template tags).

I'd classify this vulnerability as code injection leading to limited remote code execution. What made this stand out to me was the seemingly innocent django templating engine: with a naive mindset of, "it interpolates user inputs when rendering", an engineer misses the potentially catastrophic consequences of running the template engine on unsafe inputs.

Therefore my main lesson here is to always be mindful of processing untrusted input. Another important lesson is that if it seems insecure, then it's probably insecure - dig in, talk about it, and fix it. Engineering teams should cultivate a positive, blame-free culture to investigate, report, address and educate.

In Django 5.0+, get_admin_log fails outside of admin views. This appears to have been (incidentally?) patched out. Note that this is just one, default-provided attack vector. It's common to provide context to render calls, and that context is likely to have access to model objects. ↩

End of Year Review (2025)

2025-12-31T00:00:00-05:00

December rolls around once more. It's time to reflect on the year gone by, and the year upcoming. This is the third installment of my end of year reviews.

This was a year of challenges, struggle, inspiration, burnout, revelation, misery and hope.

As a friend put it, God gives us challenges the moment we're ready to face them.

Travel

I traveled to Europe twice this year, once for a work + pleasure trip to the UK and Ireland to attend DjangoCon EU, and then a vacation to Greece. I had a whirlwind time in London, trying to squeeze everything into a few short days; old town Edinburgh was rather sham-touristy and dreary; in Ireland I looked out over the Atlantic from the Cliffs of Mohre. In Greece, I walked alongside my ancient heroes, Socrates, Diogenes, Hercules. I traveled independently, facing my fears and making a splendid trip for myself.

Books

I read 15 books this year. My highlights:

Thinking in Systems
Excession
Nexus

Writing

I exceeded my goal, writing at least two posts per month. Some posts that I am particularly proud of:

Security stories series: part 1, part 2
Pytest, playwright and Django
Visualizing my Wikipedia Browsing History

Career

I am ending the year having resigned from my job, with the aim to take several months break. I have plenty to write on the experience, so suffice it to say that the greatest surprise for me is the boldness and courage I demonstrated in taking this chance for myself to spend time with myself, get to know myself, and care for myself. In the words of Lester Burnham:

It's a great thing when you realize you still have the ability to surprise yourself. Makes you wonder what else you can do that you've forgotten about.

Personal development

This continued to be my greatest theme: at one point, I estimated I have been spending about 15 hours a week between journaling, therapy, yoga, meditation, and support groups. Some highlights of this year's achievements:

Navigating the end of close relationships
Avoiding getting into unhealthy relationships
Developing new, healthy relationships
Setting boundaries, especially at work
Deep parts work
Nurturing self love and compassion
Decorating my apartment
Yoga class
Volunteering at a rehab center

So much of this work pays dividends in hard to measure ways; without direct comparison to the non-recovered version of myself in the same scenario, I have to validate myself in my new behaviors. Many times I am even taking the same action, but it's my relationship to myself and the behavior that has changed.

Goals

Looking back, I hit many of my goals from last year, notably:

✅ Physical health (joined a yoga class)
✅ Career (blogging regularly, conference attendance)
✅ Relationships (nurturing supportive community)
✅ Travel (multiple international trips)

For the upcoming year, I feel less compelled to make a list of specific targets. On reflection, this has been my trend: from a very long list, to a short, thematic list, to a more flexible approach. I've learned, and proven to myself many times over, that I succeed at strict task-oriented projects. My next challenge is to learn to navigate and live in ambiguity. Specifically, and quite generically, I want to practice living with intention. So much of my default behaviors tend towards productivity-, resource-, or task-oriented mindsets. As a practical example, rather than starting a Saturday with a todo list, I want to check in with myself throughout the day, and act in a way that honors myself, honors the "God of loving kindness", not the "God of productivity."

To that end, I want to trial the following behavioral changes:

Mindful eating: lose the technology, maybe wean off through podcasts or paper books and magazines
Mindful wake up and bed time: same deal, with emphasis not on the "perfect routine" but what works for me

As for more visible goals, I'd like to end the year working a hybrid job in the city, living in a new apartment, having continued reading and writing and traveling and making new experiences for myself.

Wrap up

This was one of the hardest years of my life, it's undeniable - as much as I want to deny it! I plan to reward the resilience that I've demonstrated with the time to process and heal. Even writing this post late (thanks stomach virus on new years eve!), and feeling the brunt of depression manifest, I feel my conscious contact being restored, and with it gratitude and hope for the new year.

OR-Tools: Getting Help

2025-12-08T00:00:00-05:00

I've been programming with google's OR-Tools library for the past few years, working on solvers for the vehicle routing problem, scheduling problem, and set partitioning problem. As a developer without a background in optimization, the OR-Tools API is not intuitive, so I've had to pay my dues over the years crawling through documentation. It's been my experience that documentation is scattered; to that end, I thought it could be helpful to share the repositories of knowledge I've used, and how I use them. As of now (late 2025), llms are generally poor at generating code for OR-Tools, and given the complexity and subtlety of the software, it's good to verify generated code in any case.

Where to look

For general searches and problems, I'll start with the github and discord. Failing that, I'll search the google group archives. Google searches sometimes surface helpful stackoverflow posts. API changes over the years mean that old solutions sometimes need to be translated into new code structures.

If I'm unable to make a dent in the problem, I'll ask in the discord - the OR-Tools developers and community are very friendly and have helped me several times. Don't expect an instant reply - it usually takes a day or so to hear back, though don't be surprised if you get a gist with a solution fully worked out.

For the vehicle routing problem specifically, the routing.h file includes tons of helpful documentation, going beyond individual functions to strategies for solving problems:

routing.h

I'll also shout-out the plethora of examples in the repository:

samples

For more in-depth reading, and some sample problem solutions:

uvx flask

2025-11-24T00:00:00-05:00

This is a neat little trick that I couldn't find documented online. I am working on a small, single-file flask application and looking to keep my workflow as simple as possible. To that end, I am using uvx (shorthand for uv run) to manage my dependencies in an implicit virtualenv:

Python

# /// script
# dependencies = [
#   "Flask"
# ]
# ///

This new file format allows me to skip requirements.txt and pyproject.toml, which may not seem like much but keeps my entire software application in one file.

To run the flask devserver, I added the following code:

Python

if __name__ == "__main__":
    app.run(debug=True, port=PORT)

And run with uvx app.py.

While this is great for development, I wanted to use gunicorn for production, which presented a challenge: gunicorn is its own command line tool and expects an wsgi application. Typically, running gunicorn for flask looks like:

Bash

$ gunicorn --bind 0.0.0.0:PORT --workers N app:app

Naively, I tried:

Bash

$ uvx gunicorn --bind 0.0.0.0:PORT --workers N app:app

but with this invocation, uvx only installs gunicorn into the virtualenv, and ignores dependencies in app.py. Browsing the docs, I found uvx --with-requirements <reqs...>, which supports requirements.txt - but moving requirements out of the single file defeats the beauty of my single file application!

On a hunch, I wondered if uvx --with-requirements <reqs...> could read the requirements from the script itself (via the inline metadata), while running a separate tool, as in uvx --with-requirements <script.py> <tool cli> .... This worked! It's possible to define my requirements just once, and run a different tool with uvx, leading to a true single file application experience:

Bash

$ uvx --with-requirements app.py gunicorn --bind 0.0.0.0:PORT --workers N app:app

This approach also works with the flask cli, meaning that app.run() can be removed. Tada 🎊

Bash

$ uvx --with-requirements app.py flask run --port PORT

tl;dr

uvx is a nifty tool for creating true single file applications (dependencies and all), extending to software served by other tools (e.g. flask web applications). The value of a single file application is subtle: easy to share with others, easy to ship, easy to work with an ai agent.

To run a tool on a script with the script's dependencies:

Bash

$ uvx --with-requirements <script.py> <tool cli>

Parameterization

2025-11-10T00:00:00-05:00

Recently I was able to leverage parameterization to great effect, and I'd like to share the experience. I maintain an internal application that is run daily and produces some necessary logistics data for our operations. In extending the application, I was asked add support for running the tool for future dates.

Easy configuration

With several years of experience under my belt, I know that time is tricky to get right with software. At work, using Django, it's typical to rely on utilities like timezone.now() and timezone.localdate() to access the current time or date. As there's no restriction on calling these utilities, they may be used in deeply nested call trees. As the concept of a local date relies on global or implicit state (i.e. the active timezone), there's necessary implicit configuration required for the code to work correctly - and when the active timezone can be manipulated in the same fashion, down the call tree, it's easy to end up in a quagmire (yes, I've seen timezone.activate(tz) used in a loop, leading to a random timezone ultimately activated at the end).

So when I was asked update my application, how much did I suffer in chasing down time related bugs? None (mostly). And how is that, you wonder? Did I leverage more global implicit behavior like overriding the computer time? Nope. I had, long ago, already parameterized the application on date.

By parameterizing, I mean adding explicit function parameters. E.g.

Python

def get_orders_for_today(today):
    return Order.objects.filter(date=today)

Instead of:

Python

def get_orders_for_today():
    return Order.objects.filter(date=timezone.localdate())

In my application code, the top level function (interface) accepted a date parameter and passed it down the call tree. The date had been previously passed to the function with timezone.localdate() - I swapped this for reading the date from user input. I verified that there was no implicit behavior based on the date by following its usage. And I was able to get this feature done in an afternoon instead of a week.

Easy testing

Testing also benefits from parameterization, as an alternative to monkeypatching. As an example, at work, we have an optimization problem solver which calls out to a native library and typically runs for several seconds. To test this code, the relevant solver() function could be monkeypatched to skip the processing and return some value. The alternative approach that I used is called the strategy pattern, where the algorithm is passed in as a parameter. Suppose I have the following interface:

Python

class ISolver:
    def solve(problem):
        ...

Then, I can implement the native solver and a stub solver:

Python

class NativeSolver(ISolver):
    def solve(problem):
        time_consuming_solver(problem)

class StubSolver(ISolver):
    def __init__(self, solution):
        self.solution = solution

    def solve(problem):
        return self.solution

Yes, this could be improved with typing and protocols, but that's besides the point here

Finally, I'd parameterize my application code:

Python

def scheduling_problem(problem, solver):
    ... # call solver(problem)

...

solver = NativeSolver()
scheduling_problem(problem, solver)

And to test:

Text Only

def test_scheduling_problem():
    problem = ...
    solution = ...
    solver = StubSolver(solution)
    scheduling_problem(problem)
    assert ...

Admittedly, I've had less success spreading the gospel on the benefits of the strategy pattern, the core critique being the boilerplate code. That's fair. I have only used this technique a handful of times, most commonly when there are multiple strategy implementations besides the test mock.

Closing thoughts

I recognize this technique isn't revolutionary, as it's really just a simple application of programming principles: don't rely on or mutate implicit or global state. As with most timeless advice, it's often ignored and quite beneficial.

Scratch the Itch

2025-10-23T00:00:00-04:00

Today I had the opportunity to practice a favorite technique: scratching the itch. By this I mean I let my curiosity take charge, quieting the know-it-all that typically runs the show.

The itch

I was working on a section of frontend typescript code. I had noticed a few days before that my editor was complaining that a type was undefined. That is, for

TypeScript

let x: Foo;

my editor put squigglies under Foo and complained. This isn't unusual, as I am used to typescript errors appearing temporarily when switching between branches: the in-memory diagnostics haven't caught up to the filesystem changes yet. Besides, this code was already merged into develop, with CI checks passed. It must be my editor.

It must be, right? I suddenly felt the itch. The itch is molded with experience, but experience isn't required: the itch may better be framed as childlike curiosity. During service at my Buddhist monastery, I realized with delight that children in the audience asked questions that I was too ashamed to ask. And how does that shame serve me, when it holds me back? Adopting a position of innocence and curiosity can be a rewarding endeavor.

To scratch this particular itch, I actually looked at the error message: cannot find name Foo. I won't be too proud to admit that I sometimes glance over error messages, jumping to conclusions with a presumed root cause. Here was my (abbreviated) thought process:

This error message was strange, how could Foo not be found? Let me check the editor: reload the file. Error still present. Okay, is Foo not defined? Indeed, it is not! How does that make sense, if type checking in CI is passing? Let me run type checking outside the editor, with the CI command. Oh that's odd - the type checking command instantly succeeds. I'd expect it to take a few seconds - it's a hint when a tool succeeds too quickly. Let me check the typescript config.. hmm, I'm not familiar with this config approach, but I recall that we recently migrated from webpack to vite - let me ask gpt quickly. Aha - gpt suggests that running tsc with this tsconfig will not perform project type checking. That explains why typescript errors are present on develop - CI was passing on broken code! Ah, with the build flag, tsc should work.. yes, I now see the typescript errors!

This took about ten minutes. There are a dozen engineers on the team, and the misconfigured typescript check had been present for three months. It's not a dig on the team that it wasn't found until now - it's a signal of how much differentiation this small technique can provide for an engineer.

Lessons

The story is representative of years of applying this technique. I'll summarize a few takeaways/lessons:

When something seems off, investigate. I ignored the error for several days before taking a look.
Double check assumptions. I assumed that the problem was local, because CI must be passing.
Time box. I stopped at identifying the broken type check command - surfacing the issue is valuable, and my teammate who configured the new system will be better suited to fix it.

Most of the time, these investigations don't pay big dividends: it's usually my implicit assumptions that are proved wrong in the end. That's fine, the active learning process itself is rewarding, as I hone my understanding of the system in addition to my investigation skills. Sometimes, though, the payoffs are big. As an example, I once found a password bypass in production when I scratched an itch.

Visualizing my Wikipedia Browsing History

2025-10-13T00:00:00-04:00

Backstory

I have been intentionally changing my internet browsing history the past few years, aiming to reduce endless scrolling on "junk food" sites. This year in particular, I quit reddit and twitter, and have instead been reading just the news and wikipedia. This has worked as a gradual reduction in satisfaction: the news, while interesting, lacks the same level of excitement as a silly debate on reddit or "hot take" from a personality on twitter. I regularly browse the Wikipedia "Did you know..." and "This day in history," finding it scratches the itch of new content, while again lacking the compulsive pull that lost me countless hours previously.

As a result, I have visited about 10,000 wikipedia articles. As a creative programmer, I was inspired to dive into this data set - could I visualize the connections between pages, for example?

Approach

Two books played a large role in this project coming to fruition. Nexus by Mark Buchanan explores network theory, explaining various network topologies and applications to the real world - hence my interest in modeling my browsing history as a network. On the creative side, I recently started The Artist's Way by Julia Cameron, with the intention of recovering the artist within me. One key challenge for me is perfectionism: I bring my professional background, but more than that, I bring high standards to what are casual and fun projects.

I think that llms can be a boon for creative blocks and perfectionism. I started a chat with a prompt that explained my default tendencies, such as over-engineering, getting lost in details, and struggling to finish and publish. I then worked with intention, describing my actions step by step, and getting helpful feedback and perspective - the llm caught a few moments where I almost went down a rabbit hole. Most notably, the llm suggested I call the project done and write this article. I believe that llms can be utilized to great benefit for people skilled at introspection: knowing my blindspots and tendencies, and communicating with honesty and accepting the feedback, I'm able to break unskillful patterns of behavior.

Technical details

The high level technical approach was as follows:

Get my wikipedia browsing history
Build a network data structure where articles are nodes and links between articles are edges
Visualize the network

Browsing history

I don't have an account on wikipedia, but I don't believe that browsing history is tracked with an account anyway. I do use firefox with sync across desktop and mobile, which conveniently syncs browsing history - important because I visit wikipedia on multiple devices. I therefore built a browser extension, which enabled retrieving history, and a convenient interface for visualizing in the browser via extension pages.

The extension uses a browser action to open an extension page - i.e. the toolbar extension button opens a new tab with content from the extension. The technical details for this approach are scattered across the mdn docs and examples, here's the basics.

Here's my manifest.json:

JSON

{
    "manifest_version": 2,
    "name": "Wikitrack",
    "version": "0.1",

    "description": "Visualize wikipedia browsing history.",

    "browser_action": {

    },

    "background": {
        "scripts": ["background-script.js"]
    },

    "permissions": [
        "history"
    ]
}

The relevant bits are defining the browser_action entry (it can be empty for a simple extension) and the background scripts. Here's background-script.js:

JavaScript

function listener () {
    browser.tabs.create({
        url: "page.html",
    });
}

browser.browserAction.onClicked.addListener(listener)

This defines a listener on the browser action, opening my page.html when the toolbar button is clicked. Great. The html is basic and references a javascript file that grabs the history (because extension pages have access to the same permissions as the extension):

JavaScript

function onGot(historyItems) {
    console.log("Found " + historyItems.length + " wikipedia pages visited");
}

browser.history
    .search({
        text: "wikipedia.org",
        startTime: 0,
        maxResults: 1000000,
    })
    .then(onGot);

Awesome! I am able to get a list of all wikipedia pages that I've visited.

Network

With the browsing history, the next step was to build a network data structure that included links between pages as edges. To get the links, I needed to get each article's content. There were a few approaches that I could've taken, the simplest most likely scraping the live wikipedia site to retrieve each article's content. At the scale of 10k articles, mindful of rate limits, and maybe a little overengineering, I decided to take the approach of parsing the full english wikipedia data dump instead.

This came with some challenges: as the data dump is 100gb decompressed, the suggested method is to use an index into the compressed file, and decompress only what's necessary. This seemed technically challenging and therefore a potential rabbit hole, so I chose to decompress the file and stream parse the xml. I tried various tools, including xmlstarlet, xmllint, and python's builtin xml parser, before settling on lxml: with stream parsing, lxml processed up to 50k articles per second. At roughly 25 million articles total, processing the entire 100gb xml file took about 9 minutes. This was perfectly reasonable for my use case: find articles that I've visited and output the article content.

Here's the script that stream processes the xml, writing a jsonl file with article contents:

Python

import json

from lxml import etree
from tqdm import tqdm

page_estimate_count = 25_040_290
visited_page_titles = set()

pages_found_in_xml = set()
pages_found_in_xml_count = 0


with open("history.cleaned.json", "r") as history_file:
    ## history.cleaned.json is a json list of article titles, cleaned and filtered (e.g. special pages removed)
    history_content = json.load(history_file)

    visited_page_titles = set(history_content)

print(f"Looking for {len(visited_page_titles)} pages")

with open(
    "wikidata/enwiki-20251001-pages-articles-multistream.xml", "rb"
) as xml_file, open("history.pagecontent.jsonl", "w") as pagecontent_file, tqdm(
    total=page_estimate_count, desc="Processing pages", unit="pages"
) as pbar:
    for event, elem in etree.iterparse(xml_file, events=("end",)):

        if elem.tag.endswith("page"):

            # get title + text
            title_el = elem.find(".//{http://www.mediawiki.org/xml/export-0.11/}title")

            text_el = elem.find(".//{http://www.mediawiki.org/xml/export-0.11/}text")

            if title_el is not None and text_el is not None:
                page_title = title_el.text
                page_content = text_el.text

                if page_title in visited_page_titles:
                    pages_found_in_xml.add(page_title)
                    pages_found_in_xml_count += 1

                    pbar.set_postfix(found=pages_found_in_xml_count)

                    pagecontent_file.write(
                        json.dumps(
                            {
                                "title": page_title,
                                "content": page_content,
                            }
                        )
                        + "\n"
                    )

            pbar.update(1)

            # Free memory as you go
            title_el = text_el = None
            elem.clear()

            # Remove references from parent to enable garbage collection
            while elem.getprevious() is not None:
                del elem.getparent()[0]

print(f"Found {len(pages_found_in_xml)} pages from history in the xml")
print()

Of note, stream processing via .iterparse() does not free memory automatically. Instead, memory for each element has to be explicitly freed via elem.clear() - however this only frees its children, not itself, so the references to the element from the parent also have to be explicitly removed. All together, this script uses 40-80mb of memory, quite impressive for processing a 100gb file!

tqdm, with the ability to specify a total and custom postfix content, provided an excellent interface:

Bash

% python graph.py
Looking for 9051 pages
Processing pages:   3%|██▋                         | 689010/25040290 [00:26<11:13, 36150.31pages/s, found=3553]

The next script parsed each article's content using mwparserfromhell, extracting links, and built a simple network data structure, mapping each article to its linked articles (limited to articles I've visited, i.e. no new nodes are introduced):

Python

import json

from collections import defaultdict

import mwparserfromhell


def is_valid_link(link):
    # Filter out images, categories, and non-internal links
    return not str(link.title).startswith(
        (
            "File:",
            "Category:",
            "Talk:",
            "User:",
            "Template:",
            "Help:",
            "Portal:",
            "Draft:",
            "TimedText:",
            "MediaWiki:",
            "Module:",
            "Special:",
        )
    )


pagecontent = []

with open("history.pagecontent.jsonl") as pagecontent_file:

    for line in pagecontent_file:
        pagecontent.append(json.loads(line.strip()))

full_graph = defaultdict(set)

for page in pagecontent:
    wikicode = mwparserfromhell.parse(page["content"])

    links = []

    for node in wikicode.filter_wikilinks():
        if is_valid_link(node):
            links.append(node)

    full_graph[page["title"]].update(str(link.title) for link in links)

interested_pages = set(full_graph.keys())
interested_graph = defaultdict(list)

for interested_page in interested_pages:
    edges = full_graph[interested_page]
    interested_edges = edges.intersection(interested_pages)

    interested_graph[interested_page] = list(interested_edges)

with open("history.graph.json", "w") as graph_file:
    json.dump(interested_graph, graph_file)

After about a minute, I have my graph data structure, ready to visualize!

Visualization

For visualization of a network in the browser, I first tried vis.js, but it choked under the load of 10k nodes and 20k edges, so I went with Sigma.js, which is better suited for this network size. Keeping it simple - no webpack or bundling - I downloaded the graphology and sigma js and loaded them directly into the html. It's amazing how fast development can be without all the cruft of modern tooling!

For loading the data, again I took a simple approach and manually edited history.graph.json to a javascript file that exports the json, and loaded the javascript file into the html. The code to render the graph is thus:

JavaScript

$(document).ready(function () {
    const container = document.getElementById('network');

    // graphData is exported implicitly via js file

    // sigma.js
    const graph = new graphology.Graph();

    // Add nodes
    Object.keys(graphData).forEach(node => {
        graph.addNode(node, { label: node });
    });

    // Add edges
    Object.keys(graphData).forEach(source => {
        graphData[source].forEach(target => {
            graph.addEdge(source, target);
        });
    });

    // Position nodes on a circle, then run Force Atlas 2 for a while to get proper graph layout:
    graphologyLibrary.layout.circular.assign(graph);
    const settings = graphologyLibrary.layoutForceAtlas2.inferSettings(graph);
    graphologyLibrary.layoutForceAtlas2.assign(graph, { settings, iterations: 600 });

    const renderer = new Sigma(graph, container);
});

Voila! From 0 to a visualization in a weekend.

Closing thoughts

I actually did rediscover some of the joy that I used to have with programming, way back in college when I "didn't know better": poorly written scripts that glue together through tedious manual processes, pulling in dependencies in the simplest way possible (pip install and downloading the js), settling for "good enough" hacks that get the job done. "Done is better than perfect" comes to mind, as I can publish this article today instead of never.

Some follow-up projects that come to mind:

Improved visualization and exploration of visualization techniques
Professionalize the project, bundle and ship it for others as an extension and web api
Building an interactive network UI, with filtering and timeline support
Clustering and labeling
"Missed connections": suggest articles that are related to articles I've visited
Sessions: visualize individual browsing sessions

More to come?

Data sources

This project uses data from Wikipedia, available under the Creative Commons Attribution-ShareAlike License. The English Wikipedia database dump was obtained from Wikimedia Downloads.

Layers

2025-10-10T00:00:00-04:00

I'm back from my trip to Greece. Three weeks of travel feels like a year; in measurements: 4 cities, 5 beds, 3 flights, 2 ferries, 1 car rental, 5 beaches, 5 museums. Greece felt to me like a land of contradictions, rich and poor, fast and slow, punctuality and delays, bustling and quiet. My experience was much the same, contradictory in many of the same ways. It all blended together to give me time, space and a mirror for self reflection.

History

I was dismayed to learn that the frescoes were re-imagined from fragments - yet how different is this from recollection and interpretation of my own fragmentary memories?

Browsing the galleries of the archaeological museum in Heraklion it suddenly became obvious to me the parallels between human history and my own. For my entire life I've other-ized myself, put myself apart from people, constructed an internal framework where I am special and unique. My ego ran the show, judging and measuring, comparing and differentiating itself. My ego believes itself to be unlimited in its potential, and views others as tools to be used to achieve a greatness that would reflect its desired self image back at it: that of power, and control.

In the history of the earliest European civilization, I was ready to be humbled. Just as the course of a civilization is determined by causes and conditions inside and outside of its control, so is my own course of life. Just as thousands, millions and even billions of people live their lives determined by causes and conditions inside and outside of their control, so do I. A part of me that was holding onto my ego was able to slacken, and I feel the pressure decreased.

Origins

For decades I thought I understood my childhood. I was able to accept that it was out of the ordinary, if only based on the amount of time it took to explain the rotating cast of characters and scene transitions. A gift of insight that I've received is a new perspective, one which has been the key to a transformation of my inner and outer worlds.

At a very young age, I lost my family. I don't know exactly when or how, partly because of my age at the time, and partly because the participants are unable to speak of it, whether through death or denial. I thought I understood this event in my childhood, despite my ignorance, but as time goes on, I am coming to see how deeply this affected me. It seems simplistic to trace the course of my life through one event, a sort of fatalism - I don't believe the future had been fixed at that point, anymore than it is right now - but I do believe that some events or processes dominate the trajectory of the future.

In response to this overwhelming tragedy, I can appreciate how my ego grew outsized, in an attempt to protect a vulnerable child. I can appreciate how my ego became reinforced with the notion that other people are dangerous and cannot be trusted. I can appreciate how I came to see myself as fundamentally different than other human beings.

Change

I struggle to feel calm, safe and stable in my life. Even simple acts like sitting in the park spiral into future or guilt tripping, anxiety taking over. As I've come to learn, this is an inability for my nervous system to relax, put another way for myself to turn off for a while. I can intellectually understand that I am in a completely safe situation, yet my body does not feel it.

Form and content: my intellectual framework and emotional reality grow and change at different rates in different times. An intellectual understanding of my childhood does not immediately yield an emotional transformation. On the other hand, consistent effort towards healing emotional wounds is not immediately noticed: travels are a reset that enable me to see my growth.

For my entire life the effort to heal has been a complete and total mystery. I was raised with unhealthy expectations around learning, I was expected to "get it" on the first try, by myself - to ask for help risked wrath instead of vulnerability. Taking this into my adulthood, I tried and failed countless times, using the result as proof of my deficiencies. I'm now learning how to learn, as I'm shedding these old mental habits and gaining emotional resilience.

I am realizing that repetition and initial failures are part of the learning process. Approaching new activities or topics with calm and curiosity serve me better than anxiety and insecurity. Learning with others can be a source of inspiration and energy, not just judgment.

Layers

Moreover, I am realizing that mindful repetition is key to healing the deep, emotional parts. A Buddhist monk shared about this recently, saying that it's now well understood how sensations build into feelings into perceptions and finally into conscious thoughts. She followed up that the opposite may be practiced: from thought down to feeling and sensation. This is loving kindness, mindfulness, equanimity, reframing, etc. - applying the mind to influence the heart.

The key revelation for me is the repetition. I am so used to expecting of myself instant results. Faith is the belief that change and growth are possible, even when I can't comprehend it. As another monk shared once, faith can be as simple as believing that someone is further along the path than I am, that if they can do it, so can I.

The training of the mind and heart takes time, effort, and repetition. For matters of the heart in particular, there's no guarantees on timeline. The Buddhists like to say that it takes decades of consistent practice to achieve any meaningful progress towards enlightenment. I think the point about the duration is not to discourage the practice, but the ego. I am coming to see, in glimmers of insight, that the practice itself is the enlightenment, both the goal and the destination. That it takes a lifetime to achieve - well, it's a lifetime of practice that I can learn to appreciate and enjoy.

Investigating the Pytest Django Teardown Race Condition

2025-09-17T00:00:00-04:00

Background

In my previous post about pytest, playwright and django, I described an annoying non deterministic failure on test teardown. I believe the issue is a race condition that occurs when the test database is flushed, while network requests are pending. The reference issue agrees with the theory but does not detail a root cause, nor are the solutions insightful or practical.

I let this issue simmer; the band-aid of explicitly waiting for network requests to finish at the end of each test mostly worked. It was far from satisfactory, however, being a guard that developers would have to mindfully add to every test. Over time, changes in network requests also meant failing tests. In short, my team reached the tipping point at which a more thorough solution was necessary to continue investing in playwright tests.

Investigation

In past investigations, I placed a degree of trust in the library code. After all, doubting the code that I depend on can lead to some significant trust issues. This time around though, I was determined to answer the question that bugged me: why are there unfinished requests at test teardown? To find the answer, I had to look into the source for pytest-django, django, and the python standard library.

pytest-django provides the live_server fixture, which exposes an http server backed by the django application on a local port. This fixture delegates to the LiveServer class, which mirrors the LiveServerTestCase class provided by django, re-implemented to work as a pytest fixture. The LiveServer class starts a LiveServerThread (also provided by django), which is responsible for actually running the http server.

Ok, still with me? The LiveServerThread is terminated at the end of the live_server fixture. In theory, then, pending requests should be forcefully terminated. Ah, but that would only be the case if LiveServerThread was responsible for processing requests itself - in fact, it isn't. The default server_class is django's ThreadedWSGIServer, which processes each incoming request in its own thread - meaning that the server is capable of concurrent request processing. The server class is powered by the python standard library, leveraging socketserver.ThreadingMixIn for the per-thread behavior.

Notably, ThreadedWSGIServer configures the request handling threads to be daemons, per the docs:

When inheriting from ThreadingMixIn for threaded connection behavior, you should explicitly declare how you want your threads to behave on an abrupt shutdown. The ThreadingMixIn class defines an attribute daemon_threads, which indicates whether or not the server should wait for thread termination. You should set the flag explicitly if you would like threads to behave autonomously; the default is False, meaning that Python will not exit until all threads created by ThreadingMixIn have exited.

This means that, while the LiveServerThread will exit cleanly, any pending request threads will not be joined / waited upon to finish. Mystery solved? If any of those requests are interacting with the database, there's the possibility for a deadlock between the request thread and the main thread attempting to flush the test database.

A possible solution

A relatively straightforward, if not pretty, solution is to override the configuration of ThreadedWSGIServer in a live_server fixture override:

Python

from django.core.servers.basehttp import ThreadedWSGIServer

@pytest.fixture
def live_server(live_server):
    # wait on requests to finish before shutting down the server
    ThreadedWSGIServer.daemon_threads = False

    yield live_server

This change will result in the server waiting for request thread completion before itself shutting down. As it's isolated to the test suite, it's not the most abominable monkeypatch imaginable, but I'd prefer a cleaner solution whereby pytest-django would support customization of the underlying LiveServerThread, to modify the server_class (I believe this is better supported by django's unit test framework).

Unfinished business

This investigation cleanly answers my original question: why are there unfinished network requests at test teardown? The requests are handled in separate, daemon threads that are not stopped / waited on when the live_server fixture cleans up.

In the pytest-django source though, I noticed another behavior: the live_server fixture is session scoped by default. This means the fixture provides the same LiveServer instance for every test in a given run. Therefore the server class won't stop until all tests have run, or put another way, those request threads won't be joined at the end of each test, so there's still the possibility for the race condition.

To solve this, my untested ¹ theory is to make the live_server fixture function scoped, which means that the LiveServer class will stop after each test case, joining all request threads.

This could be achieved by overriding it in my project conftest, copying the relevant contents from the pytest-django source:

Python

from django.core.servers.basehttp import ThreadedWSGIServer

@pytest.fixture(scope="function")
def live_server(request: pytest.FixtureRequest):
    """Run a live Django server in the background during tests.
    Copied from pytest-django source. Simplified based on project usage.
    """
    # wait on requests to finish before shutting down the server
    ThreadedWSGIServer.daemon_threads = False

    server = live_server_helper.LiveServer("localhost")

    yield server

    server.stop()

Put together, this approach should resolve the underlying race condition: at the end of each test, the http server will be stopped, waiting for pending requests to finish. When the database flush runs, there will be no concurrent access.

I am on vacation currently! ↩

Greece

2025-09-16T00:00:00-04:00

I am currently writing this from a breezy apartment in Heraklion, Greece. I'm spending a little over two weeks in Greece, traveling solo at the start and having a friend join me later.

Itinerary

I worked with Claude on my travel plans, starting at a high level (deciding on Greece as the destination) and then diving into specifics (places within Greece, timeline of stays). As a travel partner, Claude / LLMs are great - not always accurate, but typically provide a good enough starting point to figure it out. Beyond planning I've leveraged LLMs as makeshift tour guides, as they're generally more informative than museum placards, and social etiquette coaches, especially helpful as an American in Europe.

For 16 days in Greece, here's my itinerary:

Athens: 3 nights
- Settle in, visit Acropolis and museums.
Chania: 3 nights
- Explore the town + harbor, day trip to Balos Lagoon.
Heraklion: 3 nights
- Visit museums and Knossos.
Naxos: 4 nights
- Explore the island, day trip to Rina Cave.
Athens: 3 nights
- Prepare for departure, possible day trip to Delphi.

Something I've learned about travel is there's a huge difference between the abstract knowledge of the trip, and the actual reality of the trip. As an example, the above itinerary felt right when planning, but I now would changed up the distribution of nights, shaving time off Heraklion and Athens for a visit to another island, or extending the stay on Naxos. This is my general experience of travel, though - I don't know what the experience of the place I'm visiting is until I actually visit!

For transit:

Fly Athens to Chania: quick flight, took the bus from the airport to downtown Chania.
Bus from Chania to Heraklion: cheap (16 euros), scenic drive along the coast of Crete.
Ferry from Heraklion to Naxos: experience tbd, based on other travelers' stories I picked up motion sickness medication.
Ferry from Naxos to Athens: same story.

Public transit has been reliable and cheap - I haven't taken a cab ride yet (though without a phone number, I can't sign up for freenow 🙄). Public transit isn't as nice or fast as private service, but I'm in no rush this trip.

Odds and Ends

I used a luggage storage service for the first time, which was convenient as I carry two backpacks when I travel (large one behind, small one in front). Unloading my large bag was a boon as it enabled me to stay mobile and explore for a few more hours, rather than feel compelled to hunker down with my bags in a cafe.

Experiences

So far, some highlights of my trip have included the Acropolis + museum, the Ancient Agora, the Athens Archeological Museum, the old Venetian port at Chania, and a day trip to Balos Lagoon + Gramvousa.

I did a tour group for the Acropolis, which I didn't find particularly moving. Most of the information I had already read before, and the guide wasn't energetic. I rank all tours against my experience at the Greenwich Observatory, where the tour guide was clearly passionate about the history and subject matter. Tour aside, the Acropolis is an impressive monument of history, of hubris: not long after its completion, Athens was defeated, never to rise again to its golden age glory.

I was summarily impressed by Chania - I had no expectations about Crete, having seen it only from travel vlogs. The old town is charming in the truest sense, and the harbor lit up at night is delightful. The city is exactly what I imagined a European port city to be. I also did not realize how close the beaches were, nor their quality: the water is pristine, the clearest I've ever seen. Taking the bus to Kissamos Port I realized that the beaches extend endlessly, with the waterfront covered by hotel and resort sprawl, some nestled onto idyllic cliffs. On the ferry to Balos, the striking mountainous peninsula was awesome. I laid in the water in Balos, perfectly shallow.

Insights

A week with just myself, out of my usual environs, has created plenty of space for insights and awareness into myself. Specifically about environment, I see that the grooves of my daily habits and rituals induce a kind of stupor or cloud that makes self awareness challenging. When traveling outside of my home / culture, just about every interaction and situation is new enough that I am conscientious with a kind of stark contrast to view my own behavior.

I noticed that I am always going somewhere in particular, always have a destination in mind - if I am going to eat, I have the restaurant in mind, if I am at the museum, I am glued to the guide. There's a spectrum of behavior here; in the past I detailed my itineraries down to the day, while now I am more flexible with showing up and trusting that I'll figure out my activities.

Ah, that word gets to the root of so much for me - trust. I have to actively recollect that I was quite fearful that I would not succeed in solo travel, that I'd be miserable and come home early. And now that I have demonstrated that I am capable, my fears have aptly moved onto the next object for speculation.

The common theme being constant motion, direction, activity - chasing one experience after another, fearing one phantom after another. I'm constantly going and doing, spurred on by an internal, infernal drive, an innate dissatisfaction that I'm looking to resolve.

Wisdom is the knowledge that enables making better choices. It can be taught, but it's not easy to accept - I think that's an aspect of the human condition, and gets at why despite vaster and more impressive knowledge than ever before, the perfect world (for an individual or collective) cannot be educated into existence. Some truths are learned through experience. In my case, it's the experience of chasing after satisfaction through external means (alternatively, chasing after resolution of my dissatisfaction through external means).

Which leads to the unsatisfying conclusion that change, healing, recovery, takes time, and sometimes quite a lot of it. My internal structures were woven and reinforced through decades of experiences. To imagine a quick resolution is to mistake form for content: with the perfect life circumstances (girlfriend, job, apartment, etc.) I'd be just as dissatisfied as I am today, as I am the constant. Weaving new structures, unraveling the old, is a tender and delicate process, itself enjoyable as I come to realize that there is no end state anyway, that satisfaction is possible even now.

A significant element to be restructured is my perception of myself and others. This trip has once again shown me that I am constantly projecting, interacting not with the person but a fantasy of my own creation. These fantasies, stories that I've written and read so many times that I became a true believer, mirror my own inner world. When I operate from a perspective of fear and insecurity, all I see in others are fear and insecurity, or their opposite, supreme and total confidence. I treat myself as if I am unique, which became humorously apparent when I thought myself the only person to ever drop a piece of cucumber at a meal - for a moment I really imagined myself to be terminally unique.

By rewriting this narrative into one of commonality and shared experience I start to open myself to myself and view others as human beings with inner worlds just as rich as my own. I start to open myself to richer relationships with others, relationships that reflect and reaffirm in mutually reinforcing feedback that I'm okay, that I always have been, and I always will be.

I believe the creators of this artwork understood what I am talking about. Spirals interconnected - each individual an infinity, with depth and richness of existence, connected to every other, a connection that both feeds and is fed by the other; a connection small yet critical.

Django: Index Your Generic Foreign Keys

2025-09-06T00:00:00-04:00

In Django, models may relate to another, generic model, as opposed to a concrete model, via a generic foreign key, which becomes self explanatory when looking at a sample definition:

Python

class TaggedItem(models.Model):
    tag = models.SlugField()
    content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE)
    object_id = models.PositiveBigIntegerField()
    content_object = GenericForeignKey("content_type", "object_id")

A TaggedItem instance may be related to any other type of model, as opposed to being restricted to a specific type of model, say a Post. This makes the TaggedItem model quite flexible, allowing reuse across different purposes. The flexibility comes at the drawback of performance and implementation, typically.

In reviewing query performance at work, I noticed that a few of the most time consuming queries (where total time = num calls * mean time) involved generic foreign key filters. Running the queries through the postgres query explainer, I was surprised to discover that generic foreign keys are not indexed by default. That is, a query that searches for TaggedItems by their related content object like:

Python

TaggedItem.objects.filter(content_type__pk=bookmark_type.id, object_id=bookmark_id)

will do a full table scan to find the relevant TaggedItem rows. If you're lucky, the query will leverage an index on content_type to filter down the rows to scan, which may or may not make an impact depending on the characteristics of the table (e.g. if there's only one content type in the table, there's nothing to be gained via that index).

Adding an index on content_type, object_id will avoid the scanning, reducing performance in my local test by ~300x.

Python

class Meta:
    indexes = [
        models.Index(fields=["content_type", "object_id"]),
    ]

To Django's credit, this behavior is called out in the docs:

Unlike for the ForeignKey, a database index is not automatically created on the GenericForeignKey, so it’s recommended that you use Meta.indexes to add your own multiple column index. This behavior may change in the future.

though only in recent versions. If your project is older, like mine, or you weren't aware of this behavior and haven't been conscientiously adding in the indexes manually, then you may want to take a look at your queries for a potentially simple optimization.

May your generic foreign key searches be fast and efficient.

Samsara

2025-08-26T00:00:00-04:00

I'm not sure where this post will go. I want to organize and record some of my thoughts that are taking shape, in the process of taking shape, hence not quite coherent yet. I hope this helps me to synthesize my experiences and understanding of myself.

Wandering

Buddhists describe samsara as wandering, in the sense of aimlessly and blindly walking the same circuit over and over again. I relate to this concept strongly: I am becoming aware of the trends in my friendships, relationships, work, and so on. There's patterns to my behavior, patterns that I live and breathe without awareness, many of which cause me suffering large and small.

I've long struggled with the notion that I am not in control of my own life. I've related to the drifters, the explorers, the adventurers, the wanderers: people whose control over their life is demonstrated in their continuous motion. I've related to the underdogs, the heroes who've overcome impossible situations: people who've beat their fate through willpower, determination, courage.

It's come as a revelation that outward displays of overcoming do not necessarily imply resolution of inner troubles. I think there may be a correlation there, in fact: the more that control is sought in the outside world, the less there is in the inner world; in other words, the more striving, the more inner pain.

Reenactment

I learned the term trauma reenactment in therapy recently. I felt another piece of the puzzle had fallen into place. It's painfully (emphasis on painfully) obvious that so many of my patterns of behavior stem from my childhood. It's a challenge for me to build and maintain relationships, stemming from a persistent cosmic background anxiety that perpetuates a state of fear, simple but powerful fear of rejection and abandonment.

I have understood for some time that I am attracted to what I know; where what is known may or may not be healthy or wholesome for me. Easy example is food, I tend to eat the food of my culture, and within that I tend towards the same meals and food items for the most part; new or "exotic" foods don't quite taste right until they've been repeatedly ingested. A not so easy example is relationships: raised in a family where my needs were not met, I feel comfortable in my adult relationships where my needs are not met, and uncomfortable when they are met. How's that for causing my own suffering!

The key insight from trauma reenactment is not the repetition of behaviors generally, nor the repetition of harmful behaviors specifically, but in the rationalization of the behaviors themselves. While I am engaged in unhealthy behavior, that is causing me suffering, I believe I am acting in my best interest, in a healthy way - more than that, I believe that I am resolving and overcoming my original traumatic experiences. I am living in both the past and the present at the same time, merging them together, merging the people together, responding to my girlfriend as if she were my mother, responding to my boss as if he were my father.

The example from my therapist is of someone who experienced sexual abuse as a child. As an adult, they engage in sex work to feed their drug habit, and their mindset is that they have the power over their clients because they are being paid. As a child they were powerless, now they have the power - all the while engaged in dangerous and self-destructive behavior; they have internalized the anger directed at them and replay it against themself.

Resolution

Speaking with a close friend of mine, she shared that she feels she's working through her past through her current relationships: an older male parallels her father, an older woman parallels her mother. Something clicked when I talked with her: trauma reenactment is the psyche's instinctual attempt to heal from the psychological wounds of the past.

To heal, there is me, and the environment. I have spent the vast majority of my time reproducing unhealthy environments for myself, working within my "comfort" zone. In the past few years I've shifted into healthier environments, friendships, relationships, while shifting out of unhealthy environments. The healing power of my environments is slow and subtle, molecular.

For "me", I struggle to find the words to describe my experience. I can point to external changes easily. I don't know how to point to inner changes. "By their fruit ye shall know them" - I can point to the differences in my behavior, characterized most evidently by the changes I've made to my environment. But now I've come full circle. Ah, circles.

I don't have the words to describe my experiences. I just have a hunch, I have an intuition. Something about me is different today than several years ago. Something significant and profound, but not radical - a tipping point, a critical state, a what-have-you. At some point I became willing and able to change (or was I always able, but not yet willing? or was I able and willing, but waiting for the external spark?), at some point I dipped my toes into someplace new.

In this new environment, with this new me, the old me has been able to work through the old environments.

Challenges

2025-08-10T00:00:00-04:00

Several weeks ago I heard someone say, "God gives us challenges the moment we're ready for them." I can't say that I am thrilled about this, as I'd appreciate a reprieve now and again. But I have come to see the truth of it these past couple months.

Sickness

I've had a rough time recently. I came down with a severe case of the flu at the start of July. Initial sinus infection symptoms faded in a week, but fatigue and brain fog persisted for more than a month. Ugh. I have never been this sick for this long before. I've never been so limited in participation in my life.

The matter of the sickness itself was half the battle, the other half my panic at the longevity: what if the fatigue never goes away? What if I have these limitations forever? I struggled emotionally to cope, the more so as time went on. I am grateful for the support of my friends who helped me during this period.

Anxiety

Immediately preceding the illness, I was under an enormous amount of stress at work and in my personal life. I began dating a woman with whom I started falling back into old patterns of behavior. On the heels of a challenging work project I faced a test of my management skills. These scenarios played out over several weeks.

Stress intermingled with the illness, I felt like I couldn't breathe. Some small and strange phenomena was that in my low-energy days my emotions were mellow; on better days, my emotions came back like roaring waves crashing over me on a day of rough surf. I couldn't catch a break.

Recovery

I am writing on a Sunday, having had the most lively weekend in over a month. On Friday, I attended a friend's performance in a community production of Matilda. Yesterday, I went kayaking with a few close friends, spending several hours on the water. Today, I went on a hike with a new friend. I feel like I'm back at full performance.

Will I give myself a chance to breathe? I'm torn between feeling like I need to make up for lost time (a July spent on the couch!) and continuing to invest in my rest and recovery. As I learned about fatigue, over-doing it on high-energy days is a classic trap that lays the seeds for future crashes.

Humility

These past weeks have been humbling. The five remembrances include, "I am not immune from sickness; I will get sick." Hearing this I thought, all well and good, some day I'll get old and sick, but that's a problem for old and sickly me. I am young and healthy me, I bounce back fast from sickness. Well, at 31 years old, I feel the truth of this remembrance. I am of the nature to get sick, severely sick.

The fatigue was especially painful. In the past I've been able to push myself as hard as I need to, and I've long taken for granted my ability to live my life as physically as I've so wanted. With fatigue, there were days I couldn't move off the couch, no matter how much I wanted to. The brain fog has had me forgetting names, places, memories. My job is intellectual and my inability to perform at my usual level deeply unsettled me.

I am thankful for the opportunity to face this challenge. Despite years of working to curb my ego, a little scratch reveals it, seemingly bigger than ever - but I just take that to be my growing awareness seeing more of my inner parts than ever before. I'm thankful for the experience and my ability to navigate it successfully. I'm thankful for all of the challenges that I've navigated recently, some that I'm more proud of than others.

I am most of all thankful for the loving support that I received from my friends.

Travels with Charley

2025-07-24T00:00:00-04:00

It's been a long July for me. The past few weeks I've experienced passion, anxiety, sickness, humility, and deep connection. As I've reduced my pace, I've dedicated myself to reading more. This is the first month I've read three books since last July - a funny pattern to appear. I read Thinking in Systems, Look to Windward, and just finished Travels with Charley by John Steinbeck.

I've long admired Steinbeck. I remember once checking out several of his books at once from the college library, dutifully stacking them by my bed: The Pearl, The Winter of our Discontent. I remember reading Of Mice and Men, and East of Eden - the latter to impress a college girlfriend. The Grapes of Wrath radically altered my politics at an impressionable age. I suspect revisiting these books now I'd have a completely different experience.

That's all to say that I have a history with John Steinbeck, as an author. I never explored his personal life. Written towards the end of his career, Travels is a story of a man trying to reconnect with his Americana roots. Places are memories, shaped by the tides of time. The gaps in the story are written in embarassment - guiltily, as if to avoid the pain of owning his own changes from scrapping youth to fame and success. There's a contradiction between rich author staying in a luxurious hotel to camping by the riverside in Rocinante.

I enjoyed the book and found myself drifting in my imagination on my own road trip across the country. I've previously attempted two and completed one cross country road trip. At 17, desperate to get away, I set out from New Jersey - but never made it out of Pennsylvania, felled by the first road block. A second go in my early 20s succeeded, when I joined my friend moving to SoCal. We cleared the country in 4 days, going so fast as to overheat the engine. As Steinbeck says, sticking to the highways I can't say that I saw the country, just a shifting landscape.

The nature of trips

Steinbeck says that a trip takes the traveler more than the traveler takes the trip. I think this may be true. I've experienced hints of it over the years. I notice that, leaving the house, a subtle turn overtakes me: I attune to being on a trip, my rhythms adjust to the variations. When I return, I feel somehow completely different and exactly the same, as I embrace my home coffee maker and rocking chair. A trip subjects me to itself, its will is in charge, not mine - I am a passenger in the true sense.

I also relate to how trips may end before returning home. On my trip to Thailand, I recall distinctly arriving in Bangkok for the last leg, and being absolutely sure I could go home that instant and not be sorry for it. I was done, I had seen all I could possibly see, experienced as many highs and lows as possible: my tank was on empty. On the other hand, other trips beg for more, leave me aching to return, to see more, do more, experience more. I felt this way in London, not getting to do more than half of what I had planned - and that being a fraction of all available to do!

Mass production

A common observation of Steinbeck is the changing of the cityscapes. As he describes, cities are ringed with wrecked and decaying cars and such detritus of mass production. The old city centers are hollowed out, surrounded by suburbs teeming with wealth and investment. He doesn't lament the changes; rather, throughout the book, he accepts that change is inevitable, neither good nor bad, just a fact of being old enough to remember when things were different.

Another common observation is the impact of mass produced goods. Breakfast, he notes, is always freshly prepared and delicious, while plastic wrapped dinners are bland and hardly worth eating. Goods then trend towards a tolerable minimum quality, they're standardized in their blandness. The same applies to culture: as radio and television spread a mass produced culture, regional variations in speech and behavior start to fade. What was noticeable in 1960 can only be verified in 2025, with the internet erasing every last barrier to connection between communities.

On this last point I can't help but reflect on my own life. When I need anything I reach immediately for Amazon and have the item next day. The information I receive through media content on the television, phone and computer are bland, harmless and without meaning. There's little real in my life, not mediated through safety filters and quality control. It's not all bad, I am grateful that I have clean water and safe food to eat. But I think the overproduction of safety and comfort has come at a price.

America

In his humble quest, Steinbeck set out to find the real America. By his own admission, it was an impossible quest: he talked to maybe a few dozen individuals out of countless millions. I am amazed at the similarities between his observations in 1960 and today. Maybe most notably, the politic divisions. He describes family political disagreements as a kind of civil war in intensity - something that cuts against the narrative that recent times are the most political (a recurring theme, I guess, kept alive through media agitation and political interests).

I am fully American, I don't have roots outside the country. I can trace my ancestry back to the revolutionary war. When I think of the country, I think of opportunity, as a shared belief. It's a land of immigrants cutting ties and starting over. And within the country itself, cutting ties and starting over - it's a big country after all. Maybe prosperity is the shared belief, shared myth, that binds us together. I promise that I don't know either.

Thinking in Systems

2025-07-12T00:00:00-04:00

I am wrapping up Thinking in Systems by Donella Meadows, having been inspired recently to study the importance of feedback loops. I find the book fascinating and easily applicable to systems large and small, from the economy to my internal emotional world.

Systems

Systems are made up of stocks, think of it as some resource that can be measured, like water in a bathtub. Stocks are affected by flows, analagous to the pipes that feed water into or out of the tub. The simplest systems have no relation between the stocks and the flows, eg the amount of water in the tub does not affect how fast the tub can drain.

When stocks and flows are interconnected, there is feedback within the system. Think of an interest bearing bank account, where the balance yields interest, which compounds itself: the rate of return, rather than being constant, is a function (typically, a percentage) of the balance. This is a reinforcing feedback loop, which demonstrates exponential growth, ie the stock grows faster and faster over time (in fact, really fast over not so much time!). Balancing feedback loops tend towards some goal, as in the case of a thermostat in a house. When the temperature goes below the threshold, the heating unit kicks in; when the desired temperature is reached, it stops. The stock is kept at the target.

Large and small

The book makes the case that infinite growth cannot be sustained in a finite system. The author is making the argument against infinite economic growth on a planet with finite resources. In the ecological vein, fossil fuels cannot last forever; in simple models, exponential growth in depletion can lead very quickly to resource exhaustion. In my mostly uninformed knowledge, the world has kept up with oil demand through discovering new means of extraction - the USA is the world's largest oil producer thanks to shale extraction. But every finite resource has its limit, which cannot be increased as fast as demand can grow.

In the small scale, I am interested in my emotional wellbeing as a system. Let's say the stock is my stress level. In my view, stress is a reinforcing feedback loop: the more stressed I already am, the more each new stressor is amplified. After meditation I won't be bothered by someone driving erratically, but when my car's AC is broken on a sweltering day and I've already been cut off a dozen times, I am likely to start honking!

In the outflow, I think there's typically a balancing feedback loop towards low stress - unless my goals are miswired due to chronic stress, when my internal systems may naturally guide me towards more stress than I'd like. This is a point towards reflection and awareness: is my resting state anxious or calm? In terms of actually reducing stress, another key lesson is time: stocks and feedback loops tick, they do not interop instantly. In other words, there's no instant meaningful stress reduction. Time takes time, as they say, and when coming down from a high stress level, results may not be immediately apparent. Taking a walk after a brutal day at work won't cut my stress to zero, and if I react to that by beating myself up for failing to eliminate stress, well I am just reinforcing my stress rather than reducing it! A key lesson for me is that stress relief has to be a consistent practice, with results that appear sometimes slower than I'd like.

Catastrophe

In Ubiquity by Mark Buchanan, the author explores catastrophes, and challenges the prevailing wisdom that significant outcomes require significant inputs, ranging from natural disasters like avalanches and forest fires to mass extinction events and economic collapse.

Some systems, such as forests, have naturally occurring balancing feedback loops that keep dry organic material at a low level through recurring small fires. When human intervention disrupts this natural feedback loop, by policy putting out every fire, the amount of dry material builds up over time, tending towards a critical state, one in which any spark can lead to a cataclysmic fire.

Generalizing, I think systems modeling is useful for understanding crisis prone systems. Naturally occurring systems tend towards equilibrium, with internal and external balancing feedback loops. Understanding crisis as a disruption to those feedback loops is helpful to understand how to address the problem: sometimes the feedback loop needs to be restored, sometimes the system needs to be remodeled entirely because the conditions it evolved under have changed.

In the case of evolution, Buchanan describes a simple mathematical model in which evolution can lead to its own critical state, in which a small change to one specie's success can cascade quickly into a mass extinction event - no asteroid necessary! Similarly with capitalism, the boom and bust cycle is not caused by outside intervention but rather the internal dynamics of the system itself, leading to periodic crisis.

For emotional wellbeing, burnout is one potential catastrophe that results from unmanaged and accumulated stress. It may manifest as the result of this or that particular event, but it's likely due to the internal balancing feedback loops not working or not being able to cope with the pattern of incoming stress. To avoid a stress-based critical state, it behooves me to ensure my perception of my stress level is accurate, and my stress flows in and out are in line with my desired stress level.

Conclusions

I could go on and on, I find this subject to be endlessly fascinating and applicable. To keep it short, systems are a useful way to model the world - but remember, they're only models, and therefore inherently limited and flawed. Still, approximations can be used to incredible effect, and a strong application of systems thinking can provide actionable insights into a problem.

AI (2025)

2025-06-09T00:00:00-04:00

I haven't written much about AI, just a post lamenting degraded open access and a post complaining about LLM verbosity. I want to share a few perspectives and predictions here, as much to summarize where I'm at on the topic, as to give myself a looking-back point in the future.

Experience

I use LLMs at work on a daily basis. The primary interface is through copilot autocomplete in my Emacs editor. This is a love-hate relationship, when the suggestion frequency is dialed too high, the bad autocompletes generate more cognitive disruption than good autocompletes save cognitive effort and typing time. That said, I've never disabled it.

I also use gptel, typically in dedicated chat buffers, and when I remember using in-buffer rewrites. I usually create ephemeral chat buffers, but have started using file-backed buffers for persisting chats for later recall. I find that gptel has a great balance of usability and power; like my favorite tools it stays out of the way, has simple shortcuts for the common use cases, and leverages transient.el for powerful configuration.

I trialed cursor.ai, and can't wait for suggested-next-edit to come to Emacs. I also briefly demo'd claude code, and plan to invest more time with agentic coding.

Predictions

These are by no means "hot takes" - as I've disconnected from the internet's overwhelming stream of consciousness, I've lost interest in such things! Instead, the predictions are meant to organize some of my thinking about how I use LLMs today and what I expect in the near future.

Degraded internet

We've already seen major internet websites like X and Reddit close down their previously generous API access. At first, I suspected this trend to continue, as platforms hoard their precious data in order to sell to the highest bidder for training data.

With the rise of agents, or "tool-using LLMs", or "function-calling LLMs connected to other software systems", or really just "LLMs generating structured output", I predict major platforms will be forced to step up their protections, extending beyond API access to the basic web and app interfaces that humans use. This is because agents can act on behalf of humans, interfacing with dynamic websites about as well as a human can, but importantly not generating the income from human eyeballs on the omnipresent advertisements.

As an example, imagine an agentic booking a vacation for me, intelligently skipping ads and promoted content. This threatens the foundation of the internet advertising model.

What could the future hold? Maybe humans will be forced to have their cameras on, and eyes tracked, as if taking a remote examination on their computer. That's possible, but unlikely. It's much easier for me to imagine advertising to adapt and evolve, maybe injecting ads into LLM training data, or returning prompt injections when accessed by LLMs.

It's a tragedy. I predict for a brief window we'll see just what technology could do if platforms were open and inter-operable, if personal data were accessible, if my computer and software worked for me and not for others. Agents are a crowbar forcing open the door, but in short time stronger walls will be built.

At the end of the day, it's all about the money. The incentives are in place to hoard data, lock down and gate access, and prevent the usefulness of agents, in order to force humans to interface with monopolistic corporations, so that profit can be made.

Voice first

Generating structured output from unstructured input is an incredible feature of LLMs. I predict this will yield a wave of new user interfaces that are voice-first. I think back on all the frustrating, multi-step, annoyingly custom forms that I submit on a daily basis. What if that could be avoided, or at least alleviated, through a friendly chat-like interface? How nice if I could blabber on for a few minutes and have the data nicely structured for me - no formatting dates or phone numbers incorrectly, for one.

At it's simplest, imagine a browser extension that supports voice input to fill in forms throughout the internet. Given use over time, it should be able to learn/remember my data and auto-complete most forms, asking me only the unique information in the form.

This is one example of a broader possibility that I foresee. There's an incredible diversity of data entry UIs on the internet. Every time I work with a designer on a new form, it invariably includes new and unique form inputs. I think this is a waste of time and cognitive effort for users, who have to contend with yet-another-dropdown-that's-not-like-any-dropdown-I've-seen-before.

LLMs can present a simplified and unified interface to diverse form entries, saving a not-insignificant amount of time for users.

Who knows, maybe this will provide back-pressure against such complex and bespoke interfaces? Or websites will provide LLM back-channel interfaces and integrations specifically designed for the dominant LLM and agent workflows.

Just as above, I suspect this pleasant future will not come easily, and there'll be conflict between LLM automation tools and major providers. User interfaces are subject to the same incentives, and so any attempt to bypass them will be fought tooth and nail. Further, UIs are the frontlines of the battle for user attention, subject to insane levels of control and manipulation.

Agentic coding

As a software engineer, this one hits home most closely. Will I have a job in a few years? If I do, what will it look like?

I'll state that I do expect to continue working as a software engineer for the foreseeable future. I suspect my day to day will change - I got a taste of the future of babysitting agents in a recent podcast.

Agents are great at validating whatever you put into them. That is great, if I am keyed into the right idea or perspective, but if I am not, I am liable to dig in deeper into the wrong approach. For software, where engineers love to write new green-field code, I suspect there will be an explosion of new code that's poorly vetted.

As we know, reading code is significantly harder than writing code, especially when not in your style or a known colleague's style. As agents become code authors and not merely code assistants, the skill of reading code will become even more valuable. As agents then review and release code independently, it'll be the task of the overseeing software engineer to monitor and ensure the systems are running correctly - just like humans, agents have a tendency to skip important requirements!

Maybe this means software engineers working alongside or above a "team" of agents. But to suggest this future also implies an explosion in code changes that need to be vetted, and new software systems that need to be overseen - suggesting a need for at least as many, if not more, engineers.

I'd be remiss not to mention the security problems of LLMs. Agents inherently mix command and data channels, meaning there's no possibility of safely passing in untrusted data. The average sentiment appears to be, as usual for security, "who cares?" and that will remain the case until and well after the major security breaches start. When I run an agent with full autonomy (i.e. all permissions enabled, full system access via tooling), I am extending my trust to every possible source of input into the LLM.

For example, if I instruct my agent to use a particular library to solve a problem, it's easy to conceive the agent searching for relevant documentation, ending up on a page with an embedded prompt injection that steals files, code, or production API keys.

In production, the threat is much higher - I heard a story about a company's production LLM given a tool to execute code.

Beyond security, there's the practical questions of running multiple agents - because they're automated, of course I'd run as many as I can. It's a small prediction, but I foresee that reproducible, isolated environments will thrive. Perhaps we'll also see a resurgence of the "develop in the cloud", and big providers which invested in this area will get their payday if they can execute well.

At work, it takes a couple hours of annoyingly manual work to get set up and running with a local development environment. Imagine if I wanted to spin up a dozen cloud environments with agents hacking away - it would be incredibly challenging, and any updates to said environments would be quite slow.

Instead, suppose that an environment can be spun up in a few minutes or even seconds - I'm talking compiled single binary, sqlite in memory database, a totally isolated and reproducible environment. Then agents could be spun up on-demand and put to work, and die (or, retire?) when they're done. That seems like a compelling advantage.

Conclusion

Wrapping up, I am excited for the future of software, both as a user and as a programmer. My over-arching thesis is that the current economic situation will hold back the full potential of LLMs, as monopolistic companies desperately leverage their market domination to hold onto their stock values. As an engineer, I am excited to move up the abstraction layer, and become more productive with agentic tooling.

Stacking

2025-06-01T00:00:00-04:00

I believe strongly in merging small code diffs. In my experience, small code diffs merge more quickly, are released more quickly, leading to a faster feedback loop with end users - reducing the probability that work is invested into the wrong software.

There are other benefits: smaller code diffs are easier to review. Past a dozen files or a few hundred lines changed, the feasibility and quality of any code review plummets. Likewise for quality assurance: smaller diffs affect fewer systems, and are typically significantly easier to test, leading to lower risk for releases - decreasing the potential "blast radius" as my coworker calls it.

Stacking

This all sounds well and good, but is challenging in practice. Fixing a null pointer bug with a null check may be a line or two of code, but what about creating a brand new feature, or performing a large scale refactoring? How can I transform large code diffs into smaller ones?

One battle tested solution I use is code diff stacking, or PR stacking. What this basically means is a chain of git branches, a stack of code diffs that build on top of each other. In other words, break down the code diff across multiple diffs. For example:

Text Only

develop <-- feature/friends-list-api <-- feature/friends-list-frontend

Where the first branch builds a web API, and the second builds the frontend consuming the API.

Why not one PR?

This is a good question! Sometimes, one PR is the right approach. I use these heuristics when approaching the question:

How many files changed? More than a dozen: consider stacking.
How many lines changed? More than a few hundred: consider stacking.
How many systems affected? More than a few: consider stacking.

I would emphasize that stacking or splitting diffs is not a dogmatic approach, but a pragmatic approach, a tool to be used in the right circumstances.

Break it down

I typically approach stacking with the following approaches.

Vertical

Vertical stacking means each code diff is fully functional and can be released independently. This is an ideal approach when building an entirely new feature, which can be gated behind a feature flag.

For instance, I was tasked to build a new dispatchers page at work. This page was fairly dynamic, with a master/detail view of live driver routes, plus actions to handle on the road scenarios. I'll note there was a lot of uncertainty with this feature, so getting users on board by releasing early and often is key.

I broke the code diff down as follows. First, the master view, a pane displaying a list of driver routes. This was straightforward to build, as I only had to focus on building a solid foundation and displaying the list. Then, in a second diff, I built the detail view for the driver route. On top of that, more diffs followed, iterations on functionality and bug fixes from the previous diffs. This encouraged a strong feedback loop between users and devs, avoiding the unfortunate scenario where months of work go into a release that's ultimately rejected by users.

Horizontal

As opposed to vertical slices, horizontal slices are not independently releasable. Instead, the code diff is broken down into logical pieces, system by system; the small code diffs are merged together and then released all at once.

This can mean different things in different codebases and projects. A typical example is breaking down a feature into data model changes, followed by frontend changes, followed by backend changes, followed by internal changes. This is the approach I am currently taking for a project that's upgrading a previous feature, extending it to offer more options to the customer.

The first diff in my stack implements the database changes, with care paid to migrating existing data to new columns. The next builds out the updated customer user interface, with accompanying API changes. The next diff addresses necessary changes to internal operations systems. I approached the diffs this way because the underlying data model changes necessitated significant changes to several systems, that is the first "thin vertical slice" would frontload a disproportionate amount of code.

Multi stage

When reflecting on a few recent projects at work, and in discussion with other engineers, I thought of a new approach to stacking that combines horizontal and vertical. The motivation is when, as in the previous example, the initial "thin vertical slice" would be so large as to erase any potential gains from stacking.

In my example project, I realized that the customer user interface changes were backwards compatible. I could have worked on the interface changes first, aiming to release the changes, and then following the horizontal stacking for the remainder of the work. This would have meant the first code release would be significantly sooner, and the interface could be tested with customers, alongside further development.

A coworker has been working on a large scale refactor of our codebase to support multiple currencies. The first code diff was quite large, and contained a slew of different refactors, including to the data model, template display, billing, and so on. In hindsight, the large diff could have been broken down similar to my project above: for example, a first diff could refactor the template display of currencies to a universal interface - render_currency(). This could even be broken down further into batches of changes to templates, de-risking any individual diff. Finally, when the underlying data model is changed, that code diff can add support for rendering the new currencies by changing only one function - render_currency() - rather than hundreds of templates.

I think these two approaches are slightly different, and could be fleshed out further (and named?), but they share a common theme: identify changes that are in support of later code changes, either by frontloading backwards compatible changes, or refactoring to a universal interface.

Tips

Before wrapping up, let me note a few tips.

I use a rebase strategy for keeping my branches up to date. I have recently begun experimenting with single commit branches and interactive rebases to streamline this workflow further. I'll note that there are products that help with this, namely graphite.

When requesting reviewers, I may ask the same person to review the entire stack, just as if they were requested to review one large PR.

Conclusion

I find these practices to be useful and productivity boosting. I feel more confident in my code when diffs are small and easy to review and test. I look forward to continuing to hone this skill.

Scope

2025-05-10T00:00:00-04:00

This week I was asked to justify a development estimate by a product manager. As context, the project had undergone weeks of planning and proposal, and it's now under active development by yours truly. I gave an honest estimate of about 4-6 weeks, including coding, review, and testing. This estimate surprised the product team. The product team's surprise in turn surprised me. Why the mutual surprise?

What's in an estimate, anyway?

This was part of the line of questioning from my product manager: how does 4-6 weeks break down per component of the project? That line of questioning misses the forest for the trees. On the face of it, it makes a certain sense: at the end of the project, I could look back and count how many hours I spent coding each component. That would give a concrete and empirical answer to the question.

In an estimate, I sometimes use a similar heuristic. If I am tasked to build a new feature, I could for example compose an estimate as follows:

1-2 days coding the data model
1-2 days coding the UI
1-2 days review and testing

Amounting to about a week timeline. This type of estimate works well, until it doesn't. In a mature codebase, like the one I work on, seemingly simple features can take weeks or months to build. The reason for this is the complexity.

Complexity is the enemy

Complexity is invisible to product and stakeholders. From the outside looking in, my car seems fairly simple: there's a steering wheel and gear stick, and some pedals by my feet. Behind the simple interface there is a swatch of complexity: imagine my surprise that replacing my clutch pad costs thousands because the transmission has to be removed to access the underlying parts. What I believed naively to be replacing a pedal is a complex mechanical operation.

Similarly for non-technical folks, the cost (in terms of time rather than money typically) to maintain or change software features can be surprising, as the product manager can attest. The feature request - adding a second option to a selector - seems so simple! I had to explain the underlying complexity, just as my mechanic friend had to explain the same to me about my clutch. Here are some contributing factors:

The initial design was not extendable, i.e. the old feature has to be removed in order to build the new. This is more challenging than building from scratch, especially when mobile application APIs have to remain backwards compatible.
Several levers for customization: each option in the selector has customizable properties that affect both display and functionality.
Bespoke UI: rather than leveraging the design system, there are custom UI components.

Each of these adds considerable complexity, and therefore effort and timeline.

Talk about scope

It's my opinion that scope should be a top concern in project proposals. This is a lesson that I've taken to heart from my boss, who strongly advocates for a focus on ROI in software projects. What I've found in my experience is that it's easy to discuss scope at the outset of a project, when it's most abstract: "we want to launch this by end of the month!" But the devil is in the details when having to make hard decisions, that is say "no" to stakeholders. In the example of my current project, this could have meant eliminating some of the customization options or leveraging the design system.

As an engineer, it's my responsibility to inform product about the impact to scope for these types of decisions. If there are shortcuts, or 80-20 opportunities, then I will advocate for them - as I believe a feature shipped quickly and imperfectly is better than a feature shipped late and "perfect" (but let's not kid ourselves, there's no perfect feature launch!). As it goes, I did make these suggestions during this project's proposal process, and the suggestions were rejected. Hence my surprise at their surprise.

Final thoughts

I'm in a stage of my life where I am learning to let go. I firmly believe in personal excellence, in doing the best that I can. To that end, I committed myself to delivering this project according to my estimate. But when it comes to soft pushback on the timeline, I don't feel responsible, i.e. pushing myself to work longer hours/weekends to deliver sooner. The project will be delivered when it's delivered, and if that's later than expected, maybe next time scope will be prioritized throughout the project.

Attention Footprint

2025-05-03T00:00:00-04:00

I attended DjangoCon EU in Dublin, Ireland last week. I got to see a few of my colleagues, and an ex-colleague, and travel throughout the British Isles. It was an excellent work + leisure trip. I came away from the conference with several takeaways. One minor lesson: don't judge a talk by its title! Looking ahead at the schedule, my group shared the sentiment that the talks seemed uninteresting. Still, we showed up and were very pleasantly surprised by the quality and depth of the presentations. Personally, I found a mix of both technical and non-technical wisdom.

Non-technical wisdom

One such non-technical takeaway was in the very first talk about code review in the Django project. The presenter described a common open source maintainer experience: a new contributor makes a pull request for a feature they want, and invests little-to-none in documentation and maintenance of that feature.

I have found this to be a pattern in software engineering culture. It's easier for engineers to write code than it is to communicate, let alone build consensus or change processes involving people. Engineers feel safe in the realm of impersonal code and machines, in metrics and passing unit tests. Interpersonal relations have no such safety guarantees: there's no compiler check that promises a coworker will do exactly as you instructed without blowing up.

But that's a topic for another day.

Attention footprint

As I interpreted the speaker, my attention footprint is the demand that I place on other's attention.

A classic example is asking to ask. A coworker messages me saying, "Hey, can I ask you a question about so and so?" and after I reply "Go ahead" they write the actual question. Typically there are several minutes of delay between message and reply, leading to rapid context switches, or just sitting idly while "X is typing...". Asking to ask puts a higher demand on my attention as opposed to writing out the desired question.

In the talk, the speaker was focused on how we can reduce our demands on others' attention. As the talk was oriented around code review, there were several suggestions:

Writing good pull request descriptions: the aim is to provide the necessary context to review the code. When I receive a review request for a pull request with a blank description, vague title and several dozen files of code changed, I have to go digging for the relevant context, try to divine the intention and reasoning of the changes. On the other hand, when I receive a request to review changes that are thoroughly documented, I'm able to context switch in easily and spend significantly less time reviewing.

I typically perform a self-review on pull requests. I'll call out specific sections of code that are sensitive and should be reviewed more closely. I'll also describe my thought process behind decisions (in instances where a code comment would be inappropriate, e.g. choosing a third party library or refactoring a function). This precipitates questions that I expect a reviewer to ask, based on my own experience reviewing code.

When I balance what I give of my attention to what I am demanding, it leads to better outcomes for me. When my pull requests are easy to review, they are reviewed more quickly. When I ask good questions, I get good answers more quickly.

Generalizing

I believe this lesson is applicable beyond code review. In my interpersonal relationships, am I listening as much as I am speaking? Am I communicating clearly, or am I expecting others to read my mind? Striking a balance, and paying attention to my attention, pays rewards in better, easier relationships.

Awakening

2025-04-30T00:00:00-04:00

I read that spiritual awakenings typically happen in one of two ways. In the more dramatic fashion, an individual has a transcendental experience with the divine. They may see a light, hear a voice, or in any case have a particularly memorable instance of connection. The other way is slow, gradual and molecular. There's no particular moment of awakening.

I am aware that I am experiencing this slow spiritual awakening. I realized that if five-years-ago-me heard today-me speak about my perspectives and life today, old me would think new me had lost their mind! Which always brings me back to the fantastic line from A Christmas Carol, where the maid says to newly awakened Ebenezer, "You don't seem yourself!" and Ebenezer replies: "By God, I hope not!"

I feel silly even writing this. I don't pretend to have gained any secret knowledge about the world. I write to record this for my future self, that I may one day look back and smile, in love and good humor at my crude and clumsy attempts to make sense of it all. I'm going to take the risk of globalizing my experiences and understandings here.

The water of life

Every individual is interconnected. (This is where I start to sound loony to my past self.) Every action affects countless beings, probably all beings. (Seriously, what am I smoking.. okay, commentary over). I believe this is related to what the zen monks mean by "everything is reflected within everything else."

The metaphor that came to me is water. Each individual is a body of water, ranging from a puddle to a pond to a lake to an ocean. The size of the water represents our capacity for acceptance, resilience, equanimity.

By default, the surface of the water is calm. Events disturb the calm. Jump into a puddle and it'll be thrown into chaos. Jump into an ocean and the ocean barely notices.

Just so, our nature as individuals is calm. The events of life disrupt our calm. Our reactions to those disruptions ensure the calm doesn't return. The cycle begins.

The bodies of water are interconnected by streams and rivers. A peacefully flowing stream may become a raging torrent, and destroy the calm of a small pond. The disruption does not end with the pond: it rages onward, radiating outward to its connected waters, to the next bodies, and on and on, dissipating but affecting everything (I recall in a physics lecture that the gravity of a body affects another body, inversely proportional to distance, trending towards zero but never exactly zero).

Just so, as individuals, when we are hurt, we radiate that pain out to others. When we're unhappy or angry we make others unhappy or angry. Consider again the size of the water. A raging torrent may ravage a pond, but it dissipates into an ocean. In the same way, as individuals, when we cultivate equanimity, when we build our capacity for resilience, we no longer propagate pain to others.

Faith

It's a process. At first, we're ignorant of the cycle of pain. We hurt others as we are hurt by them. When we build some capacity for resilience, we no longer amplify or return what is given us, but dampen the effect, working to minimize our contribution. Later, we are able to stop propagating pain, and return love instead.

It's taken me a long time to appreciate the truth in these words. That journey has been rooted in faith, a faith that I couldn't articulate previously. Thankfully, the monk at the monastery spoke on faith this past weekend.

Change in this life is possible. It's possible to alleviate pain and suffering in this life. It's possible to heal, grow, transform. I've felt this to be true my entire life, but I have not been able to express it or truly experience it.

Humility

None of the ideas I lay claim to. After two years of attending a Buddhist monastery and spiritual fellowship, the soil of my mind has been fertilized with much wisdom, passed down from others to me. In connecting myself to these sources, to the universal source, I feel humble and blessed. I feel hopeful.

Pytest, playwright and Django

2025-04-06T00:00:00-04:00

At work, I inherited a legacy end to end testing setup based on wdio.js. It featured few tests, and wasn't run in CI, so it fell out of date pretty quickly. It also suffered from some of the usual challenges of using a non-integrated e2e testing tool, namely server setup/teardown and data fixtures. I decided to revamp the e2e test setup in order to solve these problems and encourage engineers to write more tests.

Playwright

Playwright ships with pytest integration, which made it an attractive candidate for testing our Django application with an existing pytest-based test suite. It's also a modern browser automation software, having learned from the painful experience of nigh-two decades of Selenium. Playwright features automatic waiting, a GUI-based test writing tool, a test inspector with play by play action, and more.

I strongly believe in using as few programming languages as possible within a project. This encourages and leverages language expertise for engineers, and reduces barriers to contribution for new team members. Writing tests in python, especially in pytest, meant that engineers could contribute to e2e tests with minimal friction. It also meant code reuse with our existing pytest fixtures.

With playwright in pytest, writing a test that opens a browser at a given page and runs assertions is this simple:

Python

import re
from playwright.sync_api import Page, expect

def test_has_title(page: Page):
    page.goto("https://playwright.dev/")

    # Expect a title "to contain" a substring.
    expect(page).to_have_title(re.compile("Playwright"))

Amazing! Short, sweet and to the point.

For my e2e tests I want to open a browser at a local server. Certainly I could do this by opening a new terminal, running manage.py runserver, and testing with page.goto("http://localhost:8000/").. but that means I'm limited in a number of ways. One, state must be pre-configured in the server with some manage.py prepare_db_for_e2e script to add necessary data (user accounts, etc). And two, CI becomes more complex with server setup and teardown.

pytest-django to the rescue! The library ships with a live_server fixture, which "runs a live Django server in a background thread." In practice this means an e2e test running against the django server can be as simple as:

Python

from django.urls import reverse
from playwright.sync_api import Page, expect

def test_load_home_page(page: Page, live_server):
    # Note navigating to live_server.url
    page.goto(live_server.url + reverse("home"))

    expect(page).to_have_title("My Great Website")

Magnificent: no separate server to stand up/tear down, it's all handled automatically. Now, running my e2e tests is as simple as running pytest.

Test data can be loaded through the same mechanisms as other pytest tests, supporting strong code reuse. For example, at work I use a mix of django json fixtures and factory boy - all re-usable in this testing approach. Need to test viewing an order in a given state? Create the order with a factory and navigate to its url. Easy peasy.

Seriously. This setup rocks.

Challenges

Still, no software solution is ever perfect. This setup has a few challenges that are worth mentioning.

Shared global data fixtures

First, the live_server fixture running the server in a separate thread means that per-test db transaction rollbacks are not possible. In practice this means that after each test using the fixture, every database table is truncated. If your tests depend on global data fixtures (e.g. provided in an django_db_setup override), then you'll run into problems. To resolve this, I introduced a new command line flag to pytest to separate e2e from non-e2e test runs:

Python

# in global conftest.py

def pytest_addoption(parser):
    parser.addoption(
        "--e2e",
        action="store_true",
        default=False,
        help="Enable when running e2e tests",
    )

def determine_django_db_setup_scope(fixture_name, config):
    """
    e2e tests use the live_server fixture, which will wipe
    the db after each test run, which conflicts with a
    session scoped db setup.

    therefore, we use a function scoped db setup for e2e tests.
    """
    if config.getoption("--e2e"):
        return "function"

    return "session"

@pytest.fixture(scope=determine_django_db_setup_scope)
def django_db_setup(django_db_setup, django_db_blocker):
    ...

I also separated e2e tests into their own top-level test folder, excluded by default in pytest.ini. In practice this means running the full test suite requires two commands:

Bash

# run e2e tests
$ pytest --e2e tests_e2e/
# run standard unit tests
$ pytest

Ideally, this test classification would be unnecessary, but it's a minor setback.

Static assets

Another gotcha is configuring the static assets path, so that js, css and other static content is loaded from the test server correctly. To resolve this, I created a custom live server fixture:

Python

@pytest.fixture
def my_live_server(live_server, settings):
    """
    Customized live_server fixture that configures STATIC_URL to point to the live server URL.
    """
    settings.STATIC_URL = live_server.url + "/assets/"

    yield live_server

And then replaced the test parameters with my_live_server.

Test teardown synchronization

The final issue that I've experienced in practice is a non-obvious test failure:

Bash

Exception Database test_xxxxxxxx couldn't be flushed. Possible reasons:
  * The database isn't running or isn't configured correctly.
  * At least one of the expected database tables doesn't exist.
  * The SQL was invalid.
Hint: Look at the output of 'django-admin sqlflush'. That's the SQL this command wasn't able to run.
The full error: deadlock detected
DETAIL:  Process 37253 waits for AccessExclusiveLock on relation 131803 of database 131722; blocked by process 37250.
Process 37250 waits for AccessShareLock on relation 132659 of database 131722; blocked by process 37253.
HINT:  See server log for query details.

It was discovered this is due to asynchronous requests in the browser not finishing before the test ended. For example, consider a React-based page that fetches data asynchronously on load. If the playwright test merely checks the page's title before ending, then the test may end before the asynchronous data has been fetched.

To resolve this issue, I typically add explicit waiting on the relevant network requests:

Python

with page.expect_response(my_live_server.url + reverse("my_api")):
    ...

Ideally, pending network requests could be handled more gracefully.

Update Sep 2025: see a new investigation and possible solution.

Takeaways

The challenges with this approach are greatly outweighed by the benefits. I have yet to find an e2e test setup that doesn't have issues. Writing tests in the same framework (pytest) and with the same tooling (data fixtures, mocks and so on) is a major boon compared to a different language, framework and tooling.

I'll close out by saying that, almost a year after introducing playwright at work, few e2e tests have been written! It goes to show something very important about tests: the real challenge is team culture; no matter how easy it is to write tests there has to be buy-in from leadership and team for the importance and value of tests.

Two Years of Fellowship

2025-03-24T00:00:00-04:00

This is a follow-up to my one year reflection post last year.

Changes

Recently, and more generally, I've experienced changes in my life, mainly in my relationships. Some of my early fellowship friendships have ended, some abruptly and others slowly. I am navigating changes within current relationships, some newly forming, others transitioning. These changes are challenging for me, emotionally and spiritually. I feel more aware of my dysfunctional patterns of behavior, my desire to "act out" emotionally, throw my life into chaos, bury my head in the sand.

The metaphor that's resonated with me recently is Rocky Balboa, going the distance with Apollo Creed. There's no victory in relationships, no knockout, no celebration. There is throwing in the towel, quitting, or getting knocked down and not getting up again.

I feel like I've been going the distance. If I'm generous with myself, the rounds started long ago.

I asked myself why I am experiencing challenges in my relationships. Wasn't my life calmer in the past? Ah, but it was so much poorer spiritually - I had so much less connection, so many fewer friends; naturally I struggled less.

Relationships are the causes, products and ingredients of my recovery.

Okay, that's enough getting current. I needed to get that out.

Staying the same

So much of my life has stayed the same in the past year. I live in the same apartment, drive the same car, have the same job. My relationship to what's stayed the same has changed, though. I feel more stable than ever at work, navigating the challenges of a changing role and long tenure. My apartment is feeling more like home. I'm addressing issues with my car prudently.

I'm adopting a two pronged approach to serenity. One, I aim to work on acceptance of my current situation, whatever it may be. Two, I want to empower myself to make changes and take risks. These go hand in hand for me, as I cannot predict what the future will hold: perhaps I'll remain in the same job/apartment/car for the next five years, or perhaps my company will go bankrupt/car will explode/apartment burn down tomorrow. I want to be able to acknowledge and accept life, no matter the outcome.

Mountains beyond mountains

A key learning in recovery for me is that each stage reveals the next. And there is always a next. I had been feeling steady recently, having navigated emotional challenges in recent months (dating, travel, family). Sure enough, by remaining committed and invested in healing, I soon found more peaks hidden behind the clouds.

While it's intimidating, and at times exhausting, it's the most rewarding and fruitful actions I've taken for myself. At Empty Cloud recently, the monk shared that becoming aware can be quite painful. After all, reality contains a lot of pain. As I lessen up on my emotional and spiritual painkillers, as I grow in my awareness, I am experiencing the pain. And I am coming out stronger for it. The monk said that it gets easier as equanimity grows.

Dating

Speaking of relationships, I would be remiss not to discuss how my dating strategy has changed in the past year. I joined a second fellowship that focuses on dating and intimacy. For me the programs fit together like puzzle pieces. The people that I've met there demonstrate powerful vulnerability and honesty, it's a wellspring of inspiration for me. I'm so glad that I am part of this community.

My approach to dating is the healthiest it's ever been. I am finally slowing down enough to get to know someone before committing. I am setting boundaries with myself and others. I am developing confidence in myself. While I haven't been "successful" in the sense of finding a long term partner yet, I feel that I am no longer blindly stumbling from dysfunctional relationship to dysfunctional relationship. I am showing up authentically as myself, and letting others reveal their authentic selves.

Hope

Throughout all the challenging emotional struggles of the past year, the seed of hope has grown within me. I feel that I can handle what life throws at me. And when I can't, and need to fall apart, I know that I'll have my friends there to help me pick myself back up again.

I am doing things that I never dreamed of (at times I don't even feel like myself!). I am reconnecting with my family, volunteering and making myself of service, and building healthy relationships with myself and others.

I cannot wait to write my next update in a year.

Security Stories Part 2

2025-03-17T00:00:00-04:00

Security vulnerabilities are ~~rarely~~ usually not as simple as a view executing a query param against the database. More often, they're comprised of multiple steps, forming a chain from crafted input to intermediary to - gotcha!

It takes thorough understanding of the code base and adopting the mindset of an attacker. Putting various subsystems together in unintended ways to yield disastrous consequences.

Sometimes, it just takes a little curiosity - this can't possibly work, right?

Whose MFA is it anyway?

At one company, we built support into the application for MFA - multi factor authentication - via SMS. When a user attempts to log in, they are authenticated, but have a flag in their session that MFA has not been validated, preventing them from taking any actions.

All's well and good. Mostly. There's a itch behind my knee that I just can't manage to scratch. Session management is tricky to get right. Especially when developers write their own authentication logic, instead of using the platform or library defaults.. which was the case here.

Looking at the authentication code for the tenth time, I asked myself if it were possible to set a value in a session and keep it after login. If I could set "mfa validated" to true in my session, and then login, I'd bypass MFA. How could I do that though?

Ah, right. I could sign in with a separate account, for which I can validate the MFA. If I then try to sign in on top of that, I could keep the session and bypass MFA for the second account.

Can't possibly work, right? It does! The bug lied in the session regeneration code after login. Instead of generating a brand new session, the user inherited the existing session and was authenticated on top of it. Truly "multi" factor - use any additional factor 🤓

The Great Merge

At another company (what a long career 🫠), I went vulnerability hunting when I joined. I cannot believe that this works at every company. Seriously, if you want to shine before performance reviews, do a lookup for common escape hatch code (xss escaping disabled, csrf exempt view, etc.). Developers are lazy by nature and happily commit dangerouslySetInnerHTML={untrustedInput} before rushing to clock out. The more pickins' for me.

In my search I found several csrf exempt views. Most were fairly benign, uninteresting customer endpoints seemingly due to trouble with ajax requests. Again, rather than configure csrf protection correctly, developers chose to disable it. ¯\_(ツ)_/¯

One view that stood out to is the internal tool to merge customers. You know, customer signs up with email address A and then forgets and signs up with email address B, and gets all mixed up so the customer support team merges the accounts to settle the issue.

Now, missing csrf protection is not enough in 2025. By default, the major browsers will treat cookies as same site lax, which basically means "no cookie for you" for POST requests from other domains.

Now, developers could not possibly have disabled same site cookie protection, right? Say, explicitly set same site "none" so that "yes take this cookie please"? Oh yes, they did. In order to handle some arcane single sign on flow, which used POST to provide auth creds, developers had disabled same site protection. Chain complete!

Almost. Internal tools use uuids to refer to objects. So even with same site cookie and csrf protection disabled, I'd have to figure out the relevant uuids of customers to actually do anything. Did I really come this far to fail?

Of course not, or I wouldn't be writing about this (though the cookie story is a good one in its own right). The customer merge tool uniquely does not accept uuids but integer ids. Cha-ching. I can easily merge customers together by guessing ids. Or incrementing ids..

If I can count incremental ids, why don't I do that on repeat - merge customer 2 into 1, 3 into 1, 4 into 1, and so on: I could merge every single customer into a single super customer! 🚀

I never did try this in practice (a sudden database restore would not have pleased the higher ups), but it remains a fun thought experiment.

More to come

I'm finding this quite cathartic. I derive pride and pleasure from finding and fixing vulnerabilities. I may dig a little deeper in the memory and write some more.

Security Stories Part 1

2025-03-02T00:00:00-05:00

I have a knack for finding security flaws in software applications. When I read about breaches in the news, I am not surprised, merely dismayed. Perhaps contributing to a more honest culture that shares openly about security flaws could help.

In that vein, let me share some of the security flaws that I've discovered at companies throughout my career. I've found quite a few, so this will be a series of posts.

The query endpoint

At one company I worked for, we built custom websites for small businesses. Some were storefronts for pizza parlors, others more involved applications. The founder of the company was a self taught programmer, better at business than programming. This was demonstrated in his code: get the job done and get paid.

I introduced source code management at this company. I also introduced a ticketing system. I tried to mature the company's engineering practices. It was an uphill battle.

Engineers understand the need to debug production systems. The bug can't be reproduced locally, despite trying every imaginable configuration of data states. So we add the functionality to impersonate a user's account, or hook up our local server with a readonly connection to the production database. Tools of the trade.

My boss, clever little devil, considered these approaches too complex, too much effort. He added a web endpoint for querying the production database. It looked like this:

PHP

<?php
// Execute query directly (INSECURE!)
$result = $conn->query( $_GET['query'] );

while($row = $result->fetch_assoc()) {
    print_r($row);
    echo "<br>";
}

Behind a VPN, you ask? Protected with HTTP Basic Auth, you wonder as you start to sweat? Nope. What you see is what was live in production.

Can't beat the convenience when debugging in production.

The password-less login

At another company, I worked a lot on the signup funnel. It was comprised of multiple steps, and we were always attempting to optimize conversion by making tweaks to such and such page, or rearranging the order of the steps. At a certain step in the funnel, the user enters their email address.

My usual approach to test accounts is the gmail + trick. For those not in the know, gmail will accept emails with additional alphanumeric characters after a + sign, for example if the base address is luke.skywalker@gmail.com, then luke.skywalker+1@gmail.com and luke.skywalker+test001@gmail.com will correctly forward. I am in the habit of incrementing +001, +002, and so on.

I so frequently made new accounts in the signup funnel that it became almost second-nature. Almost. One day, when testing some change in the funnel, I entered test+042@gmail.com. Hmm. I had the sudden suspicion that I had made a mistake and re-used an existing test account. I had forgotten to increment.

Then I noticed that I was logged in as test+042@gmail.com. As in, fully authenticated.. without entering a password.. the hair on my neck started to rise!

I quickly confirmed this by logging in as my boss in production. All I had to do was enter his email address in the signup funnel: boss@company.com. I was so excited when I shared this with my boss, his reply, calm and collected:

nice find, when can you fix it?

I learned an important lesson that day. Reporting an issue means that I'm likely to be the one to fix it. I'm such a professional that this doesn't prevent me from reporting, of course 🫠

More to come

I hope to write more about security. I think sharing some war stories is helpful for others to feel comfortable sharing. I hope it also dissolves the myth that security can be taken for granted.

Ride the Wave

2025-02-16T00:00:00-05:00

I posted this picture to a slack channel at work a few weeks ago:

And I got attention (as in, a slack reaction) from several in the company, including the CEO. My boss, responding to or developing the momentum, suggested I talk about the effort in the next team presentation to management. I was surprised, as I post a lot on slack and rarely receive such a response.

What is the image of? The time to load a key page on our website. In theory, an improvement of this magnitude should make a meaningful impact to the business bottom line. So how did I accomplish this sizeable feat?

Engineering initiatives

My company follows a fairly standard prioritization process. Product managers tend to a ticket queue, from which engineers pull to get their next task or project.

What makes my company a little different is the autonomy given to engineers. We're encouraged to exercise our judgment to tickets on the queue. If the effort is under-estimated, and the payoff over-estimated, a ticket can be passed back to product managers for re-consideration.

Engineers are also in charge of the more technical projects, think typical maintenance tasks like routine package upgrades. The lines get blurred at technical initiatives that may have a business payoff, but don't really fall into the realm of our product managers.

One good example is site reliability and performance. The only times a product manager has expressed interest in these topics is when there's a problem, i.e. the website is crashing or so slow as to harm our operations.

Exploratory coding

I like to tinker and poke around in the code base. Usually, I'm on the hunt for security vulnerabilities. On this occasion I was investigating the website performance of a particular page. I became intrigued by the slow performance, and eagerly tore the code apart, looking for the culprit.

This type of work is important and under-appreciated and under-discussed. Engineers need time to let their minds wander, to figure out how that one system works, to prove to themselves that bug fix from last year is still working, and to try to prove their coworker wrong about that argument in the pull request (hey, not all motivation is pure at heart!).

I was able to diagnose an obvious performance regression on this page. The caching implementation was broken, in addition to unnecessary data processing. Basically, it took me an hour to fix, and an extra hour to double check my code.

Socializing wins

My fix was released at the end of the week. The next week I checked to see if it had made a meaningful impact. I should note that one coworker who tested the fix claimed that there was no performance improvement on his machine - a classic eye twitch generating comment!

To verify the site performance, I had to jump through hoops to download production logs, and I even installed new software to generate the graph. I'll come back to this.

As I gain maturity in my career, I am putting more effort into the social aspect of my work. As an engineer, it's easy for me to respond to a problem by writing code and creating a pull request. It's not easy for me to go through the trouble of proposing an idea to the team, building consensus, delegating, and then following through and verifying results. Much easier to create a pull request, even if it never does get merged..

I took this small opportunity to post about the performance win to a public slack channel. I expected a small kudos from my boss and maybe my fellow engineers. Getting the CEO's eyes was welcome - and another opportunity!

Ride the Wave

My team has had its current logging setup for years, before I joined. Its functionality is limited, and it's not available to all engineers nor the product managers. To do anything remotely interesting requires going through its HTTP API. That is how I generated my graph above: I wrote a script to download the logs, convert them to a friendlier csv format, uploaded the csv to my local apache superset instance, and was then able to plot web metrics over time. Talk about barriers to entry!

I believe in setting people up for success. When it's this challenging to analyze production performance metrics, engineers just aren't going to do it. To that end, I seized the opportunity of the CEO and my boss's interest in performance, and advocated for the integration of a superior logging setup. The new tooling would be available to everyone on the team and make it easy to explore and investigate site performance - in addition to other metrics logged. The cost is negligly higher than the current product. Win-win!

I am pleased to say that work has begun on integrating the new logging tooling. I am proud of myself for identifying and seizing the opportunity to make this improvement for my team and the business. I did not anticipate working on logging tooling improvements when I was investigating a performance regression, nor when I was showcasing the results.

When the wave comes, ride it and see where it takes you.

Reproduce Flaky Tests in pytest

2025-02-06T00:00:00-05:00

At work, I occasionally run into flaky tests in CI that prove troublesome to reproduce. Running the test by itself does not fail, nor does running all tests in the module. Sometimes it takes running the entire test suite to trigger the failure - a behemoth at work, currently taking about 10 minutes parallelized 🫠 Clearly this isn't a feasible approach to debugging the test.

My theory for why these tests usually fail in the test suite but not in isolation is randomness, or more specifically, the randomness within the test data generation. We use faker to generate fake data for our Django models. Importantly, we fix the faker seed at the start of the test run. For a given random seed, the flaky test may pass or fail.

This has been borne out in practice. Usually the test is flaking for some specific random value. In one case, it was due to a uuid having a specific substring, b2b.

Reproducing flaky tests

Rather than running the entire test suite, I came up with the idea of re-running the flaky test using a pytest parameter:

Python

@pytest.mark.parametrize("iterations", range(5))
@pytest.mark.django_db
def test_flaky_test(iterations):
    pass

Run just the flaky test with pytest -k path.to.test_flaky_test.

Sometimes a low number of iterations like 5 suffices, occasionally 100 or even 1,000 iterations are necessary to flake. Once the test is reliably failing, debugging is straightforward, and feedback loops are much faster.

Cyteen

2025-01-18T00:00:00-05:00

This month I read Cyteen by C.J. Cherryh for my sci fi book club.

tl;dr I rate the book 2 ½ stars.

Drinking game

Okay, drink every time:

Justin is traumatized
Gehenna or Sociogenesis come up
Someone gets Worked
Any capitalized Word
Justin and Grant make coffee
Someone dines at Changes
Corain is stuck between the extremists and the expansionists
Young Ari saves Reseune's finances (guppies, horses, etc.)
Young Ari saves Reseune with a television interview
Standby. Ari, this is Ari senior..

I could go on. This book is a trip. It felt like more of a soap opera than a science fiction novel. There are so many plot holes and loose ends that I read the ending pure azi.

Waiting for the plot

From the beginning, which introduced a few dozen characters, only for a tiny few to materialize into recurring involvement in the story. But what was the story? There was no discernible plot, time just seemed to move along, events occasionally interspersed between dozens of pages of the same inner dialogue repeated over and over. By the end of the book, I realized there was no climax, no plot curve, just a steady stream of dialogue, internal monologue, and observation.

Okay, but that wasn't the point, right? The point was the character development. Justin is psychogenesis'd into becoming Ari 2's close teacher and research partner - he has the potential for genius, but it has to be brought out through vicious means. Jordan was too headstrong, independent, and in constant conflict with Reseune, so Ari manipulates young Justin's psychsets to mold him into her ideal.

Ari 2's development felt like it was on rails. There were not any surprises or subverted expectations, which may have been part of the point - she became Ari senior, just as cunning and manipulative. Something about that didn't fit right for me. It seemed that Ari 2 was becoming a slightly different person than Ari senior, which I felt strongly when Ari 2 was becoming aware of the manipulation / psychogenesis. I half expected Ari 2 to give up the fight and choose a different path - I think this would've been a more fitting character arc.

Azi

Another recurring theme is the integration of azi into CIT society. There are multiple levels, from genetic engineering to the individual psychology to the group sociology down through following generations (sociogenesis). I thought it an interesting reversal from humanoid robot to robotic human, and it posed challenging moral questions: are the azi human? Can azi consent? Are humans able to be manipulated to such an extent? I would have liked more exploration of this.

Loose ends

So why did Denys try to kill Ari 2? Did Jordan kill Ari 1? Probably not, so who did and why? How did 20 years go by in Union after Ari 1's passing, with practically nothing changing in the political landscape? Who are the rest of the Family running Reseune?

I felt quite frustrated with the loose ends and plot holes - for such a long book, I expected a cleaner ending.

That said, I did manage to finish the book.

Closed Systems

2025-01-10T00:00:00-05:00

I have an idea for an application that would keep track of my relationships - friends, colleagues, family, so on. Its main purpose is to help me stay on top of regular contact and interactions, something that's easy for me to fall behind on. Ideally, it would also help me to develop and nurture positive and healthy relationships, while winding down and backing away from unhealthy ones. I called the idea an FRM, a play on CRM (swap friend for customer).

As I thought about building this, I realized that I'd need to access my contacts and social interactions across platforms. After all, I talk to some friends on discord, others via sms, or whatsapp, and there's one group chat on signal. There's email, slack, google calendar, instagram, hinge.

That's a lot of platforms, and mostly they are not friendly to third party developers. I find this to be a real shame. I should be able to interact with my data on any major platform in whichever way I see fit. Email, an open protocol, supports this - I could download my inbox and parse my emails. However, even sms, plain text messaging stored locally on my phone, is apparently not trivial to scrape.

I believe these closed systems hurt developers and consumers. This is becoming more apparent in the age of LLMs, when my locked away data is used by the closed platforms to build their own models, used against or sold back to me.

I'd love to have a database of all of my social interactions throughout my digital life. Pictures and videos. Train and interact with models on that data set. "Talk" to myself at various stages of my life, get perspective on various events and relationships in my life.

For the foreseeable future, unfortunately, this is a pipe dream.

Django: Speed Up Tests Slight-lier with In-Memory Sessions

2025-01-04T00:00:00-05:00

An article by Adam Johnson recommended a small test speedup for Django unit tests that use the request test client. By bypassing the authentication backend and logging in the user directly with client.force_login(), and also disabling the last login signal handler, a few database queries are saved per-test run.

As I explored implementing this at work, I asked myself - could we skip the database entirely somehow? Does Django require a database for sessions?

The answer is no, Django does not require a database for sessions. In fact, Django supports several different session engines, including the cache - and the cache can be configured to use local memory as the backend!

Caveats

The documentation for configuring cached based sessions states:

The local-memory cache backend doesn’t retain data long enough to be a good choice, and it’ll be faster to use file or database sessions directly instead of sending everything through the file or database cache backends. Additionally, the local-memory cache backend is NOT multi-process safe, therefore probably not a good choice for production environments.

This is usually good and fine for an isolated test environment, but not in all scenarios. Multi-process test environments, such as running e2e tests with the pytest live_server fixture, would not be supported by this approach, since the test process and the server process do not share memory. Code that expects and interacts with database backed sessions will also (obviously) not work.

Configuration

To configure in-memory sessions for Django unit tests, set the following in your settings:

Python

SESSION_ENGINE = "django.contrib.sessions.backends.cache"

You may also want to configure the following, if you are adding a new cache entry for this purpose:

Python

SESSION_CACHE_ALIAS = "test-session-storage"

CACHES = {
    ...,
    "test-session-storage": {
        "BACKEND": "django.core.cache.backends.locmem.LocMemCache",
        "LOCATION": "test-session-storage",
  }
}

Performance

The following test was used for calculating the number of queries in each scenario:

Python

@pytest.mark.django_db
def test_login_queries(django_assert_num_queries):
    user = UserFactory(email="test@example.com", password="test")

    client = Client()

    with django_assert_num_queries(0):

        client.login(username=user.username, password="test")

        assert True, "This is a dummy test to test the number of queries."

Without Adam's changes (i.e. no signal disconnection, and using client.login()), the result is 20 queries. The base is fairly high due to a custom signal handler for user login.

With Adam's changes, the result is 14 queries.

With the local memory session backend, the result is 0 queries. Amazing!

Conclusion

While stimulating as an exercise, in practice with my work codebase and test suite I found negiglible time savings. The test suite includes maybe a dozen tests using the test client, while the vast majority test at the service layer.

Thailand

2024-12-30T00:00:00-05:00

I spent 3 and a half weeks in Thailand in the month of November. This was a spectacular trip.

Trip report

I left on the day after the election, flying through Istanbul to Bangkok, 2 roughly 10 hour flights back to back. I arrived in the morning, waited for my friend at the airport, and then taxi'd to Khao San Road, my first stay.

I have never been anywhere like Khao San before. Walking the street, I had all senses assaulted: sight, smell, sound. The music blared so loudly that I felt the air around me vibrating. Street sellers hawked their wares aggressively; eye-contact is to be avoided at all costs. The street is a carnival of sin.

I spent a few days in Bangkok, exploring temples, museums, and wandering the city. Ending a day on my feet with a foot massage was heavenly.

From Bangkok I traveled with my friend to Phuket. We stayed in Kata Beach, in a hotel primarily occupied by Russian families. Hearing spoken English made me giddy. This was my first time in a tropical island (though Phuket doesn't feel like much of an island), and I made the most of it, relaxing on the beach and watching the sunset.

We took a couple day trips to Phang Nga Bay, one snorkeling and swimming, and another kayaking. While heavily touristed and crowded, nothing could detract from the natural beauty of the bay and the islands. Snorkeling I saw tropical fish swim in the coral reefs (sadly, quite bleached). On the kayaking trip we explored caves and lagoons, finishing with a visit to the bioluminescent plankton. There was a thunderstorm in the distance, and the night sky was clear, so I saw above lightning and stars and below the glowing plankton.

I think I achieved a form of adrenal burnout, I had so many amazing experiences, I could barely work up the excitement to leave the hotel by the end of the trip.

After Phuket, I flew solo to Chiang Mai in the north. This was my first truly solo foreign trip. I spent almost a week on my own, hiking, day tripping, tasting delicious local coffee, and visiting an elephant sanctuary. Old town Chiang Mai was one of my favorite places I've visited, it felt dense yet small, busy but not crowded, and with fantastic food and coffee within a 15 minute walk.

Finally I returned to Bangkok to close out the trip for a few days before flying home. I shopped for souvenirs and had one last lunch with my friend.

Recommendation

I recommend Thailand to everyone. It is such a friendly destination, easily accessible, easy to get around. There is something for everyone, for any taste, from bustling Asian city to tropical island to hiking in mountain jungles.

End of Year Review (2024)

2024-12-23T00:00:00-05:00

It's once again the end of the year, and a time for reflection once again. This follows my end of year review for 2023.

Travel

I traveled to Amsterdam and Brussels in January/February. This was a bit of a chaotic trip, only about a week, and I was working remotely part time. Amsterdam is a cute European city, with lovely scenery (architecture, canals) - it felt like a great destination to take your romantic partner. In Brussels I attended FOSDEM, my first tech conference. I was pretty overwhelmed with the bustle of the crowds. I found the city to be significantly more grimier and grittier than Amsterdam, though enjoyable and charming in its own way.

In May, my friend/colleague visited the United States for the first time, and we traveled for a couple weeks together. First to Pittsburgh for PyCon, where we joined two other colleagues. While FOSDEM is volunteer run, PyCon has a strong organization backing it, and the difference was clear. PyCon had massive stages in a giant conference center. I found Pittsburgh to be charming, the downtown nestled in between rivers and mountains. After PyCon, my friend and I visited DC, joined by a third friend. That was a blast, I love DC, its small buildings, wide streets, and green spaces encourage me. Back in the greater New York area, we did some NYC tourism.

In July, I took my first solo trip to Gloucester, Mass. It was a quick weekend getaway, a desperate attempt to catch some relaxation and escape the FOMO of others' travel plans. I went whale watching and sat on the beach for a rare Atlantic coast sunset.

In August, I took two trips: first a weekend camping trip, then a trip to visit my brother's family in LA. This was my first time camping, and I had a stressful but enjoyable adventure with friends. In LA I got to visit my young nephew, who is just too adorable.

What a year! In October I traveled to Chicago for my work's yearly management offsite. I enjoyed spending in person time with my team. On the last day, I went on the architecture boat tour, a really unique experience.

I can't summarize my experiences in a paragraph - in November, I went to Thailand for 3 weeks. This was an incredible and life changing experience in so many ways. I will write more about this trip later. For here, I'll just say that this was the most amazing trip I've taken to date.

Books

I read 15 books this year, a decrease from the last. Mainly it was my sci fi reading group that kept me consistently reading throughout the year - I usually tend to read in bursts. My top books from this year:

The Saint of Bright Doors
The Information
Use of Weapons
The Lathe of Heaven
The Empty Mirror
In Ascension

I want to note that this was my first foray into writing reviews for the books that I read. It made for a good series of blog posts, and joined nicely with the book club discussions.

Writing

I blogged once a week for 7 months, writing more than 30 articles. Wow! That's more than I've ever written before. It has been a learning experience, and a process of trust and letting go. I had set the goal of writing every week, and when I faltered, I wasn't able to pick back up on the momentum - the overdue backlog prevented me from just writing something.

For next year, I want to write, but I will reduce my goal to twice per month. That will reduce the pressure - I would have accomplished that goal this year, actually. If I write more, great.

Career

I continued my job at Rinse. I worked on a number of projects, notably logistics efficiency, and improving system security. I was also assigned two direct reports, my first foray into people management. I have mentored and coached my reports for the past 6 months, culminating in delivering their end of year performance reviews. This has been a rewarding experience, and I'd like to write more about it.

I attended my first two tech conferences this year. Having never attended before, and attending with other conference newbies, I felt fairly overwhelmed: the crowds, the networking, the events. Working remotely it was a lot to lean into suddenly, though I feel much better prepared for the next conference (PyCon '25?).

Projects

I did not work on side projects this year. I started on several, but never got far past the idea phase.

Personal development

Personal development was my main focus for 2024. Reflecting on my 20s, I see how my mental health was my biggest roadblock to success and happiness. In the past year, I regularly attended support groups, and built a support network for the first time in my life. I've met the kindest, most caring and loving people, and nurtured positive, healing friendships with them. I am truly grateful for their support.

I continued attending meditation at my local Buddhist monastery. I sat for my longest meditation, of more than 50 minutes without break. I learned the dharma and started integrating myself into the sangha. The rewards of my meditation practice are stronger emotional resilience, presence and serenity.

I took a break from dating for the first six months of the year, a challenging but immensely rewarding experience. I went on several first dates, and dated two women for 1-2 months each, culminating in a devastating breakup in December. I feel way more confident about dating and relationships than at any point before, having really put in the work on myself. I am excited to put myself out there in 2025.

My experiences with travel this year were wonderful. I have a personal theory, something I haven't found the right words to summarize, that regular vacations and breaks (every 3-6 months) are a crucial step in development. The day to day grind, meditation, journaling, therapy, etc., builds and builds but it's when I take a step out of my normal life that suddenly all of the gains are realized - how about, like slowly revving the engine and then shifting into a higher gear.

Last year's goals

Looking at my goals from last year, let's see how I did:

🛑 Regular physical activity
🛑 Reducing screen time
🛑 Learning about finances and investing
✅ Continuing to nurture my physical health
✅ Continuing to nurture my mental health
✅ Continuing to nurture my spirituality
✅ Continuing to invest in friendships
✅ Continuing to practice meditation
✅ Continuing to eat well
✅ Continuing to sleep well
✅ FOSDEM and PyCon
✅ International trip
✅ Domestic trip
🛑 Move abroad

I struggled to engage in regular physical activity. Reflecting on that, I would like to try to socialize exercise, rather than go it alone - a key learning of mine in mental health that I want to deploy to other areas of my life.

Same for reducing screen time. To be fair, I didn't really play video games at all last year, and that's coming from someone who was a big gamer only a couple years ago. So there's a clear shift in how I engage with technology. Notably, I watched a lot more tv and movies. I also radically changed my internet browsing habits, reducing to nigh zero some really unhealthy content intake. So while the specific goal wasn't met, I would be remiss not to recognize what I did achieve.

Finances and investing - I've left this on autopilot, investing in basic index funds.

Moving abroad, when I turned 30 in a panic I started Spanish lessons with the plan of moving to Barcelona for 3 months or so. I did not follow through on that plan, which I do not feel bad about. Someone wise said to me, be careful that you're running to something, not from something. And I was clearly running away from my emotional pain. I am glad I decided to hunker down in New Jersey, where I have the support to heal.

Next year's goals

Now, what about next year. It's easy for me to load up on goals, aim for the moon in a million different directions. Instead, I'll choose a few priorities.

Physical health
- Try joining a group fitness activity (club, class, studio, casual sport, etc)
- Trial a physical trainer
- Exercise 2x/wk
- Follow up on outstanding physical ailments
Career
- Blog 2x/mth (including technical/career content)
- Attend a tech conference (stretch - present a lightning talk / submit a talk proposal)
- Launch a side project (stretch - make my first independent $)
Relationships
- Nurture and develop friendships
- Attend in-person dating events
Travel
- 2 international trips (1 wk and 2 wks, maybe Morocco, Japan)
- 3 domestic trips (camping, meditation retreat, ?)

The best year of my life

This has been a challenging year for me. I've experienced enormous pain and discomfort as I've worked on healing my emotional wounds. I no longer feel so alone, having found and nurtured a support network of loving friends. In so many ways I feel like the same person as a year ago, and yet I see the changes, the light pouring through the cracks. This past weekend I was visited by my ghosts of Christmas past. I am breaking patterns that were previously invisible. Every moment is a rebirth.

Even though this month was emotionally devastating for me, I am standing tall, weathering the storm. As in small, so in the large. This year I stood tall and weathered the storms. I lived the fullest, most intentional, most fearless, most present year of my life.

All of the effort in personal development is an investment into my present and my future. After my 2^nd best year of my life in a row, I can't wait for next year - no pressure on myself :)

Lathe of Heaven

2024-07-13T00:00:00-04:00

The 2002 TV movie adaptation of The Lathe of Heaven disappointed me. I liked that the bell hop seemed to remember George throughout the changes, but the overall plot was quite dry. So it was my surprise that the Wikipedia article mentions the adaptation excluded an alien invasion, plus all the philosophical nuance. What in the world did I watch?

My sci fi book club affirmed my suspicions. So I ordered the book to find out for myself.

Let me just say - I was shocked. This book is incredible. Reality-bending, philosophical, psychological - a perfect complement to Philip K Dick, without the paranoia and terror.

tl;dr I rate the book a rare 5 stars.

Overview

George Orr is abusing drugs alternately to stop dreaming or to stay awake to avoid sleep. Because Orr's dreams have a tendency of creeping into the real world. His first memory of changing the world is when his aunt came to stay with his family. Orr didn't care for his aunt; he dreamed that the aunt had died in a car crash. Lo and behold he wakes up to find his aunt had actually died some months ago in a car crash.

As a sane person, this disturbed Orr. No one else remembered the previous reality. The guilt, the pressure, the responsibility weighed on him. So he abused drugs to stop dreaming.

Drug abuse leads Orr to a state mandated therapist, a sleep and dream specialist, Dr. Haber. With the use of Haber's experimental new dream machine, the augmentor, Orr is given hypnotic suggestions and coaxed into dreaming. Dr. Haber comes to suspect and realize that Orr isn't just a psychotic quack - Haber, being so close to the dream epicenter, watches the world around him shift.

The characters make this story as much as the plot. Haber is an idealist, an inventor, a scientist, a psychologist. He wants to tinker to try to change, to try to improve the world. He is a large man, both physically and psychically. Orr is the opposite, small, mild mannered, meek, inward and not outward focused. Orr works a simple job as a draftsman for the state, lives a modest and isolated life.

Haber tries to use Orr's dreams to make the world a "better place." However dreams are not scientific, hypnotic suggestions are interpreted creatively by the subconscious. In an attempt to solve overpopulation, Orr dreams of a pandemic that decimates the world population. To achieve world peace, Orr dreams of an alien invasion that unites the warring nations.

Orr becomes increasingly disturbed by Haber, though he doesn't quit the sessions. Orr reminds me of the protagonist from Sheltered Lives - unable to act, to make a choice, trapped by himself in others' plans and designs. Orr does reach out to a lawyer, a woman named Lelache, to try to help him. Lelache, in attending a dream session, is thus caught up in the shifting realities, caught in Orr's wake.

The story comes to a climax when Orr finally refuses to continue working with Haber. However, Haber has finally perfected the augmentor to allow himself to dream these reality-altering dreams. In dreaming, Haber starts to unravel the world, unable to form a coherent narrative, destroying rather than creating a new world.

Orr reaches Haber and shuts off the machine. The world remains in shambles, contradictory realities mixed together. The story ends with Orr visiting Haber, now institutionalized.

Oh and Orr makes friends with the aliens who he dreamed to become friendly with humans.

What I liked

What did I not like. This story suits my fancy in every aspect. I love the dichotomy between Orr and Haber. Orr is an excellent character, not a hero, not an anti-hero, but completely relate-able, suffering from the weight of global responsibility.

I love love love reality bending stories - though that market was cornered by PKD, entries like Dream Master and The Lathe of Heaven rightfully claim their spot alongside The Three Stigmata of Palmer Eldritch.

The plot was executed so well. Orr is buddha-like when he shuts off the augmentor, at peace yet projecting his will to shape not the world but his place within it.

What I didn't like

Maybe I longed for Orr and Lelache to end up together - though it's hinted, so not truly missing. Honestly I can't find any significant faults with this book. The fact that it's short is a bonus: fitting a story to its proper length is no small feat and makes a huge impact.

Wrap up

Read the book, spend the next few weeks wondering whether your dreams are also changing the world while you sleep.

Shifting Baselines

2024-07-13T00:00:00-04:00

I was reading an article about marine population collapse and in it was mentioned a concept called the "shifting baseline."

Implicit biases

The shifting baseline is a bias whereby what's normal (that is, their baseline) is relative to one's perspective. Okay, sounds obvious enough - normal to me depends on what's normal to me. What's interesting about that?

Consider what happens over time. In the marine population article, it describes how the first generation of marine scientists noticed that populations fell 80% in their lifetimes. Later generations noticed the same rate of collapse within their lifetimes. That is, each generation of scientists were introduced to a different baseline.

So from 80% collapse to 20% remaining. From 20% remaining to 4%. From 4% to 0.8%. When each generation of scientists considers the population levels during their early careers as the baseline, they each notice only a modest relative decline, but over time this amounts to an exponential, catastrophic collapse.

Life

There are quantitative ways to demonstrate this bias in normal life. For example, inflation - ask your parents what they paid for a mortgage or to rent. Even within one's own life, the prices of some goods or services will rise or fall, such that it takes active effort to keep up to date with everyday prices.

One area I have been applying the shifting baseline model to is overload and burn out. I was shocked by a study cited in The Information in which participants elected to continue receiving more information despite it overloading them.

I notice this tendency within myself. Taking on one more obligation, one more responsibility, signing up for one more activity, all seems innocuous - the incremental effort or load is minor. Over time, however, without monitoring my overall commitments, these all add up and threaten to collapse the entire house of cards.

Along these lines, I can also grow accustomed to a certain level of responsibility, shifting my baseline to more and more activities. But I can't keep adding more - I have finite time, energy, availability. Something has to give. The patterns of the past can't keep accumulating.

Look around you

History is the antidote to the bias. Zoom out, try to see the big picture. Confused about current events? Don't understand why the world is the way it is? Don't understand how you got where you are? Look at the past, try to understand it to better understand the present. Don't get caught off guard by a shifting baseline!

Witch King

2024-07-06T00:00:00-04:00

I just finished reading Witch King by Martha Wells for my sci fi book club.

tl;dr I rate the book 2 stars.

Overview

I am writing this some time after reading, so my memory is fuzzy.

There are two plots in parallel across two periods of time. In one, in the past, we learn the history of the beginning of the rebellion against the Hierarchs. In the other, in the current, we follow the witch king Kai as he unravels the mystery of his imprisonment.

Some time ago the Hierarchs invaded the continent and destroyed everybody in their path. Literally - they razed cities, slaughtered the nomadic tribes, using a powerful form of death magic. Why did the Hierarchs invade? We never find out.

What we do find out is how Kai and a prince team up to break out of the palace where they are imprisoned. There's a strong degree of plot armor over the participants, as we follow their parallel journey in the future. Long story short, Kai manages to defeat the Hierarchs in the palace and the prisoners escape.

In the present, Kai wakes to find himself imprisoned once again in an underwater tomb - water disables Kai's demon powers. After escaping, Kai joins with his band of friends to find their missing friend, in the hopes of figuring out how they were imprisoned.

The band of heroes travels around the world, ultimately arriving back at the same palace from the earlier timeline. There the final showdown occurs: we discover the conspiracy consolidate power in the alliance that emerged after the defeat of the Hierarchs.

What I liked

I found this book an easy read, which was nice after slugging through The Information, a far more challenging book. That said, I liked the different magic systems, and I thought there was potential in some of the characters, but I didn't feel it was executed upon.

What I didn't like

Clothes. Everybody's clothes are described in detail! Why.

Who were the Hierarchs? What was their motivation to invade? What was their grand plan? So many unanswered questions.

I also didn't care for Kai's lack of confidence. As a demon among humans, capable of draining the life from victims, and later learning how to conjure magic, I was disappointed that Kai was so awkward. I mean he's the Witch King! Act like it.

Wrap up

I wouldn't recommend this book. There's more rewarding fantasy out there.

Feedback Loops

2024-06-29T00:00:00-04:00

I've been reflecting on feedback loops since reading The Information. Coming to work on a Monday morning, with fresh eyes, I jotted down a few factors affecting feedback loops in my day to day life.

Tickets

As an engineer, I expect the tickets in the queue to be appropriately prioritized, requirements explicit and clear, and reasonably scoped.

When tickets don't meet these criteria, there's friction - sometimes small, sometimes large. Perhaps this ticket is no longer relevant. Perhaps this ticket is a vague bug report without reproduction steps ("XYZ is broken"). Or perhaps the queue is full of multi month projects.

Over time, that friction becomes internalized. When I have time (waiting for code review, in between projects, etc.), I hesitate before looking into the queue. I prefer to work on developer initiatives, where I can control scope, understand requirements, and prioritize myself.

Time from code to release

Maintaining context for tickets is a significant challenge in software engineering. When I try to work on multiple tasks simultaneously, alongside overseeing others' tasks, performing code review, and planning future projects, I struggle to context switch multiple times a day and dozens of times a week.

A long turnaround time between writing and releasing code implies a lengthy period of juggling multiple contexts. I am reluctant to pick up additional tickets while several are in code review, QA or pending release.

Momentum and motivation fade quickly, and I've noticed lack of interest in monitoring and bug fixing from engineering as well as product when tickets are deployed weeks or months after ideation. The sentiment is "the excitement is gone, so let's move on."

Yet another aspect is decreased hustle. If code I write today will take days to be reviewed, then another week to be tested, and yet another week to release - then there's little difference between finishing my code today or tomorrow or even later in the week.

Shifting winds

Business priorities shift - in a healthy market, that's a sign the company is re-aligning itself towards success, responding to changing market conditions, reacting to or getting ahead of a competitor, and so on.

When business priorities constantly shift, it's a drag. How can I stay motivated on my current project, that's absorbed the attention of management, if I know from experience that in a month we'll have de-prioritized this "critical" project? It says to me that the work is disconnected from business outcomes, and that business outcomes are disconnected from the market.

Ideally, product work makes tangible impact, and failure to produce has consequences. When work is disconnected from outcomes, when success is disconnected from investment, then lethargy is a natural consequence. This applies not only to actual outcomes but perceived outcomes and company culture: if wins are objectively successful but not celebrated, if failures are not analyzed, then passion is incentivized against.

Wrap up

Feedback loops permeate my life, from work to home to spirit. Getting a grasp on them, becoming aware and then taking action, has the potential to yield large rewards for small changes.

The Information

2024-06-22T00:00:00-04:00

I just finished reading The Information by James Gleick.

tl;dr I rate the book 4 ½ stars.

Overview

This is the third book I've read by Gleick, after Chaos and Faster. I found it to be the most ambitious yet, tying together fields of study from history to biology to mathematics to communication.

The central theme is, of course, information. But what is information exactly? That is the question posed and re-posed, answered and answered again by scientists, philosophers and linguists over the centuries.

I find it a little ironic that trying to summarize the book here is to try to compress its information - an endeavor attempted broadly by new apps like Blinkist. What does it mean to summarize a book? Can the information be losslessly compressed? I think the repetition, the footnotes, the asides, in a word the redundancy is part of the learning experience for human beings. If I got anything out of this book, it's to appreciate information density and redundancy.

That out of the way, the book describes the history, the theory and the flood of information.

The history is told through the invention and adoption of new information-bearing technologies, namely the telegraph. For the first time in human history, communication could travel faster than a human being. Famously, a criminal jumped onto a train, believing himself to have escaped - only to be met by the police at the next stop, having been informed via telegraph! The telegraph, like written language before it, encoded information, now into dots, dashes and pauses.

Claude Shannon would go on to develop the laws of information theory in rigorous mathematics. This marked a revolution in the sciences, spreading to biology and physics. In biology, the discovery of DNA marked the transition to understanding life in terms of information transfer, famously espoused by Richard Dawkins in the Selfish Gene. In physics, information theory led to advancements in the understanding of black holes - eventually resolved by Stephen Hawking - and quantum mechanics.

Finally, the flood - that is, the modern day human experience of the internet, instantaneous interconnectedness to any person, book, article, movie, song, etc.. It surprises me that that psychologists in the mid century started studying information overload - without cell phones, email, or push notifications. One key takeaway was a study in which participants kept asking for more information even when it was detrimental to their effectiveness. Perhaps we have a thirst unable to be quenched, because in our history we lived in a dry desert.

What I liked

Gleick's books make me feel smart - I attribute that to his skill at science writing, I finish feeling like I learned a great deal. This book was education in breadth as opposed to education in depth with Chaos. When picking it up, I didn't imagine it'd cover Gödel's theorem, Maxwell's demon, African drum talk, Turing machines, Kolmogorov complexity, quantum mechanics, and the memetics. Gleick managed to weave it all together in a compelling narrative - I had no trouble keeping motivated to finish the book.

What I didn't like

My only complaint is that the book's ambition may have exceeded its result. In a way, the book itself is flooded with information, and that can drown out the message (how meta!).

Wrap up

James Gleick is one of my favorite science authors, and The Information delivers. Do yourself a favor and check it out.

Bonus quote

I promised myself I'd record this quote from the book, from Robert Burton in 1621:

I hear new news every day, and those ordinary rumours of war, plagues, fires, inundations, thefts, murders, massacres, meteors, comets, spectrums, prodigies, apparitions, of towns taken, cities besieged in France, Germany, Turkey, Persia, Poland, &c. daily musters and preparations, and such like, which these tempestuous times afford, battles fought, so many men slain, monomachies, shipwrecks, piracies, and sea-fights, peace, leagues, strategems, and fresh alarms. A vast confusion of vows, wishes, actions, edicts, petitions, lawsuits, pleas, laws, proclamations, complaints, grievances are daily brought to our ears. New books every day, pamphlets, currantoes, stories, whole catalogues of volumes of all sorts, new paradoxes, opinions, schisms, heresies, controversies in philosophy, religion, &c. Now come tidings of weddings, maskings, mummeries, entertainments, jubiless, embassies, tilts and tournaments, trophies, triumphs, revels, sports, plays: then again, as in a new shifted scene, treasons, cheating tricks, robberies, enormous villanies in all kinds, funerals, burials, deaths of Princes, new discoveries, expeditions; now comical then tragical matters. To-day we hear of new Lords and officers created, to-morrow of some great men deposed, and then again of fresh honours conferred; one is let loose, another imprisoned; one purchaseth, another breaketh: he thrives, his neighbour turns bankrupt; now plenty, then again dearth and famine; one runs, another rides, wrangles, laughs, weeps &c. Thus I daily hear, and such like.

All that without Twitter. Nothing new under the sun.

Code Slop

2024-06-13T00:00:00-04:00

It all started in code review

At work this week I was reviewing code when I came across this piece of code (anonymized):

Python

def can_be_foobard(self) -> bool:
    """
    Checks if the object is eligible for foobar.

    :return: True if the object can be foobar'd, False otherwise.
    """
    ...

I immediately suspected that this was copilot generated: why on Earth would a human repeat themselves 3 times over? This snippet could be reduced to:

Python

def can_be_foobard(self) -> bool:
    ...

The entire docstring is unnecessary! In terms of the information content, the original version is needlessly verbose, providing redundant explanation of the method name. As "slop" is the term for unwanted AI-generated content, this sample falls into the code slop subcategory.

From a previous discussion

Last week, my team had a discussion around AI tool best practices. We noted that copilot can be instructed, that is, for comments that describe a task, copilot will suggest an implementation. This is a great experience, as it fits entirely within my editor workflow.

However, the point was raised that developers should remember to remove those instructions, as they typically don't provide any additional informational content over the code.

While we're still in the early days of using large language models for software engineering, I foresee some patterns emerging. Engineers are lazy, and will not take time to remove copilot instructions. They may also be prone to mindlessly accept copilot suggestions, as in the docstring example above.

This suggests to me a trend towards increasing code verbosity via code slop. Combined with existing software tendencies to increase verbosity (not-invented-here syndrome, do-repeat-yourself syndrome, technical rot), software engineers may be in store for misery.

And optimism

I don't believe it's a guaranteed fate. I find it fascinating that LLMs perform better in discussions and long answers. Perhaps that's due to a low "information density" in the models. If a human can understand a question at face value, without multiple rounds of interaction, then I expect the models will catch up eventually.

And when they do, model output will be short and sweet, to the point. In the mean time, I'll hold my nose when reviewing code slop.

Three Decades

2024-06-08T00:00:00-04:00

Tomorrow is my thirtieth birthday. I'd like to take the opportunity to reflect.

Accomplishments

I tried writing this post as a narrative recollection of my twenties, but I don't feel that I'm ready to share that yet. I experienced a lot of pain and conflict that I am still processing. Instead, I am choosing to share a sample of my accomplishments, highlighting the positives of the past decade.

I returned to and graduated college.
I have worked steadily for five years, advancing in my career from junior to senior software engineer.
I have traveled to four new countries: India, Spain, Amsterdam and Belgium.
I have read about a hundred books.

I have more accomplishments, more personal in nature, that I choose to keep to myself and close friends for now. I am still getting comfortable with blogging publicly about my personal life: I find it highly rewarding and liberating, but I am choosing to ease into it.

Next decade

In the next decade of my life, I have a few high level goals:

Get married and start a family
Attempt a leadership role in my career
Launch my own company/product
Buy a house

While each of the above is daunting, I have never felt better equipped. I look forward to blossoming in my thirties.

Travels

2024-06-01T00:00:00-04:00

I watched my friend depart on the train this morning, headed back to Spain. It's been a whirlwind two weeks.

PyCon

I picked up my friend from the airport two weeks ago. It was his first time visiting the United States, and his first chance to meet team mates in person. We stopped for drinks with a few New York City locals, then headed back to Jersey.

Early the next day we set out in the car to Pittsburgh, through the beautiful rolling hills of Pennsylvania. Of course, we breakfasted at Cracker Barrel - the most American road trip diner.

This was my second time in Pittsburgh, but the first in Pittsburgh proper. The city is gorgeous: the meeting of the rivers nestled in the mountains makes for a scenic downtown.

The conference was excellent, though my highlight was the time spent with coworkers.

DC

The next stop on the road was Washington, DC. My European friend described it as "European," which is also how a resident of DC described it in comparison to London.

We toured DC with another friend of mine, also visiting for the first time. I led us on a bike tour of the monuments, along the Mall and the Tidal Basin. As for museums, we wandered the air and space museum (unfortunately partially under renovation), the national gallery of art, the botanical gardens, and the national museum of asian art. My highlight was the Wright brothers exhibit, and the following quote:

If you are looking for perfect safety, you will do well to sit on a fence and watch the birds; but if you really wish to learn, you must mount a machine and become acquainted with its tricks by actual trial.

I also met up with ex-colleagues from my last company. It was nice to reminisce - and also nice to reflect on how much I've grown in a few short years.

NYC

We drove back to Jersey, with a quick stop in Princeton for lunch. The weather was warm and sunny, the campus lush with ivy.

The next week was mostly spent in New York. One evening we dined with my siblings. Another we watched Hadestown on broadway - I enjoyed the set design.

The highlight was Memorial Day, a jam packed day of activities. I tried bouldering for the first time: there's an efficiency of movement that a newbie doesn't grasp. Then, an epic game of chess - me vs 3 friends, from which I came out victorious. Then, my first time at a gun range: good 'murican fun. After, billiards, and finishing with a burger and beer.

One takeaway

My favorite part of traveling is breaking out of my daily routines. I am susceptible to tunnel vision, to falling into a rut of the well-trodden daily habits.

I appreciate the broader perspective that comes from new experiences. I am delighted when I discover that I still have more to discover about myself.

My key takeaway from this trip regards fear. When I was rock climbing, even though I was held securely by the ropes, I could not jump off the wall. I felt the fear viscerally, I willed it but my hands wouldn't budge. Fear is deep rooted, in the "emotional brain" - it cannot be reasoned through. It has to be felt, acknowledged, accepted. It's uncomfortable. But it's not insurmountable.

PyCon US '24

2024-05-25T00:00:00-04:00

I attended PyCon US '24 with several of my colleagues. This was my second tech conference, following FOSDEM '24 back in February.

I had a great time at the conference. I was fairly overwhelmed, but thankfully the conference was well organized: the schedule was clear, rooms easy to find, and meals unexpectedly tasty.

Attending a conference just for the talks misses out on the unique opportunities of meeting in person. After all, the talks will be available online a few weeks later. I tried to put this into action by making the most of the non-talk opportunities.

Takeaway: done is better than perfect

This was delivered during a lightning talk by a Korean mother who returned to the workforce - an impressive feat in the West, let alone in a culture with stricter gender roles. She described how she started posting videos to youtube to rebuild her tech skills and reach an audience: she would record a video during downtime with her children, and upload without editing.

This lesson hits me at the core: writing this blog is a challenge, uploading without rounds of revision sounds terrifying! And that's with a readership that I could count on my fingers: me, myself and I 🫠

Takeaway: LLM keynote

Simon Willison gave an excellent keynote on LLMs. His passion is infectious. He is also a great educator, explaining various topics that I had felt I wouldn't be able to understand.

For example, he explained Retrieval Augmented Generation (RAG) as just providing extra context for a given query into the LLM input. The extra context may come from a full text search over a given dataset.

For another example, he explained the security issues with LLMs. Why can't we have autonomous agents that operate on real world input? Because LLMs are susceptible to prompt injection attacks: LLMs can't reliably distinguish between instructions and content, the data and command channels are combined.

Takeaway: open spaces

I attended a couple open spaces on security. The first, run by OWASP, was a group card game that helped teach about security. The second, run by PyPI and Github, was a discussion about security in the supply chain.

I also attended an open space on solo dev work - this took the form of a discussion about managing clients, working through timelines, and general entrepreneurship.

I found the open spaces to be more engaging than talks, due to the small groups and open discussion.

Takeaway: talks

I enjoyed the talks on:

implementing a python debugger
design of everyday APIs
extensibility in design
immortal objects
functional error handling

Check them out when the videos are published online.

Takeaway: time with colleagues

My favorite part of the trip was spending so much time with my colleagues. Meals, talks, and even just hanging out - working on a remote team, the face to face time nurtures relationship development that's just not possible via a computer.

Wrap up

I look forward to my next conference trip, solo or with team mates. I walk away each time with excitement and energy for my field of work.

The Saint of Bright Doors

2024-05-18T00:00:00-04:00

This month's selection for the sci fi reading group is the Nebula award nominee The Saint of Bright Doors by Vajra Chandrasekera.

tl;dr I rate the book 3 stars.

Plot

This story takes place in a surrealist modern southern India. The protagonist, Fetter, is in group therapy, having been groomed to murder his saintly father, the Perfect and Kind.

The therapy group is for the "almost-chosen" - the world is full of myth, and not all are meant for greatness. It is here that Fetter meets Koel, and joins her revolutionary movement to overthrow the Luriati government.

Luriat is a fascinating place: it's imbued with magic, whereby closed doors, when unobserved, will transform into doors that can no longer be opened. These doors are traditionally painted bright colors to identify them (hence the name of the book).

The country is also host to religious, ethnic and caste discrimination. I'm not familiar with the history, but the various occupations suggest possibly British and Mughal rule. Out of these occupations, and mixing with the local traditions, comes religious conflict between the Path Behind and the Path Above (both venerating the Perfect and Kind, but in conflict with one another).

Fetter's story follows his early adulthood, as he struggles to find his way in the world. He moved from his home town in Acusbad to Luriat. He dates, but is unable to lower his guard, keeping his partners at a distance. Fetter keeps everyone at a distance, including his mother (named Mother of Glory) and the therapy group: Fetter tells no one that he sees demons, antigods - the invisible laws and powers of the world. These creatures stalk the earth, bringing plague and ill portents.

Fast forwarding through the story, Fetter goes undercover to study the bright doors for the revolutionary movement, all the while making connections to the powerful of Luriat. The Perfect and Kind is coming to Luariat to reconcile the warrning Paths. An artifact is discovered which has the power to kill the Perfect and Kind. Fetter steals the artifact, and sets out on a mad dash to kill his father. He makes a final visit to Mother of Glory on her deathbed.

The assassination attempt fails - the Perfect and Kind rewrote history to avoid it. Mother of Glory explains the backstory to Fetter: originally, the Perfect and Kind was just a prince from another land. He learned the powers of magic from her people. In order to escape her curse, he rewrote history to bridge the island to the mainland, such that the island had always been connected to the mainland.

Fetter attempts to return to Luriat, but is arrested at the border. He is sent to a prison camp, which he wanders for possibly months before finally being freed by the Perfect and Kind.

Back in Luriat, Fetter seemingly submits to his father's wishes to follow in his footsteps. All the while, we discover there's a surprise first person narrator - Fetter's shadow, which had been taken from him as a child. The shadow kills the Perfect and Kind, and Fetter rejoins the revolutionary movement.

Parallels

Thanks to some reviews I learned that this story closely parallels a story about the Buddha, in which he leaves his wife and child to start the ascetic life. Thus the name "Fetter", or (worldly) "shackles." I didn't pick up on this myself, being unfamiliar with that story.

What I liked

While the first part dragged on, eventually I couldn't put this book down. I am reminded of Roger Zelazny: a world of demi-gods and magic, yet thoroughly modern. I enjoyed the reality of the mythology: the different "almost-chosen" characters living in the shadow of their individual destinies.

I liked the limited use of magic, especially world-shattering magic: the Perfect and Kind changes history just twice, each time to escape his demise. Afterwards, he acknowledges that such a powerful magic has unintended consequences: changing a river upstream may cause radical changes downstream (hello, chaos theory!). Thus his Path splits and fractures.

The prison sequence was well written. The description of the camp was evocative: the misery and the lack of coordination; Fetter's inability to find his particular zone, until he stumbles upon it; the mini worlds within the camp walls. It was thoroughly Kafkaesque, and a great representation of the horrors experienced by real people in real camps.

Theme-wise, I appreciated the author's attention on identity, childhood trauma, and destiny. Fetter is clearly traumatized: being raised to kill his father by his vindictive mother may be a simple allegory for a child raised by a single mother who hates the deadbeat father. Fetter struggles to define his own identity: he finds himself pulled into identities prepared for him by others, including the neighborhood "helper", the undercover role, the revolutionary, the boyfriend, assassin, and "chosen."

What I didn't like

While I have many positive words for the book, the ending ruined my enjoyment. Fetter's shadow played a grossly ridiculous deus ex machina, denying Fetter the opportunity to make the decision to kill his father or not. Fetter falling back into the revolutionary movement was disappointing: he remained living in the interests of grand ideals without reconciling his own inner emptiness.

That's to say that the themes, while explored, failed to be developed to a pleasing (whether positive or negative) conclusion - the catharsis was missing. Fetter's childhood trauma isn't eliminated by the death of his parents, nor does he construct a new identity through it. It's as if the author was unable to weave a worthy ending from the various threads employed.

Seriously. The last twenty pages let me down.

Wrap up

This book had potential, which makes its ending so frustrating. I would still recommend the book to those interested in blending modernity with fantasy, but with the caveat to keep expectations low with regards to the ending.

Good Software Tools

2024-05-11T00:00:00-04:00

As a programmer, I spend a significant amount of time writing and using software. I'd like to list a few of my favorite tools.

Philosophy

I prefer to use open source, cross platform, local software. Old and boring is a good signal. Community driven over VC funded projects.

I don't hop from fad editor to fad editor as popularity rises and falls. I don't worry about vendor lock-in. I don't worry about Embrace-Extend-Extinguish.

I aim to keep my tooling simple, to choose software that gets out of my way, so that I can focus on getting my work done.

Emacs

I've been using Emacs for more than a decade. That said, I'm not an elisp hacker - I use Doom to provide an excellent out-of-the-box experience, without requiring too much manual setup.

My favorite aspect of Emacs is the keyboard-focus: I rarely have to use the mouse. It's an editor that gets out of my way.

Within Emacs, a few of my favorite tools are magit, projectile, LSP mode and avy.

I love magit - for my day to day git operations, committing, branching, pushing and pulling - it's an absolute breeze.

Projectile makes working within projects, typically git repositories, so easy. My frequent project related commands are memorized - find file in project, search in project.

LSP mode - support for IDE-agnostic language servers - has been a godsend. No longer do I have to learn the specific nuances for each programming language mode, it's all standardized under LSP. Since language servers are developed independently, they also tend to be more feature complete than standalone programming modes.

Lastly, special shoutout to Avy and avy-goto-word-1. At first I didn't understand it, but when I understood the zen, that is, "look at where you want to go when you invoke the command," it all fell into place. I use this frequently (bound to M-s) to navigate.

AI Tools

Special shoutout to copilot.el for providing seamless integration with Github copilot. I am also starting to use gptel, early results are positive but I've yet to really get in the groove.

Tmux + Tmuxinator

This may be controversial, but I don't put everything inside Emacs. I know it's possible to turn Emacs into a Trapper Keeper, absorbing all use cases, but I don't think it's always the right move.

As a primarily desktop user, I turn off/on my computer every day. I quickly felt the pain of having to set up my terminal windows every morning, and found the solution in tmuxinator. With tmuxinator, I create templates for my projects, which stand up to prepare my environment. For example, at work, my tmuxinator config includes: jupyter lab server, django runserver, celery worker, webpack, and docker compose up.

I also think these templates are great for sharing with new team members: even if they don't want to use tmux, it's easily translatable to their preferred environment.

Docker Compose

Just like with tmuxinator, docker compose is a templated tool for running project dependencies. I use it for local project dependencies on linux, and my personal project deployments on my Raspberry Pi.

It's a fantastic tool that solves a tough problem: how do I maintain different versions of dependencies (postgres, redis, elastic search, etc.) when working on different projects? With docker compose, I simply don't worry about spending time installing and configuring stack dependencies. Just run docker compose up and start working.

Pelican

I can't end this post without mentioning the software that powers this blog - Pelican. It's another example of software that gets out of my way. I write my blog posts in Markdown, keep the blog backed up on Github, and upload via rsync to my Raspberry Pi. It meets all of my requirements for good software I can trust.

Conclusion

This is just a taste of the software I use on a daily basis, for work and personal projects. I am documenting it primarily for future retrospective: in a decade, I wonder what software I'll be using - perhaps I'll just be providing quality assurance for AI generated code on some locked down corporate system. Hopefully not!

Mythology

2024-05-04T00:00:00-04:00

I just wrapped up Edith Hamilton's Mythology, a staple of high school English classrooms (well, not my English classrooms, from what I remember).

tl;dr I rate the book 5 stars.

Outline and perspectives

This book details the Greek pantheon, ending with a short chapter on the Norse religion. I had previously read Hamilton's The Greek Way, which I enjoyed, and which made for a good complement to this book. Clearly, Hamilton is passionate about the ancient Greeks and their culture - and that passion is infectious. While I've been attending a Buddhist monastery for about a year now, learning and practicing, I still find myself conflicted between the two seemingly contradictory philosphies. The heroism of the ancient Greeks - the striving, the proof of excellence in deeds, is contrasted by the heroism of the Buddhists - the self mastery in renunciation. I do find overlap, there is striving for excellence and mastery in each, but it's in the way that each affirms life. More to come I am sure!

Across the various myths and stories a few stood out to me, that I'll record for posterity.

Prometheus

So many of the stories rounded out my knowledge of common myths. Prometheus, I was aware, stole fire from the gods and gave it to mankind. I was not aware that he was considered in general a benefactor of mankind, that his name meant "forethought" and he was celebrated for his wisdom. I was also surprised to hear of his determination despite his suffering. When offered the opportunity to escape his torment by revealing the one who would dethrone Zeus, Prometheus said to go and convince the sea not to break, it would be easier than convincing him. That unbreakable will in adversity is powerful to consider in my own life of comfort and ease.

Heracles

I vaguely remember watching the Hercules movie as a child. Yes indeed, Heracles strangled two snakes as a babe. But he also had a darker, human side. He was the strongest man in the world, as strong as even the gods. But he was prone to mishap: whether through divine manipulation or carelessness, he caused harm to a great many people. Perhaps the greatest strength of Heracles was his conscience: wracked with guilt, he would do anything to make amends - most famously his 12 labors, after murdering his family.

Jason and the Argonauts

Per the post image, I just had to watch the classic movie. I greatly enjoyed it - the animation was honestly more impressive than most CGI. I thought the scale of the monsters was well executed: it reminded me of Dark Souls!

Beyond the movie, I liked the myth, too. How inspiring that, given an impossible quest, Jason charges out into the world, trusting in himself that he will figure it out. That upon hearing of the quest, all the heroes of Greece join. I greatly enjoy such tales of unbridled courage and optimism.

Conclusion

This book has been sitting on my shelf for a few years. I am so happy to have picked it up. I read it at just the right time I am sure, when I was prepared to take away as much as I did.

Babel

2024-04-27T00:00:00-04:00

For this month's science fiction and fantasy book club, I read Babel by R. F. Kuang.

tl;dr I rate the book 2 stars.

Plot

This story follows the life of Robin, an immigrant to England in the early 1800s. Okay, immigrant hides the details: his father, Richard Lovell, a white British man, took Robin from China, at a young age, after the death of Robin's Chinese family. Lovell never acknowledges Robin as his own, raising Robin to become a translator in service of the British empire.

Which is a good segue to the core of the book: translation is the key to magic, in this alternate history. Translation from one language to another, some word or phrase, is never perfect, never one-to-one.¹ Something is lost in the process. That something becomes magic. The practical requirements of the magic is to inscribe the words into silver bars, and then speak the words (requiring fluency in each language for it to work). For the British, translating between diverse languages such as English, Mandarin and Urdu provides enormous opportunity for new magical applications.

Robin spends his formative years studying, isolated away from the world by Lovell. Then, he goes to Oxford, to study at the prestigious Institute of Translation, whose building is named Babel. At Oxford, Robin befriends his cohort, fellow translation students: Ramy, Letty, and Victoire. Ramy and Victoire are also transplants from their native countries, and all four bond over their experiences at Babel.

Almost immediately upon arriving at Oxford, Robin meets his doppelganger in the midst of a crime: Griffin is Robin's half brother, Lovell's previous attempt to bring Chinese students to Babel. Griffin has however left academia to fight against the tyranny of Babel and all it stands for and supports, namely the British empire. Griffin belongs to a group named the Hermes society, an underground movement fighting against the British empire.

Fast forward 3 years and several hundred pages.

Robin kills Lovell in the heat of an argument. This forces the four students to go into hiding, linking up with the Hermes society. Hermes gears up its fight to prevent war between Britain and China. Letty betrays Hermes, leading to the death of Ramy. Griffin breaks Robin and Victoire out of jail, and dies in the process.

Robin and Victoire occupy Babel, joined by sympathetic students and teachers. Without Babel, silver-magic-powered carts, roads, mills, bridges and so on start to malfunction or collapse. British troops siege the tower. Victoire leaves, and Robin destroys the tower by translating the untranslatable - translation itself - leading to a chain reaction as all the silver in the tower explodes.

What I liked

The core of the plot has potential. A group of students, overcome with sympathy for an oppressed people, hatch a scheme to overthrow the seat of magical power in an empire. As the crisis of war looms, and peaceful efforts fail, facing internal betrayal, the remaining few sacrifice themselves to blow up the tower of Babel - representing both a physical and metaphysical attack on the empire.

Unfortunately, what I liked ends there - the potential was there, but the execution faltered.

What I didn't like

The pacing of the story ruined any possibility of enjoyment. As even a positive reviewer described it, the book is dry. Several hundred pages could have been summarized and shortened to produce a better rhythm. The comparison with the most recent book I read, Embassytown, is stark: whereas Embassytown was too dense for its own good, Babel is too long.

Another annoyance is the footnotes. Some footnotes explain a particular translation - fair enough. But others provide character motivations, background, history, etc. - content that should be woven into the main text! I see it as a failure on the author's part if they rely on footnotes. For perspective, I have read some works of Richard Francis Burton, so I am well versed in footnotes filling more than half of a page. His footnotes are decorative, optional, complementary - not providing core context to the plot.

I also felt the characters were one dimensional. Either they were on the side of the good, or on the side of evil. And the author left no room for ambiguity. I had no doubt which way Robin's sympathies would turn as the novel grinded on. The betrayal by Letty felt awkward and forced, shoe-horned in to make a point. I would have appreciated more depth to Lovell, who plays such a crucial role in the story: how did he come to be so evil?

Finally, I was frustrated by the loose ends of the plot. Robin's family dies in a disease outbreak - but Lovell is conveniently nearby? It's implied the realization is that Lovell could have saved Robin's family - but I am left wondering, did Lovell intentionally kill Robin's family? The other loose end was the romance between Robin and Ramy - it's only hinted upon a few times, but it's just begging for exploration. Why not develop the romance? At least to satisfy the readers' curiosity.

The in-between

One aspect I struggled with was how to interpret the story. Robin is traumatized an early age with the loss of his family, and then continually abused and neglected by Lovell. Robin murders his own father, falls in with a band of violent revolutionaries, and sacrifices himself for the movement. This is a terribly tragic story, and arguably motivates the question of the subtitle: the necessity of violence. When I think of violence in history, of strikes, revolts and revolutions, I don't take into account the tragedies that put the actors on that specific stage. I focus on the poetry of high ideals, of justice and courage and brotherhood.

Conclusion

All in all, I don't recommend this book - it's not worth the time investment to read.

As my friend pointed out, this is true within a language, too - all communication is imperfect. The only way for it to be perfect would be direct access to each other's brains... which clarifies for me the Ariekei language in Embassytown. ↩

Lost Language

2024-04-20T00:00:00-04:00

Some weeks ago I read an excellent article by Alastair Humphreys, entitled A Single Small Map Is Enough For A Lifetime. The core of the article details how much excitement and wonder exists right outside one's own door - the thirst for adventure can be quenched without boarding a CO2 emitting plane to a far off destination.

I have heard the carbon argument against travel before, but it doesn't move me. Global warming and climate change will not be addressed through changes in individual consumer behavior. That said, this disagreement in motivation doesn't detract from my agreement with the subsequent points.

I have moved almost yearly since starting college. I live away from my hometown. I don't feel strongly connected to my locality. After so much movement, I struggle to connect to an area, as I have internalized a semi-nomadic existence. I dream of moving again at the end of my lease, somewhere distant and new.

This paragraph from the article struck me, and motivated this post:

Once I can put a name to something, like a bird or tree, I seem to come across it more often, and I also appreciate it more for knowing the word. As Robert Macfarlane wrote in “Landmarks,” “Language deficit leads to attention deficit. As we further deplete our ability to name, describe and figure particular aspects of our places, our competence for understanding and imagining possible relationships with non-human nature is correspondingly depleted.”

I can share two anecdotes which validated this perspective for me.

I recently bought a snake plant, my first house plant. Suddenly, I see snake plants everywhere, including in places I've walked by dozens of times. It's as if my world has become more real, more alive, inhabited as opposed to desolate - and just from a single houseplant.

Secondly, in reading Richard Francis Burton's Personal narrative of a pilgrimage to el Medinah and Meccah, I took away the incredible breadth of the author's vocabulary. Perhaps not surprising for a master of languages, but it extended beyond casual conversation to poetry to literature to religion to geography and geology - describing each leg of the trip in exquisite detail.

I feel this language deficit deeply. I can see how it limits my perception, and how that limited perception affects my psychology.

I don't believe there's an easy cure. The fast-paced, instant-cure lifestyle led me to this situation, so I don't think it will help me escape. Instead, I want to slow down. Read more slowly, consulting a dictionary for unfamiliar words. Write and reflect more. Engage in conversation with others. Break out of my comfort zone, before it slowly strangles itself in language deficit.

Timezones

2024-04-13T00:00:00-04:00

My company services several cities in the continental United States. For those major metropolitan cities, we only have to concern ourselves with three US timezones: US/Pacific, US/Central, and US/Eastern. Even without non-hour-offset timezones I still experience headaches on a weekly basis. Many stem from original sins within the code, the legacy of unfulfilled future-proofing, failed prophecies.

In this article I'll describe a couple experiences and lessons learned. Note that my experiences are primarily within the context of Python and Django.

datetime(..., tzinfo=mytz)

First of all, shoutout to Paul Ganssle's excellent article pytz: The Fastest Footgun in the West. It came as quite the surprise that passing a pytz timezone to the datetime constructor is incorrect - violating the principle of least surprise. It's easy enough to remedy, but it's a terrible pitfall.

I had premonitions that something was off, when I saw the strange timezone offsets, with ± a few minutes tacked onto the expected hours. I'm not completely caught up with the transition to ZoneInfo, but I believe that the situation moving forward is vastly improved, and a ZoneInfo timezone can safely be passed to the datetime constructor.

Datetimes and dates

When first starting out working with timezones, it's easy to naively try timestamp.date(). The issue of course is that timestamps are universal "points in time", while dates are not: today, April 13, 2024, is only 24 hours long from the perspective of a given timezone. Without the timezone, a date is ambiguous.

I cut my teeth on this at my internship, where I created a metrics tracking dashboard. The dashboard was designed to report the number of events in a given day. But there was feedback that the metrics were different between East and West coast users. Which makes sense, if the report is using the user's timezone for aggregating timestamps by date. An event at 1am in New York will fall on different dates for West and East coast users.

The solution, therefore, depends on the situation and the requirements. Perhaps timestamps should be converted to the originating timezone's date. Or perhaps the user's timezone. Rarely, though, should a date be extracted from a timestamp without timezone conversion.

Naive datetimes

This continues to plague me at work, original sins from developers long since moved on.

At the Django docs' suggestion, I added the following to the codebase, in order to transform naive datetime warnings into errors:

Python

warnings.filterwarnings(
    "error",
    r"DateTimeField .* received a naive datetime",
    RuntimeWarning,
    r"django\.db\.models\.fields",
)

This exposed so many subtle timezone issues, that the team quickly gave up and commented out the code within our local repos.

One of the most nefarious issues is setting a DateTimeField to a date. When persisted, this converts to midnight on the date in the server default timezone. Note that in Django, the default timezone is separate from the active timezone. At work, the default timezone is US/Pacific. I expect it would be a Herculean effort to clean this up. In the meantime, there's a long paragraph of text warning developers "here be dragons."

self.resolved_at = timezone.localtime()

This is harmless, but a personal pet peeve. Per the Django docs:

When support for time zones is enabled, Django stores datetime information in UTC in the database, uses time-zone-aware datetime objects internally, and translates them to the end user’s time zone in templates and forms.

Django stores datetime information in UTC in the database. Consider this method on a Django model:

Python

def resolve(self):
  self.resolved_at = timezone.localtime(timezone=self.timezone)
  self.save()

First, it looks up the timezone property on the object. Then, timezone.localtime() gets the current time, converting it to the specified timezone. When calling save(), the datetime is converted back to UTC when it is persisted as UTC in the database. So many unnecessary steps and translations!

Timestamps should be in UTC, and converted to a desired timezone at usage, just as Django does internally.

Embassytown

2024-04-06T00:00:00-04:00

For this month's science fiction and fantasy book club, I read Embassytown by China Miéville.

tl;dr I rate the book 3 ½ stars.

Plot

Avice, a human, grows up in the human neighborhood of a city on Arieke, an alien planet at the edge of known space. She is the protagonist from whose perspective we witness the events that unfold; she also plays a critical role in the denouement.

The natives of Arieke are called Hosts or Ariekes. They are totally alien, in biology, culture, technology, and most importantly, Language.

Language is unique in a few ways. One, the Ariekes only understand Language when spoken by a creature with a mind: reproductions via technology may make the correct sound-waves but are gibberish to Hosts. This suggests that language has some metaphysical quality uniquely created by a biological creature. Two, the Ariekes have two mouths, so the language is spoken as two streams of phonemes in parallel. This means that a single human cannot speak Language, so in order to communicate, twins are genetically engineered and then linked via technology to be able to speak Language.

There's a lot to Language; it's the core plot device of the novel.

The genetically engineered twins are called Ambassadors. A new Ambassador arrives on Arieke, a unique Ambassador: EzRa are not twins, but in fact despise each other. This contradiction between the minds speaking Language creates a sensation so strong in the Ariekes that they quickly become addicts. Bear with me, this is only a brief summary, so if it sounds ridiculous..

Arieken society collapses practically overnight as the addiction spreads. A movement within the Ariekes begins which cures the addiction by physically removing their own ears: this movement spreads by violence as the cured, named the Absurd, seek to inoculate their brethren.

Finally, Avice manages to teach the Ariekes to lie - I forgot to mention that Language is spoken Truth, the signifier and the signified are not separate - which cures the Ariekes of their addiction. The Absurd and the Liars learn to communicate in writing overnight and it's a happy ending.

What I liked

The world-building and universe-building are mostly excellent. I liked that Ariekes is an outpost colony at the edge of the immer. Instantaneous communication is impossible: "messages in a bottle" (or miabs) are sent across space physically. I liked the lighthouse in the immer, and the Wreck of a previous ship orbiting Arieke.

I really enjoyed the biorigging - the planet, the city, the buildings are alive, combinations of technology and biology, carefully crafted by the Ariekes. This alone made for a rich setting.

What I didn't like

This novel is too ambitious, it should have been split up across a series. There are multiple stories at multiple levels advancing in parallel, and it's simply too much for such a short book. For example, the empire-colony plotline, Avice's personal story, Language and the Ariekes, the spread of the addiction and the collapse of society. I'd have preferred the book ended with the start of the addiction, and then the next worked through its development and resolution.

That brings me to the ending.. I can't buy it. Overnight the Ariekes are transformed as they are taught how to lie, at just the last second.. deus ex machina as someone in the book club called it. This allowed for the happy ending, but given the plot up to that point, I think the ending should have been collapse and rebirth. The addiction spreads and consumes the Ariekes. The next human ship arrives and finds the colony destroyed: Ehrsul, a robot, the only survivors, tells the story of the collapse.

Conclusion

If you're interested in an intellectually challenging science fiction novel that focuses on themes of language, then I recommend the book to you.

Interviewing

2024-03-30T00:00:00-04:00

I have been helping my company interview, which makes for a great opportunity to elaborate on my experiences and perspectives: what do I look for when I interview a candidate?

Green flags

These are a collection of positive qualities and behaviors that I look for in a candidate.

Online presence

Typically the first impression for a candidate is a resume. A clean and straightforward resume is ideal: don't over-embellish. Next, I'll look for an online presence, such as a Github account and a personal website/blog. Bonus points for linking the accounts and websites in the resume.

I'll browse the candidate's repos and blog posts, looking to get a feel for the candidate's experiences. When possible, I'll ask the candidate during the interview about personal projects or blog posts - a good way to build rapport and demonstrate interest.

I don't expect anything excessive: a couple personal projects that are online or documented are sufficient. A website and a blog with a couple blog posts discussing their experiences is icing on the cake.

Technical competence

During the interview, I am looking for competence, not necessarily expertise or excellence. That's not to say the bar isn't high, but that in a 1 hour interview there's only so much a candidate can do!

To that end, I prefer to ask broad questions, i.e. toss softballs, hoping the candidate hits a home run. For example, if a candidate has used Django and Laravel, I could ask to compare the frameworks. A good answer would prove that the candidate has actually used the frameworks; a great answer will explain in-depth pain points or unique qualities of each. Open-ended questions are a chance for a candidate to demonstrate their experience, in a format more comfortable than trivia questions.

Red flags

Green flags wouldn't be complete without red flags: qualities that sink a candidate.

Poor communication

This is one of the first pieces of interview advice I learned, but it bears repeating because I've witnessed the mistake: communicate during the interview! If you're doing a coding exercise, explain your thought process, talk about the code that you're writing, ask questions - the interview is as much about understanding how you work as what you can produce. Even if you fail to complete the exercise, solid communication may score you ahead of a candidate who finished in silence.

Poor attitude

Similar to communication is attitude: be a good sport. Get into a good headspace before an interview by taking a walk or meditating. Advocate for yourself by taking breaks between sessions. Everyone has bad days - it may be better to reschedule than be grumpy.

This isn't to suggest putting on faux-excitement over a job you're not excited about. You can show interest by asking good questions about the role and the company, and that's sufficient.

Conclusion

These are just a few of my perspectives on interviewing, colored by experiences on both sides of the table.

One key piece of advice: interviewing is a skill, practice makes perfect.

Remote Team Building

2024-03-24T00:00:00-04:00

I've had the privilege recently of organizing a few team building events. I am writing this to share my experience.

Welcome

The first event was to welcome a new team member.

I chose Friday, as it's typically an easygoing day. In Google Calendar, I added the dozen team members, and stared at unavailability across the entire day. Thankfully, most meetings on Friday afternoons are 1 on 1s, easily rescheduled or rolled into the broader team social. I picked a time with few overlaps and sent the invite.

I considered including an activity, versus letting conversation flow. I decided to include a game: with a remote meet of a dozen people, big personalities can dominate to the exclusion of others. A game provides the structure for everyone to participate.

I searched Google and asked ChatGPT for help, finally settling on two truths and a lie. I didn't inform the team beforehand, to encourage spontaneous answers.

The game went over very well, easily taking up over half an hour. I emcee'd, leading the votes for each round: "Raise your hand if you think XYZ is the lie."

Throughout the game, people explained the stories behind the truths and the lies, and that naturally led to further conversation from the group.

Goodbye

The next event was saying goodbye to a team member. They weren't leaving on bad terms; they were well integrated and appreciated on the team.

As before, I chose Friday afternoon - coinciding with the team member's departure.

With invites sent, I faced the challenge of organizing a more somber event. After scouring the internet, I found a promising idea: each team member would share for a minute or so on their experience with, or advice for, the departing colleague.

This went better than I expected. The emotion was palpable. The shares covered favorite memories, accomplishments, positive feedback, and advice. It was after this social that I came to appreciate the strongly positive team culture.

Afterwards, I received private feedback from a team member asking me to organize the same event for them when they depart. I can't imagine more positive feedback.

And Welcome Again

Finally, I organized another welcome social for another new team member.

Time slot: locked down on Friday afternoon.

Game: out with two truths and a lie, in with a different guessing game. I wanted to switch things up, given how recently we had just played two truths and a lie. The game I settled on involved each team member sharing privately with me a fun fact or story, and during the social the team would guess whose story belonged to whom.

There's a lot of startup effort involved in a game with setup like this. The team all shared their stories with me, but it took prodding.

I put together a slideshow, with AI generated images for each story, using the story as a prompt.

The game went well, but didn't last as long as I'd have hoped. To make it longer, I could've asked for multiple stories from each team member, or added in fake stories to the mix.

The format of preparation did encourage different story telling than two truths and a lie. Some team members shared longer stories, others a simple fun fact.

Conclusion

In organizing just a few remote team social events, I have gained appreciation for event organizers and planners. I also appreciate my team's strongly positive culture, without which it's not hard for me to imagine suffering in awkward silences!

I'm thankful for the opportunity to cut my teeth. Organizing is a powerful skill, small remote events being just the start.

Fantasies

2024-03-17T00:00:00-04:00

At the monastery yesterday, the monk explained how people form images of others in their minds, and those images crystallize. For example, older family members may treat you as if you were still a child, because in their mind they still see you as a child.

Change and attachment

This example is great, because time gives a concrete representation of change. As people change, the images we have of them in our minds lag behind.

We grow attached to the images, subconsciously, without intent. It takes constant effort to keep the images in sync with reality. This effort is rarely applied because people tend to change slowly.

At a certain point, though, the gap between the image and reality may become too great, leading to disillusionment, a disturbance, conflict.

Conflict

This conflict is natural, and most of the time is resolved unconsciously. Sometimes, however, it bleeds into the conscious.

One possible outcome is denial: pretend that the person hasn't changed, like the family members in the example. Denial is insidious and powerful, unconscious and invisible.

Another outcome is confrontation: "You've changed!" spills out in the midst of a fight started by something unrelated. The confrontation expresses anger, in an attempt to pull the other person back into line with the image.

Yet another outcome is acceptance: understand that one's perception is delusional, that one's mental images approximate but do not mirror reality. Accept that people change: impermanence.

How well do you know your friends?

This discussion is topical for me, as I am realizing that the images I have of people in my life are incongruent with reality. It's painful: I feel grief at the loss of the images, the fantasies. Loss is a perfectly apt description, even though I am not losing something real: it's my attachment to the fantasies that causes suffering.

I am grateful to have gained this perspective. In my past, I acted out, tried to coerce people to stay in line with my images. It never worked, though that didn't stop me from trying again repeatedly.

Now I choose to accept that the people in my life are constantly changing, coming and going.

Lottery Ticket

2024-03-09T00:00:00-05:00

"You can't win the lottery without buying a ticket" - wisdom shared with me this week.

Buying a ticket

I have spent my adult life minimizing risk in favor of the safety of routine. I have bought few "lottery tickets," whether for small and large "jackpots." When I have, it's usually at the behest of another, which allows me to mask my own vulnerability by projecting it onto another. Or, it's driven by the winds of fate, which absolves me of the vulnerability of making decisions.

This has been the playbook by which I did not live my own life, but rather tagged along in others' lives, buffeted by forces and wills not my own. I avoided the risks involved in making decisions, taking chances: both failure and success.

In finding an emotionally safe and supportive community, I have developed the resilience necessary to experience my own vulnerability. This has enabled me to start taking chances. This has given me the courage to challenge myself, to risk not knowing myself, to chance changes to my life that might just get me where I want to be.

Winning the lottery

And where do I want to be? Only in digging deeply into my psychology have I been able to start to authentically ask and answer this question. "What do I want?" is so readily answered by my false self that seeks to protect my vulnerability, guiding me towards the safety of the known and the routine.

The work of uncovering my true self involves seizing opportunities to play the lottery. In playing, win or lose, I discover myself, I forge myself.

Sure, it would be great for my wildest fantasies to come true. But so many of those fantasies are the stories that nourished and protected me when I wasn't willing to risk buying a ticket.

As I start to buy tickets, I surrender my desire for any particular outcome. I accept the experience, the vulnerability, the uncertainty, just as it is. And that makes every ticket a winner.

One Year of Fellowship

2024-03-02T00:00:00-05:00

I am approaching one year in a 12 step fellowship. As fate would have it, I also just wrapped up Step 12. While not easy to write about publicly, I want to take the opportunity to reflect on my experiences in the program.

Hitting a bottom

A nasty breakup led me to the church basement. I was miserable, full of anger and resentment. I had just read a self-help book that suggested that my unhappiness was due to my own behavior, and in spite of heavy denial, I was open to try something new.

I didn't even know what a "bottom" was at the time. A lot of terminology was new to me, as I started navigating the program. I felt awkward, running out the door the moment the meeting ended.

In the shares and the readings I found a reason to keep coming back: my own story being shared by someone I've never met.

Fellowship

My sponsor told me that fellowship is half the program. It was months before I felt comfortable enough to hang around after meetings. I remember how anxious I felt when I first asked someone to meet for coffee. And how nervous I was when picking up the phone to call or text. The anxiety hasn't disappeared, but it's dissipated over time.

My sponsor's words proved true: fellowship is indeed a critical, and awesome, part of the program. The relationships that I've developed with fellow travelers stand in stark contrast to those outside. There is an emotional safety that I haven't experienced elsewhere. There is so much power and healing, whether reaching out and being heard, or picking up the phone to lend a friendly ear.

Step work

I started the steps after about 6 months. I was nervous, and understandably so: I was intimidated by the prospect of amends, I didn't feel I had the spiritual "credentials." I managed to start despite the fears, and I am so happy that I did. If attending meetings is opening the door to a new life, the steps are walking through it and living it.

I took my time, not aiming to rush through it. I called weekly with my sponsor to discuss the reading. I took breaks for vacation. I became willing to do what the program asked, on faith, and the guarantee - if it doesn't work, the program will refund my old life in full.

I have begun making amends, which I believe are a reflection of my own growth and healing, as well as continued growth and healing. I will be honest, it isn't easy. It's required a ton of self reflection and humility. So far, I've noticed an enhanced peacefulness in my life as a direct result.

Balance sheet

So what has this all been for? I have spent a significant amount of time and energy on working the program, attending weekly meetings, engaging with fellowship.

I have heard from others that I sound lighter now than when I walked in the door. They've noticed the changes, the growth, the healing within me.

I have noticed it too. I have cultivated self-love, humility, honesty, and surrender. I try to keep my focus on myself, and let others live their own lives. I try to make space for others emotions. I try to feel and honor my own emotions. I try to ask for help when I need it.

My relationships have changed, too. I have parted ways with close friends. I have a new awareness in relationships, and a sense of intention for selecting and nurturing positive relationships. Most importantly, my relationship with myself has improved dramatically, and that has improved all of my relationships.

Next steps

Finishing the steps isn't the end, but the beginning. I will continue to attend meetings, make inventories, and make amends. I can't wait to see what my life is like after another year in the program.

The Only Harmless Great Thing

2024-02-24T00:00:00-05:00

For my science fiction book club I read this novelette (note: a novelette is shorter than a novella). In the meeting I was surprised to hear that a few members couldn't finish it. I rarely stop reading a book - though there are countless unfinished books on my shelf.

Anyway, to the book. There are three plot lines told in parallel. The primary story is an alternate history where elephants are intelligent enough to communicate with humans. The elephants are brought in to US Radium to take over after the health disaster affecting humans. That plan is spoiled when one elephant kills the foreman, leading to her execution by electrocution (based on the true life story of Topsy the Elephant), which she spoils again through explosive revenge.

The other two plotlines are closely linked thematically. One is the story of the racial unconscious of the elephants, who maintain their history by passing down stories from generation to generation. The other is a modern day scientist who devises a plan to make elephants glow near radioactive sites, in order to provide a warning for future generations about the danger. The author suggests that humanity lacks the memory of the elephants, which leads humanity to repeat its mistakes. Interestingly enough this is something I had been thinking of recently: the role that community plays in distributing knowledge, skills, stories. The book does not approach why the memory of humanity is seemingly so short. Certainly, human communities build collective knowledge and stories and pass them down, I'm no historian but I'd presume in some cases for thousands of years or more. The loss of that community, its destruction, should be understood as a terrible tragedy, a result of catastrophe (war, famine, etc.). At the same time, it plays a rejuvenating role, leading to the creation of new communities, new stories, new memories.

The author could have done a better job at explaining the real life history, because it's necessary context to appreciate the story. I had to read about US Radium and the "Radium Girls" who were lied to about the safety of ingesting radium. Fun but also disturbing fact: I live near the location of the old US Radium factory, and several Radium Girls are buried in a cemetary nearby. A member recalled the superfund site and excavation of mountains of contaminated dirt. Additionally, the real life electrocution of Topsy the Elephant. And a member mentioned a real life attempt by scientists to make cats glow near radiation.

Overall I enjoyed the book, though it felt a bit trite, between the trope of a scientist's research being used for unintended and unethical purposes, and the orgy of violence in the finale.

Indecision

2024-02-17T00:00:00-05:00

Indecision plagues me. I am planning a trip to a conference for my team, and have to pick the hotel. Thankfully, the conference recommends a few hotels. Not so thankfully, the options are mostly identical.

Why not choose?

Let's start at the beginning. I believe there's a spectrum of indecision: in some scenarios, it's easier; for some people, it's easier. I currently lie towards the generally indecisive end: picking a restaurant, ordering household items on amazon, reaching out to someone, or writing a blog article. Rarely is the choice obvious.

At the heart of indecision is fear. Admitting to fear is looked down upon, especially for a man, but a quote has stuck with me recently: courage is not the absence of fear, but acting despite the fear.

Fear of failure is fear of making the wrong choice.

If I buy the wrong household item, it'll break down and I'll feel like a fool.

If I approach this stranger to talk, they may reject me.

Fear of success is fear of making the right choice. This can be difficult to appreciate. It's been described to me as: when the fear of making a change is greater than the fear of the status quo.

If I book this trip, I'll have to give up the safety of my daily routines.

If I try this new activity and find that I enjoy it, I may have to face that I don't know myself completely.

How to choose

I read recently that it's widely believed that first comes confidence, then comes action - in reality, it's the opposite: first comes action, then comes confidence. So, how do I make choices? Let's look at a few tools.

Goals

Setting intentional goals is one way out of the impass. For example, with health and wellness as a goal, I will limit my intake of unhealthy food: this helps with choices at the grocery store as well as dining out.

If frugality is the goal, then purchasing an item based on its cost effectiveness will guide my choice.

Goals provide a metric for filtering and sorting: exclude the options that don't contribute to the goal, sort the remaining by how much each contributes.

Experts

Thanks to the internet, there's a review of practically everything under the sun: travel destinations, local car repair shops, and movies (cheers to watchmojo top 10 videos).

This can, paradoxically, become overwhelming: attempting to read the review for dozens of choices does not help with decision paralysis, but contributes to it via increased stress.

Expert guides, on the other hand, like wirecutter, are fantastic resources. This is another method I use for cutting down a broad field of choices to a select few.

Letting go

The above heuristics take a broad set of choices into a limited or ranked set of choices. Chances are, the final or top few will be roughly equivalent, just as in the conference hotels. I could spend untold hours pouring through reviews, endlessly sorting and re-sorting the list, but it won't make the choice for me.

All that's left to do is let go. There's no certainty in life. There's no way to know the outcome ahead of time. The hotel choice may result in any number of possible horrors; or it may work out just fine. The choice really boils down to the paralysis of fear, or taking a leap of faith and deciding.

Embrace the choice

Reframing the experience of choosing from anxiety to empowerment can help to build a positive self-image and confidence. A life without choice would be dull and present no opportunity for self development. Embrace the chance to fail, to learn, to grow.

Travel Practical Tips

2024-02-10T00:00:00-05:00

As a follow up to my travel guidelines, I want to share some practical tips from my experience.

Research ahead of time

I'm paying a nice chunk of change to travel, so I will invest some time in planning ahead: tourist attractions, tourist pitfalls, etc.

Reddit and youtube travel vlogs are fantastic for this. I was so nervous about traveling to India that I even found a video of someone arriving at the airport and booking an uber to the hotel (warranted anxiety - it was overwhelming!).

Book but don't pay ahead

This bit me on a recent trip, where I booked the wrong dates for the hotel, and had pre-paid. Lesson learned!

Hotel breakfast

Be mindful of the price and reviews, to be sure, but in my experience, a decent hotel breakfast is a great way to start the day.

Travel esim

I was recently told about purchasing an esim for travel, and tried out Airalo on this trip. While pricier than purchasing a local sim card, it's also far less hassle. Landing from the flight with data access makes traveling to the hotel a breeze.

And there's no risk of losing the sim card for home, since it never leaves the phone. Nice.

City mapper

City mapper is a directions app that specializes in certain cities. I found results to generally be higher quality than google maps. Compare and contrast - in some places, neither will be great.

Google maps list

This is a super handy feature: make a list for the trip, and add tourist attractions, restaurants, the hotel, etc.. This is great to help guide explorations around the city, and also serves as a historical list of places visited.

Look up

Speaking of map apps, I was amazed that in Amsterdam, there were restaurants and cafes missing! I learned my lesson: open my eyes and look around, don't tunnel vision to the internet.

Tourist transit passes

Some countries have info desks at metro stations - I asked at Amsterdam and was pleasantly surprised to find a great public transit pass (covering bus, tram, subway, train) geared towards tourists.

Go forth

This list is non-exhaustive - I will come up with more ideas as I continue to travel.

Travel Guidelines

2024-02-03T00:00:00-05:00

I am traveling in Europe, and feeling inspired to reflect on some of my guidelines for travel. These are not hard and fast rules - they are guidelines, or philosophies.

Don't go back for seconds

In Barcelona there was a restaurant that served the most incredible hummus (in fact the restaurant was __called_ Hummus). I decided to go back a second time, and felt underwhelmed - it just wasn't as impressive the next time around. I don't fault the restaurant - the quality was there, but the novelty had faded.

My takeaway is that it's best to make the effort to enjoy the experience fully just once. Be present, be engaged, and be bold.

In future travels, I'd like to explore the "upgraded" experience: for example, choosing the tour at the museum.

Make plans, but leave room

It's recommended by some museums to book ahead, due to tickets selling out day-of. This encourages a rough structure for the day: for example, if the museum is at 2pm, then that may suggest brunch/lunch before, or stopping for a coffee/beer to relax after.

I like to do some research before hand, making a google maps list of tourist attractions and restaurants. Local subreddits usually have a decent guide for tourists. I like to keep it loose, though, to leave room for spontaneous suggestions or invitations.

Break your routines

I don't want to recreate my home life somewhere else. Even this short trip, working part-time, has been a great opportunity to break free from the routines of daily home life.

That includes eating habits, work habits, social habits - putting myself in an unfamiliar environment and adapting.

Push yourself but don't exhaust yourself

In Amsterdam on this trip, I landed from the red eye flight at 7am local time not having slept well.. and went to bed at 2am after a full day of adventure, and night of drinking. The next day I was so exhausted I barely left the hotel.

I believe it's good to push myself on vacation - a fresh environment means old habits and self-imposed limits are lessened. I surprised myself already on this trip with how much energy I could muster.

That said, I don't want to miss out due to exhaustion: I want to pace myself.

Wrap up

I have taken five trips in the past year, and these are some of the lessons I've learned and plan to apply in the future.

Django Database Performance Tips

2024-01-20T00:00:00-05:00

I've wanted to share this for some time now, but felt that it would be too large an undertaking. So I will keep it straightforward and avoid diving into metrics: I encourage you to test out the suggestions for your use cases.

Note that I presume familiarity with web development and databases: think of this list as my step by step guide for diagnosing an issue, not a primer on database indices or schema design.

Diagnosing

To identify database performance issues, I will first try to use the django debug toolbar. At work, I even created a dummy view where I can plug in scripts and tasks (e.g. non-view-accessible code), just to use the debug toolbar.

Sometimes, however, the overhead of the debug toolbar is too great, and a request won't finish in a reasonable amount of time (or in my lifetime!). In those cases, I use a context manager that prints out the SQL executed within. I use this within a jupyter notebook, which makes it easy to iterate.

In either case, after this step I'll have pin-pointed the slowdown to a single slow query or an N+1 issue.

Profiling

In the case of a slow query, the next step is to check the query plan. I use a local copy of pev2, which can be downloaded here. Follow the instructions to run the query with EXPLAIN - the exact SQL should be available from the previous step. I use dbeaver or pgadmin to interact directly with the database.

Caveat: using EXPLAIN against a local database may not produce the same plan as in production. In my experience, ensuring that I have a similar amount of data helps, but if I can check in production, that's ideal.

Common Issues

Missing index: may address full table scans.
Too many joins: try prefetch related instead (see below).
Slow join / subquery: experiment with subquery vs join.

Sometimes, the best solution is to just Write Another Query. It may feel awkward, and I'm sure there's some level of raw SQL-fu that could be performant in one query - but I stick to the ORM.

N+1

N+1 is the most common performance issue. To resolve, I'll reach for select_related() and prefetch_related(). Simple enough, most of the time, but beware select_related(): adding joins can cause a query to slow to a crawl. In such circumstances, I will use prefetch_related() instead - yes, even for a "*-to-one" relationship! A second query may be orders of magnitude faster than a join.

Rare: Avoid Loading More Than I Need

Sometimes, I'll find that the problem is that the query is just loading too much data. Think of what has to happen to get a field from the database to an object in Python: the data is read from disk by the database server, deserialized into memory, then serialized over the wire, then deserialized by the database driver in the webserver, then loaded into Django ORM objects: the cost adds up.

Experimenting with only() and defer() can lead to surprising performance improvements.

Caveat: only() and defer() are always maintenance burdens. Any field that's not initially included will require a new database query to fetch. I've seen it happen repeatedly: a field is added to the serializer but not to only() (typically, they're defined in serializers.py and views.py respectively, so no locality of behavior to help!). Thus the performance boon becomes a performance penalty.

Rare: Don't pay the de/serialization cost

Suppose that I am trying to load a significant number of objects from the database. Scale is dependent, but let's say about 10k. The price of creating ORM objects is the difference between finishing the request in 30s or not (yes, still using Heroku).

In such a case, dropping to values() can be great - the queryset will return dictionaries with the specified properties. The performance gains can be substantial, even orders of magnitude.

Caveat: values() is only an option if I don't actually need model object instances.

Summary

Follow these steps:

Diagnose
Profile

Then try the common solutions:

Add missing database indices
Split the query into multiple
select_related(), prefetch_related()
only(), defer()
values()

Senior

2024-01-13T00:00:00-05:00

I was recently promoted to senior software engineer. Which warrants an examination of, what does it mean to be senior?

Please note that this is a personal answer, based on my own experience - there is plenty of excellent literature on the subject. The key differentiators of seniority may change between individuals and companies, remote-first and non-remote cultures, etc.

Technical

Certainly, I expect a high technical bar for seniority. This doesn't necessarily mean in-depth knowledge of data structures, algorithms, system design, etc., etc. But it does mean a strong breadth of knowledge, especially throughout the relevant domain of the company stack.

It also means the capability to plan and execute on large scale projects. Note that scale doesn't always imply thousands of lines of code - in fact, I consider it an accomplishment to "thread the needle" on a complex, high impact, broad reaching project in a succint amount of code changes (though clever solutions can also be bad... as usual, it depends!).

To be sure, I believe in continuing to develop my technical skills. But it's not the sole focus of a senior engineer: what's more important is that I feel confident that I could pick up a new technology when necessary.

Mentorship

Mentorship is an important part of seniority. At the moment, I invest time in regular 1 on 1s with other engineers, in particular new team members. Setting aside regular scheduled time creates an open floor for questions and rapport-building that can otherwise be awkward to initiate when working remotely.

In addition, sharing resources for team education is an example of mentorship. For me, this includes articles, tutorials, and conference recordings, that I think would interest the team. Note it's easy to get carried away here and think that just posting links is enough - I think that demonstrations, for example "brown bag lunches", take education to the next level.

Leadership

With increased rank comes increased responsibility, and that responsibility will include coordinating within the team and across teams. A senior commands the respect of their peers and throughout the company: it's a subtle but valuable resource, to be used wisely.

To be more concrete, a senior is expected to plan, propose and execute on projects, process improvements, etc. This involves coordinating others, striking compromise, and developing relationships.

One final note that I think is important: leading by example. Other engineers will look at a senior engineer's behavior and emulate it: they set the bar, so make sure to set a high one. This comes into play for development (writing tests, writing clean code, responding well to PR feedback, etc.) as well as soft skills like communication and friendliness.

Business

Last but not least, seniority raises the bar for business impact. A senior may now have access to more information about business priorities; they are now expected to align their work with making an impact.

This can be challenging, because it goes beyond the day to day work. Understanding business context means expanding my perspective to broader trends within the company and even beyond (e.g. the broader market). It means keeping my finger on the pulse of the business: growth, efficiency, and changing priorities.

This is a skill under active development: I am susceptible to tunnel visioning into my current project (good for focus, not so great for managing changing priorities over time).

And beyond?

I look forward to developing these skills, seizing new and challenging opportunities. Next stop: lead engineer 🚀

Humility

2024-01-07T00:00:00-05:00

To me, humility means being teachable. That is, I don't have all the answers, I'm not immune to making mistakes.

Recently it has helped me to pay attention when my instinct is to drift off: the impulse of "I know this already, there's nothing new for me."

One instance is at the monastery, when a layperson asked the monk about working through the loss of a loved one. I hadn't personally experienced this recently, so my instinct was to let my mind wander to something more interesting (what a cool movie I watched yesterday...). Instead, I paid attention, and it was highly rewarding. The monk explained that grief should be faced directly, i.e. not avoided. Be present with the feelings, understand the feelings, process and work through the feelings. As an example, the monk shared a story of someone who still grieved for their late spouse after a decade: certainly, this person was still working through the feelings, or possibly in denial about the need to finally move on and let go.

I was surprised with how relevant the teaching was, despite my initial hesitation. I was not grieving the loss of a loved one, but I was grieving the loss of the "life I could have had" (e.g. missed opportunities, failures, so on). The same techniques and principles applied: face the feelings, be present with the feelings, understand the feelings, and process and work through the feelings. I want to work through my grief in order to let go and move on, as the grief will keep me anchored in the past, whereas I want to be firmly in the present.

To be fair, I don't always find something relevant or interesting when I pay attention. Just as often I will struggle to concentrate during a meeting or lecture.

But it ties into another idea I've been thinking over: setting the intention. When I try to pay attention in instances where I believe I have nothing to learn, I am setting the intention of humility: maybe I am wrong and I will learn something.

With enough repetition, the intention becomes second nature. Humility, openness, "teachable"-ness, becomes second nature.

You Don't Have to Do It All

2024-01-03T00:00:00-05:00

At work recently I helped lead an effort to solve a technical problem. I had been aware of this problem for some time, at least a month. However, I was hesitant to take action.

Now that I'm on the other side, I feel I have a better understanding of my state of mind.

Getting the right people in the room

This problem required the coordination of several team mates. I took the initiative to put us all in a (virtual) room together, and we were able to make significant progress.

After the technical progress, further coordination was required to wrap up. I took the initiative again and the problem was finally solved.

In hindsight, I can see that the challenge was primarily coordination and not technical. I just had to get the right people in the room!

You don't have to do it all

Which gets to an understanding of my state of mind: because I didn't feel that I could solve the problem on my own, with my own technical skills, I hesitated to take action. The problem also went beyond my own typical responsibilities, which gave me pause.

Certainly, others with the right skills, responsibilities, and context should step in. Right?

Not so right! While I could not solve this on my own, I could play an important role in coordinating the team to solve the problem.

This has been a great, albeit small, opportunity for leadership: I was not instrumental in solving the problem with my technical prowess, but rather in coordinating others.

Management is a skill

I feel good about my role in this effort. I have also identified areas of improvement when future leadership opportunities arise: recognize the leadership gap earlier, take bolder action, and communicate effectively.

This small example has redoubled the importance of effective leadership, and the difference it can make to success - even in small problems.

Management is a skill: some have natural talent, but most have to work to develop it. Do so - it makes an impact!