Streaming

visit pushcx on Twitch (RSS feed)

⊕ Stream Motto: If it’s worth doing, it’s worth doing badly. This is primarily Lobsters office hours and dedicated development time.⊕ See the announcement for more info or scroll down for the archive. I take questions about the site, community, and codebase, or even run your queries against the production db. If you’re watching live, you can ask questions at any time.

I stream Mondays at 2PM and Thursdays at 9 AM (US Central). The schedule on Twitch should note if I’ll skip a day. You can add it to GCal or most other calendars.

Solve a bug before I do to be marked as a Channel VIP.

FAQ

What am I even looking at

Probably vim in an alacritty terminal. The roman font is Incosolata, the cursive is Operator Sans Mono Italic.

No syntax highlighting!?

There is. I find color helpful when I use it sparingly. Watch vim for:

cursive on comments and literals (strings, symbols)
git status in the left column
red > for linter warnings in left column
red for spellcheck in comments
yellow for TODO and FIXME comments

Streaming Setup Todos

rss/atom feed
give aws transcribe a custom vocabulary so it is less awful at technical terms and usernames
fix transcript for multiple speakers (‘heap dump’ post, 2 with Joel Drapper)
~~OBS: install browser plugin; most every extensions for audience interaction depends on it~~ blocked, arch does not have a working package due to the complexity
~~make fun integrations~~ blocked because Twitch won’t let me set up 2FA for an API key
- copy scratch to live panel on twitch
- track lobsters usernames (handy!) and pronouns (I can’t remember everyone!)
- allow viewers to spend bits/points to shift the terminal HSV by 1
- allow viewers to open popups to point out bugs (and more)
- allow viewers to insert popups to point out bugs
- vim: visual fun when a bug report is correct
- vim: visual fun when someone follows

Archive

Titles are out-of-context quotes from each stream. Streams are also included on the tag listing.

⊕ 2025-03-10

I didn't undo my undo

⊕ 2025-03-06

disconnect

⊕ 2025-02-27

I am going to do the unthinkable for a programmer and read the docs first

⊕ 2025-02-24

it's been one year since my last push -f

⊕ 2025-02-20 Lobsters Ruby on Rails UK OSA refactoring

There's more than one way to get mad at rails

⊕ 2025-02-17

There's a hole in my bucket

⊕ 2025-02-13 Erb Lobsters Ruby on Rails SVG cabinet data URLs validating HTML

Erb is string soup

⊕ 2025-02-11

In the U.S. we have a system of mandatory football

⊕ 2025-02-06

Let's just break master again, it's what the GitHub UI wants me to do

⊕ 2025-02-03 Lobsters Ruby on Rails UK OSA story merging

The UK government has a secret list of prohibited sex acts

Wrote my last planned update on the UK OSA. Showed Zulip, a chat platform I’m experimenting with for supporting sister sites a bit more; maybe as little as being able to make security announcements. Adding an icon and count after titles when they have stories merged in.

⊕ 2025-01-30 Joel Drapper Lobsters Phlex Ruby on Rails UK OSA YJIT goaccess guest stream pairing

I like you but not enough to leave you unsupervised on my machine

Most of this stream was pairing with Joel Drapper to experiment with rewriting a partial to Phlex. Some discussion why I no longer expect anything good out of Ofcom or Parliament about the UK OSA. Playing with goaccess, dealing with the Enumerator.

⊕ 2025-01-27 Literal Lobsters Phlex Quickdraw Ruby on Rails dark mode performance

I'm suspicious of any type signature with a newline

After a quick romp through issues and PRs, I was joined by a surprise guest (surprise even to me), Joel Drapper. We confirmed the dark mode bug is fixed (though, spoiler, it reopened the next day). Then we discussed SQL performance in sqlite and toured Joel’s Phlex and Literal gems comparing them against how Lobsters handles those topics.

⊕ 2025-01-23 Joel Drapper Lobsters Ruby on Rails UK OSA YJIT guest story merging

there is no gift someone can give that obliges them to give more gifts

Surprise guest Joel Drapper talks through his libraries Phlex, Plume, Yippee.

⊕ 2025-01-20 Lobsters Ruby on Rails UK OSA story merging

I'm still writing 2024 on my mod notes

Some PRs are ready to go but I’m waiting for the weekend to deploy because the asset pipeline always breaks in prod for me. Then a not-so-quick small update on the UK OSA. Finally, categorizing more story merges to make sure I’m not missing anything as I work on improving those features and docs.

Here’s the updated TSV of merged stories.

⊕ 2025-01-16 Lobsters Ruby on Rails UK OSA

I should be doing this in production

Big PR review. Minor news about the UK OSA, which has caused performance problems. Obviously the best way to fix it is logged into prod as root.

⊕ 2025-01-13 Lobsters Raz Ruby on Rails UK OSA

it's the fourth thing I would do if I had a spare million dollars

No real news on the UK OSA; big PR review. Raz drops by the office.

⊕ 2025-01-09 CSS Lobsters Ruby on Rails UK OSA checkbox hack

it's been 25 years, I should do that again

Rounding up a lot of PRs, finishing a stale one, filing new issues… then on to the UK Online Safety Act and what’s happened in the last couple days. To have fun after that, writing a silly CSS kludge for a desired feature and then beating it into submission.

⊕ 2025-01-06 Lobsters Ruby on Rails UK OSA

Hello, I am your constituent from Chicago

A big review of merged and open PRs with a lot of Rails 8 feature work from a new contributor. Then working on the user CTA in response to the UK OSA.

⊕ 2024-12-19 Lobsters Ruby on Rails UK OSA

it would be much more fun to tool around in a golden sub in the 70s than deal with the online safety act

Taking stock of stream inventory, why Peter doesn’t play lawyer, and figuring out how to deal with the UK Online Safety Act.

⊕ 2024-12-16 Lobsters Ruby on Rails moderation pattern recognition

now we have nothing, which is like progress

Chatting about good Rails intro books, an app for recognizing art scammers, quick queries to get at whether we’re seeing a lot of drive-by github promos, chat about moderation and training pattern recognition, and then a big dive into revamping ~/standing into a ~/context as side norms have improved.

New VIP: calebhearth

⊕ 2024-12-09 Lobsters coding story merging

as if I don't have enough software bugs, I have actual bugs

All things story merging! The limitations of the db model (bug factory), some UI hassles (metadata, knowing which comment is where), unclear practices. So I started reviewing all the merges to build up understanding while working towards better documentation and UI.

⊕ 2024-12-05 Lobsters git gpg

I don't think the data model forbids it, but it should

Fixed a bunch of my bugs, like:

comment stats from last stream
my git misconfig that was checking gpg signatures
accidentally-committed if true

Some puzzles around PRs, then building out an admin feature for reparenting.

⊕ 2024-12-02 Lobsters browsers

When I say 'your' bug -- I'm the one who breaks things

Starting to think about story merging.
How technical writing is like product design.
Why journalists can’t spell.

⊕ 2024-11-28 Ruby on Rails social media censorship story merging

From civil penalties import unit

Short Thanksgiving stream.
Australia’s new social media law with a $32M USD penalty demands age verification that would require some kind of ID check. I won’t comply with it and lots is still up in the air for a few months.
Reviewed two PRs from Ashwin Mullankandy: PR #1388 for search highlights and 1387 to fix /stats
@chaelcodes dropped by (check out her recent Dead Code podcast appearance)
Some comparisons of Phlex and Erb that really highlighted what I like about it.

⊕ 2024-11-25 Lobsters Ruby on Rails api keys credentials json logging story merging

Once per site, in the way that marriage is that one special day

Fixing up json logs by not having nice things, once again kludging a workaround for a Rails feature by crashing the site at boot. Talked a bit about story merging, but didn’t get to code.

⊕ 2024-11-21 CSS stacking contexts Lobsters Ruby on Rails self-promo

Rails features that we had to invent before Rails had them

As feared, the PR to fix reloading extras/ broke prod, and the fix would break sister sites. Looking at fixing the fix by leaning into Rails features and conventions that have come along since Lobsters started. Also a discussion of temporal fairness and the self-promo rule. Getting close to PR Zero.

⊕ 2024-11-18 Lobsters RubyConf open soure contributions self-promo

It's YAML all the way down

Upgrading to Rails 8, though there’s lots left to do.
Configuring GitHub Action for stale PRs, planning rest of #1368.
Shipping the CommentStat table for finding above-average participation.
Discussing APM/telemetry options
Lots of RubyConf 2024 stories sprinkled throughout, maybe the most interesting being that Lobsters is now a standard(ish) Ruby/Rails performance benchmark.

⊕ 2024-11-11 CSS CSS nesting Lobsters MariaDB self-promo

Very obvious or very obviously nefarious

Hat request approvals were rejected (bug); manual log + db groveling to recover the records #1371
CSS nesting is now widely supported in browsers! Learned in PR #1376
Self-promo prompts motivated reasoning; voting rings; why invites mean we don’t have to be as sophisticated as HN
SQL query optimization for detecting above-average comments; I love Joe Celko’s books
bots.in.space is shutting down so our bot must migrate to a new instance
My cat knows 2.5 tricks

⊕ 2024-11-04 GitHub Lobsters MariaDB open source contributions self-promo

Let's do a different kind of evil

Reviewed PR #1360 for link shortener detection - merged after confirming non-ASCII domains are allowed; so many different URL validations.
Reviewed PR #1359 about UI/accessibility improvements for story disowning - needs changes to handle CSRF and no-JS fallback properly.
Started on self-promo detection improvements - exploring query performance for checking if users have well-received comments (score > daily avg) in last 6 months. Added date-based index but still hitting performance issues with correlated subqueries.
Discussed challenges with GitHub’s limited UI customization forcing use of bots like Zulip to manage contributor interactions. Really don’t want to keep nodejs in prod.
LLM slop HaikuOS post.

⊕ 2024-10-31 Django LLM code Lobsters Phlex self-promo

If it doesn't come from the spam region of the Balkans, it's just sparkling self-promotion

Looked at some PRs, mostly stalled. The new one is LLM or junior code.
Fix up a test failing from fixing up some length validations.
Fun Rails code from fractaledmind.
Deciding between Rails vs Django
The big wooly problem of self-promo, the limitations of coding social rules, the limitations of mod judgement, trying hard to not create bad incentives.

⊕ 2024-10-28 30x500 Chicago-style pizza Recheck invalid data worker-owned coops

Recheck demo / a correct but wrong description

Three important facts about Chicago pizza.
Demo Recheck, my database validation tool inspired by Stripe’s internal tooling. Found real invalid data in Lobsters comment threading from 2005. Issue #1236
Impossible/invalid data in production DBs is a common issue in any DB with >500k records. Gave examples from my KCDC 2016 talk
Launching Recheck as worker-owned cooperative. Explained co-op equity model vs traditional startup equity
Recommended 30x500 and Stacking the Bricks for developers wanting to bootstrap businesses
difference between Lobsters codebase vs community re: IPFS/magnet links feature request

⊕ 2024-10-24 Aider LLM coding tools

No, I don't want to run the linter

Catching up on the Sorbet bugfix (merged!) and avatars (more Active Storage stumbling blocks!), then from 19:06 a demo of Aider, which I think is the best and fastest-improving LLM coding tool available. Where the risks are, the useful parts, the failure modes.

⊕ 2024-10-21 ActiveStorage Lobsters Ruby on Rails open source contributions

I'm doing a lot of sighing on this stream

Working on adding direct avatar uploads to Lobsters, lots of surprising/unsafe defaults in ActiveStorage like not validating that images are images. (PR #547)
“cookie licking” from Sumana Harihareswara’s post, considering Zulipbot to help manage stale issue claims
Reviewed several PRs:
TheCoppinger raided, so I explained Lobsters’ invite system and code

⊕ 2024-10-17 ActiveStorage Lobsters avatars open source contributions

I want to yassify the StoriesController

Filed issue about StoriesController complexity - controller has grown too large, needs breaking up into resourceful routes and separate controllers for different story actions
Reviewed PRs:
Avatar uploads: got Active Storage basically working; dealing with metadata stripping, file naming, and variant generation.
Added some copy to usernames/invites to help trans users avoid footgunning
Documented IRC vs Discord discussion in discord-vs-irc-notes
Requested books on managing open source projects, got rec for How to Open Source by Richard Schneeman

⊕ 2024-10-14 ActiveStorage Lobsters Story tags images moderation

Stepped on a rake of my own devising

Added new chat moderators companion_cube and EBPF to help with increased trolling
Followed up on cloud-init outage from 3 weeks ago, Digital Ocean support identified cloud-init bug, see update
Reviewed several PRs:
Started replacing an unnamed third-party avatar service with direct uploads using Rails ActiveStorage.

⊕ 2024-10-10 Lobsters Ruby on Rails sqlite sidecars

I am unreasonably attached to ellipses

Merged Origin branch after commenting out validation that would block new users’ submissions; want to watch behavior in production before activating
Extract Links from Comments, show recent on story submission form
Looking forward to Rails 8, especially sqlite sidecar databases
Ran query showing most common dupe comments. Found a wholesome pattern of thanks/congrats, with only a few “+1” type comments that we can nudge to upvotes (and discourage ‘bump’ comments)

⊕ 2024-10-07 Lobsters self-promo sorbet

That test wasn't supposed to pass

Changed “archive” link text to “caches” (l33t)
Added retrocomputing
Bugfix to Sorbet for type signatures of Data.define with no arguments
Building “Origin” model for Lobsters to detect self-promotion from new users posting from common domains (mostly github.com, medium.com, dev.to).

⊕ 2024-10-03 Lobsters Ruby on Rails self-promo

Where did it come from, unstructured control flow

• Implementing Origin feature for recognizing self-promo. Implemented domain/origin routing and views, hit issues with Rails routing conventions and migration rollbacks • Found edge case with github.io subdomains, may need public suffix list integration; found some normalization bugs with www • vim/neovim config and the need to learn slowly • The Checklist Manifesto because humans are inconsistent, especially experts

⊕ 2024-09-30 Lobsters self-promo

That really collapses the special cases down

Origin model to recognize that github.com/bob and bob.github.io are the same author powered by regex because the users are developers
Fix duplicate domain records bug in database migration - found two domains (codinunicorn.dev and aaronhawley.livejournal.com) that needed cleanup

⊕ 2024-09-26 Lobsters Ruby on Rails self-promo

at least I can assume my users know regular expressions

Working on Origin model for Lobsters to better handle domains like GitHub/Medium/Dev.to where multiple authors share a domain. Spam patterns and prevention, invite system is critical for Lobsters, vs example of Reddit’s sophisticated spam ecosystem with paid accounts/upvotes/tools. Hard to maintain a good new user experience/onboarding though!

⊕ 2024-09-23 Lobsters Ruby on Rails

Loudly disappointed in the whole world

Handled Lobsters outage yesterday due to DigitalOcean hypervisor/cloud-init issues. Wrote up postmortem. Thanks to Hunter for debugging help.

Merged PR #1316 to enable Rails 7.2 defaults and PR #1327 improving setup docs
Worked on PR #1322 about clickbait titles. Analyzed ~80 title edits to develop clearer guidelines around hyperbole, missing info, and moralizing.
Started evaluating pagination gems (pagy, geared_pagination, order_query) vs current cursor-based pagination branch. Need to decide between building custom solution or using existing gem.
Looked at at PR about porting comment replies to Action Mailbox

⊕ 2024-09-19 Lobsters RSS Ruby on Rails

If n is 82 I can write any code I want

Merged PR #1320 to rewrite odd RSS feed design with XMLBuilder. Reviewed PR #1322 about clickbait guidelines; looking at moderated titles to define “clickbait”. Added caching for RSS feeds since they represent ~25% of site traffic. Implemented 120s TTL.

⊕ 2024-09-12 Lobsters Ruby on Rails float scores

We fixed seven out of the four bugs

More #1318 fixing comment confidence/sorting bugs - found float precision issues causing negative confidence values. Switched to BigDecimal; dithered about clamp.
Switched calculated_confidence to use BigDecimal instead of Float to avoid precision errors
Fixed admin-only ban enforcement in controllers to match view restrictions
Migration to fix cached comment scores/flags

⊕ 2024-09-09 Lobsters performance

the goldilocks diff

Following up on the last stream a viewer found a whole bunch of bugs in the performance-oriented code that sorts comment threads.
Work through those bugs and find several more.
Banning these stream pages from the site for inarticulable reasons.
A potential upgrade to the Domain model.
Should we have confidence in confidence, or drop it in favor of score? Channel VIP: dzwdz.

⊕ 2024-09-05 Lobsters SQL comment threading performance recheck

You're allowed to be evil in performance code

Bug #1313 prompted a deep dive into the most dangerously clever code on the site. We explored why the code works that way, figured out why the performance improvement in #1308 was putting comments in the wrong place, tried to fix it, realized more problems, and ultimately reverted it. (And then fixed a small bug right at the end for funsies.)

⊕ 2024-08-29 Lobsters Twitch Affiliate leak pagination

Twitch may or may not have an ical timezone bug, depending on the meaning of “standard”
Reviewing and merging three Lobsters PRs
Rspec is its own language; optimistic about Quickdraw’s API
Don’t maintain Rails migrations
Refactoring to lean into the CRUD

⊕ 2024-08-19 ActiveRecord Lobsters Twitch API

Numbers that should have decimals in them

Fixing a bug with comments missing from user_threads - which turns out to only appear to logged-in users who are not moderators (though mods get their own unreported bug)
Complexities in ActiveRecord scopes: leaning on Rails more
Investigating the ability to search comments for c++
Gumption traps; Zen and the Art of Motorcycle Maintenance
First cat cam
Trying outreach to get access to Twitch API
Exploring the db schema
Bug VIPs: dpk0, jameslittle230

⊕ 2024-08-15 Lobsters online communities

A healthy amount of bugs

Adding a ‘last read’ line to /newest
The complexity of story list templates
Big “fun” with the mysterious mic cutout problem; jameslittle230 identifies a workaround!
Revising the work from last stream on story scoring that didn’t work out
Code is the physics of online communities
dlamz requests an endpoint, Nogweii volunteers to add it
what deployment looks like
Bug VIPs: arh68, dr3ig

⊕ 2024-08-12 Discordianism Lobsters

Use hide for story score

First stream!

First appearance of the mysterious mic cutout problem
Changing story scoring to use hides instead of flags
Updating the UI for fading out problem stories
Discordianism, and the difficulty of finding a “real” copy of the book
Updating the replying_comments view, perf considerations on join
Arch packaging of OBS and its plugins
Stream follower goals; daydreaming about fun viewer interactions
Bug VIPs: nogweii

⊕ 2024-08-10 Lobsters awesomewm

Test stream, fixing bugs

An unannounced test stream to shake out bugs in my streaming setup.

add spec for long-fixed #1274 missing story previews
afoocado catches a layout bug cutting off the bottom of the terminal; live fix
first stream spammer banned
fixed #1242 don’t show comments on deleted stories
Bug VIPs: afoocado frici