Rails features that we had to invent before Rails had them

01:08The Lobster's Office Hours, Thursday edition for the EU folks. I mean, especially, but honestly, the Monday streams just seem to be way more popular. Lots of folks like the, it's easier to get to stuff after work than something that's kind of in the day. So, I don't know.

...39So I am happy to take any kind of questions about Lobster is the site, Lobster is the community, Lobster is the breakfast cereal, Lobster is the flamethrower. Hold me. And generally try to encourage people to understand what's going on and get involved. When people don't have lots of questions or lots of stuff to discuss, I tend to do coding for lobsters because if I touch the code base every couple of days, twice a week, really, I notice I get a lot more pull request contributions. So as I'm just getting started, a running stream joke has been joke, running stream tragedy. You know, in the sense that all comedy is just tragedy reversed. A running hassle has been that when it rains, my bitrate goes to hell. I start having drop packets. So I've been, this is my 27th stream. pushcx https://push.cx/stream
And I know because I just cleaned up the stream archives. So if you look at yesterday, I caught up. Some of this stuff was done and I checked it off. But all of the previous 27 streams now all have titles, summaries, tags. The tags are a little hit or miss, but it's getting there. So anyways, I feel pretty good about that. It's one of those things that was hanging around over my head of like, oh, you got to clean up the thing. Oh, you got to clean the cat box. you got to take out the trash, you know, you got to write the stream summaries. So I actually wrote a, I've demoed, rechecked the thing. I'm the business I'm starting. I wrote a version of it for my blog here. I'll just show you where I sit. and there might be something private in there. Let me, I'm going to pull it up off stream. just to make sure there isn't anything especially embarrassing. One of the fun things about my blog, which is now just a static site generator using Jekyll, is because it's in Ruby, I can just hack on it. And I do. So very similar to recheck the tool I've been making for production databases. I just made a checker for my stream archives and it loops through my stream archive pages and it says things like, hey, you are missing a summary. You are missing a permalink. You are missing the scratch file. Yeah, there was one from, I think the second stream was missing a scratch file or the literally, oh no, the very first one, the tech test I did before I first announced them. and i caught that anyways this habit of writing code that checks that all the things are done is so much better it's just a basically an automated checklist actually one of the one of the streams i just summaries i just wrote and i guess pushcx https://bookshop.org/p/books/th…
Didn't get my markdown indenting correct for. I mentioned the checklist manifesto on stream, what, six weeks ago? I still love this book, so I'd like it again. But one of the nice things about writing code is I get to automate my checklists, or at least automate not the process of doing them, sometimes that, but also just the process of checking them. So that's been fun. I feel pretty good about getting that up to date.

06:13So if nobody has queries they would like to run against the production database or questions about the site, I will hop into development mode. chamlis_ the recheck manifesto coming soon?
There was, I think, two streams ago, this pull request here. recheck manifesto I mean not to be too much of a shameless capitalist about it but I think the recheck manifesto is just buy the recheck pro license because I'm going to do an open core thing with it so the main engine will be open source freely available to use and just drop it in and start writing checks and that's probably what lobsters will use And then stuff that a business needs like a schmancy dashboard and how there's some code strategies you have to be clever about when you have tens or hundreds or billions of records in your database.

07:27So I'm very much following the sidekick. We know sidekick. sidekick pro model so sidekick is the very very popular background job system for rails there was one of those for anybody who doesn't know the rails history this was actually kind of a funny story where like every 18 months to two years somebody else someone would introduce a new Rails background job library. And each one was like a response to the previous one of things that didn't quite work or turned out not to scale. And you could watch from... I think I saw the first one in like 2007. But you could watch every two years as another one of these came out as the community was just kind of learning from experience. And Mike Parham apparently wrote dpk0 some questions you probably can’t immediately (if at all) formulate a query for (in part because this really needs analyses of correlation strength and statistical significance): how strongly does the *total* number of votes a story gets correlate with the amount of time the story is on the front page before getting its first upvote?
Mike Parham is the primary author of sidekick and he apparently had written a couple of job cues internally and based on those that experience and the public ones wrote sidekick and sidekick solved a problem. Probably pretty huge hassle for the rails ecosystem. And then it just became the standard and so the open source core does these things and then pro. dpk0 how strongly does this correlate with the time of day a story is posted?
has some more stuff, the schmancier web UI. Anyways, I'm really inspired by this because Mike being able to afford to maintain Sidekiq is why it's good. It's not that it started off perfect. It's that the open core model allowed him to work full time on Sidekiq. And you haven't met mike he goes to a lot of ruby conferences and is really generous with his just his time and his experience whether you talk to him in person or just email him and he let me use his license for recheck so that'll be a very familiar license ah so definitely hey dpk i see you have a couple of questions query for How strongly the total number of votes gets with the correlate number...

10:09I see what you're getting at.

...16How strongly does the total number of votes correlate with the amount of time the story is on the front page before getting its first upvote? dpk0 and, if i’m correct that these have a significant impact, how could the ranking algorithm be improved to be made fairer to stories posted during busy times of day when it’s harder for a brand-new story to stay on the front page? and the really hard question: would it be desirable overall to make such a change?
Actually, I'm not sure I can parse your question. I think I see the direction you're headed, but we don't see a lot of stories land on the homepage without any upvotes anymore. I guess there's one right now. But this is pretty uncommon.

...52Nowadays, we tend to see enough dpk0 when i post a story in the morning my time, in Europe, i think it does better because it’s not quite such a busy time
stories submitted that things don't often start on the home page if you're correct that those have a significant impact how could the ranking help them so i see where you're going of there's a really when you post a story in the morning your time in europe i think it does better because it's not quite such a breeze time I think that is almost certainly the case. dpk0 most users are in GMT-5 to GMT-8, let’s say …
I think you're seeing a real thing, but I'm not sure how to get at it because GMT-5 to 8, yeah, I don't think you have to let's say it. I would say America, and I am specifically saying the USA rather than just those time zones, but Because I've looked at the server logs a little bit. I think Americans are a majority of users still. They are certainly a plurality. But yeah, I'm pretty sure they're still a majority. International stuff has grown in the last couple of years, but it's not something I've really kept an eye on. So you're really getting at that discontinuity of, does something start on the homepage? Because there's a big difference between... starting at 25 or starting at 26. Because you get a heck of a lot more attention if you're here on the homepage. I mostly read the site through slash newest slash active. Well, actually, I mostly read the site through the mod dashboard. And then after that, newest and active. So I don't actually think about the homepage a huge amount. And I think... One of the things that you're asking about is kind of a perspective thing that comes up a bunch, but the scope for stories that should go on the homepage is using this hottest scope. And as you can see here, hottest is the only filter it does is a story can't be flagged to a negative score and appear on the home page and then well if you if you click page two page three that still counts as the home page or the front page for purposes of this scope because it's how it's implemented and then the hotness ranking is the votes So where I'm going with this is a lot of people assume that there is some editorial intent involved in what makes it onto the homepage. There is not. It's just this straight ranking. And then also there isn't any kind of flag or tracking of has something appeared on the first page of the homepage view. on the hottest really. And so getting back to your question on how long it's on the homepage before getting its first upvote, I'm actually not even sure we could reliably query that out of the data because that hotness column is a moving value so as a story gets upvoted or flagged or hidden we update that one database column and so looking back there isn't a way that i can query and say what did the home page look like a day ago what did the home page look like on this particular date because to do that i would have to have the hotness values from those dates dpk0 yeah. i’m happy to open an issue for the question of should the ranking be made more time-zone-egalitarian due to how difficult even that first stage is …
And I suppose, so this hotness is based on, we've shown it on stream. I don't remember, I don't think you were around for it. But in a couple of early streams, I did a bunch with looking at this formula and especially the version of it that appears on comments.

15:54Should the ranking be made more time zone egalitarian?

16:05I'm not even sure what that would look like in practice. I kind of see your point of... So there's a feature on Hacker News I'm a little jealous of. And I'm going to just... Please don't let this be a flame war. The site is very spicy this morning. We have the same issue inside of comments where we sort all of the comments by the score of the top of the thread. So this guy says 17, 9. This one doesn't yet have enough upvotes to have a visible score or it's not old enough. Yeah, it must just not have enough. Same here. but there's a very rich get richer effect where if you are at the top of the page, you're more likely to get seen and more likely to get upvoted. Hacker news has a feature where at least on stories that are in their first, I don't know what the time is. It feels like 12 or 24 hours. They interleave comments so that. if you were to sort top level comments chronological first and newest comment that'll go at the very top for i don't know a half hour or so and then the second thread will be whatever the most upvoted thing is by their hotness calculation and then the next thread will be the next newest comment and so they kind of go back and forth of exposing new comments to View I would love to do that, I think that would be an improvement. i'm happy to steal good ideas from other people. it's a little harder with the homepage but it's the same kind of rich get richer problem.

18:12And I guess the other version of the time zone egalitarian is the code that I've written over the last couple of streams for calculating the average of comment upvotes over the last, you know, by day, because that number has increased over time and is significantly higher on weekdays than weekends. that got deployed last stream and is in production or maybe two streams ago and is in production fine i guess what i'm saying is i'm not sure how we would make stories more egalitarian like we have recent that tries to highlight stuff that doesn't yet appear there, and I know a fair number of people read this page. But. I'm not even sure, so since I can't recreate the historical.

19:27So this is one of those bits of code that like it's fine that it's in Ruby, but this could be a sequel query. And if it was a SQL query, no, we still can't recreate the homepage because votes are also mutable. I was thinking you could kind of run the clock back and look at the historical homepage by saying, well, let's just look at all comments and all the scores and votes as they were, you know, and we'll just select with an updated at or created at before whatever date we're interested in. But the database doesn't quite work like that. If you create a vote, say you upvote a comment and then you later unvote it, we just delete the vote record. We don't keep the history of it.

20:26So That's fairly uncommon. It feels, I would say like a, I don't know, like a 1% kind of thing. So I'm saying, I feel like if you ordered this to a SQL query, we could recreate historical front pages with like, you know, a one or a 2% inaccuracy. Maybe one or two stories would swap a little. maybe something would be on the home page that would otherwise not be visible but it would be like basically the home page you saw on that date yeah actually no that's fine yeah that it feels like about how accurate that would be but i honestly don't think we need To recreate historical home pages and evaluate how strongly the total number of votes gets with being on the front page before getting a first up vote. Because we already know with really high certainty that that discontinuity exists that. Like this story that was posted 29 minutes ago, aside from being an interesting topic is going to get a lot more. upvotes than the php release announcement that happens to be just below the fold so there's kind of a big brain dump do you feel like i've approached your question does that make sense is it helpful

22:34dpk0 well, yeah, you’ve explained the sorts of difficulties i expected to arise in answering this question
maybe you've had to step afk i want to say this is late in your work day i'm just guessing at your time zone because you've commented about it being late on the monday stream so probably you were at work okay i've explained the kind of difficulties expected well it's also just we don't don't think we need to query to answer this question i think we can say with really high confidence that there's a discontinuous effect of does your post start as number 25 or number 24 or number 23 well not that but you know 25 24 as opposed to 26 27 like and then how how closely does it correlate with time That one actually, maybe we can just... Let's see, what data do I have locally? It's a recent-ish backup and... Oh, still in the blog folder. Come here.

23:56So... dpk0 more interesting is probably the underlying motivator/last question: whether, and if so how, should ranking be changed so that my posts at 9am my time don’t have any better chance of spending a whole day on the front page than some random Silicon Valley guy’s post at 9am his time
let's just bucket it out so more interesting is the motivator last question whether and how should ranking be changed supposed that your time don't have a better chance of spending the whole day on the front page in some random silicon valley's post at 9 00 a.m his time Yeah, I was going to do a query and just kind of bucket out scores from the last year by time of day, maybe to the nearest 10 minutes or something. And

24:55I just want to I think eyeball data for a second. Because I'm still trying to get a good feel for what fairness means in this context.

25:20Yeah.

...33dpk0 that’s a reasonable way to start yeah (with just bucketing by time of day to total votes)
Let's see. Somebody's written the Stack Overflow post for this. I don't have to think about it, right?

...57That's I guess we could do it as a string, you know, for this kind of one off query. It's a little sinful, but that's OK. And then.

26:26dpk0 stream title: it’s a little sinful, but that’s okay
Oh, this is going to be. This doesn't do what I want. This is You know, so many of my, I'll put it in the list, but so many of my titles are jokes about sin and evil. chamlis_ do you want avg(score) instead of score?
Where, especially when I'm doing performance code or kind of kludging around something, I'm like, ah, it's bad, but it's okay in this context. Man is a rationalizing animal. Do I want average score? Yes, yes I do. Let's see if somebody has the query I want.

27:21frici All sin & evil sounds like one of the basic intros to programming. Evening 👋
Yeah, here we go.

...41easeout code quality relativism huh
dpk0 grouping by hour might be easier if you’re doing it with string stuff, and possibly also more likely to show up a correlation in a comprehensible way
Because I don't really care about which day. We can do a day of the week thing later if we want. But I also have a pretty good idea what that's going to say. So let's say that. And then we will group by. I'll come back for that average score. I didn't forget the edit.

28:10Making it harder to copy and paste than just retype, huh?

...32All sin and evil sounds like one of the basic interests of programming. Yeah, fair. I mean, I'm not writing in, what, like C sharp? Code quality relativism? That's a fair description. All right, what did I get wrong? Something at group by. Oh, group group by. If I'm grouping by hour, it might be easier if I'm doing it with string stuff. Yeah, let's see what comes out. All right, so... Maybe I have to pull this into a graph, but I thought the difference would be stark enough. We could just kind of eyeball it. I kind of just expect that there's going to be a bell curve here. And there sort of is around 13. So this is minutes 20 to 30. We're getting a big peak and then we're dropping off. Like I can pass this into calc, but. What is 13 UTC now? So yeah, that makes sense that things that are posted right at the beginning of the US workday tend to get more upvotes. That's just kind of confirming my preconception.

29:55I feel like the difference here is stark enough. We can see it just by eyeballing because in the us we hours we're seeing numbers in the high 30s low 40s and then there's a pretty continuous gradient up to the three digits and then back down the other side so

30:36Honestly, even, you know, dpk, to your point, even slash newest is biased this way, where if you are, because this is a fairly popular page, a lot of people like to read the site this way, not just moderators who are keeping an eye on off topic stories. But I think folks will tend to do better if they are posting in low busy times in part because people come back and read the page again. And if you see a thing a second time, you might get more curious and look into it.

31:33So I feel like we have a pretty good understanding of what the skew is here like what the difference is but i don't know about what fairness means or how we would i guess what i'm saying is for most definitions of fairness we can assume it's kind of unfair But we have to be a lot more specific before we can say what it is. So I'm going to do this subtraction just to try and nudge back to the US workday or back to US time zones.

32:30Because if I don't, there's going to be a hangover on to Sunday.

...45It'll be a little fun to type two different times, though. Yeah. So you can see the U.S. workday effect real strong here. I'm actually surprised. That's a little weaker than my intuition is. I had thought it was more like double, but this is only showing like 50%. I wonder if this is an averaging thing, and if I summed, it would be more in line with my assumptions.

33:28Yeah, that's where I'm seeing. This is more like what it feels like to me, where the weekends are like roughly half as busy as the weekdays. And there's also an effect you can see here pretty starkly where monday is the busiest day i mean tuesday is a nose ahead of it but i bet if i did the group by for comments too we would see the same effect and then it falls off during the week this is super common for social sites where people take fridays off or otherwise site traffic tends to peak on monday tuesday slide during the week and just fall off a cliff for the weekend The shape of this is very, very common. So let's see it as comments. And actually, that's the only change. That's easy.

34:39Yeah, a little weaker than Stories, but again, you do see the Monday effect, that Monday is the strongest, and then less, less, less. And then weekends are, here it's more like 60% instead of half, but I suppose if I'm being fair, that's the case on Stories as well. I'm a little off on my calibration on that. Because I really did think it was pretty close to 50%.

35:19So DPK, can you say some more about what fairness means or what kind of ideas you would have for for fixing it because maybe that'll also help us get at a better problem statement even if it's just an idea for a fix oh and to explicitly answer your question i think this one would be much better as a meta thread on the site rather than filing a issue request on github because it's not really well defined and the value is unclear Because, I mean, we can average some stuff out, but we can't... If we really wanted to average things out, we would have to move a lot of Americans over to other time zones or, you know, redefine the legal work week. I suppose the work week is just a convention, right? It's not actually a law? Well, at least not in the States. We don't really have labor laws in the States, right? Ah, shit. Sorry, politics. I will slap my wrist.

36:44So yeah, if you think of, it doesn't have to be on stream, but if you think of something to do with what it means to be fair or how to handle it differently, You can just message me, email me, or put up a meta thread if you feel like, especially if you have like a pretty clear idea of the problem statement. dpk0 constitutional protection of Sundays and public holidays here … anyway. yeah this is too complex a question to just immediately answer it here. but i think it’s worth thinking about, and will think about it
Because, boy, and I just mentioned politics, but this is sort of that, where's that meme image? Constitutional protection of Sundays and holidays here. Yeah, yeah. dpk0 i’m in Germany
Like I said, I had the idea you were in the EU because of time zones, but if you have constitutional protection of Sundays, I mean, that narrows down which EU countries it probably is because that just says you're in a Catholic country, Germany. God, I don't know if Germany is Catholic or Protestant. Oh, wait, maybe I should know. Martin Luther was German, wasn't he? Yeah, I don't know. God, please, let's not talk too much religion. Just had to prune a thread yesterday where someone was like, this thing is always wrong. Like religion. Like, really, guy? That's going to prompt a good discussion? Please don't do that.

38:16Where is that?

...22dpk0 half and half Catholic and Protestant. but the constitutional protection of Sundays is actually a labour/socialist/left thing as much as a conservative-religious thing – everyone gets a guaranteed day off work a week
There's a meme image. There's a... Ha ha. I've seen a lot of half and half, but the constitutional production is actually a labor socialist left thing as much as conservative religious thing. Ah, okay. Neat. I mean, it's neat as long as you don't need groceries on a Sunday, I suppose. So there are a lot of edits of this and I have no idea what the canonical one is, but I think this is part of what you're getting at is we have a, an equality thing going on of the homepage treats all submissions with equality, but because of time zone effects, there is different equity. chamlis_ queue up all the stories submitted in a day and publish them simultaneously
Yeah, this is, I don't love these particular definitions, but the images here are roughly queue up all the stories submitted in a day and publish them simultaneously. Yeah, that's something we could do actually. because we are not lobsters is not particularly news oriented and i wrote stuff in the site guidelines specifically some rules of thumb will it improve the reader's next program will it deepen their chamlis_ the lobsters digest
of the last program and then especially will it be more interesting in five or ten years just trying to get away from daily news of x happened a point released happened like are we getting at stuff that has long-term value and honestly if we if we grouped up stories by day and they went in a queue for a day and like all posted at midnight in whichever time zone, it would make the marketers really, really mad by default. And I kind of like that. dpk0 making marketers mad ++
Excuse me, got to clear my throat.

40:52Because we get a fair number of people who come by and are like, look at my neat startup. And I just Oh, man. Yeah, I'm okay calling this guy out on stream. So this guy I just banned this morning. I'm not sure how much I can, what's left when I'm logged out. But he had this startup that he called Gitroom or Postiz or Gitroom is the startup and Postiz is the tool or he's rebranding, blah, blah. But all he wanted to do was submit links to his projects and resubmit and resubmit. Like these two removed ones are just reposts of his other projects. And this guy was very explicitly just here for spam. And part of the reason I know that is I'm in an entrepreneurship slack for MicroCon. I believe I... I talked about this a while ago. Okay, shush. So many pop ups, I can't even look at the website. MicroConf is a nice offline conference for bootstrapped entrepreneurs. And they have some online stuff like a slack that kind of grew out of being the background slack for the conference back channel. I'm not super enthusiastic about their remote conferences, even though I went to one yesterday. And their online community, I don't know that I would really endorse it or say it's worth the time. But I was hanging out there on Slack this morning and saw a very familiar name. This guy, Nevo David, private messaged me and I'm going to guess dozens of other people linking to his startup on Product Hunt saying, please come promote me. targze Could this be something that could be solved through probability, reserve N slots on the front page to promote less engaged stories from the last M hours? lots of params to play with (lookback, engagement ceiling/floor, num slots, etc) but would help temporal shifting
Product Hunt being a really not particularly useful anymore startup dick measuring contest.

43:21dlamz I don't funny understand the self promo rule tbh. e.g. if someone like Hillel only submitted his own content I think lobsters would be better off than without.
dlamz fully*
targz i think i think that's a really so d lambs you have a good point let me let me address targz's things and then come back targz i think that's a really interesting idea and if nothing else you and dpk could talk about what fairness is but we could do that with reserving slots on the home page or even just have a second section down here that says like things in the queue or newest submitted you know you you kind of throw a couple of options out there we could definitely do something like that and see how it goes the home page doesn't have to stay just that list of hottest stories we could add some more controls or we could change what hottest means to include some more of that And then DLAMS, the self promo rule is...

44:32So I've talked a lot about it on stream because we've been doing the stuff to get at whether people are positively contributing to the community and meaningfully contributing along with their self promo. I've written up this and then there's the longer stuff that's in that mitigating self-marketing post or what is it? Mitigating content marketing, just kind of general stuff around The heart of the problem is there are folks like Hillel who are wonderful contributors to the community, who are here to be part of the community because they have the same interests. And Hillel, who is a Chicagoan, I've met him offline. He's good people, I think. He... particular writes a lot of really good thoughtful novel stuff and we want to reward that with attention that's kind of the best part of a gift economy where if there is any competition people are competing to be more generous and more helpful and the bad version is folks like Neville David who just want to try to take from the site and drive traffic to whatever their startup is they don't give a about any amount of discussion or anything except in that it might create links to their startup and dollars in their park pocket and so i think i think hillel is You know, I don't know off the hand, off the top of my head, if he meets this quarter of one's stories and comments. But I don't really care because I know his heart is in the right place and he does a tremendous amount of generous stuff. And his stuff is very well received as opposed to... dlamz seems more like a problem with quality of submissions than self promo imo
know if he was just posting here's my project and here's 0.01 of my project and here's 1.002 of my project like we've had people do that it doesn't prompt discussion it's not serious contributions and so this rule you know i say it's a rule of thumb because it's very fuzzily enforced there's another guy we were just talking about yesterday in the lobsters chat room who has pretty much only submitted his own stuff but then also it's yeah quality is a you're getting it what i'm trying to say and not doing a good job of expressing it's not just quantity it's also quality and When someone is submitting consistently really high quality stuff, I'm less worried about whether they're fully engaging with the community or submitting other people's stuff. And like all places where moderators are kind of eyeballing and having rules of thumb and judging quality by gut feel, there's a bunch of personal judgment and possibly more potential for errors here. I do think this year we're seeing a lot more self-promotion like we haven't since... I think the last time it felt like we were seeing this much was maybe late 2019, early 2020. Hmm.

48:47One sec, I gotta sneeze. And it's just tickling me instead of... So yeah, there's this fuzzy notion of, are you participating in the community or are you exploiting the community? And then it's really hard to say, is your stuff good or not good? Although we've kind of been getting at it with that, are your comments above average or not? And the big value of are your comments above average is the really exploitative people like Nevo David don't want to leave comments because it is prohibitively expensive on their time to participate with novel things in a community. So... targze Proof-of-work but for humans
So I've been doing the, are you doing above, are you writing above average comments that are not on your own stories as a way of getting at participation? And I'm still eyeballing those results I haven't tried to build a leaderboard or a worst of leaderboard both of which would probably be familiar names proof of work but for humans yeah actually and it's you know what the thing I really want is proof of don't even know how to summarize it i guess it's you know participate but not to exploit like i don't this rule is is not enough like i have just brain dumped a bunch of stuff about quality that is not at all present here and i don't have great ideas how to include that without i've tried a couple of times to write it and not been able to produce anything coherent And when I write sentences where they're like a hundred words long and have the word but five times and I'm like reaching for semicolons and parentheses, I'm just not, you know, it's that writing is a tool for thinking. When I can't write something reasonable, I know my thinking is not totally coherent yet. dlamz I think the quality stuff is really well done, but the more I hear the self promo rule described the less I feel like it's necessary
One of the values of having the stream is I hope folks understand that I'm doing a lot of talking off the top of my head and trying to get it gut stuff. And it's been really valuable for me running the site because it's, you know, forcing me to look at stuff and forcing me to talk through stuff in ways that are making my opinions a little more coherent and then hopefully legible on the site. So... dlamz seems like a rule designed to justify banning people for other reasons
The more you hear the self promo rule, the less you feel it's necessary. Do you mean the rule, the whole thing or this like one quarter rule of thumb? A rule designed to justify banning people for other reasons. God, I hope it's not that.

52:19I don't think I've done that.

...27So I've talked about how there's, I can't pull it up right away, but there's related to this. What is it called? There's the check for, are you a brand new user? So for our context, that's 70 days that they can't submit stories from domains we haven't seen before. And I've talked about this on stream, but a chunk of the value of this rule is when folks show up, especially people like Nevo David, to do self-promotion, they want to do it right out of the gate before they have any other understanding of the community or any kind of participation. Because for them, Lobsters is a traffic source, potential customers, not a party they want to join, a conversation they have something to contribute to. And so they want to go right to promoting their thing. And this has false negatives when, you know, we've had one post to their company blog before. But a big chunk of the value is this sentence here of, yeah, yeah, we know you probably could duck this rule. And there are a bunch of technical ways to duck this rule. We are going to recognize you and toss you because that's very obviously, you know what the rule is. You see the rule. This literally appears as a red box on the story submission page. You can't proceed with submitting a story if you see this. And so when people deliberately end run about it and are deliberately breaking the rules, it's kind of evidence of bad behavior and bad intent. Waiting for a minute is not a big deal.

54:35And if someone is going to break a rule that is very bright line, very explicit, like you didn't miss it, my experience has been that they are going to break a bunch of other rules too. Even though, like, number one, they're probably posting crap, they're also going to be posting a bunch of other stuff that's like, let me pick a fight, because... folks learned from twitter hey let's let's get mad about stuff is great engagement it gets you upvotes it gets you content so that's about where that's at of it's not about

55:26I care a lot that the rules are legible in a way that people can internalize them and predict the likely outcome. If I were using a rule of, you posted about the programming language I don't like, I'm going to find an excuse to ban you, I think that would be real obvious, and I would have a real shitty reputation for it. Ah, excuse me, I'm getting froggy again.

56:05ymeynot45 More Tea!
I don't know. That's part of the value of the mod log is you can look over my shoulder and ask yourself if I'm full of shit. More tea? Yeah, I have some coffee actually, but it doesn't really help. At least not with the throat clearing. It is just black coffee. I didn't put milk in it. I know that would make it worse.

...31Maybe like that time I had... By root come in. hair on the stream I should get somebody else to.

...47We started with topicality right right.

...58If I don't add to the topics, as we go writing summaries is a heck of a lot more work and I don't want to get behind again. Oh, hey, why me, why not, by the way. Good to see you again. So, oh, to finish the thought, yeah, I gotta get somebody else to stream with me so that my voice can take a break. I'm still actually a little scratchy from RailsConf last week.

57:32chamlis_ did you get a chance to investigate the custom vocabulary I sent over a while back? I should probably update it to incorporate newer usernames as well
arh68 some of my neighbors have those "No Soliciting" signs in their yard lol. self-promo talk reminding me
Anyway, if there aren't more questions, did I get a chance to investigate the custom vocab? No, shameless, I haven't. I have totally dropped the ball on that one. It is on my to-do list, but it's been behind a lot of stuff. chamlis_ no worries, definitely been there
No soliciting signs? Yeah. I have no idea how effective those are.

58:09chamlis_ I wrote a JS snippet to scrape them from your archives hehe
the i've been thinking about that custom dictionary because getting usernames out of the twitch logs is pretty trivial so again i'm sorry i haven't wired it up it's the get it over to amazon that's harder and then i guess i have to re-transcribe those i don't have any redo pipeline but i could build that pretty quickly i think The harder part is it also gets every, you wrote a JS script to snipe them from Microsoft. Ah, nice. I mean, honestly, the Twitch chat. So I still don't have the Twitch API, but there's a couple of sites that'll let you download Twitch chat from VODs. And so what was my last stream? So they just come down as these CSV files. So that's why I say they're easy enough for me to grab. So this is the inputs to the blog archive. There's another directory that has the transcripts in it. I don't have any principled reason for why I put this CSV in here with my writings versus the other. It's just, you know, my personal blog is allowed to be sloppy. See, there's that justification again. I didn't say sin or evil, but I was like, eh, sloppy. I guess a big part of it is I think a lot about ROI on coding of, will spending an hour on this thing actually, like, it will make it better in a theoretical sense code sense, but will that better be visible to anyone but me? Will it actually deliver value for what this project is trying to accomplish? And that's kind of always running in my head because I'm otherwise really prone to I want this code to be perfect. And a lot of the time not being perfect is just fine. It saves a huge amount of effort.

01:00:30Speaking of cleaning up after slopping things, so we have this extras folder that was never loading quite properly. It worked in prod, but reloading in dev didn't work. And Krydos, if you're around, please say hi, investigated it a bit and made this PR for, well, really, we should just put extras in the autoload paths And we should remove this code in the initializers that loads the files in these two directories at boot. pushcx https://github.com/lobsters/lob…
I'll throw the link here in the chat if anybody wants to be able to click on it. Ooh, Linux. I accidentally selected half of the Twitch stream manager as I was going to paste. So I wrote, I'm going to merge and deploy this when I have a chance to babysit production, because this strongly feels like one of those PRs that's going to act different in dev and prod, even running in the Rails production mode locally. Well, guess what? I deployed it, and it was one of those PRs that acted differently in dev and prod, even running in the Rails production mode locally.

01:01:57So... Let me show what this ended up looking like.

01:02:09Where did I put it? Let's just...

...30So in that 00 Zeitwerk initializer, and the 00 is just because initializers are loaded in alphabetical order, so if you want to mess with the loader, you have to do it first. I ended up writing this big kludge.

...53But the gist of it was

01:03:02Not proscite. We have this initializer called production RB. And this is the sample because the actual one is full of, what do you call it, API keys. And when I say the actual one, I mean the actual one on prod. I don't keep API keys on my local machine. But the way it works is if we're in production, this initializer tries to set these values. But the problem is these initializers run before Zeitwerk loads. And so when Rails boots, it's like, oh, on the pushover, let's set this class variable API token to secret. And then let's immediately throw an exception, a name error, because the pushover class isn't loaded. And dev mode will let you get away with it. I think the boot order is a little bit different. Or it is possible that when I was tinkering with this, I failed to stop and start the server. I thought I did, though. So I added this kludge, which is very similar to the code that was removed from the initializer, but at least it is in the initializer rather than... in another file. And I tried to... So the hassle is basically we have this pattern for setting API keys. And it made a lot more sense 12 or 13 years ago, and Rails has matured a little bit with how you're supposed to handle API keys. And maybe all of it should have just moved into end bars. That seems to be mostly the way things go, but it's definitely not the way the code looks. And since we want them available at boot time, we were kind of fighting to... If I could wave a magic wand, I would put this stuff in the boot order after Zeitwerk, but before anything else. pushcx https://guides.rubyonrails.org/…
and if you look at the where is it there's a whole guide about the rails initialization process of this config file loads then this other part then these other config files then your initializers It's a little vague on when it gets to the initializers, but I believe it's... Where are we? Back to the environment application. It's here because it has the call to run initializers. And it doesn't... Like it says the Railtie initializers, but it doesn't specifically say, I guess that's your config initializers directory. I wish it was a little more explicit. And when Zeitwerk loads, I don't know. It is not called out in this doc, unfortunately.

01:06:47Because what I would really like is if Zeitwerk, I keep saying it, but I haven't defined it. Zeitwerk is the Rails autoloader. If you use a constant that's not defined yet, Zeitwerk follows a set of rules to say, dpk0 okay, i have to go now, for a weird reason: i have to do some testing that involves making sure some code handles bad/silently lost network connections correctly, which means activating Network Link Conditioner on my computer, meaning Twitch isn’t going to work very well. i’ll have a think about the fairness issue and maybe chat with you on IRC about it
You know, if you try and load user, maybe that is a user model. Can I just go load that for you so you don't have to have so many requires? You have to go. No worries.

01:07:22arh68 best of luck dpk
OK, well, good luck with your weird connection issues. Those are certainly a trope of the stream. And as you have more thoughts or even just more of these good questions, yeah, ping me on IRC or email or site message. Or if you feel like you got a handle on it, just go straight to a MetaThread. You don't need my permission.

...57I feel like I need a streamover of whey for when I'm blowing my nose. It is very wintry weather here in Chicago. We've got our first snow of the season. It's a lot earlier and thicker than the last couple of years, which is nice. I love winter. Out the window right next to me, it looks like there's about an inch, inch and a half, and it's coming down pretty strong. It's lovely to look at, but I guess I'm sympathetically getting a cold and a runny nose.

01:08:37Oh, I didn't show the end of the kludge. So the kludge here is, this is the sample production RB. In your actual production, this is a separate file where lobsters and the various sister sites that reuse the code base have to fill in their API keys And a hassle is that shouldn't be in Git because you don't want to commit your API keys. Never do that. But that does mean that if I add a fix to ProductionRB, sister sites aren't going to get it because their production copy of ProductionRB won't yet include this code. And I can't force them to get this code. because it's not even tracked by Git. So that is one more sign that this structure of class variables on these extras classes is not a great pattern because we can't update it in production without breaking production. So I added code to break production. So this is where that require was removed from. And this is my ugly hack it says if we're in production let's look in the rails root let's load production rb as a text file let's break it into lines let's look at the first five lines and if it doesn't mention extras which is that fix i just had to make let's raise with a explanation of what I broke and what the kludge is. And I'm pretty unhappy about this. Like, so the upside of it is when the sister sites hit this issue, they will at least get a reasonably clear explanation of what's broken and how to fix it. But this is worse than having the require in this exact same place.

01:11:06I did this because it feels like an incremental way to get out of this situation, but maybe I should have just reverted the pull request. I don't feel great about it.

...29Part of the hassle here is it is hard to clean up boot issues, especially if they only show in prod, because it's hard to test those sorts of things. And then overall, I think all of this is just, it's all there to just work around the lack of autoloading in the past and then this slightly odd pattern. I think all of this code, all of this hackery is probably unneeded. pushcx https://github.com/lobsters/lob…
So I filed a bug, an issue about this when I deployed the previous PR. And I tried to, let me throw out the link. I tried to get at my best understanding of the problem. And having had two nights to sleep on it, I am more certain that this is just the API key pattern is not A great way to structure the Ruby we should probably just have those classes. Pull from envars and then maybe all of this goes away. So on an early stream. By route. I never remember his last name busier.

01:13:22Alright, this is getting ridiculous. I'm going to step away from the thread for 60 seconds to grab a Sudafed.

01:14:23gsora_ hello chat
You know, washing down a pill with black coffee feels like one of those little character moments in a script that just feels underwritten, like it's kind of hacky character establishment. It's not terrible, but it could be a little more subtle. Oh, hey, Jisora.

01:15:01So I wrote this pull request, and I tried to nerd snipe Byroot, Jean Boussier, who is from our last stream, did our Rails 8 update, because I believe he's a Rails core contributor? Yeah, Rails core Ruby committer. And he showed up on an early stream helping with our memory leak that turned out to be memory fragmentation. And he seems to like some of our weird plumbing issues in that he's done a couple of other pull requests for that. Is it for some uncommitted files? Oh, here's...

01:16:02Ah, wonderful. Kratos did a great job of explaining what this problem is for Jaune.

...52That starts this whole contention of whether the extra, the utility class is loaded or at the end of the analyzer.

01:17:22Pounded? Hidden? Yeah.

01:18:23we don't have a deprecation policy or

01:19:58The. yeah so just adding to what krydos wrote so i saw that jean submitted a couple of pull requests and i only just glanced at them before stream which is why they don't have that blue notification tab and knowing that he's a very experienced rail course committer who has done a great job repeatedly of diving into stuff. I almost feel like I could just hit merge, but... Oh. I know which one this is. The prohibit stuff? This one is the... There's a...

01:21:03form tag and there was a big changeover in rails years ago of replacing form tag and form four and we still have one all right got rid of the form four and they were replacing it with form width and so i tried to write a cop to enforce it

...37Because that kind of, there are three methods that differ by a preposition. You have to just memorize which one to use. Like I can't remember that. I can't enforce that. I can't communicate that to potential contributors. So I tried to write a cop to get at this. And I thought I've seen it work.

01:22:35Or what was it? It's rails.env.production, right? Yeah.

01:23:09It's definitely been broken for a long time. And then moving from form tag and form for to form with is far enough in the past that I'm not too worried. Going to reintroduce them.

...45You know, let me look at this before I write about it. I think this one form tag is all right. I think this one form tag is one of those places where redundant class in the submit tags, the form tag isn't printed if nested in another form. Yeah. This helper who uses this.

01:24:16This one is...

...22This is that thing with Ahrefs. Yeah. So we've been cleaning up this code anyways. But there's a whole issue that wonderful Eduardo Lopez was starting to work on.

...58What's the number here at 1382? Oh, is it app helpers link? No, app views helpers link hosted.

01:25:42Trying to centralize the hassles of styling and, or trying to solve styling and, or styling CSRF and HP verb in one place. Has the added complication that some of these appear inside of, other forms and when you nest HTML forms, generally the wrong thing happens.

01:26:32I'm not sure off the top of my head whether the right thing to do is, remove link host or try and prove it so that we can centralize

01:27:31So that one issue, link post, this whole thing is, or the logout path is what immediately made me think of it. But there are a few of these, if you're on the settings page and you want to disconnect GitHub, Mastodon, what's that other one? Keybase. Well, in the same way that at the top of the comments, We have the voter, the edit link, the undelete, the moderator delete, the delete and disown links, like all of those. pushcx https://github.com/lobsters/lob…
Most all of those because Eduardo Lopez has fixed a couple now. Yes, you see the two PRs. I'll share the PR link.

01:28:34Those all currently depend on JavaScript and I would like them to not depend on JavaScript. And Eduardo cleaned up a couple of them and there is a to-do list of remaining ones. And so it's kind of fortuitous to run into this one use of form tag

01:29:51all right so yeah i am going to just merge and i will also just use github's Odd styling of merge message. So they're cleaning up libmonkey. He is, yeah. All right, so he caught the other reference to it. I want to say that if I look at it, I suspect he didn't dig deep. But I wanted to say, come here, tig. Yeah, we used to have more in libmonkey.

01:30:58And I've simplified it with time. And also, everything about Unicode has gotten a little easier to work with. Oh, yeah, there was the other big thing that was in there. Did I write those? I don't know who wrote those, but they were kind of working around queries that shouldn't be writing raw SQL and interpolating values in any case, because that's the setup for SQL injection bugs, or just really, it's just the setup for unreliable queries. It's nice to use the active record interface as much as possible.

01:31:51So I'm actually happy to delete that. Oh, why did he touch these?

01:32:07So the script has to load. I don't know why the flagged cache script would load that. I don't know how he would notice these either. And loading config environment starts the whole boot. Well, let me just bring back from the Twitch chat this tab. that makes sense so if this is the entry point from rack passenger puma and starts everything off i can see why he can just load the file and get the whole rails boot process that's a lot cleaner i bet he has a comment explaining this and i just worked it out for nothing I mean, you know, my own edification. OK. I guess I merged these in the wrong order or no, he's a.

01:33:32He's just touching the same stuff I broke. So this can just get deleted. But then I merged his other PR that also removed that extras. So let me pull things down. And if I look in the extras directory, what do we have left over? So we have the utilities and now we have no cops. Right, good. So that is the correct fix, is just delete all of it. Always nice to delete code.

01:34:41Today feels like the day that I'm going to miss the mute button and you're all going to be really glad that I properly configured a noise compressor. So if you've never done audio engineering, a noise compressor is like a like setting a maximum volume on the input channel so that if I talk or cough or sneeze or clap in front of the microphone, it won't really get any louder. It'll get a little bit louder than my voice is right now, but not the actual loudness it is in the real world. So this is not one of those movies where you have to turn it up to hear the dialogue and then the explosions blow out your speakers. I try and keep it to a fairly tight volume range, lifting up my quiet parts and very much limiting the loud parts. That also means that I can sit and talk in a fairly normal voice. I don't have to put on my I'm standing on stage projecting voice because I can do that for, you know, a talk that's a half an hour, 40 minutes. But if I did that for three hours a week, I would need to take some proper voice training like an actor.

01:36:43jmiven hello
Oh, hey, Jay Mevin. Thank you for your messages this morning. I appreciate that you pointed me at a hotspot. The flags do work, but a lot of the times when someone is just Sometimes there's just bad vibes that don't line up super well with the flagging. So I appreciate you highlighting stuff to me. Thank you. You know what? I mostly do it for people who find bugs ahead of me on the stream here, but I'm going to just mark you as a VIP of the channel because you've been incredibly helpful in spotting site issues and moderation issues.

01:37:44jmiven oh, thank you :)
so with that and i don't know if you know the stream archive pages but we had a big discussion this morning around how timing works in story posting and self promo that might be interesting to you and if you weren't tuned in earlier the The stream archives typically go live a couple of hours after I stream. I say typically because the pipeline for that is just a hacky shell script because it doesn't matter if it's perfectly reliable. And so sometimes if it throws an exception, I sometimes don't notice until the next morning because it's not super urgent. Okay, so this. jmiven interesting, I will look at it
This actually said extras got smaller hold on let's pull down the other thing here and let's look at what's left in live. So now live is just the tasks and these two utility methods. which could probably be in utils extra.

01:39:10Right, or extras utils. Yeah.

...20For historical reasons, well, just the Rails pattern has always been that rake tasks go in lib tasks. And I don't know that we have a strong reason to have an extras directory instead of shoving these in lib. I think JCS did it that way because there was originally just stuff for these different services. But then we've, well, that's not true because we've always had the markdown code and then sponge is not a service and email parser is not a service. Yeah, maybe they should just be in lib. I don't know that we get anything out of reorganizing them right now. All right. So I think, yeah, so that's both of the pull requests that I nerd snipe by root into. And I think they're both improvements.

01:40:33Well, Sudafed is kicking in. So if I, especially mid-sentence, go dead for about 10 seconds, I haven't fallen asleep or something, that's just me mucking a loogie. All right.

...56So this is fine.

01:42:01It's not a, what is it called?

...44Initializer. All right. Organization. Here we go. I feel pretty good about this, though. The boot stuff has been a hassle for a while. pushcx https://github.com/lobsters/lob…
and i mean a while the entire time i've been running the site what seven and a half years seven years and change because it was october of 17. oh so the the other pull request i wanted to look at here was 1326. oh actually before i jump into that i will Repeat the station identification. This is office hours for lobsters. If you have any questions about the site or its code base, feel free to pipe up. And when people don't have questions, I work on the code to make sure that it's moving along. And aggregator.

01:44:15Didn't mean to close that. So this other one was by GMem, who is sometimes here in the stream. You can say hi if you're around. You don't have to. But we are trying to lean into using Rails features for things that we have hardcoded before. and one of the fairly scary bits i say scary because oh it's such a hassle to set up in prod and it's so easy to break in prod is the way that incoming emails go into the mailing list mode code so if you subscribe to get emails you can just click reply in your email client and post comments straight from Mutt or Neo Mutt or some other less popular email client. I was using Neo Mutt at a Ruby conference. Somebody happened to walk by at a table and was floored that my email was Mutt. They were like, I haven't seen that since I was in college in 2000. And I was like, well, still here. Never had much of a reason to update. So GMM started adding Active Storage, which is the, well, started adding Action Mailbox, which will hopefully get to subsume some of the work that's done by our parse incoming email script and my terrible postman shell hackery that tries to deliver it into the app.

01:46:12And part of the reason to do this is if we get action mailbox working, we get better exception handling in prod, which is always nice. We get more testability. And ideally, we get more reliable code generally. So GMEM also did a hand with breaking out the active storage config so I could start the avatar code. So the PR, stale PR bot showed up and said, hey, this has been a minute. And GMEM pulled in master, which fixed the build. The build had been broken for a minute. I think it was either a merge conflict or something not too particular.

01:47:10All right, so I'll have to configure that. So I want to do one more review of this pull request, but I think it is ready to merge. Especially if it's a second pipeline that doesn't break the existing pipeline, then I can do the Ansible stuff. off stream because I know it's another one of those pull requests that raises the hair on the back of my neck for as soon as I deploy it, it's going to need some prod tweaking, much like the one that I started this stream with.

01:48:00So.

01:49:22I wonder if this is going to change because of that monkey patch change.

...34Because wasn't that UTF stuff used in email?

...43It's giving me the darn merge commit, not the merge. Is it?

...57There we go. It took a second to read the graph. So this is the forcibly convert that's in the email parser. And does this guy use the email parser? Re-implement? I don't think so.

01:50:25Yeah, this is sort of an alternative to the email parser.

...34I'm not sure what. So Rails 2.

01:51:00David Price- Oh, so what happens here is when you reply to an email from lobsters to post a comment, it comes back to an email that is like lobsters dash and then a unique token for you. David Price- I see what I see what GM is doing here, so I think what this wants to be is mailed to dot. Select. We really want to just.

...58Anything that is starting with.

01:52:42I really want to, let's just say, this probably has to be a, surprised I don't have to escape this hyphen.

01:53:24I guess really what I want to say is I want the part of the string between the hyphen and the at, which is... Oh, actually, maybe that's a good place to demo something. I wrote a hacky little script for a Ruby gem called Quick Question. I've been playing with it as I've played with... Claude Sonnet. And so it is a Ruby gem that just takes stuff from the command line and slaps it into Claude Sonnet and then gives me an answer. So from the command line, I can say something like Ruby one-liner to parse an email like email string like boo-bar at example.com into just bar.

01:54:29Yeah, and so sometimes you get this stuff. To parse, let's say to regex.

...44This kind of quick lookup is not bad. Returning nil if it doesn't match. Because that's going to throw an error, right? Yeah, rescue now. Yeah, sometimes you get this silly stuff back.

01:55:10pushcx https://push.cx/qq
So the link for that is here. It's just a tiny Ruby gem. I kind of like including these costs because occasionally I tell it to generate some complicated bash pipeline where it's like yes i could remember those args to ffmpeg and tar and nine other command line utilities and pipe things through awk and sed or instead of spending like 15 minutes on that for a tenth of a cent i could have an answer out of the lom there is a There's a bit of value sometimes, but you have to read these things I like it for small things where I can immediately tell whether something works or not and fairly low stakes. I say empty string. get another oh overloaded I guess too many people are asking songs to do their homework for them. that's. Wow, that's a little more complicated than I expected. Huh, I didn't know 2s. So a string, we can just square bracket a two bar square bracket bar? Huh. Did not know that was part of the string standard library.

01:56:56so then oh and if we say this we get a capture out of it so if i just said hyphen followed by anything followed by an at sign that is succinct ruby what if we don't match then we get a nail that's fine because we can 2s it so this is Give me a dash or any character. That's weird.

01:57:40Not sure what that less than is doing in this context. I could just ask the LLM, but that's all right. And then maybe, oh, so it's more optionality than I wanted. All right, so let's just say It's one of the banes of regular expressions is grabbing too many things. So here, and if we say user who, great. So that's a little example of asking a short question.

01:58:28easeout I do like LLM codegen for that specific aspect—showing me bits of the toolbox I didn't know about, that you might only learn by studying bottom-up or cover to cover
Oh, come here. That's not the... LM Code Gem, showing me bits of the toolbox. Yeah. Oh, I got rid of the...

...57arh68 I believe 'lookbehind' is the (?<= ...) thingy
didn't put the i'm going to just keep revising this so this wants to be dot first look behind oh probably yeah you know especially when you're one-shotting with like a 10-word prompt and no context it just sort of assumes things about the context that are not always super accurate. This one is also going to throw exceptions here if there isn't a to email address that matches the short name.

01:59:54And so we'll just use the nil propagation operator, right? Oh, God, that makes so much worse I mean this is I don't love this but I guess it works so if I said let's grab that I said mail equals empty list and I run this I guess what I want is that, okay? If I said mail equals paidExample.com.

02:00:46Yeah, I started a regular console, not a Rails console.

...57And it's got its own IRBRC. And it doesn't have a mail. Oh, I typoed. This should be .shortname, shouldn't it? OK.

02:01:45This were invalid because people can put anything in a to line. We'll just get empty string. That's fine. All right, so let's fix that typo. I guess I'm sure about that backslash now because it runs. Let's look at how he uses it. Maybe I should say they. I don't think I know GMEM's pronoun. UserToken. It's actually fine if it's nil. Let me take that 2s off the back then.

02:02:39Because if we pass nil into the query, it'll just not find any records. And that's fine.

...53Glad I don't have to shovel. So I'm looking at my window here in downtown Chicago. And there are now, I don't know, call it 2 and 1 half, 3 inches sticking. We're getting a real good snow. All right.

02:03:16Add action mailbox, configure it, add the tables.

...28See, this is a beautiful thing to have a couple of tests on these. Mailing list mode doesn't break often, but that's only because it changes so infrequently.

...48It's hit rate for changes that break it. It's a percentage that approximates 100%. And so getting to put a couple of tests on these things feels great. Getting to use the Rails feature for it instead of it's a shell script that pipes off a dot forward, that feels good.

02:05:10Did I?

02:06:36Great.

...42So if I've got that, ingress post fix. So I just want to review this while I've got this code in my head. So this we say no. configure post fixed type inbound emails and the ingress password you previously generated okay so i'm definitely gonna have to run some stuff in prod yeah you know this rails credentials that's it isn't it that's the

02:07:34That's the new feature that we want to use. Can't be edited directly, has a master key. Other secrets, such as access keys for external APIs.

02:08:01Speaking of leaning harder into Rails features that we had to invent before Rails had them,

02:09:14Great. That's funny how closely related some of these things are. It feels like there's been a really high hit rate on, oh, these two issues that we just talked about are actually related and inform each other. And this, I don't know, I think it's probably the two big contributors here are leaning more into rails features and cleaning out the junk drawers i suppose there's a small aspect of i've been working for forever to try and minimize the number of open pull requests by you know keeping them active and getting them merged and then also the number of what's the called the number of open issues or at least just making them easier to work on and as people touch stuff they bring in the rails defaults and the desire to use them over our weird solutions not so weird arh68 can I ask: what was the original Rails version for lbosters
or effective solutions that predate Rails going, oh, everyone has this hassle. I wonder if I even have this set up in prod. What was the original version of lobsters? I want to say 3.2.2. So let's look. Oh, and if you haven't seen it, the... There also is one of my fun facts about the site. The very first commit message is initial work on conversion from PHP tree. So little known bit of lobsters trivia is that JCS started out in PHP, and then after some amount of time, chucked it and rewrote in Rails. I've never asked him about it, but boy, am I glad he made that call. Come here, Tig.

02:11:46Ah, yeah. So there it is. I did remember. It's Rails 3.2.2.

...56And I'm pulling the... Terminal off screen because I'm going to look at production real quick to see do we even have the credential store set up? And. Yes, the file exists. I don't know that there's anything in it. So I can run rails credentials at it. What are the other command? Rails dash T credentials. bundle exec. Alright, so there is a rails boot times are getting a little slow.

02:13:03I have to start the server to list the tasks because they could be in any code. But that takes a minute. So I haven't touched the credentials file in a couple of years.

...24So the date is like early 2021. And if I say edit,

...37should get a vim with the credentials let's mark this as not streamable so that i'm not you know one dumb keystroke away from showing keys on stream yeah okay so as i'm looking at the credentials file we have a secret key base which is basically the default there is a commented out I guess it looks like YAML, like AWS. And then subkeys, access key ID, secret access key, we don't use AWS and never have. They're literally just the one, two, three, and three, four, five. So there are a couple of default values. All right, so we'll make that streamable again. Nothing too exciting. But yeah, ARH, to your question, there are a bunch of places that either because it was JCS's first Rails app or because we predate it. There are a bunch of places where Lobster does not follow the Rails convention for something or does not make use of a feature. And over the years, I have tried to use them more and more and become more of a standard Rails app. It makes it a lot easier for people to contribute. We get a lot more bug fixes just for free when it's not our custom code. So I will set an Ingress password. Yeah, let me put this on my, given that GMEM came and did the, or pinged on the action mailbox PR, I wanna say within an hour or two of the stale bot pinging. Fingers crossed that they come and finish off their PR.

02:16:04Where's that? Let's just grab that. PR four. All right. So we'll grab that. I will do this after the stream. This is nice. I'm so glad to have these tests. And do I?

...41Just trying to peek at GMM and see if they listed pronouns. He. Great. Then I wasn't wrong. I thought so, but it's hard for me to remember these sometimes. I'm just outnumbered.

02:17:09Man, theme of the morning is using the features that we predate.

...22What's this pull request that's hanging out? Oh, it's this one that I didn't see that poster came and replied.

...45Oh, does it actually work? Oh, I read the flow request and I didn't get that. Let me clone this locally, because if this is fixed, I would really like to merge this. Let's pull it locally.

02:18:09So this is... When you click flag or you click the caches link, this is local git setup, I have to say. I set it to force me to think about whether I want to rebase or fast forward, but it does mean that I constantly get this warning instead of the task I expect. All right, so if that's set, let me start the Rails server in the background and pull up the localhost.

02:19:04Actually, I think I set my password to... I thought I set my password to some test thing. I was dealing with a... some password login thing in prod so i don't remember what state this is in so i'm just going to blow it away grab me the where's my user do i only have that alias in prod

...44This is me checking that I'm not on production right now, and I'm not. All right. This darn definer thing. This is... Did I clean this up? I did it off. When I restore from a database backup, the view has the wrong definer. And so I have to run this little fix up shell script. There we go. So now if I click caches, OK, that's good. And now I need a story.

02:20:41Yeah, here we go. Story that's been flagged a bunch, but not down to a negative score. So let me actually just grab that.

02:21:11So if I give the story a negative score, since no one's going to vote on it in my local, I can just slap the story into an impossible state. And then, yeah, it just works. If I can't flag in. All right, great. Nope.

02:22:10Oh, I do understand why it works. So the whole hassle here is in CSS, and I get it, there's a concept called stacking context and the 3D conceptualization of HTML elements along an imaginary z-axis. So the question is, Which elements are on top when they overlap and when you open this menu you want the menu to be on top of the text area and on top of. Basically, everything else on the page, but the hassle is. If you set an opacity on. an element, it gets its own stacking context. And so if we set the story to be less opaque, the flag menu was showing up underneath. But if this person has set the opacity, they are making another secondary stacking. That's actually a really simple fix that should be reliable. Yeah. Okay. All right.

02:24:06And I bet that has a z-index, doesn't it? The menu? Yes.

...39And I believe there might even be a thing with text areas where they get their own stacking context. So there might be a third at play. I don't remember offhand the text area as one of the early The MDN, which lists number of things out back to this page. So yeah, see, so there's the opacity and his text area. One of those things that creates a no does not create a stacking context. It's got a, there's something along those lines. speaking of things that are old and predate other features hyrum's law definitely applies with regard to web pages so hyrum's law is any observable behavior is going to become part of your api because someone's going to depend on it and textarea is a little bit different from other form controls and The early form stuff, it had a little jank in it, but it's become load-bearing jank.

02:26:16Let's just merge. Let's fix the message though.

...30So let's fix.

...54so it's fixing and i don't know if i'm at 80 characters or what but i don't want to think about it this is not like it's going to go to some piper mail archive where someone is going to explain that i have violated free software principles by having a commit message that's longer than 72 characters so we'll be okay oh that's great i'm so happy to merge that it also I'm especially happy about that. You can probably hear it in my voice because fixing it by setting the opacity is probably going to be a reliable fix. Fingers crossed.

02:27:47All right. Man, I can almost... If GMEM comes back to this pull request in the next day or so, we've got a real chance of getting down to pull request zero, inbox zero, pull request zero. And I would really like to have all of them merged. And this one, I talked about it last stream where I had said, they offered to add tests to this cleanup And I said, you know, Hey, yeah, even a smoke test would be great. And then a couple of days went by and they didn't. And so I was like, well, you know, you don't, you don't have to volunteer to do more work. And I said, do you want to close out the remaining break, man? Cause that did feel like, it would be nice if they finished it. It is part of this work. that if it trips the security scanner, it should probably get fixed. But if they've gotten real busy and they can't finish, I think I would rather grab this to the command line. I would rather sneeze.

02:29:23All right, having handled that little meatspace drawer, let's grab this code and bring it down and finish off this pull request, because I would rather finish it than let the stale bot close it in a bit. And again, come here. Good. Oh, this prompted some little tweak. All right. Look at what changed here on the branch.

02:30:20arh68 ? cmonBruh what was that lmao
What was what? My gripe about meat space? arh68 "building a dream ... "
I muted to sneeze. I've got a little bit of a runny nose. Something about the... Changing the weather is getting me today, either that or that. Oh, building a dream. So I'm using Vim and you can see the tabs across the top for my various pages. In Vim, you can open a terminal and I do, that's how I keep opening all these terminals. But if you start vim from a terminal that you run from vim, there is some weird freaking config that gets a little bit mad. Something about shelling out from a vim that's in a terminal in vim. And I didn't want to debug it. arh68 ohh ok. vimception detected
And so I hacked in some vimrc that says like, hey, if we are starting vim and we're already in a terminal that started by vim, let's just not because it's so easy for me to type like vim. And so this is reminding me, yeah, vimception detected. That is exactly it. And so I made it a goofy quote from Inception. I just watched this movie again a couple of weeks ago, and it's pretty, but boy, it just gets... It is not a movie that gets better with watching again. There's a thing in sci-fi where one of the genre conventions is... It's okay to start out by just dropping people into your different setting, but at some point you have to explain like the world building or the rules of the world or like, oh, we have robots here or, oh, we're actually in a virtual reality or, oh, there are spaceships that make you go crazy if you're awake. Like you have to explain that at some point And, you know, there can be a cold open where you don't explain that, like in The Matrix or like in Inception, that's totally fine. But once you've set up the rules, you just really want to leave them alone and let people enjoy the setting and get into it. And if your world building is so complicated that you have to keep explaining what it is, people can never really get into the narrative. So, like, The Matrix, fairly early on, they explain what's going on and what the setting is and, you know, if you die, you die. And it's good. Like, it works. From there, we can enjoy the rest of the movie. And Inception, I swear, every ten minutes, and it's like a long movie, every ten minutes, they're like, oh, well, But if you die in the dream, you go to limbo. But if you die in limbo, you go back to the dream. Except in this dream, because we have the extra sedative. And then it goes to double limbo. And it's just the whole way through the movie, they're just making up rules and making up settings. And it's incredibly distracting. And... ymeynot45 LOL, now I want to hear you explain Tenet
It's really weird because they could have just been like, yeah, it's a dream, there aren't any rules, fuck it, and let the whole thing run on dream logic, and it would have been fine, but it kind of wants to be sci-fi. It just... I want to like that movie because it's so darn pretty. chamlis_ exactly what I was going to bring up, never got any of the limbo stuff
You want to hear me explain Tenet? Sure, so the first thing to know about Tenet is Christopher Nolan has no idea... how to write a female character so inception has a good example where the female character arachne ariadne ariadne is just there to have things explained to her the whole movie and then there's maul who like actually is sort of an interesting character but yeah that gets on the plot but like ariadne the primary well not primary I guess secondary female character. She's just there so that men can explain things to her. Tenet is also a movie. What's the... So there's the Bechdel test in movies. Like, does your work of fiction have two women with names who have a conversation that is not about a man? And it's just kind of like, it started as a joke in a comic in the 90s. it's actually kind of a useful bit of media criticism like it's a fairly low bar to clear of do you actually have two women in your work and it's useful because not a lot of movies and books actually cross this line and christopher nolan's movies are especially bad about it and there is here let me There's a version of the Bechdel test that is... Oh, this is a reasonable page. That is like even more extreme. It's sort of like the Bechdel test was already literally a joke from the beginning. A joke of this is such a low bar to clear. Can you clear it? ymeynot45 Sorry for hijacking the conversation further. jerdLURK
And since so few works can, someone created the sexy lamp test, which is if you could replace your female character with a sexy lamp and the story still basically works, you should probably work on it. No, this is fine. Tenet is one of those movies, like many Christopher Nolan works, where the female lead could be replaced with a sexy lamp. and the plot wouldn't really change. And I start there because it's such a clunky thing in Christopher Nolan's works and it just gets worse with time.

02:37:00The plot of Tenet is that a guy is recruited, a secret agent undergoes an unclear recruiting into a secret organization that is dealing with time that runs backwards, and he has to help a sexy lamp save her kid from the big bad guy. And there's not a lot more plot worth explaining. arh68 i saw inception just the once, in theaters. really memorable. saw tenet twice can't recall much :|
I think Tenet actually does a lot better than Inception on the eye candy front and on the, look, let's set things up and then just run on dream logic. Yeah, Tenet...

02:38:01Tenet also really leans hard into Christopher Nolan's dialogue where, or maybe it's his brother, because I believe he writes a lot of these with his brother, where characters have to do exposition dumps in a really clunky way. And I'm not going to try and find a trope for it offhand, especially because it's a video and I don't have the system wired up so that video can, well, maybe video can play into the stream. I don't know. And then there's the whole copyright strike thing but. there's an early scene with the protagonist in tenant where he meets a sexy lamp who explains the rules of time travel to him and they they it is like the quintessential Nolan bad dialogue where. he just kind of throws the character into the scene because it doesn't make any sense how the character gets there. And then she explains some stuff and they have a conversation, but it's just him trying to ask questions that would elicit the exact answers the audience needs to understand the movie a little bit. And then she says some cryptic shit and then they just cut out of the scene because there is no way to end that clunky fucking scene. yeah i don't know i think the i think maybe the secret of tenet and inception is that they're fine if you're stoned because then it's just lots of pretty lights and colors and it doesn't really matter that the movie can't set up a coherent world and just kind of lurches from scene to scene all the time Speaking of lurching, let's lurch over to this pull request review. so we have all this where's my diff right I am here to fix brakeman oh the other reason you don't have to apologize for like hijacking is I'm not gonna ramble about this stuff if I don't want to And this is Lobster's office hours. This is not watch Peter code for three hours. The code is how I'm going to fill the time if nobody's asking questions. You can ask me fun questions like that. All right. So there's that. How do I get Breakman to just show me this note? So break man watches for things that look like sequel injection and it has this signature concept and. This pull request broke the build because one of those signatures changed because they touched the code around the. The query. And it is wonderful that break man works this way, it is exactly the kind of. be a little paranoid that I like out of security testing tools. And I can say, hey, this is safe. We understand this query and what it's doing. And I want to, rather than write that note, I want to just grab that note that already exists because this is a false positive. And I thought there was a way I could just pass the signature in and print it. Let me just grab it out of the file. Where are we? What is Brakeman's file name?

02:42:07Is it in config? Is it hidden? Come here. How do I show hidden files in this darn? Capital I. That makes sense.

...28And it's not in the root. Is it in config? Ah, here we go. Breakman.ignore. All right. And so if I search for that, here we go. So this is the one that we're getting the false positive on. And if you were paying a lot of attention, you might notice this query exactly matches the one that we're seeing a warning for. And my note just said I didn't leave an extremely informative note. That's fine. So we're going to run BrakeMan interactively, a different capital I, to inspect the new warning. And we're going to say, let's write a more informative note than the one I just spent a minute schlepping to find.

02:43:40So the concern here is at view stats index, but it's, it's triggering the query, but the actual query is up in the controller and it is, it is this query. Unescaped model attribute. I don't see any unescaped model attributes.

02:44:43thought it was saying that I'm using oh oh I get it it actually is so this is the query that runs but what it's concerned about is that the output of the query is just getting printed raw to the page it's not but because the query gets evaluated here brakeman can't know that

02:45:19And the reason it's concerned about unescaped model attribute is, oh, I want to say, and for making a note, I don't need to say this is safe. That's what the presence of the note means. Query output, come here.

...56And then there's the old fingerprint and safe. So break man is saying, Hey, if you have a query and you directly print attributes from that model without escaping, which is what calling raw does, it is possible that someone could like imagine somebody is, About on their personal profile page if they filled in something that could be html or javascript and we just raw print that out of the users table, maybe we are opening ourselves up to XSS injection cross site scripting and so.

02:47:00And so Brakeman would just like to prevent you from put-cutting yourself. It can't really know that the data we're selecting out both doesn't use any user input and then also know that it's going to the SVG library and we're printing the output of that. Just the limits of static analysis in a highly dynamic language. What's the name of this branch? Great. Push that up.

...43Please, please, no merge conflicts. Great. So now if that's pushed up, the PR should close. I mean, the PR should know that I merged it. Maybe GitHub's just moving a little slow this morning.

02:48:07Maybe following its exact instructions in the command line did not produce a merge message where it knows that I merged.

...24Except that this is this person's commit. GitHub, you should know by now. didn't edit a commit message and it shows them as the author so they have the commit id 982c and something i must have accidentally rebased it onto master well i haven't

02:49:04Oh, I haven't merged to Origin Master, so I haven't pushed it yet. Huh, thought I just pushed. Now it's pushed. Now do you recognize it or now do I have to leave my comment that says I merged it? Now I have to leave my comment.

...35So there isn't a button I can say, yeah, I merged this. So it's just going to show as closed without merge.

02:50:35Wonderful. So click on this. We get the yes. Oh, GitHub thinks I am a co-contributor on this? What does it think I changed? This is rebased. That's what. One of these decades, try the new merge experience. -oh. But it has AI sparkles. And it doesn't do anything. Well, that's AI. Okay. So now we are down to just one open pull request. What a great place to be. I don't know where GMM lives, but I would assume he works as a coder and so is probably at their job, right? At his job right now. So maybe as early as this evening, we will get down to pull request zero, which would be very nice.

02:51:47Oh, there's activity in this since I left a comment here on stream. Y'all can watch me refresh my... What is EJSON?

02:52:11Cryptid secrets, unique way of retrieving secrets you can set.

...22So this is a big serious install thing where maybe you're pulling your secrets out of LDAP or some other enterprise-y secret store that does not involve pasting things around in the prod console in Vim.

...44I'm going to double check that.

02:53:31arh68 i am also curious as to what they prefer bout it
Also curious what they prefer about it. Yeah. I believe Sean works at Shopify. Yes, he's on the Ruby and Rails infrastructure team.

02:54:16So aside from my general aversion to adding dependencies that are not part of core Rails, I suspect this is, yeah, they're just operating at scale. And part of the reason I immediately jumped to that is it has arbitrary code and this unique way of retrieving decrypted secrets. And I remember that When I was working at Stripe, they had their own very sophisticated secret store, and I knew I barely interacted with that thing, but I used it once or twice and just wanted to make sure I didn't F it up. I assume Shopify also has something big and mature and enterprising.

02:55:23Alright, so with all of the pull requests that I can merge, which also it's wonderful that I get to merge them instead of close them or let the stale issue bot close them, and really promising prospects for GMM's remaining pull request. we are coming right up on my usual target of three hours so i guess i will say like one more time this is lobster's office hours if you have questions about the site the community the rules really or what goes into good sci-fi movies and why so many movies are bad this is the lobster's office hours and you can ask these questions if nobody has any questions here in the next minute or so i'm going to roll off i did some tv and radio production stuff in high school and i was always impressed upon me the value of you know you can't have dead air in your broadcast or in your production But I also know that the Twitch stream is necessarily like five to 30 seconds behind. And so then whenever I ask like, hey, does anybody have questions? Aside from the usual, it takes time to think of people questions and add them to the chat to type them up. I also have to vamp over the lag. There was a Yeah. And so that habit of filling time is something I've been thinking about a bunch.

02:57:20arh68 PopGhost no further questions yer honor. i'm just leetcoding
There's a, where was it? Was it 2019?

...402018? There was a conference in Chicago a couple of years back, pre-pandemic. No further questions. pushcx https://www.youtube.com/playlis…
This is it he's got the suit on, so I will I will roll out with this, but this is a cute playlist from and for conf 2018 and the organizer here track glow glow lap that getting tired glow wacky. Fine Polish name probably. In between all the talks he gave little. Chicago history trivia to kind of vamp while the presenter was setting up. You can see people kind of milling around and setting up the projector and debugging the, why doesn't my laptop show up on the screen or giving people an extra minute to come back from grabbing a cup of coffee and get seated. And this is just. incredibly nice slightly overproduced you know in that this is a lot of extra work for a conference organizer to come up with clever anecdotes to use to fill the dead time in the presentation schedule but i saw these and i was really impressed by them and then also you know i'm a big chicago history nerd and a big fan of the city and if you like these pushcx https://www.chicagohistorypod.c…
I will recommend the Chicago History Podcast, which also has a whole lot of fun stories about, and pop-ups, fun stories about the history of Chicago. And there are plenty of episodes in the back catalog. Oh, very murdery recent couple episodes. They're not all murder and horror and such. Anyways. On that note, I'm going to roll out because nobody's got more questions for office hours. But it was good chatting about meta stuff. hacking out some code getting stuff merged so let's look at the the topic list oh yeah let's did i grab those to the clipboard and then i selected other stuff not a lot of need for the scratch this time i wasn't working through any painful technical problems it's just a couple of links to share put that on the stream archive page I showed this to a stream archive page to a friend that runs a podcast, and he was like, oh my God, I need to upgrade my show notes. So that was promising. I hope people steal any good ideas I have and make their own stuff good too. Better. Alrighty. Well, take care, everybody. I'm going to roll out. arh68 see y'all monday DinoDance
I hope you have wonderful snowy days yourselves.

03:01:06Oh yeah, Monday. Next stream will be Monday as usual. Thank you, ARH, for reminding me. So that's 2 p.m. U.S. Central and earlier or later in other lesser time zones. Take care.