It's YAML all the way down

01:13jangomandalorian 👋🏼👋🏼👋🏼
Happy Monday. I'm Peter, and this is the Lobster's Office Hour stream, where I work on... Ah, hey, Django. Nice to see you again. Where I work on anything related to lobsters or answer questions about it. Got a pretty light topic plan. My notes get set up. Hope I don't sound too bad. I have a nice herbal Chinese tea. I can't read what's on the box, but I know it's nice when I beat up my voice by talking for days and days, like I sure did at RubyConf on Wednesday, Thursday, Friday last week. So that's why there was no stream last Thursday, and... There's not too much to catch up on, although there is one really pleasant surprise in the PRs. I'm already coughing and clearing my throat. That's a bad sign. We'll see how this goes. So if you haven't hopped on the stream before, this is the office hours for lobsters. You're welcome to pipe up and ask queries that you would want to run against the prod database or any kind of questions really about the community or the code base or the site, how stuff works, why decisions happen the way they did, what's next. Someone just this morning in the chat room was talking about a query regarding homepage. I don't remember your username, started with a T, so if you pop up, We can work that out here. Otherwise, I have some pull requests and issues to pull through and talk through. I believe, just to look at a small one that we've looked at on stream before, I think this one's still waiting for another nudge or another test, really. I am trying to volunteer Jason Mock very politely. to add some tests. This kind of thing is always tricky because I appreciate anybody's contribution. It is really nice to have tests, and I would rather... I mean, I literally... The build is broken right now because they have the chore of running Breakman. I would really appreciate if they add tests, but I would settle for a passing build. Actually, let me just say that on here.

04:40Just trying to make it really easy for somebody to help out. So there's no movement on the others, and so much of my time was spent at the conference that I didn't do the hot code reloading. But there is one small thing to note here. I spoke with, who was it? I spoke with John Hawthorne, who, I think he's on the Rails core team, while I was there, and we were chatting about zeitwerk with somebody else and i said oh i have this weird issue where it seems like our code is not reloading properly and then we have this stuff in the extras directory and it was like yeah if it's a monkey patch it kind of can't reload properly and i we talked a little about that and it makes a lot of sense but the other stuff the utils should be And I explained that, well, we think maybe it's this require initializer that's firing before Zeitwerk gets a chance to load. And he did one of those things where he kind of looked up and found the answer on the ceiling tiles, like he thought about it for a second. And am I not on the, sorry, I'm not on the right scene here that whole couple of minutes, sorry. so this is the request i mentioned i haven't merged yet about reloading the extras folder and so john said yeah you know it really does sound like if like this would be the issue like loading those libraries explicitly before zeitwerk has a chance to load would cause it to ignore them and basically fail to reload them in development mode And now that I recount it, he did say the behavior is different enough between dev and prod because prod does the eager loading that we probably don't really have to worry about this breaking prod because we're not hitting this code path anyways. So I'm going to go ahead and merge this. And actually, I'm going to merge a more extreme version of this, because I don't think we need it for lib either. We just need that in the autoload path.

07:17And so they have us noting the extras.

08:41There we go. Nice little comment saying, thanks. And if they want to do that update, I'll get that merged pretty promptly. So the other open pull request that was real interesting and I have some activity on was Rails 8. We have a, I opened an issue for it, and this is just the core of it. So it's always kind of funny to me that I put a lot of issues in the issue tracker that are small bugs or well-contained problems or feature requests. And then people pick whatever is most interesting to them. And I'm trying to make an easy on-ramp for getting started of, oh, okay, well, if I just copy this pattern from here to over there, that's pretty easy. albynton Hello
And, you know, hopefully it'll only take a contributor an hour or two and they'll decide to keep contributing. then i put these things in that are like here's a big chore hey albinton welcome back and sometimes people just jump on these big chores so this one actually sounds like not a big deal especially because Because Rails has been... Yeah, I kind of wonder if Ada is a marketing release. It has some new additions, but no... I guess I'm just being a little snarky. And it's fine. So let's... Don't need to see the details of that. Shadows. Typo.

10:39Existing, so we need to explicitly disable that. Good. Well, that really is a small upgrade. Yeah. It doesn't feel much like a major release that we get out of it with basically a two-line code change, but I'll take it. I wish I could just... Suggestions cannot be applied. All right, so hold on.

11:19Yeah, okay, I can just commit that. Great. And I guess I don't need that pending review anymore. So I can just go ahead and approve it.

...46So I'm going to let that build finish, but it will probably be fine because the core libraries of Rails, the behavior of action view and controller and active record didn't change existing functionality too much. So I can see how we get out of this without a big PR. And...

12:25What was that number? 1,366. 1,366.

13:02All right, so there is a nice little cross list. I'll give that another minute or so to run, but it'll probably run. So the big thing in Rails 8 has been that it got a bunch of these new libraries, like SolidCache and SolidQueue, and then Kamal2 and Propshaft. Even if we do the core update of the framework, there's all of these things to add on. Oh, Chaotic Job reminds me. Maybe the best talk I saw at RubyConf was from my review. Oh, it's an outdated bit of Flash. Little GitHub bug. Stephen Margheim here gave a talk at Ruby Conference that was about these two libraries of his chaotic job for testing background jobs and acidic job for making very durable and reliable jobs. And I think it was my favorite talk of the conference. He very clearly has a lot of experience with the kinds of complexity you get running jobs at scale and how much hassle it can be to work towards reliability and deal with jobs that fail partway through. And he came up with really just nicely factored solutions for cutting through the complexity. I think he really does get at the heart of What's happening in those so that to that he talked about wonder if that's been updated. kind of curious about that one. pushcx https://github.com/fractaledmin… https://github.com/fractaledmin…
we're chaotic job and acidic job, so if I grab these links just to share for anybody who's curious.

15:20The talks are not going to be available for a while. I would expect months. RubyConf didn't make a promise of when they'd be available, so I'm just kind of guessing. On a default, conference recordings usually take two or three months to get everything transcoded and edited and cleaned up and checked. The videos were done by Confreaks, who do a whole ton of the Ruby conferences, and I think they really started out of RubyConf. And I was chatting with someone who got a real kick out of the stream archive pages for this stream that I'm sure you all have seen. There's a transcript made out of the video kind of automatically, and then that turns into a written transcript with paragraph breaks with the chat alongside and the timestamps. you can kind of click on stuff or scroll to see activity i think some of that javascript is broken right now i have to admit but just that general functionality and confreaks used to have a website called confreaks.tv they may have accidentally let the domain lapse or archiving videos and i shared the code with folks there so maybe Maybe in the new year, we're going to talk some more about making better archive pages for Ruby conferences or just stuff done by Confreaks. I've gotten an enormous amount over the years out of watching recordings, enough that I know the name Confreaks of who's recorded these videos and who's taking care of them. So I really do hope that I get to contribute back a little there. That would be very nice. So these two gems, acidic jobs is about durable execution workflows. And it adds a bit of, it almost comes down to syntactic sugar. The main functionality of it is this block method execute workflow. So you break your job up into several steps that are like one atomic read or write or one side effect, ideally. And when I say side effect, I guess I mean like, you know, the unit of work pattern, one logical connection of side effects, plural. And then it records, kind of annotates your job. Each of these is just a method. So you would have def step one, step two, obviously with better names. And then if any of the jobs crashes, workflow has automatically checkpointed which steps it's completed and which ones have yet to go so that it can resume or when it re-enqueues and tries the job again, it picks up from exactly where it left off instead of redoing too much work. Here's a better example of the code where have named their steps and you see like you know create two records all right that's probably two side effects especially knowing the active record interface but it's one logical chunk of work that are either going to be in a transaction or otherwise succeed and fail together and then create stripe charge hitting a third-party api is always going to be the riskiest thing the You know, not to toot my former employer's horn, but Stripe is pretty reliable. But if you're considering comparing talking to your own database that's next to your web server versus a third party API, it's always going to be more reliable to talk to your own database. So I really like that. It lays a little bit of syntax on top of a best practice for tests. And then chaotic test here, which I would like to add to lobsters as we start using active job. I mean, it does make sense to use the acidic job gem, although for the moment our background jobs are very, well, they're very non-existent. They are cron jobs. chamlis_ good afternoon/evening
but they're reliable enough that i don't feel like we need a huge amount of extra support we'll see it would be great to get more of that stuff ported over to a little job queue oh hey shameless i was thinking about you because i mentioned oh i ran into so-and-so and i saw this article and then nobody popped up to say like oh here's that link chamlis_ hehe
that's usually a thing you like to take on which i very much appreciate so this chaotic job that i would like to add has a bunch of test methods to simulate that crashing jobs can get retried properly and it has a nice small api and i'm not it's been a minute since i've written background jobs So I can't say it super confidently, but looking at it, I was like, oh, I think this roughly fits all of the hassles I've had testing background jobs. So though I can't be certain because I need some more practice and I need to actually just use the darn thing, it feels like a really, really clean and powerful API to test the actual things that go wrong in jobs. so i can't say enough good things about this talk here from rubyconf and i will have to link it on stream and obviously submit it to lobsters when the recording is up live because i liked it quite a lot so this other repo from this same author stephen markheim is solid errors and i'm curious about this one i've talked a few times about having that idea of well what do you do if you have a A local database running sequel light that separate from your production database you don't really have to worry too much about. Right convention or giant joins and it's just local and it's intended sort of as a cash and I was thinking about what you could do with that and it occurred to me that application performance monitoring something like scout or. or the other one I'm blanking on totally right now. I only remember Scout because they were a sponsor at RubyCon, and their logo was everywhere. Scout donated use of their APM to lobsters, gosh, maybe it's two or three years ago, and we used it for a couple of months. We were going to get a performance analysis by Nate Berkapek, but then were some scheduling hassles and nothing much came of it but using scout apm per minute was neat it does know that it's a business product for businesses and charge accordingly so it's always been more than i want to pay for lobsters but then i saw this repo solid errors and i realized oh fractal mind stephen markheim here has had the exact same idea i did of bsandro ehlo cyberpals
what if we logged exceptions to a local sqlite database because then you can have some nice lightweight analytics on top of them we'll see how it goes this is i don't know that i would even call it alpha software because it sounds like from the bug here that it doesn't quite work and just looking at the the repo in general it's not seeing significant work to build out its functionality but i think this is really promising and i want to keep checking in on this one i would love to add it in you know hey bisandra welcome back maybe bisandra are you a smtp server saying hello instead of hello So maybe in a bit, this will be ready to merge and we can use it. I'm hopeful. This is really interesting.

24:18bsandro yes I'm an old smtpd :D how are you today good sir?
So by now, good, I can just go ahead and merge this. Yeah, you're an old SMTPD. Well, I'm very, very scratchy from talking for days at RubyConf.

...36I hate the default merge messages by GitHub. I just never liked them. I want this in the headline always, the description of what's the PR and what's the branch number. It's nice if it fits. It's more extended description than top level. Good. Nice to get a clean merge there. So then that just leaves this stuff for Rails 8, which is mostly let's configure stuff for these new features. And I've chatted with folks. One of my big projects for RubyConf was talking to people who have already started using ActiveJobs, SolidQ, SolidCache, et cetera, in production, and especially Kamal 2, to understand the maturity level, because Rails 7 was a little vapor-weary, where somebody needed something to talk about in a keynote, and some features got done very briskly to make that keynote deadline. And the general consensus on these libraries is that they are pretty well factored and pretty well reliable at this point. ActiveJob is the only one I still heard a little bit of hesitation around. And it sounded like folks were looking forward to Rails 8 as having the fixes that they were expecting. So I think that'll probably be pretty good. I don't think I want to go too deep onto... It's tempting. It's very, very tempting to do this stuff because it's fun plumbing, which I suppose perhaps that's why people keep contributing stuff like Rails upgrades rather than little feature requests. But I'm not going to jump into this on stream. And hey, if this is what folks are going to contribute, I'll take it. So I don't want to do that. Over on the issues, things have been pretty quiet. Last week, I talked about how the JSON logging was a little bit wrong. And I added a feature for highlighting search terms. Since then, I added one more feature. Let's talk a little about this this morning. And then there was this bug report. Oh, it's from DPK. I didn't recognize the author. DPK, if you're here, hello. She's a stream regular, so nice to see. The logic of relevance, when you do a search on lobsters, You have the option of sorting by newest or relevance or score. And newest is pretty obvious. It's just reverse chronological. Score is pretty obvious. It's just number of upvotes descending. Relevance tries to be clever.

28:25where is it so take the firm sql so it basically counts up i mean it's naive but it used to be more complicated i forgot that it got simpler it counts up how many matches there are of your terms in the title description and body which means if there is a long post that says foo your search term if a long post says it 10 times but another post has it in the title you will see the long post that just kind of mentions it a bunch so this is a very very late 90s search engine because the web search engines in the late 90s were this kind of immature where they just did basic term counting and we're just shoving it at MariaDB so it's not surprising It used to be, I redid the search engine a couple of years ago, maybe two or three. Relevance used to do a weighting of if it's in the title or tags, it counts as like 10 points. And then if it's in the body, it counts as one or two, something like that. But I couldn't find a good way to do this. And there were so many moving parts that I simplified the relevance So hopefully someone would like to do an interesting plumbing job and improve the relevance. I should probably leave that as a note. I left a small comment. Oh no, I guess I just tagged the issue.

31:59espartapalma Hi folks, nice to meet you again
Hey asperg palma welcome back so i was just reviewing this bug report that came in since the last stream from regular stream viewer daphne and yeah they suggested his features work better when i don't delete them as i'm in the middle of upgrading stuff so either i will have to dig that out or someone can you know i should give the The reference there's enough search code.

32:49And where's.

33:36And on this one, there's going to be another hit for comments, which is this. Yes.

34:01I spoke with someone at RubyConf about encouraging contributions, and they said a coworker of theirs likes to leave really detailed messages. At some point, leaving, and I looked up one or two of these to see an example. At some point, I think the level of detail just becomes I'm implementing it myself in this really roundabout way of explaining every last step of it and then having someone else type it in. So I don't know that I want to get down to that level of detail, but I'm trying to do more of this. Hey, here's the code. Hey, you can just click on it. I hope that'll lead people in. So here is the other feature request I filed over the weekend, or no, this morning.

...55This is actually pretty uncommon abuse. We got a It's just an outright spam bot. It was doing, what do they call it? Backlink building? Link building? So it left these comments that said, that were very clearly chat GPT with a sentence inserted. And so whatever the topic was, I don't remember what topics it picked. Let's just say Apple Vision Pro. If there was a story about Apple Vision Pro, there would be like four sentences of the most generic LLM slop you can imagine about how great Vision Pro is. And then in the middle, there would also be a sentence that would be like, for the best Miami vacations, visit pissedconsumer.com. We don't get a lot of these because of the invite system. So I'm real curious what happened. I think for seeing this kind of outright spam link building, it's been at least five or six years since I've seen one of these. So I looked at, of course, the person who invited the spam bot, and it looks like it was just an accident on their part because they're a very Very normal member they engage in discussions they submit links they vote on stuff they don't seem to have any connection to the spam. So they certainly weren't punished in any way, but I did drop them a line saying. hey there was a bad invite here, can you tell me anything about how. This came about and i'm very curious about that one because. It's uncommon to see that kind of deliberately malicious behavior where somebody wires up their Spambot. Because this definitely wasn't Baby's first Spambot. And we're usually pretty small and below the radar for somebody to make a Spambot. I mean, they certainly didn't make it for us. They just customized it a little. All right. So there is a mod feature for being able to immediately create domains and ban them. jmiven hello folks
And I need that for origins, too, because this site, I think I saw one post had a subdomain, like blah-de-blah, whatever the product it was shilling, blah-de-blah.pissedconsumer.com. And if it did it once, it's going to do it again. And so that's not a good fit for our ban code. So this would be nice.

37:55All right. So here's, let's see, I talked about RubyConf and PR. Talked about the issues, stale issue config. Oh, hey, Jim Evan, welcome back. So along the lines of contributing I have mentioned a couple of times that I would like to try and make sure that pull requests don't go stale or that people who show up on issues to say, hey, can I take this, either do take this or I get some kind of reminder at some point to come back and remove their comments so that potential future contributors don't get confused. frici Hey folks
oh before i jump into that because i've been running for 40 minutes or so now and there's a whole bunch of folks watching this is lobsters office hours if you have any questions about the site or the code base i'm happy to talk through it or run queries against the public domain or public database hey freaky i was just I blew somebody's mind with a stream chat log where I was talking to you. I talked about doing a little streaming and they were like, oh, that's neat, but your work kind of gets thrown away. And I showed them my archive pages with the chat and I just scrolled down to interacting with someone and it happened to be you. And they were just kind of delighted reading back and forth between the chat text in the right column and me talking in the left, because that's kind of what the stream is for. frici Nice LUL
so so there's this stale issue config and where did i put it i put it in workflow oh and i pasted it so When I was at RubyConf, they have a Hack Day in the middle. Really slack byte order marks. Slacked a little bit of a number to this. So I went over to the ThoughtBot table, because I've been really impressed by their code, their libraries. And they're really smoothly jbrancha hey -- first time popping in to these
managed open source projects that get a good number of contributions and some of that is just they're wildly popular but a lot of that is smart things that thoughtbot does hey jbrancho welcome and i'm especially impressed by bot bot running standard rb because it's kind of a it has the potential to be an unending bike shed flame war but it's actually a super pleasant repo where, if you look at the issues and the discussions it's just people figuring out. How to paint the bike shed in a productive and collaborative way has a good vibe. So. I chatted with them and they grabbed over Sarah Jackson who spoke at RubyConf. I didn't see her talk, but she had some experience setting stuff up for their repos and she shared with me their GitHub action for stale issues. So I had filed a while ago an issue about having a pull request bot to help impose process and encourage people to complete issues that they put their hand up for. And so I kind of worked out what all the functionality would be because I didn't want to have... I didn't want to use Zulip Bot because I don't want to keep running Node in production. It is on its way out, as noted in the Rails 8 issue that we saw just a few minutes ago. And after, like a day or two after I wrote all this up, I ran into GitHub Actions because Mozilla stale closed a bug I reported. And I feel compelled to mention that it was stale because they never acknowledged the bug, not because I abandoned it. But I looked at the docs for GitHub Actions, and it looks like it actually does this close stale issues I don't know what to call these sub things, feature, bot, actions. It feels weird to call something that, to call it an action when it takes several actions, but I guess I don't know branding like Microsoft. So Sarah Jackson gave me this demo, which is a little bit more complicated, not a lot, but well, a little different organized also than the default on the page. It also just kind of helped to know that I was getting one that someone actually used and copied out of their production config rather than something on a README, which I don't have any knock on GitHub here, but just in general, it's easy for docs to slip out of date or out of sync. And so seeing one that was a very close parallel is promising.

43:50They're really generous about the activity time. In my issue here, I had written, well, let's give it 11 days and post a reminder and then close after 14. They have their bot set to never close issues, but to close PRs after quite a lot of days. I wonder why they interleaved these instead of going the two issues and then the two PRs. Makes a little more sense to me the other way. Because first it is considered stale and it gets a warning. Warning is maybe a little harsh. Comment, reminder, nudge. And then... It gets a close. So I think rather than go for 40 days, what does 21 sound like? It's closer to theirs. So we'll say we'll give it 14 days. And then after 21, we'll go ahead and close it. I think we need a label. I don't know what benefit you get out of labels. It feels a little bit like branding the pull request or issue is bad. And I care a lot about how this comes off. I think one of the big benefits of having a bot here is it doesn't come off as a maintainer saying, hey, you're not good enough. You didn't move fast enough. The other thing is, and I think the reason they do this minus one on issues is, yeah, it's not, it doesn't have the claim feature that I would really like. So the thing that's been frustrating me in our issues is when folks swing by and they leave a comment that says, hi, I'd like to take this on. And then they never open a pull request. But when someone comes by and sees those comments, it looks like someone is working on it. Unfortunately, the hassle here is I only really consider an issue stale. It's not the issue I consider stale, it's the assignment of the issue. And it doesn't look like this bot can quite do that. Yeah, so I have to... The whole way that GitHub actions work keeps puzzling me because this is official GitHub code to work around limitations of the GitHub product. And yet, I still have to configure an action separately and give it permissions. It's odd to me to have to give GitHub permissions to itself.

47:37yeah and everything about assigning assumes that you want to manually manage assignment and i really don't i don't know this could work process only assigned issues

48:04only the issues of the pull request with an assignee will be marked as stale automatically that if i can tell it that a stale issue should be unassigned that's actually pretty promising so let's go ahead and do that and i'm going to copy that 14 and 7. oh and let's

...35I guess that's the purpose of this label. Yeah.

49:33I'm trying to be encouraging and say, hey, it's great.

...44Please do keep working on stuff. It feels like it logically makes sense to put all the issue config and then above the PR config. It's not just the alphabetical sorting, but we usually have issues before PR, so I'm going to bring that up. up and then looks like i'll need a stale issue message this is one of those like it feels like ansible where i'm very clearly writing code in yaml but it's strange and unstructured because it's in yaml and it's not in a real programming language And so, instead of these being known config options or structured it's just kind of like yeah there's this hash with a bag of magic keys and they combine in odd ways.

50:46Alright, so for this stale issue message.

51:25yeah please leave a comment if you're still working on it or in another week it'll be unassigned so anyone can pick it up that feels oh i don't want close what i really want is this i'm going to call it include only assigned And then you just have to know that this applies to issues. Oh no, it also applies to pull requests. So I can't do them separately. That's so... This is why YAML is not a good programming language. This feels a little bit like... This bot would really like to take structured input. but then it can't provide a nice API because it just has to take one flat bag of values as a hash.

52:38Can't read that. All right, hold on. Let's make this bigger. Swap it up.

...49Oh, and now there's YAML. to list the YAML options that make up this config. This is YAML all the way down.

53:07I think this is the same material as this. I would hope one of them is... I would hope the text is auto-generated off of the other, but... Can I unassign? See, that's the thing I want. albynton That's a nice title
And I can't do it, is I want to unassign an issue when it goes stale. What's a nice title? YAML all the way down? Oh, it is. Yeah. Yeah. All right, so let's bring that up.

...53albynton yeah, "It's YAML all the way down"
So if I can't, with its, all right. So there's this. There's this part. Just getting my windows properly arranged in the window manager. So here, I can run only on assigned things, but I can't actually remove an assignee, which is the thing I want to do. I can just say, if it has an assignee, ignore it, which is kind of the opposite of what I want to do.

54:58Accept all assignees.

55:16There is a special kind of frustration to software that does half of what you want it to do and can't be edited to do the other half. Am I allowed to have comments here? I hope so.

...43Let me grab this URL. Let's put this down here because we're only going to do that part.

56:53So it occurred to me that looking at that, that they explicitly listed days before issue stale as negative one. The number of days old an issue can be before marking as stale set to negative one. What's the default though? What's the default? Issue stale. There's no default listed. 60. So override first. Okay. So yes, I do have to explicitly turn off issue stale or it would come along and delete everything.

58:06Boy, I could tell I'm still a little tired from the conference and social stuff because I keep reaching for snarkier mentions than I want to be describing this actions features as limitations. It wasn't designed to do the thing we want, and it just doesn't support the thing we want. But I'm pretty frustrated by that because I'm just generally frustrated at GitHub about this whole situation. So I'm trying to make sure to be kind about it, that we have to configure it.

59:07I guess it's a very... I don't know about very. albynton The extension has no feelings, you don't have to be kind
It's an interesting choice that the default for days before stale is 60, because that implies that the extension doesn't have feelings, but the authors do. Both of the feature requests, or I'm sorry, both of the documentation and the code itself. And if they're... Giving me stuff for free, even if it does only half of what I want, that's a half I didn't have before. And the right attitude to have towards someone giving you half of what you want is to say, thanks. Not to say, but give me more. So I don't want to come off as entitled. really I don't want to act entitled it's I'm frustrated just in that general nerd way of oh it feels like it almost works so the odd thing here about 60 is it's really for a project that sees a lot more continuous effort and a lot more false bug reports than lobsters. We have issues that hang out for years, especially when they're feature requests. We don't want to close them. And so I guess this is sort of assuming that you don't use GitHub issues as a backlog for future work, where we very much do. And then also that you're seeing a steady stream of bug reports where the reporters just kind of flake.

01:01:15And you're also seeing very busy maintainers who are making sure that no bug report or feature request that's in the issue tracker hangs out for two months without being worked or discussed in some way. So all in all, it's a default for a much more active project with a different workflow than us.

...44All right. What did I say, 21? Well, actually, yeah, let's be more generous. Let's say 21 and 7. So that's a total of 28 with no activity. Good. And then I think this magic config file works by getting put in a magic location. This github.token also doesn't match what this sample does. So let's take a look at that.

01:02:41Do we have another mention of our token here? Oh.

...55think maybe this secrets thing is how they're doing injection and we want the default yeah with no mention i think we need the default so i'm going to just delete that and then let's figure out the magic place to leave this file All right, so you make a workflow that uses the action. Do I just put it in that workflow? All right.

01:04:12just kind of reading these descriptions to make sure that we're not doing anything too odd and i think this is all appropriate just gonna say please because i try and be explicit when i have an action in this kind of copy like ideally please just do the thing, which is keep moving your PR forward.

...55All right, I think that's pretty good. I'm going to give this table of contents a quick glance, but otherwise I think this does it. This run at 6 p.m., that feels fine. I would assume it runs in... UTC. So that's probably going to be what around noon Chicago time.

01:05:25All right. Sweet contents, right? Oh yeah. We're not doing delete branch. And then there's the whole list of options and then nothing extra at the bottom. Who needs debugging? Surely it'll work the first try.

...54I kind of wish that if a pull request had an assignee that I could just say, give them an extra two weeks, or let me add a label to add them two weeks, but then that's really getting into stuff that's hard to do when you're just writing YAML. jmiven I think it adds a "Stale" label by default, so you might want to change that also
all right that might be one of the most annoying programs i've written in a while because this really is a kind of programming where i'm building and processing a state machine you think it adds a stale label oh let's check so it adds a label yeah and i i did explicitly say i don't want to label because i don't feel like i want to do anything that sounds like it's going to shame people and marking their pull request as you know oh this was so bad that the bot closed it it's not but sometimes people take things this way so There's a saying in American football that it can be a game of inches. So in American football, two teams are competing to carry the ball to the opposite end of the field. And it has a really unusual sequence of you play and then you pause and then you play and then you pause. And every time there's a break in the game play, the referees come and set the ball down preparation for the next play and so football is a game of inches because literally every little bit more progress you get is a benefit and so when the people are getting knocked down they will like carry the ball to the ground and then just before they hit the ground reach out to hold it ahead of them which you really do have to do a ferocious amount of training to override a reflex that wants to get your hands under you like that it's just a wild bit of sport but i feel like attributing what am i saying i feel like encouraging open source contributors is also a game of inches there is less sacking people and getting folded into a pretzel by a 300 pound man ideally but There is a lot of this, I look at and I polish and I polish and I polish and I try and keep encouraging things. So how do I turn off the labeling?

01:09:09Because I can change what the label is.

...19Add, specify, no. Labels to remove when stale. Man, so many fiddly things, just because they have to explicitly list everything you could want to do. jmiven if I understand correctly the label is how it keeps its state
And even so, we immediately ran into a couple of things we can't do.

...42The label is how it keeps state. Oh no! That's... I see how you get there, Jay Miven. And that would be really frustrating. Because it is a thing I don't want to do. I mean, I would think you could just add like add days before pr stale plus days before pr close and then if it's well over stale you know but i think you're right about that and i can see how they would get there for if you configure this action to run more than once per day you could end up in the situation where an issue gets marked as stale and then immediately close. No, maybe not. Maybe just if you were... I guess it does enforce a grace period because if you install it on an existing repo that has open issues, you would prefer it not immediately close everything that's 67 days old.

01:11:11Yeah, since adding the state label will alter the last update date, we can calculate the number of days from this date. I think you are definitely right that this is how they're doing it. Damn it.

...33It's funny, you can skip the comment by passing an empty string. The comment is the thing I want. The comment is the valuable part because it's for the human. The label is the bookkeeping part and I don't want to bother the human with a kind of irrelevant label.

...57If you wish to speed up the stale workflow, you can add this label manually. So I guess I just have to live with the label. Lose that inch.

01:12:14That's frustrating.

...35arh68 u can't customize it beyond `Stale` ? i don't get it
Well, I can change what the label is, but I can't get rid of the label because it uses that to differentiate between being stale and then getting into how many more days to get to close.

01:13:07dpk0 shame (in a sense) you can’t make a label that’s hidden from everyone except repo admins
I almost want to shame you can't make a label that's hidden from everyone except repo admins. Oh, hey, DPK. We were talking through your issue earlier of relevance in the search engine. And I pulled up the code and poked around a bit. dpk0 but only in a sense, because people would definitely abuse that to have labels like ‘pebkac’, ‘user clue-impaired’
And the short version is when I redid the search engine two or three years ago, I completely broke that feature. And there isn't any code that ranks and weights by relevance. It really is just do you match or not, and then whatever order the database gives back in.

...50Oh, yeah. I could see people being that kind of mean with issues. I can't. imagine the foolishness that goes into wanting to mock the people who are trying to contribute because even people who are mistaken about bugs are very often it is a pet peeve of mine that when someone reports a bug and it's you decide it's not actually a bug a lot of times they are right for the wrong reasons like it is not technically a code bug But it is a misdesign, or it is a failure to set expectations, or it is a failure to explain things in the term they understand. So as an example, I had one of these, I don't know, a week or two ago, I was using a service called Publer to try and post to both Blue Sky and Mastodon. And I wrote a post and it had a link. And then underneath the posting box, there was a text that said, yeah, links are going to be shortened with URL shorteners. And I didn't want that. And I contacted support and I was like, hey, even if I unchecked the box that I don't want URLs shortened, it still says they're going to be shortened. And they basically said, oh, that's not a bug. Even though the checkbox turned itself back on every time I edited a draft. What actually is happening is that's just a general warning. It's not about your specific tweet, even though it only appears on your specific post. And you have to go to another side of the dashboard entirely to configure URL shorteners in the first place, or it never happens at all. arh68 "not a bug" and bug bounties go hand in hand lol
And so I said, okay, so it's kind of like two bugs. Number one, I see it around three. Like I see the message on posts with no links. I see the message when I don't have any URL shorteners selected. Oh yeah. And to say nothing of, security bounties. So it was a really frustrating interaction with the customer support there because Even if they don't see it as, oh, that text is just always there, so there isn't even any code to have a bug, the fact that the text looks like it is applying to the post and it's just general stuff was the bug, and that they couldn't explain how the future worked, that was the bug. There are a whole bunch of polished things wrong there. I kept finding bugs in that app actually it was one of those apps where like every time I log in and do something I find another bug and I reported the first couple and I kept getting brushed off and then I stopped and just gave up entirely on them it's yeah all right so I'm trying to I was thinking about anthropomorphizing the bot like This bot doesn't see that you have any activity. But I don't think that's needed. The bot is closing this PR. Because I see this passive voice of the PR was closed. Well, who did it? Obviously, the bot did it.

01:17:41I think that's about as good as that's going to get. And given that it shouldn't be closing issues, I'm not too worried about it running amok. Yeah. You know, ARH, you've got a point there about bug bounties. I think the incentives all change when you attach thousands or more dollars to bug reports, especially when $1,000 for a bug report goes a long way outside of the United States. So I can see why the bug bounties end up overclosing issues, because they are probably just drowning in invalid issues that are hoping that they'll get a bounty.

01:19:10What's the number here? 1368.

...19Let's rebase this onto the Rails 8 stuff that I merged at the beginning of the stream and push that up. And if that's pushed up, I should see it on the issue after a minute. All right.

01:20:17doesn't, what's the right way to say this? Just doesn't allow users to assign and unassign themselves to issues or allow unassigning issues when they go stale. So very topically, I'm going to keep this issue open.

01:21:15Yeah. So I will take half a solution over no solution. Although sometimes it feels like I'm taking one shoe over two shoes, which is almost worse.

...43So there's that.

...50And that brings back around to maybe a good time for the bumper that this is Lobster's office hours. And if you have questions about the site or the code base, feel free to pipe up. Happy to run queries against the production database like somebody was asking for this morning. And then otherwise, I work on Lobster's code as I was just doing with the stale issues action. You know, looking at this, is there a search engine on this actions marketplace? Can I search for sign? A bot for managing your issue assignments. Switch labels and assignees. Are these, okay, so these are just by random people, right? So I don't totally... I've used GitHub Actions a little bit. But in my head, it's GitHub CI and GitHub stale issue thing. frici Indeed not github issued, just people publishing their things on the "marketplace"
But this is very clearly some random person. And I don't understand how all the security issues get handled. jbrancha anyone can write an action e.g. https://github.com/BoundfoxStud…
Just people publishing things on their marketplace. Right, so where does it run, Fritchie, if you know? Anyone can write an action, e.g.?

01:23:36Oh, I got a typo in there. No? Come here.

...47Welcome to the Peter Struggles with the Linux Clipboard section of today's stream. frici wrt security practices It's GitHub so highly likely your answer is "they don't"
It appears every stream. Come here. Yoink.

01:24:08Oh, that is the exact... So, Jay Brancher, you found the exact... feature that we're missing so with regard to security it's github so it's highly likely your answer is they don't i don't know github seems to be pretty good about designing to minimize foot guns and grab random code from anywhere on the internet is a little foot gun shaped especially when it's given access to your repos. So who actually runs this code? Does it have like a Docker file and GitHub runs it? marvooy tha fuck is that UI size?
Or do they have a server somewhere, like Manuel Raber has a server somewhere?

01:25:15The fuck is that UI size? I don't know, man. jbrancha I think these run in github managed containers
It is a font. We're a pretty chill stream, so you might want to ask your question a little more politely if you want an answer. They run in GitHub managed containers. marvooy ok leaving then :P
Oh, so that's the how many action minutes thing you get? All right. Mystery solved about the pricing grid on GitHub. Cool. Thanks, Marvoy. I'm glad you figured out this wasn't what you were looking for. And then this is just TypeScript and JavaScript, the thing I don't want to put in production. So I suppose I could review this code and pin a specific version instead of latest. Is that the best practice? So we grab these, we can unassign issues. We have an issues model. No, it's some kind of database too. Our API.

01:26:39Alrighty. And then we got inputs. Oh, they're just making a module. Fine.

...51jbrancha I've not, just searched
So Jay Branca, have you used this one or you just searched and you found it faster than I did? chamlis_ looks like you can pin to a commit hash to be ultra-sure
Because I had the marketplace open here. So presumably with a search for the word assign, it's going to show up here somewhere. You've not, you just searched. jbrancha I didn't even go to the marketplace, just googled
I can pin to a commit hash to be ultra sure. I guess I could roughly trust it because that's the permission section that we just saw on the... frici Should be fine yeah
stale workflow so if we said you can touch issues and we can touch pull requests and then it's pinned to a specific migration that seems fine right like it's not terribly different from adding any gem into the library okay So I guess now the thing is, oh boy, there's nine pages. That's probably like a default max. Okay, so there's 170 things that mention assigning. And the features, hold on. So I wrote it all up. Let's go back to the feature request. agustinraven Hi, whats up, what are you doing?
I don't love it, but I guess I'm willing to... combine multiple actions to get the behavior we want. So which ones do we have? Hey, Augustin Raven. pushcx https://github.com/lobsters/lob…
We are trying to, on lobsters, have a bot that make sure that people who say they are gonna work on stuff, both that they can get issues assigned to them so they feel like it's theirs, and then if it's inactive long enough, that things get marked stale. So let's say we've done the PR. This one is a nice feature that we don't really need. That's just a polished thing. Assigned issues. Oh, not should be closed, should be unassigned. Issues that are inactive. OK. So these two sections are done, and I have to edit that text.

01:29:39All right, so it would be really nice if there's just one of these GitHub actions that wants to handle this assigning. And I can live without if the user has one thing. But I would really like to figure out this link between issues and pull requests so that if someone is assigned to a issue, it recognizes that activity on the PR is activity on the issue. I hope that makes sense. And I guess I guess part of this thing with labels is a way to communicate state between these actions and kind of hammer that into being a what do you call it an api between the bots this really is just yaml all the way down right so if the one like if the pull request bot leaves a ish what is it, leaves a label on the issue, we could say, oh, this issue has an open PR and then configure that bot to ignore it. agustinraven So you are resolving a issue created by github actions? sorry if i didn't understand
So then it's almost like they're working together. It's just all tied together with duct tape, which is how we work as professional software engineers. So where do we have, God, I don't want to go through 170. Let's limit down to verified creators. Let's see who has jumped through hoops to get a blue checkmark. Friendly fire pull requests. I don't actually want to have friendly fire against contributors. Friendly fire is a very bad thing. What a weird fucking name. Why would you call it that? What is this image? No, don't save. I don't think somebody understands what a friendly fire is. Yeah, I think maybe there's a little ESL happening there.

01:32:10All right. So nobody verified sounds like they do the thing we want.

...23So this might end up having to be two actions. So I'm resolving an issue created by GitHub actions. No, it's on this repo. pushcx https://github.com/lobsters/lob…
If you read the issue here, 1368, and I'll paste you the link because I jump around between tabs a bit. Sometimes people say they're going to work on an issue and then never do end up working on an issue or they work on it a little bit and they leave. And so we just wrapped up configuring an action so that if a pull request has been inactive for 21 days, it gets kind of a nudge comment. And then after seven more, it gets closed. And then the other thing to do would be, all right, well, if a issue doesn't have a PR open and it has someone assigned and they haven't done or said anything in again like two or three weeks we'll go ahead and leave a nudge and then we'll close it well not close it obviously because we're still going to have the bug unassign them all right so so j branch did you search for unassign is that how you found that one auto assign And issue a signer, which is the one I just had open a minute ago.

01:33:58All right. And this switch labels and assignees like that's the thing I want is I want people to be able to say, like, put their hand up and say, hey, I'd like to do this one. And it would be really. agustinraven and you do all that with extensions?
didn't even list it in the issue but the thing i would like is if someone asks for an issue to be assigned to themselves and then they don't do it and the bot unassigns them i would like it to hide their comments assigning themselves but yeah apparently we have to do that with extensions augustine the there is not an option on github to do that

01:34:46or actions as they call these extensions.

...54Users can request. Great. Max number of assignees allowed per issue. One. Assigned issues, number of issues a user can be assigned to at a time. That would be nice, especially if you could be smart about knowing whether they have pull requests open or not. You can unassign. So this does, and it's free, OK. Why do I have to install it? I don't just reference it? I'll come back to that. agustinraven and somebody can develop a extension to do that task not? it remember me like discord bots, i think they are called Apps too
So this OWASP guy, OK, has the assignment, and it has the automatic unassignment. But it only says if they're resolved, and I am happy to settle for any kind of activity. Yeah, it's quite a bit like Discord bots, and I would prefer not to write one from scratch. If no longer maintained. That's a shame. This one sounded kind of promising from the brief description. but it just does pull requests okay then who's this this is a different bot so this is more like what i thought we were going to have to do run a standalone service but this one is python instead of ruby so i'm just going to pass on it because i don't want to put another programming language in prod and this one is a little aggressive in that an issue has to be resolved rather than just have activity. And then does it? So this is a repo. So there's probably some magic to make these things a, to get this header on here. Like either you register the action in the marketplace or you add some JSON to configure it. And then GitHub recognizes that it's a action. All right. Best used with action stale. See a running example here. So I think a combination of these last two covers the feature set of allowing people to assign themselves and then unassigning them if there's no activity. The only thing missing is recognizing that a pull request counts as activity on a issue, but I do, I guess I could compromise there because I do try to be pretty aggressive about moving PRs forward and if I just added a couple of days to the issue, you know, so if I had said if the issue doesn't get any activity on it or its PR for 14 days or 21 days, well, if I just basically said you don't get activity on the issue itself for 30 days, then that covers the whole period that PR is. Or maybe that's got to be 37. I'm not adding things together, but you see kind of the outline of it. It could just go ahead and be real generous on the issue. Okay. It's a shame I can't. So this one is kind of a one stage thing. So it's only stage is that It immediately unassigns people as opposed to leaves a nudge.

01:39:28But I think this, yeah, using this is nicer than, and it's barely been touched, not in the last year. That's a little rough. But on the other hand, it's doing a very narrow, specific job to a well-defined API, so I wouldn't expect a ton of churn. So then what? I just have to read the whole repo and evaluate for secure code are there any mentions of eval in here all right let's give it a skim oh this was the one i already clicked into wasn't it yeah so we got the issues we got the assign

01:40:33figure out when they were done, grab the date. chamlis_ with javascript github actions it seems you have to vendor your dependencies or prebundle them, so I think a year of inactivity is also a year without even minor dependency updates, if that's a concern
And I guess this is where they would have that extra label in the same way as the official GitHub action to be able to do the two-stage process of Nudge and then actually deactivate with JavaScript, GitHub X. And then it seems if you have to vendor your dependency or pre bundle them. So I think a year in activity is also a year without even minor dependency updates. If that's a concern. Hmm. You know, it's not really a big concern to me because this thing is gonna run in an isolated container in its own environment, speaking to GitHub's pretty well-defined and stable API. I trust GitHub not to break it. And none of these are, well, the assign yourself box is actually looking at user input. But all the inputs to these functions are just things that I control or the clock. And so I don't really have a lot of fear that this thing's dependencies have some weird bug that someone could exploit because they don't really take any inputs besides who's the assignee, and then I guess what labels there are, but then only I can create labels on the Lobster's repo. So I'm not going to like SQL inject this bot. So I think we're okay. I would really...

01:42:39chamlis_ yeah, I can't imagine anything too terrible coming of it, just thought it worth mentioning
Is there a... At this point... Is there a Ruby gem for writing GitHub actions? The ones for Dependabot? Because at this point, like, the decision is wire a couple of things together and then have a little nervousness about them. Or do the custom input from, or the custom coding that's described in the issue. Some of these sound like they are, no. Testing GitHub actions.

01:43:54chamlis_ the github docs talk about using docker, javascript, or yaml. I don't know if you could replicate whatever their js library is doing, but it definitely doesn't seem officially supported
it's just some guy's thing where's your where's your repo sir so what i'm looking for is if there is a github docs talk about using docker javascript or yaml i don't know if you could replicate what their js does but it doesn't seem officially supported That's possible, but if it's Docker, that's not... I mean, I have deliberately avoided using Docker, but that's still not so bad.

01:44:40agustinraven why avoid docker?
And so if I stuck a Ruby script into a Docker container and it did everything we want and nothing we don't, that's not so bad. I've avoided Docker because I run a Linux desktop and I'm running the same operating system as I run in prod. So I haven't had a need like I would on OS X or windows to virtualize, to recreate the POSIX or the quasi POSIX environment that is Linux.

01:45:28What does this do? Bumping GitHub actions via... Oh, so Dependabot could trigger actions. All right, that's not at all what I want.

...48Somehow I've heard of OCRan before. Ah, yeah. Okay.

01:46:03So I'm not immediately getting the impression that anyone writes GitHub actions in Ruby.

...19arh68 that's funny cuz I always thought of GH as a ruby shop lol
And, you know, Augustine, to finish out answering your question, it's not that I really have anything against Docker. It's just there hasn't been a strong use for it. And there are so many things to learn that I have to prioritize. oh yeah i mean github is very much a ruby shop and i met a whole bunch of github folks at rubyconf just a couple of days ago so i hear someone from two years ago writing about github actions

01:47:20agustinraven i understand, there's no time
that's this is all like not bad i can see where labels really come into it because if this action runs in a ephemeral docker container that runs does whatever its work is and then gets thrown away It doesn't have anywhere to remember, oh, I left a comment on this issue last week, six days ago versus seven days ago. And so if it looks and it sees the label, then it can act in a, I mean, I almost want to say stateless way, but in a way where it doesn't manage its own state, I guess.

01:48:20Well, I have learned a lot about GitHub actions today. I think I'm gonna note that.

...34Oh, here's a demo that has been touched in the last two months that runs the linter. So we have inputs, that's that, documentation I saw for the GitHub action. And then we check out the code. Ooh, we just run gel scripts in the middle. Okay. Call that setup Ruby. chamlis_ huh, my message failed to send. yeah, I really bungled reading those docs, you can do ruby with the composite actions like this
And then by running the commands, so this is really promising because it clearly has a checkout of the repo.

01:49:32And not that I want it to have access to source code, but just getting an idea of what it can do. And the fact that you can run arbitrary shell commands inside the container instead of having to wrap them all up in you know, one process.

...53Hmm. All right. So.

01:50:09That looked like it committed.

...16Yes, actually, it does commit. OK. Log leak. More clipboard hassles. All right. All right. arh68 i am both impressed and scared of YAML full o shell scripts LUL
So we got that.

...51Yeah, YAML full of shell scripts. I mean, we already have an Ansible repo if you want Ansible full of shell scripts. I'm sorry, YAML full of shell scripts. I have that same gripe for Ansible that it started out kind of declarative as YAML, but very clearly just became its own programming language with its own weird syntax that's built it on top of YAML. And despite their claims of being declarative, it is just an imperative programming language with the weirdest interpreter and runtime environment. So at that point, why not replace it with a very small shell script? agustinraven You know, i know a little bit of programming but i recognize that even the way you navigate in tabs is like a pro haha, is a lot info to me, probably and frecuently i open a tab and i have necesity to read almost everything so i waste much time in that, a shame
I mean, the answer to that seems to be string interpolation of being able to insert values into templates, I don't know why, but that's the rocks that all of these deployment tools seem to crash into. The provisioning is fine, the deployment is fine, but orchestrating to get multiple containers or multiple VPSs working together where they have the shared state of each other's names, that has to get interpolated into various config files. That just becomes the biggest source of pain.

01:52:28Yeah, Augustin, I'm also just kind of skimming because I don't know a heck of a lot about Docker or GitHub actions or this marketplace. And so I'm just kind of kicking around trying to find things. And so the skill there is not programming, the skill is skimming. And I do think that's a real skill to have, because There are several more books in the world than I have time in my life to finish reading cover to cover. Alright. So, I don't... God, I don't want to write a shell script, for sure. And there is this... Who is CI Cube? Oh, okay. This is just a write-up of Setup Ruby. Alright. So if I grab that.

01:54:17frici The biggest problem with Ansible is people misunderstanding what to use it for or what scale to use it for. whih is how you end up with all sords of frankensteinisms that create situations that even stevven kind would be afraid to write of.
frici which* sorts* cold hands, many typos
Because problem with Ansible is people misunderstanding. agustinraven i need that skill memaggFine
Yeah, Ansible is maybe it's funny, we have the sort of opposite problem with Ansible, where we're really using a tool that's bigger than we're designed for. Because we have lobsters has basically two VPS is one for web one for database. There's the other one for the IRC bot, but it's not really super relevant to our hosting setup, so I don't mention it all the time. But if you have exactly two hosts, Ansible is overkill for your problem. Augustine, there's a good book called How to Read a Book. Where is it? agustinraven i fell just like a baby
meaning to look things up here. agustinraven feel*
pushcx https://bookshop.org/p/books/ho…
This, this isn't the same thing. Here we go. There you go. frici How to read things, the only useful skill I ever got from my master
albynton How do you read the book "How to read a book" ?
I realize a book on how to read a book sounds a little bit like the bootstrapping paradox, but arh68 i'd just skim it SeriousSloth
agustinraven @Albynton hahahaha
It's a justifiable classic. Albington, the answer is very carefully. The first page has an A on it, and once you've worked that out, the second page has a B, and it kind of builds up from there. Yeah, no, this is not the little schemer or the SICP of literature. It's just a nice way to look at There is more to get out of a book than just reading it straight through cover to cover. And some of that is things like skimming. Some of that is learning to read critically to understand what arguments a author is making and who their assumed audience is. agustinraven noted
It's kind of a, you know, level two literacy. Level one is you can read street signs. Level two is, do you really understand what you're reading? I don't know. Good book. Yeah, it's been ages since I read that. Maybe I should give that a reread as if I don't have enough books in my queue already. All right. Let's close out this. I think I do just want to kind of bolt these together.

01:57:16So one question for anybody who knows node projects, the dist directory, that's build output, right? They committed all their editor config. Weird. So I believe that dist is an output directory, and if someone was doing something malicious, they could edit it into the disk directory and it would be, that's the code that actually gets run, not what's in the source directory. Do I have that correct?

01:58:03chamlis_ that's my understanding
And so if I really wanted to be a hard ass about watching out for vulnerabilities, I would have to read everything in the disk directory. chamlis_ this is what I poorly communicated by "prebundling" earlier
which is clearly minified or linted in some way.

...27At the same time, this is, oh, I was gonna say this is a short amount of code, but no, this is 15,000 lines. Oh, it's lovely the way, I wonder if somebody didn't run a tree shaker on this Oh, Shameless, I see what you mean about pre-bundling. Yeah, your earlier comment makes a little more sense now. That's a lot of code to review or take on trust.

01:59:07And this one also is just kind of abandoned. arh68 i see why ppl write their own yikers
So I guess I'm kind of using the issue as my scratch file here. arh68 just 17,000 lines of JS for like hello world v2
Yeah, there's three.

...40Yeah, 17,000 lines for hello world. So that's why I mentioned tree shaking ARHs. I get the impression that they depended on some library, or that could even just be the TypeScript runtime, because I noticed this project was TypeScript rather than just JavaScript. And since TypeScript has to kind of inject its runtime enforcement, some enormous number of those lines could just be the TypeScript runtime. arh68 ya the transition beyond ES5 to babel and beyond has just disappointed me i guess
I may not be using the right jargon there for it, but...

02:00:37agustinraven how can you mantain 17k lines code?
goreng hello
Hmm. Actually, I don't know about that. I'm going to think about whether I like that more or less than actions too. Fold this together with actions. So Augustine, that 17,000 is probably a whole bunch of dependencies. But for what it's worth, where's lobsters? Because I just updated to eight. I want to say lobsters is like five. 14,000 lines of code if you include our templates. And boy, do I. Oh, okay. 17,400. So pretty much right on. But this is input code. Excuse me. This is code that arh68 does it not count comments ?
chamlis_ not counting tests?
A human has actually typed, rather than something compiled by a build tool and minified or uglified or whatever was going on in that disk directory. Does it not count comments? I don't think it does. Not counting tests. I still have to maintain the tests. I would count them. Oh. You're saying the... Yeah, it doesn't see that I have any test lines of code. I don't actually know how that Rails stats is configured. I'm a little surprised that it doesn't count specs for our spec out of the box. All right, so here we go. Let's go to the user and say, this is an example comment. Does the LOC change by two? So if we get 17, so it looks like ARH to your question, it is counting comments. So this is pretty naive. There is that tool, was it slot count? Do I have that installed? Guess not. And I don't remember if slot count has been replaced by something else. No, where's, Checking something off screen real quick. Okay. That's not bad.

02:03:36So slot count counts 15,000, but that is going to include the specs. Yeah, which it counts as 4,800.

...54agustinraven well, time to go, see ya dudes
arh68 later agustin
agustinraven good luck
frici cya around AgustinRaven
so yeah like these are numbers that are in line with the rails stats that we saw above plus a whole bunch of pro warnings but yeah that's roughly right see around the total estimated cost to develop is kind of laughably low though

02:04:37espartapalma it try to count only if you got in the "test" folder, like RSpec never existed
chamlis_ 👋
It tries to get us. So if I just move spec to test and run Rails stats, then what do we get? No. I regret to say, as Perk Palma, that that was not enough to fool it or to hint it. Oh, I bet it's... Could it be... Could it be plural? Because Rails is awful about singular and plural, so... No, that's not it. I don't know. espartapalma I'd think better how it was
If you figure it out, I'm happy to run it again, but... We'll put that back. Oh, what did I do? Let's restore the spec directory. Oh, I put the spec directory in the spec directory.

02:05:55Did I have some... Yeah, okay. There must have been a... Oh, that's right. I have a... espartapalma yeah they probably detect minitests
so rails calls out to i think unit test or mini test but i guess it detected that so i made i guess that's how i wired it up by default rails expects your tests in test because it prefers the unit test framework at least out of the box and then i wired it up so if you call rails test It runs RSpec instead, because I wanted it to do something useful. That's why I kept this dead test directory around. Yeah, so whatever they're doing to detect mini tests didn't like me just naming the file there. Wasn't good enough. So...

02:07:15Okay.

02:08:40goreng that is some good issue commenting
Oh, hey, Goring. Thanks. Yeah, I'm trying to write out the thinking process. It also helps because as I'm streaming, trying to keep up the banter chews up a lot of my attention, and I make, I suspect that I make a significant number more bugs, but also I work a lot slower and just a little more spacey, so trying to write stuff out is a good way of Getting my thoughts in order. What little thoughts I have.

02:10:30Similar ones.

...53So either we set up a standalone service that pulls GitHub's API, or we bolt it together with actions, or we hybridize one and two and write our own action, which I guess I might as well just keep in that paragraph.

02:11:56All right. So where are we at here?

02:12:17I guess where I'm at for the stream is either

...30Continuing down this little bit of a rabbit hole to try and set up a GitHub action. But maybe this is the kind of thing that, as I was saying at the beginning, feels like bolting together functionality or doing interesting plumbing, and somebody would want to set this up. So maybe I just want to leave this for a week and hope that somebody picks it up. because it's pretty well contained. It's clear what I want to do. And I think I definitely don't want to do the hybrid approach on stream. I don't want to do the hybrid approach period, just because the workflow for developing a GitHub action seems kind of, painful, especially one that's based on waiting a week. So I can write some amount of tests, but I can't really, I didn't immediately, I didn't poke around the docs, but every time I set up a new runtime environment, like now we're going to run this in GitHub's Docker container. There's that cavalcade of, oh, there's random typos. Oh, there's, you have to access Ruby like this. There's a path issue. And that kind of config stuff is so fiddly and painful. The nice thing about the first service strategy one here is that it just goes directly into code we already have that's in a running Ruby environment that's very comfortable for me, both for production and development.

02:14:40So I always try and make ROI calculations as an engineer of if I'm gonna spend a couple hours coding this, am I gonna get what I want out of it? goreng Just choose one that is fun to do
And setting up a GitHub action feels a lot like, it feels like a big risk because it's a programming environment I haven't used before, especially it's a remote programming environment and the iterative process of pushing new code to it is painful. So I don't think I want to do number three. I think if it lands on me, I'm going to do number one. Or honestly, maybe just number two, because I don't have huge confidence that adding these actions is going to make an enormous difference in the number of contributors and the number of active contributors and the number of successful contributors. I think this feels like an incremental help. And so I don't want to do a big project. All right. So yeah, there's my thinking. I'm going to punt on this for a week, and maybe somebody will pick it up. But if it's not done by next Monday, I will bolt together these other two to roughly do it. And if that gets a good amount of value... which looks like more contributors and more things moving along and fewer stale issues. Maybe then I would revisit and think about, honestly, number one. But that's good for now. Yeah.

02:16:31Also, waiting a week gives me a chance to see if the... existing action that i just plugged in for labeling stale issues wants to work i wonder is there a way i can manually kick that off can i make it run right now actor status event No, these things, it just runs on the clock. Pin. Hmm. That's a shame. I'd really like to manually run it and see if it works.

02:17:28I guess I will just have to wait until... Tomorrow, because 6 p.m. UTC has already passed here in Chicago, right? What do we got? Yeah, 12 hours, 14 hours. More than that. Yeah, just missed it. Ooh, can't do math. I guess that's a sign I'm getting a little tired. Let's take a look at where, oh, let's take an advantage of the break to remind that this is Lobster's office hours. If you have questions about the site or the community, eventually this is going to become a very polished elevator speech just for how many times I rattle it off per stream a couple of times a week.

02:18:37So let's see what's different. Right, where we left off on last Monday's stream with self promo was trying to find above average comments, and I had made the migration for daily comment stats.

02:19:07And then this join worked fine, as I recall.

...22Oh, yeah. And I got 23, which felt pretty good. But then I saw something else that gave different data. didn't feel super great about if the joins were working correctly and honestly i feel a little bit like i'm run over by a truck i'm trying not to yawn on stream but i haven't slept well the last couple nights because i was out late at a conference having long conversations and then up early to get back All right, so what's next steps on self-promo?

02:20:14Honestly, I think I can bring this branch back into master if I just surface it for admins. So let's go over to... So this really is a scope. So I'm going to make it a scope.

...49So then it becomes composable.

02:21:00And it still has the to do in it, which is fine. Where's that diff? What was the comment stat? All right, I have to populate the comment stat table on a regular basis, I think. So let's surface this and then goreng just read the `about` page on lobster. Kinda shocked that lobster is run on a relatively small vps
edit a cron job to make sure the comment stats are getting updated and then merge it down to master i think that's a good way to spend the rest of the stream and if i'm aiming at my usual target of about three hours that's about 40 minutes from now and it feels like something i can do in about 40 minutes depending on how many interesting questions and stories people can or interesting questions people ask and how many stories i can be persuaded to ramble on to oh gring so yeah that that kind of distraction so you say run on a relatively small vps it's actually accidentally large so there's the web vps and the database vps And a while ago, we were having some slowdowns on the database. And I happen to know that the easiest way to optimize database queries is just to have enough RAM that the whole database fits in RAM. goreng the whole DB fits in the ram PogChamp
And so rather than think about stuff, I just pulled up the DigitalOcean VPS and clicked to add RAM to the database VPS and added, I don't know, four gigs or something. I think I doubled it. It's been a minute. And then like a week later, I realized I had misclicked. Yeah, the whole database is only like 5 million rows. And of that, like 4 million are votes, which is a very narrow table. So I realized a week or two later that I had misclicked in the DigitalOcean control panel and added all of that RAM to the WebWorker. And one of those hard to take totally straight-faced limitations of the DigitalOcean control panel is you can add more RAM to a VPS pretty trivially. But if you want to decrease the amount of RAM and thus decrease the amount you pay for hosting, you have to rebuild the box. And so I was like, ugh. And so I resized up the database server as well. goreng I see
And it's been running happily. But our web server is actually really over provisioned right now. And like, I'm happy to over provision it and spend a lot of time tuning it because it's like 20 bucks a month added. It's not a big deal. I want to say the whole hosting bill including backups comes in around 100 bucks a month, which is not a big deal. But it does mean that we have more web workers in production than we really need especially if we can add solid cache and that seriously allows us to improve our caching strategy because we see something like a quarter million visitors per day and most of those are logged out so we can actually be pretty aggressive about whole page caching

02:24:48arh68 wait is there no like, Varnish ?
There's one quirk. Is there no varnish? No, there's no varnish. We do a page caching to disk, and there's a cron job that cleans those up, but it's a little bit flaky. goreng Also no docker I assume? since I see no dockerfile. How do you usually do the updates etc
So Nginx can serve cached pages. It's fine. But I think using solid cache would probably be an improvement. We do not currently use Docker. I do the updates. If you click up to the lobsters organization, there is a lobsters-ansible repo. asrael_io who needs docker!
So one of the biggest negatives for caching is on comments. When you load the page, it says approximately how long ago the comments were. Hey, Azriel, can you think of three reasons people would use Docker? asrael_io yes
Just as an example. So if we cache this page for, say, four or five hours, this goes badly out of date. goreng @asrael_io VC backed startups
I'm occasionally tempted to put in a raw timestamp here because then we could cache these logged out pages a very long time. asrael_io that's where i learned docker
It's this and it's this. And the story pages are very popular.

02:26:25Yeah. Docker is also really nice for, I mean, a big chunk of this theme has been me trying to encourage contributors. And if it is literally one command to set up a development environment, if we look over at the readme, asrael_io i've been promising my team at work i'll setup a one command dev env setup using docker
There's a whole bunch of steps to set up a database, make sure it's running, configure it to run or configure it to have the right permissions. And then you have to run our setup and then add some fake data and then run the Rails server. And then in production, you have to know some extra stuff. goreng @asrael_io they wont use it, I tried
But this has been enough to dissuade people from contributing to lobsters. ellg I appreciate that aspect of docker a lot, I've had terrible experiences setting up rails apps locally in the past and its made me wary of touching it again
And I have had a number of issues over the years where people find troubles in these setup steps. asrael_io but this is the part i am afraid of
ellg especially around Bundler
So I mentioned over in our issues in the Rails 8 issue that I showed earlier on stream that I would like the Rails 8 feature of adding Docker even if I don't use it. If it makes it easier for development and I get one or two good PRs out of that, that's an enormous win. So that's our caching.

02:28:00goreng Kamal 2 demo by DHH was cool
Yeah, setting up Rails apps, especially the first time, can be real frustrating. I also recently had an issue with Bundler where in production they changed the way well i mean i had the issue in production they don't know but they changed the way you can override where gems are installed to and we use bundler to install gems as the lobsters user on prod not as root just to simplify things and keep things together yeah the kamal 2 demo was neat i don't know i Like I said, all the complexity comes in for orchestration. asrael_io that's where we're at, 2 SaaS apps in different VPC
As soon as you go from one VPS to two, and they have to know each other's names, and the database server only lets in connections from the IP address of the web server, and the web server has to know the host name of the database server, you can end up in circular dependencies pretty quick with doles. So a lot of complexity sneaks in. I'm really curious to see how Kamal handles it. asrael_io i've only used docker and docker compose
Asriel, have you used Docker, or I'm sorry, Kamal 2, you're saying, or just Docker on your two different apps?

02:29:33Okay, only Docker. Do you have any opinion on... What's the name of it? So this is where I admit I don't actually know Docker, but I understand there are a couple of management tools you can use. goreng Kubernetes?
goreng Docker Swarm
Like there's a service called Docker or there's like another service for managing your running containers. I know there's a couple of options there and I'm getting, I am just butchering the terminologies. No, not Kubernetes. ellg theres docker swarm, but you dont really use that nowadays
because that's lower level. asrael_io yah docker compose let's you run multi-container setups
Swarm. Shoot. What's the... Do you have a name like controller or composer? All right. ellg docker-compose is for orechestrating locally
goreng docker compose
Docker compose must be what I'm thinking of. I was thinking of there's like a system process. So like I would run a CLI command to start up a container asrael_io dockerd
goreng containerd?
And then what's the name of the daemon that runs in the background that actually runs the container and manages it and kills it if it runs amok? Docker D? ellg have you looked at systemd + podman? systemd has built in container support now
Yeah, aren't there multiple options for Docker D? Container D? That's what I'm thinking of. ellg thats what i usually gravitate towards
Oh, yeah, Podman is one of them. I know that's one word. I can't remember the others. I guess Podman is the thing I'm curious about. Elg, why do you gravitate towards it? ellg mainly just because it ships with most servers I use
ellg systemd is pretty prolific
frici podman is something people started going for a lot cause its sort of docker without docker at this point. and systemd is everywhere anyway
goreng @ellg tbh docker-compose is enough for most people. For our on-prems client only need it for HA, no need for k8s
lg i don't know how to say your name sorry and i guess part of why i'm curious about it is just taste chips with the most servers you use a thing i've seen with a lot of system services is the first one the official one like in this case i guess that would be docker d tends to be really broad and include a lot of odd ends that don't quite get used, but they get used just enough that they can't delete. ellg docker-compose is good for development but systemd integration is really nice for logging and other nice to have integration stuff
And so like a second version or an alternative tends to strip them down to just the needed features and then also have the benefit of learning from everything else.

02:32:12ellg around observability and restarting and such
Compose is good for development, but system deintegration is nice for logging. goreng nah dont
Yeah, I spent a lot of time thinking about logging lately.

...27goreng logging is
goreng expensive
All right, well, thank you. I appreciate folks sharing your experience on this because I'm still really reluctant to put Docker into prod because, hey, it already runs on Linux. At the same time, if it's going to have a development container, maybe that wants to be the same thing, just to pin down odds and ends. Honestly, at this point, some of it is just being an experienced Ruby developer. ellg if you dont need it for production, then I wouldnt bother
I've seen where the foot guns are, and I don't tend to step on the booby traps around things slipping out of date or out of sync. ellg but docker + docker-compose for local dev would be worth it
The other one is Ruby has learned a lot about that tooling. So we have stuff like a Ruby version and the gem lock file. And that's like 90% of it.

02:33:22goreng @ellg +1
ellg ha, perhaps
Well, LG, if you're feeling really sprightly, you can whitewash the fence. ellg I am a fan of the site
So earlier on stream, I made the... asrael_io same
initial update to rails 8. i say made i should have said merged because another contributor oh thank you for being a fan another contributor just bumped us to rails 8 and sorted things out and now we have all of these nice features that we have to set up so if you want to set up kamal 2 that would be pretty cool i would love to see a worked example of it ellg does lobsters use an otel stuff currently?
goreng how are you liking the sqlite stuff on rails 8?
ellg you mentioned thinking through logging
So I'm debating whether I want another section on the user profile.

02:34:13goreng or not yet
I don't know what an hotel is. goreng @ellg opentelemetry
targze open telem?
Oh, is hotel short for observability and telemetry open telemetry? OK, I got that one wrong. ellg ah ok, it might be worth reading up on opentelemetry + tracing as a means for logging if you are reading up on this
No, the thing I've been thinking about lately and just talking to folks about is setting up LogRage so that out of the box Rails gives a whole ton of... Its logs are really verbose and one request generates like eight to ten lines of log output. i realized in running the site i look at the logs all the time to find things but i only really care about a limited amount and i saw someone evangelize logging in json and just doing one line per request and i gave it a shot and there have been some minor bugs there's an open issue on it right now for anybody else who wants to contribute but it's been really valuable to be able to grab these logs and pipe them into jq to sort and filter and just not have to write another grep basically.

02:35:58ellg otel is even cooler, you can see a visual representation of the full request path and dig into metrics for each step
ellg im a huge fan
Oh, I just saw somebody talking about OpenTelemetry at RubyConf on Friday. They were talking about wiring it up to have, well, basically, traces, metrics, and logs funneled together into one place. Yeah, I was saying that I would like to have an APM for lobsters, but I would like to not pay an APM bill because They charge a serious business price and lobsters is not a serious business.

02:36:41So I like, can you use. ellg grafana's saas option has some really great free tier stuff
targze otel hcan be consumed by prom
Actually, maybe that's a better way to put it. Can you give an example or share a screenshot of. goreng the tracing stuff?
targze prometheus
targze https://opentelemetry.io/docs/c…
what this looks like targz i don't know what prom is either i only know enough system administration to be dangerous i'm not actually good at it prometheus

02:37:30targze (sorry dont know if links are allowed in your chat)
Oh yeah, links are very much appreciated. No, thank you. That's what I was trying to talk LG into giving. ellg you basically can set it up so you can do stuff like this https://grafana.com/static/asse…
Just some idea of what this looks like. So I'm a little... I have a broad general reluctance to add moving parts to production. But at the same time, it would be nice to... have a little more insight. ellg where you get full end to end tracing and can drill into things like, how slow your database queries are, etc etc
And yes, LG, this kind of waterfall graph and tracing is pretty lovely.

02:38:13asrael_io dang
Does it also do slow query logging so that you get what was the actual SQL query for some of these bars?

...32ellg yup, you can see the full sql query
goreng yeah
goreng tempo is such a treat to use
You can see the full SQL query. Well, that's pretty beautiful. asrael_io that's pretty nice
goreng not so much to manage
That's a thing I want. LG, you want to set that up?

...46Not so much to manage. ellg ha, ill bookmark the repo, kinda swamped at work but im tempted now
Gorang, what's Tempo? Can you share a link, please?

02:39:05goreng the screenshot you are looking at is tempo
Yeah, LG, I mean, especially for stuff that involves services, you'll have to work with me pretty closely because I'm the one with root on prod. goreng https://grafana.com/docs/tempo/…
Oh, this is Tempo that I'm looking at. I thought this was Grafana. Is Grafana the database and then Tempo is the front end?

...27goreng grafana is the frontend, tempo is the "backend"
Source, high-scale tracing back end. Ooh, pretty charts. I'm a charts. Requires an object storage. Oh, man. I mean, distributed tracing is a little overkill when we only have two boxes and there's only app code on the one. But yeah, this is the kind of thing that would be quite nice. We are mostly... ellg this is a pretty good tldr on how to set it up in rails, youd just replace the exporter url's to point to something like grafana cloud https://help.sumologic.com/docs…
So I have added some performance stuff to the LogRage output. And I have peeked at it a little bit to make sure there aren't any insane spikes. But all of that stuff is so much easier to see on a visual chart than me kind of glancing at a list of numbers. or piping it to sort and eyeballing the distribution. goreng also for logs, you can keep it cheap by only keeping it for a short amount of time, for example only 3 days.
And since the web server is over provisioned anyways, this might actually be pretty nice to set up.

02:40:55Yeah, logs, actually I have the opposite thing is that I wanna keep something like three months of logs. And I don't need this kind of performance data for months or tracing for months, but our action log that I showed here, this one line per request, this is the thing that's useful for months because when I, I literally saw one today because I had to boot out a spam bot. And so looking at it and looking at how it's doing requests and seeing all of its stuff is really useful. And it signed up actually goreng other than grafana, theres alot of other OSS solutions. for example https://signoz.io/
two months ago and then left an innocuous comment and then came back and tried to start spamming maybe a day and a half ago and so at that point it was really nice to be able to look back in the logs and see all right what did you do did you click around can i get a feel for you so that when i see you in the logs in the future because spam bots always come back I'll notice it. But this kind of detailed performance stuff, yeah, I can see tossing it after a couple of days. Other than Grafana, there's a lot of other OSS solutions like Cygnos. Who names these things? All right. OpenTelemetry native logs, metrics, and traces in a single pane. ellg otel is an open standard so once you implement the collection stuff, you have a lot of wiggle room on where you send it
Okay, so this would be an alternative front end to that logging. ellg its really nice
Hmm. What tells an open standard?

02:42:41goreng this one is a more "all in one" solution
ellg could be signogz, could be grafana
Where is your little slider? ellg could be datadog if you get $$$
Okay. Yeah, lobsters is not a... goreng with grafana, you need to install alot of stuff. (prometheus for metric, loki for logs, tempo for tracing)
$3 signs kind of restaurant. Lobsters is a $1 sign kind of restaurant. I am not going to add Datadog or Splunk for these kinds of tasks. ellg @Goreng ya although their hosted solution is great, and very generous with the free tier
I have a shell server and grep and jq for these kinds of tasks. But some of this kind of application performance monitoring would be real nice. I don't want to accidentally start breaking things. ellg grafana
arh68 all this stuff looks cool, I've just never used em SeemsGood
LG, are you saying whose hosted solution, Datadogs, is great with the free tier? Grafana. Okay.

02:43:52ellg https://grafana.com/products/cl…
targze @ellg do you run into cardinality issues with traces like you do with metrics?
ellg you get access to hosted prometheus
goreng yeah, alot of the provider provides free cloud tier
Hey, LG, what's your username on GitHub? We're talking about assigning issues.

02:44:05goreng newrelic provides generous one
targze or whats the ops thing to keep in mind?
A lot of the provider provides a free cloud tier. Yeah. targze from a sizing perspective
I try not to take too many. What's the ops thing to keep in mind? The ops thing is KISS. Oh, from a sizing perspective, I think it's like aim for your Aim to have 80% utilization? I don't know. An actual person would have much better rules of thumb. goreng btw, this is signoz setup for rails https://signoz.io/docs/instrume…
ellg https://github.com/ell https://lobste.rs/~ellg
And I suppose if you were making money off of something, you'd be a lot more conservative with that utilization. What is this measuring? Ah, thanks, El. ellg ill star the repo, not sure if ill have time soon but ill start looking at the code
Oh, so they're set up for Cignaw's Cloud. So that sounds like a hosted thing. OK.

02:45:18goreng scroll down, theres self hosted one
ellg also dang hit 10 years on lobsters, didnt even notice
ellg thanks for the site!
Oh, I like your VG Music site, LG. I've seen this before. So I clicked through to your profile. ellg awesome!
I've seen this vgmusic.fm before. It's nice for background stuff. ellg i use it for programming a lot, its why i made it lol
Where do you source your music? ellg ocremix and various torrent sites
goreng @ellg daaang 3 letters username
yeah i also like music without words for background for programming growing lobsters is oh it's three letters on l yeah that's true or on github yeah looks like they signed up in 2010. when did i sign up on github

02:46:20ellg ive got https://ell.dev too
ellg sniped that
Oh, I'm even older. 2008 sometime. You got l.dev? Yeah, that's a good get. I think I've got pushcx.dev, but I live on push.cx.

...39Oh, you're up on Twitch? I will follow you. ellg oh ya i do some rust coding on twitch occasionally
Do you actually stream here on Twitch? I don't see a schedule. Last stream nine days ago. So it sounds like you're going to stream on Wednesday, maybe? ellg no real schedule, just when the mood hits
All right. espartapalma your ID is 6262, pushcx https://api.github.com/users/pu…
So I'm a little sideswiped from the self-promo stuff, but this has actually been a really interesting conversation because I don't even know the terms for it. for all this telemetry kind of stuff. What are we looking at? My ID is 6262. Oh, I see. Yeah, I actually consider it a bug anytime that lobsters shows a user ID or a story ID or a comment ID because people, it becomes like a weird, Coolness measuring factor, it was so bad on Slashdot. goreng btw, is there a way to receive invitation to lobste.rs?
This low number is just that I've long been, well, number one, I'm older. Number two, at the time GitHub came out, I was really connected to the startup scene and Ruby development in general. And so GitHub really took off first with the Ruby community was my understanding of it. So that's just right place, right time. The best way to receive an invitation would be to look at the user tree. So if you go to lobsters slash users, you'll see a list of folks. You can control F for a username or two that you know. But the alternate thing would be to, at the bottom of the page, there's a list for the chat room, which of course is not the Twitch chat. It is IRC, because again, old. And You can drop by there and share a profile or two so people understand that you're on the level, and somebody will invite you. All right, so I want to plug this in probably up here. Let me split out the temporal aspect of it.

02:49:14Yeah. We're just going to say recent.

...36And then I'm going to grab that part, but not with the syntax error and then here. See, every time I see a scope and I end up with multiple adjectives in it, I want to split it up into multiple scopes and compose that. So instead of being recent underscore above average comments, it will be recent dot above average comments or really recently on the last but I find when I replace those underscores with dots I end up with much more reusable scopes so we got that and then I will say let's add a br and then

02:50:38targze @Goreng https://lobste.rs/chat FYI. I've been to coy to just ask there for an invite but that's how folks seem to do it
And then we will say, let's just put a comma.

...54Yeah, the other thing you can do is if I'm streaming and you catch a bug, I will send you an invite and make you a channel VIP. Or if you... send a pull request to the site to implement something i will certainly send you an invite to the site but otherwise i tend to invite people i know right so now here there we go you know i gripe about active record a lot on stream but when i compose scopes oh i'm so happy with it so much nicer than schlepping around sql strings

02:51:44So let's see if I broke any tests. I think there is a spec for loading profiles, but maybe not as a moderator. goreng @targze aight
And my database is full of, so one of my practices is that I restore Lobster's backups to my development database because it forces me to test the workflow for and the quality of existing backups. but then also, yeah, it's plural. That does mean I have a bunch of sensitive data on stream. So I'm not gonna pull up a moderator login real quick. All right. Come here, baby. Yeah, cat. Oh, broke some stuff. What do we get? Undefined method above average comments. why is this not a method it's a scope so i can count the recent comments you know i guess as long as this is on comments i should just move it over so if this becomes above average yoink

02:53:19It's going to have to have that.

...29So here, that looks good. Let's just run this one file that had all the fails.

...59OK, just some typo or something.

02:54:10All right, so if I have that.

...23That's fine. That's fine. That's moving around. Good. Let's add all this stuff. And I will squash those whip commits later.

...55And then there's a daily cron job that I can use for filling in the comment stats. I think I want to do a left outer join so that today's you get a null. And then I can, what, coalesce that down? That also protects against anything that's missing a day. With coalesce, the default comes second, right?

02:55:49Value. Oh, yeah, it's like a list. OK. So that ought to do it. Let's see if that still wants to run. I just really only need to run that one profile. This is just a smoke test. Yeah. What are we mad about? I need to see the actual query.

02:56:35So you are mad about on this left outer join comments stats. Oh, I think I have to. Rather than interpolate it in the string, I think I have to call left outer joins. That's right. Let's reload. Let's do reload without typos. Only hash, symbol, and array. Fine. Let's join the comment stacks. reload again this is going to get teased you don't have an association oh that's the all right then i do want to just do it like this because it doesn't know that there's an association because i can't really key it on the date like i want

02:58:00Interjoin stories on blah. Left outer join comments stats where? Oh, do I have to have an on? I think I might for left outer join. So let's grab this. Do it up there. Let's see if that's all it wants. Yes. Excellent.

...31Good.

...38So if I've got these, now I just need the, let's see that that spec runs.

...53Good. So where's my daily crown job?

02:59:04This does run, right? So we're going to jump over to Ansible a second, right?

...20Yeah, we run this lobsters dash daily, but that's not the right task.

...33goreng anyway, gotta go guys/gals. cya
Huh, I wonder if this rate task is even running. Nice to see you, Gerang. Thanks for chipping in your experience or your knowledge on the telemetry discussion. I appreciate that. Ah, it's running here. Yeah, this is the file I want.

03:00:00OK, so the daily cron job runs this stuff, does the sitemap. And I will put it in here. So we will say, I put it in the migration, didn't I? But I just did raw SQL. So what's the incremental update here?

03:01:06Fillmore, that's a terrible name.

...25All right.

...36That's fine. So what I want to do is just this insert, but for anybody that any date that hasn't happened yet, and honestly, also any one extra.

03:02:20But I want it to be... What does this look like? Is there a... On duplicate key update. So what I'm thinking through is failure modes. If the daily task should fail to run, I don't want to leave gaps in the comments stats. And then I also don't want to... I also want to kind of override the last day or maybe even two days because I don't expect that the clock the stat is going to run yeah saying it wrong I don't expect the job is going to run at exactly midnight and I want it to I'm fine with it doing an average for a partial day as long as we insert at some point. So, so looking at this, well, right off, this probably wants a low priority for production.

03:04:21So if I did this, it said, oh, well, it's going to be this code plus the on duplicate key stuff.

...51This goes, oh, you're OK. Insert into set, select. So this goes here. arh68 insert .. low priority .. (then) into ?
I'm trying to think of.

03:05:26Insert library into, yeah, there's so many forms for this and I do it so rarely. So I want the insert into select form, but I don't want to have to say, I guess I have to say that update again.

...52I guess I'm fine with it. I'm kind of used to... What is it called? To sequel being verbose...

03:06:20I don't know that this is actually going to do what I want of selecting the data out, though.

...37Well, let's see what it does now. Yeah, it's fine. You're OK again.

...49Oh, not in my director anymore. JLo.

03:07:00Actually, before I run this, hold on.

...16So here's 11.6 when I ran it on a previous stream, and then it hasn't run since. So I'm in a good place where what I want is for this to run over the last 30 days, basically.

...39So let's break this up a little. From comments where it's greater than, what, can I say today in SQL? Where am I?

03:08:20say now. So this should touch all the averages for the last 30 days, inserting them if they're not there and updating them if they are there. So the way that would appear here is we'd see What, 12 more days of records after the sixth, and maybe this average for the sixth will change? So let's see. Or I could just get a syntax error. Oh, it's the low priority part it's angry about. Oh, low priority goes before into. arh68 ya
Is that what you were saying, ARH? arh68 no worries only 1 way to see if it worked
didn't realize thank you for catching the bug 19 time channel vip arh all right so now we've got a new syntax error two comment stats no worries only one way to see if it worked yeah i just misread the docs there so we're gonna near well it's mad about something in the early query insert whatever into table name select Oh, it's not low space priority, it's low underscore priority. Okay, good. Invalid use of group function. Not good, but understandable.

03:10:18So what do you dislike about my group function?

...44I wonder if it's thinking that the unduplicated P is part of the select. Like what if I slap parentheses around that?

03:11:09No, still not. arh68 idk i'd "just" try rewriting in terms of CTEs . err msg sux
Maybe this calls for a common table expression. So if I broke that out and said. Yeah. You're getting there the same time I am. With. Add by date as. Select and then here we're going to do this nonsense.

...48Oh, read line. Read line. Let's do it over again.

03:12:28This is going to work with both of these named average. The left one has to be for the insert, and the right one has to be from the CTE, but I don't know that that's anything like valid syntax. Yeah.

03:13:03So the as.

...14No. All right.

...35You must use the CTE as select. Whoa. So the, it goes here. All right. So we say insert into these columns with that as not syntaxing. And then I select the fields out. So let's select out date average. But then on duplicate key, let's just call it A, just to differentiate between them.

03:14:40Not a syntax error? Five seconds, long query. 19 rows affected. How do you get to 19 for 30 days? So there were 12 days unaccounted for. Hmm. Where'd that select go? So let's change this back to 20. I have duplicates for 11.6. So that's bad. And it's actually fine that it didn't add more here because There haven't been more comments on these days.

03:15:49arh68 so "on dup" never triggered huh
How do we say which keys are being duplicated? On dupe never triggered. Clearly not. I think it's using the ID key.

03:16:20How do I specify that I care that it's the date column that's getting duplicated? arh68 join in n]the id i guess ?
This is missing the index, isn't it?

...49Yeah. It's a different problem. I don't have an index.

03:17:00What is it?

...22So let's roll back. Oh, good. I got an error doing a rollback. Why? Because you're rolling back the duplicate origins? That's not even the one I want to roll back. I've written that since... How do I specify which migration to rollback? I can't. All right, let's do that by hand.

03:18:20espartapalma you can specify which migration rollback by using VERSION=xxx
All right, now let's run that migration.

...27by specifying version equals. I didn't see that in the output from the rollback help there. I can specify steps, but maybe it's just not documented very well.

...59Oh, it's the low priority thing.

03:19:13espartapalma rake db:migrate:down VERSION=20100905201547
And now it's the migration only partially ran. espartapalma it's on migration:down
Migrate down. Does that? espartapalma will rollback the version only
reverse that specific migration with version or does it go to that version or does it go one past that version runs down okay for that specific all right well thank you i'll put that in my back pocket That's especially nice when I cross myself up by juggling so much in-progress work.

03:20:08OK, so that looks good. Now let's grab this and run it. Let me think about it for five seconds. 3.9 seconds, fine.

...30And nobody changed. All right, so I'm going to just drop out one of these values.

...43and I'm gonna update the other one.

03:21:03Okay, so now if I look, we have a ridiculous value and a missing value. And if I grab the query, Can I run the query? Should have healed up. Do we think it worked? Hey, it did the right thing. Excellent.

...38I know it's a big, redundant, kind of clunky SQL query, but I like it more as just a clunk of raw SQL than I do trying to translate that into active record. Yeah.

03:22:29Boom. Bam. What did I just push? There we go. Lost that sentence for a second. All right, so I'm going to do that. I am also going to say from comments where this and comment.isDeleted is false and is moderated. Or wait. On comment, moderated just means a mod touched it. So yeah, that's fine. It doesn't feel fair that you get credit for a comment left on a day where someone managed to hit min combat score, or combat score. Wow. I am glad to not moderate a site with live combat, min comment score. So that's a lot nicer there. All right, so if that wants to work, let's just call it once from the console.

03:23:45And that's why we run it once from the console, because I tweaked something.

...56It's not called is underscore deleted. Oh, it's not comment singular, it's comment plural. Thanks Rails. All right, try that again. Just want to see these things.

03:24:23All right. So let's take that, which is a nice tidying. I'm just going to rebase this on a master rather than make a merge out of it because there's a couple of commits and all that, so. So let's just rebase into master. Yay, no merge conflicts.

03:25:07Let's see, can I just grab it like that? Yes, excellent. And now I'll give it a commit message

...29All right.

...36I brought all of them over. I want to actually just squash them.

...47So I'm in the terminal in the vim, and when it tries to open another vim, because that causes so much hassle, I set it to just try and quit that vim. So I got to remember how to get this little plugin to let me squash these. Okay. All right. Squash, squash, squash. And what is it? R to reword? Yeah.

03:26:35That's concerning. All right. So if this is going to be. So I'm making this moderator only just because i want to see it run in production for a couple of days but if anybody is curious i am happy to tell you what your number is off abroad yeah

03:27:29All right, let me check that I got the git history I wanted. This is not exactly jujitsu where I can go back on all this. Oh yeah, that's weird. 12 days ago? Fine. And then it pulled in the other commit messages because I said tidy instead of fix up. So how do I edit this commit? not that amend and edit all right there we go yeah there's one more thing doing this reminds me of This could really use an explanation. comment dot above average needs to join needs to compare against the average which is an expensive dependent sub sub query one word or two to maria db i think it's one i don't know

03:29:34What would you call this I guess it's. it's not a materialized view, this is a cash value of that data.

03:30:05All right. So that gets across. This is for performance specifically. This is for that expensive dependent sub query. Cause otherwise I can see coming back to this in a couple of years and being like, why on earth do you do this? So we will just amend it. Great. And then I'm going to push that up to GitHub.

...38Boy, I'm tempted to deploy right this second, but I could actually use a minute to stretch and everything. It'll deploy, right? It's just Rails 8 and adding a new PR or adding a new migration that has a bunch of raw SQL and a workflow. which I've barely done. I mean, what are the odds it doesn't run? Besides screamingly high.

03:31:32Yeah, I would love to... to have like a Twitch poll here that runs while Ansible chews on this stuff. So this will probably run for 90 seconds plus the time it takes to run the migration, which I don't know, you could call that a minute or two in prod.

03:32:01It feels good to merge this down. arh68 what's that xkcd number "it's compiling!"
I still have the active storage stuff to read from that previous stream. What was that? I think that was also last Monday's stream. And I'd like to move that along. I got a couple potential beta testers at RubyCon for Recheck. So now I'm looking at it with the like, oh God, can I ship it to them? Oh, there's, I got to clean that up. I got to fix that readme. kind of like last minute task that I knew I had to do, but kind of stopped seeing after a while. But now that I'm looking at with, is it ready for Alice and Bob to see? I'm seeing all of those little issues. jmiven 303 :)
Yeah, it's compiling. I don't know. 386 is the only one that I know by heart because XKCD386 changed my life. And I say that without 303. Thank you, Jamie. pushcx xkcd.com/386
386 is the one that made me stop and go, what am I doing with myself? So if I just throw the link in there. So it's this one, Duty Calls.

03:33:22espartapalma a classic
And I saw this when it first came out, which has been a minute, right? When did this get published?

...36This one must have been published a while ago, right? arh68 just like... yesterday ! LUL
xkcde3d6. The deploy is progressing in the background, but now I want to know the date.

...53Is there no date? No? Because it's at least a decade old. Oh, here we go, 2008. Yeah, so I read it. And I was like, oh no, it's me. This is the thing I do. I am wasting my life arguing with people on the internet. And in 2008, I got a lot better about, you know what? I don't need to win these conversations. Even when someone is wrong, I can just put the right thing there and let them keep being wrong. arh68 a strange game. the only winning move is not to play
And I also do think that that's a heck of a lot more persuasive to people who end up seeing your posts is if you don't get into a slap fight where you're fighting in the mud and insulting somebody and you know explaining 19 comments deep how they're wrong about everything i think it is more persuasive to state your case and link some things than just go forever yes thank you arh that is god i should watch war games that's been a minute all right so This is lobsters running Rails 8. Looks pretty similar. Cool. So I'm pulling up a browser off stream. Give me one second. Would anyone who is present like to be featured and see how many above average comments you've had in the last six months? arh68 u can roast me
Because I don't want to pick on anybody. But if you do, I will pull up your profile and show it. arh68 i'm `unwind` on lbost
But not if you don't want to. I can roast you. You have the same on lobsters, right? Or are you just ARH? Yeah, you're just ARH. Okay. Okay.

03:36:02You're unwind on... Oh, you're not ARH?

...16arh68 i'm arh68 on HN, but i switched it up
All right. Well, actually, this is a little bit of a bust. Not just because... So this didn't work. The numbers are missing from your profile. So the deploy did not work somehow. Oh, you know what? I don't know. Hold on. It did the bundle. So we touched the Rails bundle. And that has to do a, instead of the phased restart, it has to do the cold restart. And so the Ansible playbook got that wrong. So we did not actually get switched over. Let's take a look.

03:37:21frici task "get recent versions, aykm ubuntu" hits so hard and relatably...
Yeah, so this service has been up for a couple of weeks, one week, two days ago. And it needs to be, what is this that's from right here? yeah fritchie because i chose ubuntu long-term service and it's really behind on ruby versions so that's why i had to do that it was a little frustrating because like i get why they have the old package that's kind of the point of long-term support but i was really frustrated when i wrote that message

03:38:05Oh, if I run this on stream, it may end up kicking out somebody's IP address or something. So I'm going to actually run that off screen.

...32yeah so just to summarize what i'm seeing off stream the bundle didn't run or lobsters didn't puma didn't pick up the new bundle and so it didn't switch over so i'm running bundle in production

03:39:05And it's updated. So now I'm going to bounce lobsters puma and try not to, let me go tell the chat room that I'm about to bounce lobsters puma.

...23Bounce the site for the Rails 8 deploy. So 60 to 180 seconds.

...39All right.

...58We are now restarting. Oh, look at the site not load.

03:40:15All right, I'm watching the... What do you call it? The journal CTL output. The web workers are booting. And there we go. This ought to load any second now, because Nginx will hold the connection for about 30 seconds. There we go. We are live again. Now we're on Rails 8. So now on that personal browser, if I load up ARH's page, there we go. That's exactly the data I wanted to see.

03:41:05come here let me i am rearranging windows a little off stream to make sure i don't accidentally throw arh's email address up on the screen there we go so take a look arh is not recently flagged a bunch is not a self-promoter and recently has left 38 comments above average for the day four so What's the meme? ARH, those are rookie numbers. You got to get that up. Oh, wait, is that meme obscene? I know it's from Wolf of Wall Street. arh68 lol it's fine LUL
I actually, maybe that one's obscene. Excuse me. Forgot about the context. arh68 i think of the glengarry glenross
I just, you know, you see these things disconnected for so long. I think I'm pretty sure that one's a sex joke. Well, I mean, says the guy who's named 68 to show off that he's not nice. Glen Gary, Glen Ross, what about it? I mean, not that I am inviting you to give the Alec Baldwin speech about big brass balls. I'm just not seeing the relation to Glen Gary, Glen Ross.

03:42:33arh68 oh it's just what I tihnk of re: "rookie numbers"
arh68 never seen the toher one in total
Oh, rookie numbers? Yeah. No, there's a meme from Wall Street that I'm not going to repeat totally because after I said it, I remembered what the start of that conversation is. arh68 ya it's weird lunch convo
All right. So, Lobsters is now running Rails 8 in production. I should tell the chat room. Yeah, it was actually... So I'm going to wrap up stream if anybody has any... Yes, it's the weird lunch conversation. If anybody has any last questions, you should get them in now. But I was just reminded of a little story. One more from RubyConf. I had obviously a very exciting and busy couple of days there because I have all these fun stories. My spouse has waited very patiently through several. But I was chatting with... Aaron Patterson and John Hawthorne, who are two, if they're not Rails core, they have been, or they're heavy contributors. I can never remember people's, you know, proper titles. The, and I mentioned that when I had considered installing YJIT, it was really neat to look at the performance benchmark that they had and be like, oh, I bet we'll see about a 26 to 28% improvement in speed. pushcx https://lobste.rs/s/m9ivcf/we_t…
then that's exactly what we saw because they had tested with lobsters which where was that there we go this was a fun one so i learned of this when i was literally like scrolling down mastodon and i saw espartapalma both are more ruby and Rails core developers, but more recently leaning more on ruby performance
aaron write up a blog post about using lobsters as a y-jet benchmark and aaron suggested lobsters when they were thinking about testing y-jet and they looked at and i i heard a little bit more of the stories at rubyconf apparently they were thinking about mastodon but it's pretty heavyweight and it has a whole bunch of Thank you. It has a lot more services where lobsters doesn't expect to have anything besides the rails service process. You know, there's no queue worker, there's no sidekick, there's no Redis, there's no, there is a database, but that's pretty simple. And they got it working with SQLite pretty easily. So Apparently, Lobster continues to be in use in their code base as sort of the go-to benchmark for Rails performance and Ruby performance because it's got real-world complexity. It looks like real-world code, and it leans pretty hard into being a vanilla Rails app. Like, I have resisted adding... I started to add Sidekick and Redis. You can see it in the Ansible repo, but then I pulled it back out as ActiveJob got to mature. But I have really leaned away from adding any kind of external dependencies. And so it was really easy for them to get running in prod or in this perf testing mode. And so Lobsters is having a little bit of life as a benchmark there. That's pretty fun. I really like the idea that it's kind of standard. I also really like, speaking of, I've played some more with Flex, this nice view library. And boy, would it take us away from standard to drop out an action view in favor of Flex. I keep liking it. So I probably, I don't know, I'm not quite so pumped up by lobsters getting to contribute back in this way that I want to stay only on the default stack. I mean, obviously we used our spec. I don't know, just kind of neat. Thought I'd share. I hope you all have as much fun writing Ruby as I do. espartapalma thanks!
Thanks for hanging out as I moved along some of our own features and our meta stuff, like what version of Rails are we running and how do we encourage contributors? arh68 PrideLGBTea peace y'all VoHiYo
I hope folks decide to be contributors, especially Elle, who I heard publicly commit to getting Telemetry and everything work. I'm sure everybody heard him commit and promise. Yeah. frici totally LUL
espartapalma I heard that
Anyways, in any case, if anybody can volunteer, I really do appreciate it. Happy to explain stuff and encourage. There are plenty of open issues. frici Have a good time and catch you on Thursday folks
All right. Yeah, I will be streaming as normal on Thursday morning. So that'll be 9 a.m. Central Time. Yep. I'll see you then, Fruity. Especially with your good hearing. All right. Take care, everybody. Bye.