The user who usees the HR tag most on Lobsters

01:43Why do I keep doing that? Anyways, I guess I'll get started a second earlier than I wanted. That's fine. Hello, this is Lobster's office hours. That is my terminal. This is my browser. And this is Lobster's. There we go. Ah, so, It is also nice and oasty-toasty hot in Chicago. So for the first time, I'm running my AC for the summer. And the layout is a little different than the last time I ran it. So if you hear it, let me know, and I can tinker with the noise gate settings. But I think we'll be OK. pushcx This is Lobsters office hours, drop questions into chat anytime.
So this is Lobster's office hours. questions to chat anytime there we go all right so i've got that cool so let's see where i'm at here i have yeah so it's i didn't finish the spiel so this is lobsters you can ask any kind of questions about the site or the community or the moderation or anything else that's going on and when folks aren't asking questions i work on the code base and that's been pretty quiet these last couple of days though we did just get a couple of suggestions here in issues and then pull requests are Pretty much where I left them last stream. This one about postfix is about handling known bad senders and dropping mail from them. I'm waiting on one more round of Q&A. CSP. I believe he's waiting on me, yes.

04:08pushcx https://github.com/lobsters/lob…
Yeah, so this one, this one's hanging out.

...29So for anybody who doesn't know, CSP is a security control because, I mean, it is very easy to create XSS bugs in websites. ERB doesn't help too much. And if we shift to tighter policies, not only are we protected from the inevitable errors, but As I recall channels was working on something that would benefit from this that I am blanking on. Because I. haven't looked at it for a second. Right, so this one was pretty straightforward so i've skim this PR.

05:28I don't know CSP so authoritatively that it immediately looks right. And I don't know that we have an endpoint here. That must be why this is a draft.

...52Maybe we do. So we picked this up. All right, we do. We started adding this a few years ago, but the contributor didn't carry it through. So rather than do that, I'll just log it to a file because it'll never get said there. If we log it to the bug system, it will get looked at. So there's one suggestion that she could not have guessed. And then I don't totally understand what this is doing with nonsense. So I am going to have to learn that. Yeah. There is a bug. An edge case on the signup page where a guest user is served JavaScript for the markdown. Ah, there goes my IC. So if you hear it now. No, good. My little levels aren't bouncing. You shouldn't be able to hear my IC.

07:21For the markdown formatting drop down, you know we talked about. yeah.

08:17twitchtd hi pushcx
hey td i was just going to click over to your issue where i left a comment a few minutes ago let me finish up this pr but we can talk about anything you're up to i'm especially curious we currently send forward violations to the main rails log where do they vanish into the infinite A stack of hits the year to use. What is the. note right in this. box. Right so installation and then I think it gives an example of logging a first message.

09:31So this reuses the sentry. API.

...53That is actually mildly obnoxious that this is titled sending your first error report. of these instructions you literally do not send a first error well i guess there's like this but

10:27grayhatter_ CSP in ephemeral, you can just enforce without report
This is probably because Sentry is used on the front end. It looks like tracking telemetry. grayhatter_ the report option makes no sense
Yes, but if I enforce without a report, I don't actually know that we're fixing anything. We need to have those violation reports so that we can actually Do something with it. So the value of a report is if something fails, the content security policy will know it.

11:16As opposed to. It's broken in production and we've just broken site functionality and we have no idea which. I mean. We're often oblivious, but I would prefer not to stay oblivious.

...36All right, so.

...45The. What is this?

12:07twitchtd csp report is useful for detecting issues in a known working csp, cause if it's working and it sends a report, could be a xss or other attack in your data and then you can handle it
Did I wrap this song. You know,

13:02Yeah, actually, Tom makes the excellent point there that if it sends a report, you also know if you have just suffered a security vulnerability.

...29marcoroth_ HeyGuys
Oh, hey, Marco. I was I've talked about it on stream, but I am eagerly awaiting your ERB linter and formatter. And if you would like a beta user, I would really like to get shot in the foot less by our ERB templates. marcoroth_ oh lol
pushcx https://github.com/lobsters/lob…
And we might not be a great first case, because if you haven't seen in the code base, we have a terrible, sinful thing for performance. This comes up all the time and I joke about it on stream. So this is the sinful thing because partial lookup is slow in Rails. We have this feature I called heinous inline partial so that it is very conspicuous what it's doing, but basically what it does is one template can name another and it is included in. grayhatter_ yeah, that's a very good point; my problem is I often forget that security systems often aren't well tested
So for us, the hot path for the site is this view, the single story view. twitchtd oh that's what heinous inline partials warning comes from when I run anything related to rails!
And if we rendered out the comments over and over and over, we do a lot faster with heinous inline partial, something like 10% faster, if I recall. marcoroth_ that's quite involved
by basically inlining that partial into this page, which is a lot, but yeah. Yes, TD, that is me. I put that in there because So it has some boots and suspenders to make it a little bit less dangerous, but it is a super confusing feature because it will lose you work if you don't know something about it. So if we go look at view thread, yes. So it has a big comment that says, hey, don't edit this code here because this is the partial getting inlined. And I do, I have once or twice lost like a minute or two of code because if you control F or do some kind of multi-file search and that comment is off screen, it is easy to not realize you are editing the compiled version. But Marco, I swear the performance numbers justify this. It's just that partial lookup is slow in Rails. And the most irritating thing, I wonder if it could have a cache or something, because otherwise, right here, it would say render individual comment. But we pay the tax of looking up that comment partial every single time. twitchtd cached partials make sense to me, I wonder why it does a lookup each time
It doesn't seem to have an internal cache about that.

16:55marcoroth_ I guess if it would be low hanging fruit Aaron or Jean would have already implemented it
Yeah, and it's possible it's something that it's fixed, or it's also possible that I mismeasured and the cache is off in development and I measured in development mode, but it's on in production, but I sure didn't. I'm pretty sure I was running in production mode, but it's been years now. And yeah, a lot of this stuff that is so high touch core Rails features, you can be pretty Pretty sure that it's going to be reasonable. There are a couple of... You know, there's also the opposite thing, though, where when something is an absolute core Rails feature, it's been there since... What was that first one? 0.6? Sometimes it has some weird edge case or other... thing that seemed like a good idea back in 2005, and nobody wants to touch it and break compatibility. And Rails is usually pretty good about deprecating things and moving the whole thing forward. But there are a few things that still feel like they work that way.

18:28marcoroth_ From just running a quick `analyze` on the repo there seem to be 18/97 files with "errors"
So does this?

...33If I say... Yeah, Marco, one of the other issues we have had is

...55with ERB because ERB is just, ERB doesn't know anything about HTML, it's just string soup. We have had basic errors of construction, like there's an open div tag and there isn't a closed div tag.

19:17And I had an interaction with Byroot, Jean Boussier, Let's see if that's in my history. Yeah. I think it's going to be this one.

...39marcoroth_ yup, this is exactly part of why I'm building this tooling
pushcx https://github.com/Shopify/yjit…
Came in at the end. Yes. So

...53They maintain a sort of fork of lobsters for performance testing-wise. and other MRI things. And every once in a while, I see a PR go by where they merge in our master and I try and keep in touch and tell them about big changes we're making. And on this one, he pulled in changes and from my perspective, this is sort of nice where a bunch of Ruby experts look at our code base and sometimes they spot significant performance regressions. and a lot of the stuff that they do assumes that you are running a very large rails site which makes sense given how many people who work on widget work on work at shopify but we're not we're like a single web server vps run on very little budget And so a lot of things like, why is prosopite on in production? Well, because that's the only way you can catch n plus one errors, which are another performance problems, because we don't have a whole staging situation that we can replay broad traffic to we're nowhere near that level of sophisticated hosting. And if we had, honestly, if we had consistently 200 hours of developer time a month hell if we had 50 hours consistently a month we could do things like run prosopite in production for a week fix those errors come back six months later and turn it on but we don't have that level of contributions so it's just on and if that's slow That's going to be slow, but less slow than having 1 plus n errors. More and more the last couple years, Rails feels like it doesn't scale down. Oh, so the reason I brought up that PR was my comment at the end here is that he said that There was a big regression in the RSS route. The new stories builder is much slower than the old ERB, but I guess that's expected. Can't build ERB with Ruby DSLs. And I tried to be polite. But the very short version is I can't reliably get ERB to produce valid HTML. And I am aware that sounds kind of silly, like, oh, I've only been working in Rails for 20 years, and I've only been making websites for 33 years, and I can't get all of my close tags to line up with my open tags. And I have an extra close tag, or I'm missing a close tag. And that basic level of construction, after 33 years, either I got dropped on the head or these tools are insufficient because I am way past. grayhatter_ why not both?
Well, let's just blame pilot error for all of these things, because if you just blame pilot error or user error. You never actually produce things that are good. So. marcoroth_ I'm also planning on building a new prototype HTML+ERB engine that's fully HTML-aware
Having this. run well i am willing to write a lot of silly code around it like heinous inline partial to compile templates in to work around a strange not strange just a surprisingly slow code path in rails i don't know i have been hopeful that your herb tools will be a little bit better or like give us some more support for things like avoiding basic errors of HTML construction. That sounds like a big project and possibly very excellent. Yeah. So I was hacking on this and I was trying to see if the

24:44If there's a good way with the Sentry API, which... Where's the darn sidebar? So I'm struggling with nonsense off screen, hold on. All right, light mode warning, folks. So this one... Yeah, we could just say a message.

25:17Or we can say capture exception, but we're not gonna have a trace back. That CSP controller just sends the body. Yeah. Okay.

26:06marcoroth_ Since we can understand the views very well now, we might even be able to do specific optimizations because of that. Or something like Phoenix LiveView-style rendering of just the delta in-between renders. But all very early on, focus is on tooling first.
Yeah Marco it is maybe worth mentioning that. lobsters does not use any of the. Basically, any of the front end features from the last decade of rails so there's no hot wire turbo prop shaft asset pipeline. I guess we use import maps now yeah that's The one thing we do use because it just means. have a very negligible build step but i haven't been especially impressed by any of the rails front end integration stuff i don't mean to speak badly of your work it just solves problems we don't really have because twitchtd I used to program phoenix liveview stuff, it's really neat but also a ton of complexity imo
the median page is like three kilobytes of html just render it server side and serve it we don't get a huge win out of well we've served you two megs of javascript if we take a whole page load you have to reparse two megs of javascript even if you know none of that i know browsers are very smart about caching now but i i hope you see where i'm coming from my own master yes

27:30marcoroth_ Fair enough, but even just the HTML+ERB tooling should be helpful
All right, so there's that for chinless. Yeah. Yeah, my big gripe with ERB has just been because it doesn't know anything about HTML and there isn't too much tooling around it, it's hard to keep it in good shape. And I'm aware that I make it harder to keep in good shape by doing stuff like heinous inline partial, but it's... Like one controller, this one controller action is over 50% of Lobster's traffic. And we are so lopsided on that that I am willing to do a fair amount of nonsense to make it zippy. Because Ruby still is not particularly fast. And even with having spent a bunch of performance time on it, dlamz Django templating is also a disaster. the community sizes for those tools are a bit rough :-/
I mean, there are not a lot of pages on the site.

28:38How do I want to say this? So it's been a minute since I've profiled in production, but I want to say typically we render pages in something like 100 to 130 milliseconds, which is just so slow for how much data we're pushing around. You know, we can go look. Let me pull this up off screen because I have it in the logs and I just need to write the mobster shared log. Anybody know JQ really well?

29:28If I say, so jq strip out one field, exclude column, I can say del, I want to see the whole, right, so little jq. So let's head the action log into jq-r del,

30:05see if I get something out of that. Okay. So I have our action log, I have our thing, and you can kind of I don't have a okay, the duration timestamp. Well, 500 milliseconds. That is just slow as heck. 1900. So we recently did a server move. If we've been running way the heck slower than the last server, we took on some big performance regression now. So this looks pretty typical, like a hit. We respond in 150 milliseconds. Yeah. So what if I say. Do this and.

31:11Give me just the path and the duration. This is not going to be right. I cannot write. Ooh, I am so bad at writing JQ invocations that I'm kind of shocked I got that right. All right, so if I said, well, and I mean, this just, These are painful. What did I do to the RSS that these are taking 900? Yeah, so we're watching the the live tail log, but. All right, a bunch of these are a little more reasonable, maybe the. You know I wonder if the head of the log was right as one of the daily crown jobs was running because i've got to look at log rotate and it would be at the top of the hour. yeah Okay, so a lot of these are having the ones that are coming out of the cash are the ones that are in the zero to 50 milliseconds range. A bunch of the rest of these, though, are. Like to load a tag page we took at. To load a single story, we took 80? I don't know.

32:58I don't want to turn into Casey Muratori, but sometimes I look at these numbers and it feels like we run slower than we should for no particular reason.

33:15So I'm kind of running down my to-do list here. So Marco, I've actually just kind of brain dumped a whole bunch of our setup at you. But if there's anything about HTML and ERB tooling you want to talk about, I am generally happy to be guinea pig or talk about why I do silly things. Because occasionally I talk about something silly I do and somebody says, yeah, you haven't need to do that for three years. And I go, really?

...46And that's wonderful. Or sometimes they say you misunderstood. marcoroth_ I'm looking at the errors and it already found a few edge cases, so this is already helpful
You never need to do that. Whoops. All right, so.

34:02Oh, great. Well, I'm glad our complicated front end and my poor maintenance have given you fertile grounds. I guess one of the nice things about lobsters is it's not very big, but it has been old enough that it's dealt with real-world complexity of both scale and years. And so we have a lot of oddities. As hard as I try to delete stuff out of the code base, it feels like a lot of what I've done in the last six months, especially here on stream, has been deleting our odd features so that we can use the new Rails stuff for them. Stats. That's going to be wide. Yeah, the views are 4,900 lines of code. And that's going to be, of course, a miscount because it doesn't know about heinous inline partial.

35:26Oh, and Marco, if you want to open bugs or fixes against the repo, I am happy to have even just basic error reports or... I love PRs. PRs are my love language. Lean towards making the...

36:11marcoroth_ I'm about to open a PR with the low-hanging fruit
Oh, wonderful. Thank you. We can give that a review on stream. I was going to do some deployment cleanup, but if somebody wants to talk code and make contributions live on stream, there goes my to-do list. All right.

...55vikrant55 !github
pushcx https://github.com/lobsters/lob…
So. Which won't actually give me an API key to let me make integration, so I will be your chat bot. I'm presuming you're wondering where I'm editing. Also welcome Vikrant.

38:16marcoroth_ There are some fun edge cases LUL
Marco, if there are any especially weird edge cases, you can just ask me about it, and I'm happy to spelunk the code history and answer questions about it. It is also possible we're just using features in a weird way. I don't know.

...41Interested in.

39:08marcoroth_ oh no, it's mostly my parser not accounting for these things, like emoji's in CSS property values
marcoroth_ about.html.erb
marcoroth_ yeah
emojis we have an emoji and a css property value oh i bet the upvote arrow yeah i changed the upvote error arrow to be a emoji fairly recently oh the lobster down in the trivia section right so that's going to be pushcx https://lobste.rs/about#trivia
Yeah, I have it on my, so for anybody who's curious, that's this. This code actually also got touched just a couple of weeks ago, I want to say by Chemlis, who figured out a way to cut down the CSS here, because we had kind of kludgy hack to have no dot on this one and then put a negative margin on the lobster and i think she found a way to just write in css that you should use the emoji instead of the yeah yeah that was so much simpler i wrote such terrible code sometimes of it works but it misses something like that In my defense, I am pretty sure that list style did not let you put in an emoji for the first decade, which is when I learned CSS. All right.

41:11pushcx https://github.com/lobsters/lob…
marcoroth_ One thing I noticed: there's isn't much indentation. Is that on purpose too to save bandwidth?
so there's that and then we've got redesign notifications so thomas this was your place where you were active so let's throw this over here and get the windows sorry

...56So, Thomas, I tossed you. Oh, you already responded. You know, I also do this like, you know, I'm probably going to only run the script once. It's okay if it takes 21 hours. If you're... Generate the votes. You know, taking a backup is a lot smarter I would not have thought to do that I would just end up wasting days by like. Four days in writing to start again. No Marco so.

42:41it's funny we were just talking about this in the irc room this morning but we don't do any kind of minification step because it's such a negligible improvement if you have gzip or z standard encoding i don't twitchtd I've already used the backups once cause I didn't know a particular behavior of lobster's activerecord callbacks
When you say isn't much indentation, I'm not sure what you mean because, and I just kind of pulled this one off the top of my head. Oh, look, is that a tab? Where's my linter? Yeah, there's, aside from the, unless you mean that it's using the two space indentation, that's about it. marcoroth_ in some files a lot of tags are on the same level, even if they should be indent one more level
But did I let other tabs slip in? That's literally the only tab in the front end. We'll fix that.

43:47No, don't do that. Oh, okay. marcoroth_ maybe it was also done to limit the git diff
I did manage to run JJ new. Oh no. After I bumped the bookmark. So I'm still getting the hang of jujitsu. Marco, maybe you're looking at about, there is a, So there is one style thing where at views about, and I know it's here. A lot of the times when we have P tags, we don't indent their contents. And the about page is the only place I have tried to minimize git diff because it has a lot of, how do I say this without sounding silly? A big problem that all communities have is understanding themselves and where things came from. And so if I have diffs that are just formatting diffs, especially if we do them incrementally, they make it harder for people to read the git blame. And I know this is like a fix your tools kind of thing, but people don't. And I care a lot about that marginal ability To track down when a sentence was added or deleted from the guidelines that kind of thing.

45:33marcoroth_ that makes sense, thanks for the context
Anyways, Thomas, I do hope that the transaction invalidations might help. Good luck. I'd be really curious to see your script. Not that I need to critique it, just you're doing something interesting. We talked about how I'd like the fake data task to create more data or have the option to create more data. so if you get to a point where you feel like that the data generation script that you're working on here is shareable please do even if it's just you know you paste it into a comment or attach a file that would be nice to have yeah yeah when i wrote this there's so much of this code is like You can't nest heinous inline partials. twitchtd https://gist.github.com/thomasd… it's still in progress
You can't have something that is a parent template and a child template in different places. I tried to keep it simple to avoid foot gunning. Oh, a gist. Thank you. I'm curious.

47:17twitchtd yup
So it looks like you're adapting this from the fake data task, right? This is in the same order as that other script, roughly. And I can see one or two of the strategies it uses, yeah.

...52think i mentioned it in the other issue to you but one of the rules of thumb for user behavior is everything is log normal distributed and so the i am probably just beating a point to death for you but for anybody else a lot of the times we do things like in the fake data we say well grab a random existing comment and reply to it so that's going to be uniformly distributed if we have a thousand comments there is a one in a thousand chance that each comment gets a reply everything users do is log normal approximately and so the comment that has the most replies has like a hundred replies and then the median number of replies is zero which is not at all a uniform distribution and everything is like this once you get into user behavior the user who writes the most comments submits the most stories receives the most votes or flags or words in a comment or hrs i could name you the user who uses the hr tag most on lobsters Because there is one specific person who does it 10 times more than everybody else. And this isn't to pick on that person. This is everything in user behavior is log normal distributed. There is no normal, median, average, crash test dummy, median human. grayhatter_ it's more likely natural log
Everyone is odd in a particular way, especially once you start looking at the idea of most or least.

49:42Yeah, you know, you're seeing the limits of my math knowledge. It is totally possible it's natural log instead of log normal. grayhatter_ they're so close, I'm just being a pedant
I just think of it as that one curve. I could totally be misusing the basic stats because I taught myself basic stats. I never taught a course on it. grayhatter_ :D
Yeah, I've picked up on your hobbies there, gray header. Just a little teasing. grayhatter_ so has lobsters
No, actually, the only thing I noticed in how you express yourself is you did it earlier this conversation when we were talking about CSP reports. grayhatter_ blunt?
there's a thing in how you express your opinion and experience where instead of asking a question, so that's how you think of it.

50:52grayhatter_ and you want to say I'm wrong?
Let me grab what you wrote. So you wrote, you can just enforce without report. The report option makes no sense. So you're doing, It's not that I want to say you're wrong. I'm saying you are suboptimal. grayhatter_ yes, I've decided I don't want to be
You can be a lot more influential and rhetorically persuasive if you would quit foot gunning your rhetoric.

51:21Ah. So you've decided to fail?

...37Like at that point, if you've decided to not be rhetorically effective, why write comments?

52:01See this? And log rotate I don't know why log rotate is the unit utility that fights me the most.

...15We have this but something in. The hatch box log rotate is fighting us so that's.

...44All right. So hatch boxes log rotate has this rotate seven and daily. And we copy our log rotate config in.

53:19We do not. Why is it? Oh, okay. Wait. So we have it. It must be this one is ours, right? Just looking at the file name. grayhatter_ no, I know how to apply social engineering tactics to get people to feel and behave the way I want... I choose not to because when I see other people do it, it feels very manipulative to me and I hate it. I choose to behave the way I would want others to behave because I know how lonely it feels on this side, and I don't mind my blunt nature annoying people who can't tolerate it, because one day, I'll meet another one of me and they'll be happy to talk to someone who's willing to be blunt
So the root deploy script.

...41copies it. But the gist of it is so that I got distracted by there's the small bug that we left it named log rotate, but that's still alphabetically after hatch box log rotate. grayhatter_ I'll point out, you're assuming bad faith with the way you asked that question
Even if we didn't remember to call it lobsters. And I thought log rotate, let me tease an ASCII medical order. I'm not assuming bad faith.

54:31I'm saying you're falling into a trap that's very familiar. And I don't just say that as a moderator, I'm saying that as an author. because I used to write like that for a while, where it was like, I feel like I'm so right that the less time I spend explaining how right I am makes my statements more emphatic. And it just actually made them irritating. And it made me an irritating author to read. And I didn't run into other people with communication styles and suddenly mesh with them They just thought they were being blunt in different ways. grayhatter_ you've read the way I write publicly, did you get that impression from my post? or just in casual conversations?
And so we would get into really tedious time wasting flame wars because neither of us could communicate effectively. I see it in your conversations here, but then also in IRC and also on lobsters itself. I don't know that I've seen you do it in comments on the repo. I think I've only seen you there once or twice. Like one of the things you do that's a very small thing is you write very universally about things where you say, yeah, like that's kind of building on your example here of The report option makes no sense in regards to CSP.

56:20grayhatter_ yeah, that one I regret, because I didn't include report-only
And you wrote so universally that your comment wasn't correct. And so you may pride yourself on being blunt, but you're also priding yourself on being so blunt that you've lost the nuance. Yeah. But this is not a, like, just once. This is a lot of topics. grayhatter_ no, I didn't convey the nuance, I didn't lose it
I see you state a universal opinion, especially that's really dismissive of my experience is universal, or I have so much more experience with this, I don't need to hear what anybody else says. And you never explicitly say those things. But that's kind of the gist of always speaking bluntly. Hmm.

57:15grayhatter_ again, that's an assumption of bad faith
Yes, he could you break that down for me because I'm not I actually am not trying to assume bad faith. I'm not saying you're doing it to wind people up or irritate them or frustrate them. I don't know that I could guess why you do it except that It's a pattern I've seen before in people and it's had a few expressions.

...59Global options do not affect proceeding includes.

58:26grayhatter_ I don't think you're trying to do it, I'm saying you're doing it... you said, (please correct me) the way you behave is such that it feels like you don't think others have anything to contribute
grayhatter_ did I misunderstand?
I couldn't tell you the exact words I said, but that's kind of the gist of it. Yeah, when I say your comments are often dismissive, that is the gist of what I'm trying to say, even though I can't recall the exact words I just said out loud two seconds ago.

...52grayhatter_ sure, can you identify any specific thing that proves that assumption?
Yeah, see... Wait, most of the files are in alphabetical? Those which are not regular files, okay.

59:07grayhatter_ irreverent != dismissive
So something odd is happening here with log rotate where it is probably including those files.

...26grayhatter_ or is it just an assumption that people most people who are dismissive behave that way, because I also behave that way I must be dismissive
I think the big difference there is being dismissive is about speaking universally and saying not just that something is not wrong, but that it's not worth paying attention to, that it was totally thoughtless. And irreverence, humor requires trust and rapport to understand that somebody is being silly. They are saying something that is not explicitly true. And that's a shared experience you can't have with someone unless they trust you.

01:00:19Did I just get the path wrong maybe?

...27manateemon Your rationality and approach to discussions are inspiring.
No, I copied and pasted.

...50I have no idea what helix is. Can you share a link? Yeah, so either our file must be totally ignored. because even the features that aren't overrides are getting ignored.

01:01:45Sounds good, Thomas.

...54So I grab that Helix Editor link. As I look at this, I realize I think I have actually heard of this. I just sure didn't recognize it. I don't know. I live in old Vim. I couldn't get NeoVim working for me. So probably I'm not in a hurry to replace it with something else. Marco, you are welcome to drop by twice a week and hijack my stream with bug fixes and feature requests. I would not complain once about it.

01:02:49Oh, and welcome to Podota and Scott B, neither of whom I recognize. So let's see. Yes. Oh, you've expanded this since I last looked at this repo. Great.

01:03:27So I'm really curious as I'm skimming these, how many are going to be like, you know, minor white space error, not a big deal. The invitations view also not too big. So gray header, one of the ways that trust in humor is really apparent is things like sarcasm, where in sarcasm, you state almost the opposite of what you believe but you have to be able to expect that your reader will know you're being sarcastic there's sort of it riffs into i think it's pose law should be posters law right yeah that in part because so poe's law i think is a little imprecise here because it says without a clear indicator of the author's intent product or sarcastic expressions of extreme views can be mistaken for a sincere expression of those and poe wrote about as if there could be an emoji for sarcasm you know and we've kind of settled on a oh yeah sorry i didn't give the dark mode light mode warning Can I save that? Like now, since Poe has written this 20 years ago, we have like slash S. I see that on the tail of some lobster or not lobsters, although it's showed up there, Reddit comments especially. But the other part of this is this even happens offline when people are talking face to face. You either make a very big expression to explain sarcasm, but you have to trust that someone knows you're being sarcastic. They have to know enough about you and your personality and your views to understand when you are going for humor. I don't know.

01:05:50Yeah, that's some pretty heavy stuff there, Grey Hatter. And I'm not trying to tell you your life is wrong. Hmm. I'm trying to think of what's useful to you here.

01:06:15Oh, yeah. Podota, it sounds like you've had the... the difficulty I'm trying to get at by talking about Poe's Law.

...35That's a recent typo.

...46You know, it's interesting how many of these are paragraph tags.

...57And I wonder if they were... I wonder if it's just our style of not indenting text in Ps biting us.

01:07:25So I think this idea of not being responsible, however, is received the message only how you send. I see where you're going with that, but it's phrased so universally that I have to disagree with it because at that point, I would wonder what communication even is. Hey, Melnath. Yeah, and we have a Rails core team member who has dropped by to find bugs in our templates even, which is really neat. Huh. Oh, that's just a... I mean that's nicer style but that's not technically wrong.

01:08:38So many of these are key tags all right well i'll take them oh you're not excuse me Marco I assumed you've done so much stuff with it's hot wire right. I don't actually know too much about how the Ruby and Rails core teams work. I just sort of consume things happily. I should have clicked start on that when I started reading this. But this looks great. Thank you for this. And I'm going to, if you're actually here watching, this is going to be very redundant, but I leave a comment because more people will see this comment than watch the stream.

01:09:56Yeah, so it's worth mentioning I've watched all of your talks of the last year or two. I watch a lot of Ruby talks on rubyevents.dev or .org. I thought it was, I would have guessed dev.

01:10:31Ah, that must be what I'm misremembering. Yeah, there's a, there is a significant, like at least single digit percentage of this stream where I'm confused about something and it changed like six years ago and I'm just out of date. We do it all the time with Rails features and HTML itself. So doing it on particular websites is not too shocking. And I would like to see the, The see I go green I don't expect any of these to fail. that's why we run the build because my expectations are often violated. So we are, I was planning on this stream being 60 to 90 minutes and we're at around. 65 now so i'm going to head towards winding down so greg hatter you and i are hitting some fairly heavy topics can i ask you to message these to me on irc and we can pick it up there because i don't want to start talking about it and then blow you off but i do have to wind down the stream before too terribly long or email or dm on the site if that's easier

01:12:00Hello, sir. The cat has stolen a pen off my desk and is fighting it around the desk leg. You can't see him because I can't actually get the camera down there, but you may hear the meows. Because when he's a very fierce hunter, he has to loudly scream about it.

...25The issue...

...38Speaking of markup.

01:13:11hunger hunting i don't know the phrase hunger hunting he he gets breakfast at nine and dinner at six we actually have an alarm clock for him because he was doing the thing where he was trying to wake us up earlier and earlier and earlier because the sooner the humans are out of bed the sooner he gets breakfast right so we have a Let me make sure that the desktop, where is it? YouTube Sam song. We found a, ah, if I walked over to his food bowl, I guarantee you there's food in it. No, Marco, it is absolutely not code golf. this one. We grabbed an old phone in a drawer and used it to make an alarm clock for the cat. So he is very well attuned to this exact song. And if I were to play it, he would leap up and run over to his food bowl because that would mean that it is time for a meal. But he doesn't bolt his food. He tends to graze. So, no. I wonder, so my usual thing is, like, if I show up at YouTube with a blank profile, how many videos down before I go into, like, flat earth? And it's especially funny when you do it on something, like, totally not emotionally loaded. Yes, he is an indoor only cat. Yeah, so we're into... magic music for negative energy. Okay. Oh, you know, I looked up the Captain Jack thing. I actually have a cookie, so it's not quite a clear experiment.

01:15:32Yeah, great, Hatter. Go ahead and throw it in, but let's see. So I've got this PR and I've touched hatch box. Not at all, but that's all right.

...47I don't need to squash. I can just throw that on top. Great. And then get fetch. Good.

01:16:10Zap. There goes the deploy. This log rotate thing is really irksome. Because we exactly match. Could it be a permissions thing?

...27No, it's named root root. Huh. Odd that the Hatchbox one has no read.

...40Greyhatter, that seems fairly plausible. That's one of those things you can accidentally... An organization can, you know... What is it? Goodhart's Law itself into doing stuff like that, where... It comes up with a simple metric of, I don't know, watch minutes or something. And then it promotes really dumb or offensive or outrageous things. Facebook. Oh, man. There's that Facebook book I regularly recommend. Oh, what's the title of it? Broken Code. Yeah, there we go.

01:17:29this book is about facebook just repeatedly doing that kind of thing to itself just constantly coming up with the most naive metrics for things and having horrific second order effects which go up to and including genocide hmm So the conspiracy theory is that it is not incompetence. It is actually malice masquerading as incompetence.

01:18:20Oh, you know, Mark, if you're still here,

...32I do hope you write a formatter that is as opinionated and option-free as standard Arby. One of my biggest changes in styling the last couple years One of my changes in code opinions is boy, do I want a formatter like Python's black or JS standard or Ruby standard RB that just, just format it and don't have options. I disagree with some of the options, but none of them seem worth having an argument over intending.

01:19:29yeah all right so i would really like to make progress on this irksome log rotate thing i guess my next debugging step would be what if i so every time i deploy which that just finished 26 seconds. Every time I see those. I'm so happy. And it would be nice if it was 2.6 seconds, but boy, will I take 26 over 260 or the 2,600 that Ansible took. I am really loving JJ. Jujutsu is really thoughtful about UI design because i've gotten comfortable with git's quirks and i understand its data model and then every time i show someone git i spend like hours justifying weird terminal things from the 70s or unix file later issues from the 80s and i would rather not do any of that and jj it's got a lot of thoughtful stuff and the op log is

01:21:00So it took me a minute, a couple of days to get comfortable with sort of reversing the workflow from do the work, git commit to create a commit and then do the work. And then you can describe it anywhere in there. That's a little odd, but then otherwise everything has just been an upgrade over the Git version, especially the Oplog where I think the first time I did a merge or a rebase, I goofed it and it took me, like I ran JJ undo and it was like, great, you undo. And I was like, wonderful. And then I tried to do JJ undo again. And it was like, okay, now you are redoing your undo. Maybe you want to look at the op log. And I was like, oh no, I am, this is going to be their version of the get ref log. I'm about to go to hell. And instead I ran this command and it just took me 30 seconds to work out exactly which change I wanted to roll back to. And I could apply it and I lost nothing. And it was so obvious and pleasant. I don't know. I'm really impressed by jujitsu. It's... it's not just technically nice it's clear they care a lot about the users and a lot about the use cases and they have written something for people to use hmm maybe there's a metaphor there greyhatter for what's the ui like between jj and git

01:22:47Marco, and I don't mean to ignore your previous comment there, Marco, I can understand why you might want to make two, but I don't know, that seems like that seems like much more than twice the pain for not as much benefit. Because The thing that is neat about Ruby standard, do I have it here? Yeah, it's, no, yes. The most interesting thing about the standard gem is not any particular line of code or the way they've organized the code base. It is running the issues and the pull requests really well and managing a community that can have productive debates around style rules. And so if you maintain two tools, it feels like to me, I don't mean to downplay the work of writing herb or a linter or a formatter, yeah and that that's certainly true on the other hand that also was the faster way to write any of those if the linter and formatter had not existed we would just be in the situation of the golang fmt tool which i guess people say format i don't know i don't really know golang as much as i just hacked on it last week But in that case, they did not make a configurable tool. They were like, hey, that's it. We're going to format the code. It's going to ship with the language. And I think that was a great decision.

01:24:59Podota, I have no idea. Probably someone in here does.

01:25:07So that actually is something I meant to bring up here on office hours. So I will at least stick it in the scratch, but... Zig format, okay, same guy.

...30So we've switched over to using Caddy in production now that we're deploying with Hatchbox, because that's what it ships with. i am very glad to have that decision from somebody who knows more about rails deployment than i do and i hacked up this tool that is by someone who is i think they're like a founder of caddy maybe the person who started caddy or just They're heavily involved. If anybody knows who M. Holt is, they said something on their profile that made me think. M. Holt. Author of the Caddy web server. Okay, yes. So presumably he created the project. So he wrote this module for Caddy. to basically serve files out of a SQLite database, and we do full page caching. And it would be an improvement for us if we could do this. So if anybody knows Go, especially if you know Caddy, I would love a read through of this code, because I have just sort of not knowing Go, cludged it into working the way we would need a read-through cache to work, where if the database goes away or gets rotated out or a dozen other things, it just returns nil and then recreates the database connection on the next load. because I would like our full page caching to save to a database because we'd have less file thrashing. Our job for expiring the page cache has occasionally caused performance issues, and it certainly leaves empty directories behind. So anyways, if you know Go, and I think I ran Go format so that this would be formatted correctly.

01:28:13solid cache is what enables this marco well no the the related work done to sqlite over the last eight nine months is what enables this but this is if you look we have where's it configured why am i here instead of

...53Yeah. So there is a gem that was extracted from Rails called action pack page caching. It's like one line of config, but when, or no, there's more.

01:29:20When we have, Here it is. When we serve pages, if nobody is logged in and they don't have filters, when we render it, we save that full page. So this is the reason Lobster's hasn't had any pain with aggressive AI scrapers that are badly behaved in the last year is because, I don't know, it's been a while since I've checked, but like rule of thumb, Something north of 90% of our traffic gets served out of this page cache, and it is Very, very cheap. Yeah, Melaneth, if you click through to the page vac, it is in Consolata plus Operator Sans Mono. And the value of this over solid cache is Rails is not involved in something like 80 to 90% of our hits. Caddy sees the file on disk, Caddy Pre. Where is it? Yeah. So here it says, okay, so if you're coming to lobsters, you don't have a cookie, and the file exists on disk, just give you that file. Right, so you heard me griping that I couldn't get Rails hit times down under, really under 60 milliseconds. We did see a few things in the live log there that were lower, But just as a practical matter, there's only so fast you can get rails where caddy is basically like and map that file to the socket done, these are so cheap to serve so wonderfully cheap and. I don't know why full page caching has fallen out of favor with. The rails ecosystem. This is also true of Laravel and Django. But if you have a site that is read heavy, and boy, is Lobster's read heavy, this is such a ridiculous performance improvement.

01:31:51So if I could tweak this caddy config to pull in full page caches out of SQLite, Yeah, but I want to say it happened earlier. It was just this idea that everything will be served by this one web framework rather than... I sort of feel like maybe it happened just because you have to be able to write the web server config of some kind. And this was originally for Nginx. I ported it over with Hunter's help. to caddy. You can write versions of this in pretty much any web server, but you do have to be able to write some version of this. I think Rails lost it because Heroku was so universal for hosting for a couple of years there in the late 2000s, early teens, maybe like 09 to 14, 15, somewhere in there, everybody was on Heroku and they have ephemeral hosts. But they're not so ephemeral that this wouldn't work like nobody's business. But I'm pretty sure that Heroku didn't let you tweak the web server config. So that is a pretty strict limit. I have, when we were seriously talking about getting off of Ansible before the ship totally went under the sea, We looked at things like Render and Heroku and others, and none of them would let us do full page caching like this. I don't know, maybe if I get the caddy plugin, not plugin, module, caddy module working, because you have to recompile the whole dang thing.

01:33:59We can bring it back, get back to the old ways. There is nothing like serving files off disk. Scales like nobody's business. Speaking of scaling, I am going to go scale myself a late lunch and wind down this stream. So the next scheduled stream is going to be Thursday morning at 9 a.m. Chicago time. yes there is nothing on the calendar cool and i will probably resume by fighting log rotate some more but if you want to put that off you can submit more pull requests oh thomas that's great thank you i'm really looking forward to what you do with notifications I feel like that very vague issue is in good hands with you. Thank you. Issue feature request? It's kind of both. Yeah. All right. Hope to see you all on Thursday. Take care, folks.