there is no gift someone can give that obliges them to give more gifts

02:51I dearly hope the microphone is not picking up the sound of my neighbor schnuffling. We got some nice snow here in Chicago, so... It is a winter wonderland. Not, you know, New Orleans ten inches of snow, but two or three. Oh, shoot, you know, I meant to most the, eh, I don't need to have a going live now kind of thing. That's fine. Usually I post a note on Blue Sky and Mastodon when I'm actually sitting down at the stream. I posted that I was two minutes away. That's good. So, hello. Good morning. I am streaming Lobster's Office Hours. This is Lobster's, a website about computing. Lots of discussion, bunch of fun links. You can read all kinds of things. There's a lot of serendipity here of reading interesting stuff you would never have thought to look up. And Codebase is open source. If you have any questions about the Codebase, which is a nice Rails app, or about the website, which is a nice community, feel free to pop in and ask any time. When folks don't have questions, work on the code base i also work on the community so like lately that has been a lot of the uk online safety act just kind of making sure that's not a threat to the existence of the site because that's what it's doing i want to Playing the code base a little bit. I'm still organizing my thoughts about the Online Safety Act, but I have some updates from last stream, and I want to write a comment about it, and we'll see how that goes. But the first thing I wanted to talk about is actually here on the homepage. This post by Beirut. So if you have been on the stream for a while, you may remember there was a stream where, let's find it. pushcx https://push.cx/stream/2024-08-…
Did I yeah there we go, so I will throw the link in the chat here. Just for the archive and all that but john busier here is a rails core contributor we had a. What looked like a memory leak in lobsters when we enabled why jet, which is a. let's just call it a big performance improvement to Rails apps. It is a more general change to the Ruby compiler, but for our purposes, it's just Rails go faster. So we ran into this memory leak and he came on the stream and we figured out, he figured out, obviously, not so much me, that it was memory fragmentation and we fixed it. It was a very fun session. And then he just wrote this post about YJIT. And in there, he's talked about, has a bunch of links to how people have seen a 15 to 25% speed up, depending on the workload. And one of the things that he talks a bunch about is whether wide joint performance, because it does significantly improve Ruby performance, really matters if Rails apps spend all their time waiting on IO. And so he notes that on benchmarks that are tailored towards JIT, it speeds up Ruby by two or three X, which is stunning to me. On more realistic benchmarks like Lobster's, it's more around 1.7x. I should throw this link in the chat in case anybody wants to follow it. pushcx https://byroot.github.io/ruby/p…
Like, obviously, it's on the homepage. But he linked to it here. But up at the top here, a year or so ago, they grabbed the Lobster's open source code base and used it as part of their performance suite for evaluating YJIT. This post by the recently lost Noah Gibbs. God, he was a very kind and generous person.

08:01Yeah. God, those sneak up on you sometimes. So...

...16So... Sean said, Oh, it's enough to me to consider that most rails applications aren't IO bound. Because if they weren't, then if they were IO bound, they wouldn't really get any performance improvement out of yjit. And someone posted it to lobsters here, of course, but I had already seen it by the time I logged in this morning, because you know, when I was doing my morning scrolling, I was looking at blue sky. And He linked to his post and someone who I don't actually know this person, I would assume they're involved in Ruby and Rails performance because they went immediately deep into the weeds of, do we know what these numbers mean? And can we kind of allocate performance between how much time we're spending in the database, waiting on IO, on how much time we're doing calculations that yjit can improve and then if we make some reasonable assumptions about the speedup we get onto these different parts what sort of output numbers do we get and of course he's using the lobsters performance numbers I posted which are kind of neat and then there's some mention of what the conditions are and then Sean says that this figure still contains some IO, just less because it was migrated from MariaDB to SQLite. So, you know, hitting the local disk is going to be faster than waiting on the network. can't say the two figures are perfectly comparable because the benchmark focus arbitrarily on some endpoints so it's not necessarily representative actual live traffic on lobsters oh it's funny when i'm logged in i actually see the whole tree of responses so it's hard to tell the exact io ratio but you definitely got a point that is why it gets better the io ratio gets proportionally higher and then i kind of like jumped into the conversation of like do you want to just measure pride like you don't have to theorize about this split you could just measure the actual split between IO and CPU time. And we could even turn Y jet on and off, which is a thing that we did while chasing the memory fragmentation. One of the neat things about lobsters that I said on that last stream with John is we're in this very odd situation where we have lots of real world traffic, but Nobody's going to get too bent out of shape if it runs a little slower in prod for a while. We're not a business that's losing money. We're not violating an uptime guarantee of some kind. Oh, hidden that. We're going to come back to that.

11:17And then, yeah, this person who was kind of theorizing out the numbers also was like, well, yeah, I guess we could just finish here. I don't know. The whole interaction was funny because there's a very familiar kind of nerd disagreement where people have dueling assumptions and kind of build castles in the sky and have very long and specific arguments about different points of this, but this is an empirical question that we can just measure. So, kind of funny to cut that conversation off. And it'll be neat if we get to contribute some more numbers back to Rails.

12:11So then, before I jump into pull request review, which is another usual staple of the stream, I wanted to talk about, not my issue, This comment by Rahul1990gupta, who has recently been contributing quite a lot to Lobsters.

...40And I don't know, I would like to try and drag him on stream sometime because he has really leapt in with both feet and opened a whole bunch of nice pull requests and has been very kind about when I have very strong opinions about wanting code or error messages to be a particular way so he's been a great contributor this last what month or so six weeks, something like that. and We have this old issue hanging out, let me show you the top of it, it used to be called migrate to postgresql with a question mark where. Time flies. What is that? So six and a half years ago, I was like, yeah, you know, we keep kicking around this idea of should we use PostgreSQL instead of MariaDB? And the backstory there is that JCS, who started the site, was most familiar with MySQL. And so he started with that. And then after he handed off the site to us, we ran MariaDB because we were more familiar with it just rahul1990gupta Hi Peter. I just dropped in to say hi
a couple of people who were doing DevOps at the time. Hey, Rahul, thanks for dropping in. I was just talking about you. I was saying what a kind and generous and tall and rich person you must be. Thanks for dropping in. I've really enjoyed your code. And I have on my to-do list to email you about the idea of doing a video call or just otherwise chatting. because I've really appreciated your many contributions. If you have any kind of questions that you want to talk through, so this is your first time on the stream here, usually I talk through pull requests and I do reviews live on stream. If there's anything specific here you'd like me to address, I would really like to use this time especially to help you because you've been such a great contributor lately. Thank you.

14:59And then otherwise, you know, I kind of talk through decision-making and stuff that's happening on the site and I write code. But seriously, if you have any kind of questions or interests, let's dive in. So we were kicking this idea around for years and it was pretty inactive. Like there was a little burst when I first filed it and then it was quiet for years. And then a couple of days ago, Rahul, you know what, let's, I'm going to immediately, there we go. Let's mark Rahul as a channel VIP because he's been such a great contributor.

15:49So usually I mark people VIPs when they catch a bug before me on the stream, but you've made so many contributions to the code base that I feel like I should just do it right now while I'm thinking of it. Oh, there's a small stream note. Frici a torrent of accepted PRs sounds like a reasonable alternative LUL
So if we look back at the last stream archive, I was just tinkering on this because I wanted some light, fun coding. Yeah, very much so. Frici Greetings !
So actually let's... Oh, did I not? I broke it in prod? Let's redeploy. I was just working on this yesterday. So I'm off screen. This is my personal blog and it uses Jekyll. And so I have it deploying. All right, so let's reload. It must be, there we go. So let's look for Fritchie. Oh, Fritchie didn't talk on this one. Here we go. Here you are, Fritchie. So you talked about time zone appreciation, and I have now updated the channel archive to put a little gem next to your name, like the one that appears here on Twitch, because you're a VIP for catching bugs. So thanks. rahul1990gupta Appreciate your kind words. Postgres migration is on top of my mind lately. So, that's very interesting to me.
Yeah, I noticed that you really jumped into this Postgres project for us. Is that something that you work with? Actually, if you don't mind asking personal questions and note that these streams get archived, have you been doing Rails development for long? I got the impression you're fairly new to Rails. Do you have more experience with MariaDB or PostgresQL? I'm really curious about your background. But of course, since it's archived and everything, only share what you're comfortable with. So while Rahul is considering typing, I will give more context here that he woke up this issue a couple of days ago with some surprising business news. I didn't realize MariaDB had been acquired. I missed that back in September. And then ran through our list of things that PostgreSQL might be better about. And what's really interesting is some of these, a bunch of these like JSON and I believe PG vector and PG stat statements, all of this stuff is newer in the last couple of years since this issue was originally filed. So he made a big case of, yeah, actually PostgreSQL has a bunch of stuff you want, And MariaDB is headed in a direction we don't want. So I left a big comment. I will leave it to folks. pushcx https://github.com/lobsters/lob…
Oh, I didn't try and put these issue links and such in the chat for folks to make it easier to follow along and read at your own speed. And I am not, since I'm not streaming for eight hours, I don't feel the need to pad out the stream and like read every comment. But the gist of it was Rahul made a solid case to me that Yeah, we do actually want to migrate to PostgreSQL. We would get a benefit out of it. It has, yeah, you know, I try and compliment folks on stream. So I will mention, where did I put it?

19:39That PostgreSQL over the last 10 years, it's overtaken SQL, MariaDB and features and popularity. rahul1990gupta I have been in a data role (data engineering, ml)for last 10 years. New to ror. It's very fascinating to me so excited to learn and work on it
But then also, I'm really just impressed by PostgreSQL, the way it keeps growing its breadth of features. And not losing performance, reliability, or development speed. PostgreSQL is really just a hell of a project. I would love to read a book about how they're working the last 10 years. You know, not how to write SQL, how to run a project that's as successful as PostgreSQL. Oh, cool, Rahul. No wonder the database stuff has caught your attention. Well, I've really appreciated that you're jumping in. Yeah, some of your first poll request or two, I could tell that you were new to Rails, but I really appreciated your enthusiasm and that you were willing to work through any kind of bugs to get things working. And, you know, put up with my weird technical opinions. So I made a checklist of things that we ought to clean up to be ready to migrate to PostgreSQL, because there are a lot of things the codebase does where either for historical reasons or developer convenience, we just drop to raw SQL and The spoiler alert is that Rahul has already jumped on a couple of these items. So that's 1437. So let's put those in here. This one is 1437. Yes. And then what was your... So I haven't even looked at this one yet. Peeked at the other one in my email. Oh, is this all of our, yeah, this is all of the ID desk kind of stuff. I thought it might be. Okay. So this is 1441. Cool. So I will add those references to show that they are in progress. So I've been meaning to... I linked our attitude towards minimizing dependencies. I should write this up in the contribution doc or in the readme, but it's really been a change to my development style for Rails apps the last 10 years. Maintaining this code base and seeing the value of being very strict about keeping dependencies few and small. And one of the nice things about PostgreSQL is, I mean, you can't call it small, but it replaces a whole bunch of other stuff that we might get tempted to use. Like, the site for a while had, shoot, what's the name of that? Elasticsearch? It's a whole database search service. It was hard to keep running, and eventually we dropped it. I say maybe 10 years ago we dropped it. And then search was pretty bad for a couple of years. And then I re-implemented it two or three years ago, and now it's not great, but it's okay. And it's in a state where it's improvable because it's not just a giant pile of potential SQL injection. So, Rahul, what's the... this 1441 ready to go it's hard to tell because there are so many of these that i could see this being a work in progress or i could see all of them and you didn't one thing that would be helpful in your commits or i'm sorry in your pull request is if you left a comment saying when they're ready to review or what their state is because sometimes you push partial work and sometimes you push Completed work and it would be nice if you called out which was which All right, so I got this Yeah, this is the kind of there are just so many of these little things and These are such standard sql 99 syntax that they're fine, but the more of them we can get rid of better.

24:56We if you include tags, I never remember this API is includes the same as well, you know, if the if PersoPyte didn't blow up any tests, this is probably fine.

25:21rahul1990gupta This is ready to review. I have been replying on github's "draft pull request" and "pull request"
This could be. This is ready to review okay yeah i've seen you are the first person who's done that draft pull request and full pull request against our repo it's a fairly new. feature of github. I think it's.

26:07rahul1990gupta ddraft -> WIP while regular pull request is ready to review. is my understaning in line with oss community ?
Yeah, I think that's a reasonable workflow. It's just one we haven't done on Lobsters before.

...46All right, so yeah, there's just so many of these. Oh, this. So there are more of these. Where do we do it? There's somewhere in the code base that I recently introduced this.

27:16Yeah, here we go.

...52And it would be especially nice to, why am I just saying this instead of just editing it in?

28:18Less standardized.

...39There we go.

31:40rahul1990gupta I wouldn't be able to stick around much longer. It would be nice to get your thoughts in person on https://github.com/lobsters/lob… another issue with migration is some sql for search between mariadb and postgres is not matching and we don't have a way to abstract it through activerecord. this means we reply on migration flag kind of thing.
dr3ig Muted?
rahul1990gupta I am also not able to hear peter's voice
Oh shit, I'm sorry. I had to cough a minute ago and I have been unmuted since. I'm sorry. So Rahul, thank you. I'm really glad to dive into the thing you're asking about. Let me back up. So your strategy is entirely right. I think this is correct. Yes. There is a Rails feature called countercache. Here we go. So here, hold on, how do I?

32:33Here. pushcx https://guides.rubyonrails.org/…
So let me share this link. gj1118 Can I ask you a question please? are you using a usb mic or a condenser one ?
so counter cache allows you to have reply count automatically maintained by rails and this is the kind of rails feature that's newer that i would like us to lean into doing because there are places where we do this sort of thing manually and i don't want to add more manual so yes gj i am using a usb cardioid mic it is an atr it is This microphone, I'll show you the link. pushcx https://www.audio-technica.com/…
I should put this in the fact, this is like the third time it's come up.

33:24And then the other part of it is

...36gj1118 Thanks
So we got reply count and depth. Yeah, that's fine. As long as we... Those are fine. So Rahul, your general strategy of materializing those comments is sound. There is a nice Rails feature. Oh, the other one was, what is it? Is it async count? Or is it count async? Yeah, here we are.

34:13This is a new one that I'd like to, I think I mentioned it in the Rails 8 issue. Where'd it go? Oh, I didn't mention it.

...39pushcx https://api.rubyonrails.org/cla…
So there is a Rails feature available for basically counting records in the background and for your depth. This would be especially nice because when initializing a comment, we can just say in the background, set the depth to be the correct thing. It doesn't have to be something we, well, but if we don't block on it at the time of, All right, yeah, actually, never mind. That doesn't want to... This is a neat feature, but not something we want to do for depth, because then we would insert comments at a depth of zero, and people would see the page while this ran in the background, which would be bad. We already have enough comment sorting issues. And you had one more comment that's worth talking about. Where did it go?

35:43rahul1990gupta the proposed state is implemented in the PR. it ended up being a smaller change than I thought
So this confidence order path is, I see what you're saying Rahul, but you didn't use the counter cash or the, yeah, the counter cash on the association. You're doing it manually here.

36:25Thank you.

38:03rahul1990gupta yes.
rahul1990gupta I will need to look into counter_cache. new to me.
so rahul do you understand why i say this have you seen this issue with we do this in voting and i'm i'm grudgingly okay with it in voting because i put so many tests on it but it's very hard to get these correct and Even with all of the tests on it, I know we have a race condition in voting where if two people vote for the same comment at the same time, it's possible for the vote score to get out of sync. rahul1990gupta understood
And so doing it entirely in the database forces a consistency. Cool. Then I will stop explaining something you already know. So this is good. Oh yeah, I remember this one. I am so amazed by how much shorter this gets to use things correctly this way. It's lovely.

39:33So confidence order path is yes. rahul1990gupta Could go explain how we use confidence_order_path
So it's a performance issue. And it's very evil code. Where do we have it described? Yes. pushcx https://push.cx/stream/2024-09-… https://push.cx/stream/2024-09-…
i'll link you here to the two streams where i talked about these and i realize i'm giving you like five or six hours of video so i will give you the very short version the most pushcx https://push.cx/stream/2024-09-…
expensive performance thing that the site does oh no it's it's this one is the big description the others the other two are bug fixes to it because it's very hard to get this comment this code correct and it still has a subtle bug in it right now but

40:51sql doesn't work well with nesting and with recursive comment table expressions it's gotten a lot better at it but we want comments with the thread nesting so that you see the tree of replies and sql is not a good fit for that and it is something like 60% of our page views are to view comments. That's a very rough number, I'm just estimating, but it's most, it is majority of ours to view a story and see, just to pick one, this tree of comments. And we used to sort comments in ruby there was a function in the comment model that would fetch all of these comments out and then build this tree structure in ruby and it was computationally expensive and if a big site like hacker news or reddit linked to a long discussion the site would crash and so i moved all of it into the database and so it is this this kind of evil performance code where we build up a string. So we sort comments by confidence and we have to encode confidence in a string because we can't, this is a MariaDB limitation, we can't build an array in a recursive common table expression like this, but we can build up a string and then the string is lexically sorted so that comments come out of the database in the order they appear on the page because SQL will return them flat. rahul1990gupta So, move to postgres may simplify the logic by leveraging array type ?
It can't return a nested data structure. Yes, moving to Postgres would probably allow us to simplify the array. There is the question of performance. So if we could use the array, We could both fix the bug here because there is a bug that Confidence does not have enough bits to correctly sort comments 100% of the time. And if you look at those streams or skim the transcripts or look at the commits around them, because all of those streams link to the issues and pull requests, rahul1990gupta https://mariadb.com/kb/en/json_…
Replacing with a PostgreSQL array would be excellent, I think, as long as it's performant, because this is by far the most performance. This is where we spend all of our CPU time on the site, or this is on the database. JSON array. There's a limitation with this JSON array. I don't remember why I didn't use it. But I think with MariaDB, I couldn't use it here. And I don't remember what happened because it's been a few years. But I did this insane string because I couldn't use the arrays. rahul1990gupta https://guides.rubyonrails.org/…
Yeah, this would be nice. Oh, and if Active Record has nice syntax for it, that's great. This definitely... Well... It can be a problem for the performance people on their own. rahul1990gupta activerecord also supports arrays. so maybe we implement in activerecord and hope it plays nice with trilogy
I say that it can be a problem for them because they want to run their test with SQLite, but that's too bad for them. Our goal... We don't have to play nice with Trilogy. We don't have to support MariaDB. We're not maintaining support for MariaDB. And you said... I forget where you said it, but... Oh, maybe it was in the issue... that you opened that I already closed. pushcx https://github.com/lobsters/lob…
So there was this issue that Rahul opened about whether we should have a staging environment. And you said, increase confidence. We don't have to run MariaDB and PostgreSQL in production at the same time ever, period. When I deploy this, we will take downtime rather than try to make a smooth perfect cutover from one to the other. I do not expect that we will maintain both. Period. That's one of the nice things about not being a business and not making money of this. Rather than spend 20 or 30 hours of development time trying to make a seamless switch we will just take an hour or two of downtime to dump and reload the database it's fine and i appreciate your professionalism it is totally correct if you were doing this as a job but we're very limited on how much developer time we can spend on the site so let's spend that wisely are there any other big questions you have about your current pull requests or the code base or anything?

48:13This.

...40rahul1990gupta that covers all my burning issues. I will just sit back and enjoy the stream. Let others chime in.
Okay. Usually the streams are pretty quiet, especially these Thursday morning streams. So if you have any other questions or anything, just pop up. You also seem to have a lot of lag. You might try reloading the stream. If you reload the Twitch page, sometimes you will catch up. And instead of being 30 seconds behind, usually people are only three to five seconds behind. I've noticed you're replying to my questions pretty slowly.

49:16rebelelder is in lurk mode
Oh, hey Rebel Elder. Nice to see you again. Why do you have to do this?

...30Oh, I see. That's fine.

...45Yeah. So I will finish up this pull request review. I'm going to write a lot of the things I just said to you out loud into the review just so that we don't lose them.

50:07Yeah. This

52:13Hope that makes sense. All right. So, Oh, this is I'm reviewing the wrong thing.

...42So there's all of that. Oh, I guess I already left you the review. Was it the other one that was ready to go? All right. So you'll have that pretty soon. Great. rahul1990gupta at line 58 in comments_controller is only building comment comment = story.comments.build comment.comment = params[:comment].to_s comment.user = @user comment.hat = @user.wearable_hats.find_by(short_id: params[:hat_id])
So line 58 comments controller.

53:15So I have comments controller up. Comment is comments.build. I see this is line 22, it sounds like you're talking about. With the parent comment.

...55rahul1990gupta line 22
What's your question, Rahul? In the building.

54:16rahul1990gupta as we are only building, does it make sense to run update_column at this point, as suggested in the review comments
Oh, your question is probably going to be that reply count won't be valid because it won't be persisted until, or it won't be correct until this comment is persisted.

...50What part of your already closed the pull request. So this is come here, which PR this one, right? 1437. I see what you're saying. I didn't have enough context here.

55:52This is. There's a race condition here where if someone. Well, I guess it's a potential race condition because I haven't actually added the feature that I'm talking about in my review comment. But. If we set the depth here. It'll be wrong. Yeah, Rails is just, Active Record is not expressive enough to say that we want to set, we want the database to set a column to be the value, or not to be a value, but to be a SQL expression that the database interprets at the time it's inserting the comment. That's what we really want, but we can't say it in Active Record.

57:41rahul1990gupta I understood your concern about race condition. will see if i can find a way to avoid it.
I think don't worry about it. We're not so busy that it's going to be wrong. And what I would rather do is write a prod checker for it and deal with it later. Because we can reify it, or not reify it, we can... correct it with a simple update query in a background job that just says, you know, go and set the correct number of replying comments on anything, go and set the other thing, but I can just wait until it's an actual problem.

59:33So let's just add that comment and copy a link. So I'm going to put this in my notes.

...55I have to write a checker for those. If, oh, you know, Rahul, this might be interesting to you. pushcx https://recheck.dev/
What I'm referring to is the software product I'm working on is this kind of thing where rather than be perfect and spend 100 hours of development time being perfect, we are going to risk a little bit of inconsistency in part because Active Record just isn't expressive in the way we need here. We had a tool at Stripe called a checker that I am re-implementing from scratch with permission. And of course, I'm running it against the Lobster's code base, but it's a way of saying, well, if we have this count of comment.replyCount and comment.depth, we can compare those cached values against A live query to know if they are correct and in the one in the million case where they're wrong, we can escalate that to a developers attention and so for the cost of writing a small database query. We can save 100 hours of developer time trying to make that perfect and trying to avoid a theoretical race condition. And I am so far behind releasing that, but yeah. I think countercache handles it. Did I link to the wrong thing? Yes, I did. A frequent feature of this stream is that I mess up my clipboard and I need to unify them. I think in this case I just misread the chat history and selected the wrong link.

01:02:23Yeah, I don't remember the specifics of this. But I think Rails handles the issue of when the, yeah, so it does all the filling. So I think just specifying a, Countercache will be correct there. Oh, yeah, that's a note per.

01:03:21rahul1990gupta Will recheck.dev be a commercial product ?
Yes, recheck will be a open core product. I think I did I say it down here? No, I didn't get into it So I talked about it on stream because I gave a DEMO of it lobster stream. can't remember my own thing. pushcx https://push.cx/stream/2024-10-…
So I. I demoed recheck a couple of months ago and then got distracted by life and holidays and being sick and the online safety act so it hasn't been released yet. But. The plan is that it will be an open core product. So any Rails app can pick it up and use it. If you are using it in a very large commercial project where you have millions or billions of rows in your database, there are some extra features that will be worth upgrading for. But this is basically the sidekick model of It's useful for everyone, but if you are very large, there are features you will be very happy to pay for because of your size. Mike Parham, actually, the guy who created Sidekiq, has been really generous with his time and everything for me. So there's that. Yeah, so this comment can stand. And then those are fine. Those are fine. All right. That's that's everything. Did I save my review? I think so.

01:05:09Yes.

...20All right. So there's that change.

...35All right, I was looking back at this pull request to continue working through the backlog. All these nice little fixes. So Rahul, I know this must have been a fairly tedious change, like the rename you did for our time ago in Words Helper. And I really appreciate that you have been willing to take on these very simple, very boring changes. I know they're not exciting, but they are the kind of investment in the code base that keeps us maintainable long term. They are never very fun. And so I like to say very much that I really appreciate these changes you've made. So.

01:06:40There are. So ActiveRecords or syntax was added after the codebase started, I want to say. So we started with Rails 3.2. And I think or arrived in maybe Rails 4.1 or 5. But we have these littered throughout the codebase. There are more of them because my mental model of active record is very weak on or I just haven't written enough of them to feel confident, and so I introduce more of them. Because it's just faster for me. yeah this.

01:07:35This one can be.

...46So we have to nest, but then it's flags.

...58rahul1990gupta is there any advantage of using `.or()` other than this being more idiomatic ?
Just to give one more example of this practice. Yes, the advantage of using or is that when we have complicated includes or otherwise join table, So in this case, there's an or to the user table, which is fine. But when you have a join or an eager load, I want to say your other expression. So this one, on some versions of, what's the right way to say this? Rails has many approaches to combine queries into one for performance reasons. There's eager load, there's includes, there's preload, there's joins, and there's left outer joins. And sometimes when you use these, and I can't be more specific because I don't have it memorized. Sometimes when you do these, it will try and fetch all of the records in one query, or it will be forced to alias a table name. So if this said moderation dot moderator user ID, or it said tag dot tag ID, sometimes Rails will alias those join table names and then your query breaks because your query is a string An active record doesn't try to look into strings to figure out which tables you're referencing. That's a level of complexity it really can't get into. And so if we use OR and always use, you know, this hash interface for these things when we add joins or includes or eager loads later they don't blow up in surprising ways because these things it's fine and it works but months from now or years from now when the query becomes more complicated or you compose two scopes in unexpected ways string queries blow up.

01:10:36So that's why I keep wanting to get rid of these even though they're very simple is because slowly these kinds of things migrate into scopes and if our scopes are using strings that include names and I don't think there is a active record way of encoding like rahul1990gupta Noted. Thanks for explanation.
which is why I don't highlight this line. We're just making bugs down the line.

01:11:14This one. This one is a little painful. How did I handle these in the time helper?

...39I didn't handle them in the interval helper.

...57So all of these things, take the interval and then just interpolate it into strings. Yeah, I didn't design a great interface for that. And now there's more strings hanging out.

01:12:39So if you play with these, like preload might be the one that does table name rewriting.

...53There are so many of these. Here's one. Yeah. So I did some kind of subquery here.

01:13:18What are we sorting by? Oh, latest comment ID. Can I not? Oh, right. This is a GitHub limitation that I can't comment on this.

01:14:07What's the, There's an active record feature here for touching the associated parent.

...54Not countercache. Not dependent. Touch.

01:15:24Can we specify? Yeah, use. Let me be more specific.

...56I hope this makes sense.

01:16:36This one is that same thing again of saying, I think I may need parentheses around it. I don't remember.

01:17:44It's funny that it lets me add a comment on an unedited line if it's next to a changed line or two lines away from a change line, but not in the middle of the file. Sometimes the get UI is strange.

01:18:17Thank you.

01:19:02Thanks.

01:20:19So, Rahul, if you're still here, one thing that I try to be very deliberate about is when I'm reviewing pull requests, sometimes I'm making the project bigger, like these ores and especially things like what we were just talking about, this pushcx https://github.com/lobsters/lob…
migration for slash active and I try to say it every time, but I always appreciate the volunteer aspect of your code contributions. I know you never have to make them and you never have to make changes, So if I ever propose a change that's too big or that you feel is not something you want to work on, you can always say no. And I appreciate if you say no instead of abandoning a PR. I should put that in the contributing doc because it is a big part of My philosophy around open source is that people's contributions are gifts, and there is no gift that someone can give that obliges them to give more gifts.

01:22:16Maybe I'll add it to the contribution doc on stream because I wanted to write up that philosophy. All right. So I think that's all the active pull requests. Yeah. Well, it ran a little longer than I expected, but it was really nice to get to talk to you, Rahul. Thank you for dropping in. Again for anybody who's joined us rahul has been a very active contributor to lobsters the last. month or so two months. So you can see, he has five open pull requests now and he's merged five. So. Oh, this one was a merged for this one was a rename that we decided not to do but. He's been a great new contributor, and he's especially done a bunch of things related to us making better use of new features that came with Rails 8. So I'm very happy to have it. All right. So I'm looking at my topic list for the stream. Let's see, for time, we're an hour and 20 minutes in. That's pretty good time. I typically stream for about three hours. So we've talked about this, I talked about PostgreSQL that led into pull request review, there were two pull requests and I didn't know either of them. Because there's 1441.

01:24:07and recursive CTE. This one is so nice. So one thing I didn't, you know, I mentioned the serendipity of just dropping by a conversation on Blue Sky where two people were discussing Lobster's performance in the theoretical abstract. I mentioned

...41has ActiveRecord improved on its support for recursive common table expressions? We have a bunch of them. I'm a big fan. I want to add more. And then I talked a little about approach to dependencies. And then a comment that came in very quickly after, like, what was the timestamp here? So I wrote this. Come on, give me the full timestamp, GitHub. 10.33. And then six minutes later, We, the Mastodon project, contributed to this recently, and our work is upstream in Rails 8. How great is that? Like, the folks who worked on it and contributed it back to Rails happened to be reading our issue tracker, and they got to pipe in and say, oh, yeah, we added that exact thing you want. It's in Rails. And recently, too. This is only last May. So that's so lovely. Nice little open source moment.

01:25:49All right. I touched the cat, so I got to mute and sneeze for a second.

01:26:12I remembered to unmute this time. Thank you, Dreg and Rahul. So, before I jump into the OSA and that impending disaster, I will reiterate that this is Office Hours for Lobsters. If you have questions about the site or the codebase, you can ask them anytime. And I will Even if I can't immediately answer your question, I will stick it on my topic list to talk about. And when there aren't things to talk about, I work on site maintenance. Mostly that's code. Sometimes that is stuff like writing comments. Yeah, actually, let's put that on the...

01:27:09Let's just put this up here. And this up here.

...29Because I would rather spend time doing it on stream. One of the points of Lobster's office hours is not so optional. Not so obvious, but I really love working on lobsters. It wants to take over my life where I could be hacking on it all the time instead of working on my entrepreneurial project or doing other stuff. And if I have office hours with designated times and designated durations, I am at least time boxing myself a little bit. And I do especially try to look at pull requests and issue comments pretty fast because if I ignore contributors, they go away. And the faster I respond to contributors, the more likely they are to finish pull requests. We've seen on this stream that pull requests can go stale. But then otherwise, I try and keep my big lobsters development stuff to these streams. And the off stream time I do is when I'm doing maintenance stuff that I can't show on stream. Like if I have to rotate an API key in production, well, I'm not going to do that on stream because the chance that I accidentally paste the API key where folks can see would be very high. So like when I was doing the credential changes, a big chunk of that development happened offline. So let's talk about the UK's Online Safety Act. Anyone in the chat in the UK? We can ask you to call your member of parliament live on stream here. So for context, let's put that in here.

01:29:43chamlis_ cor blimey
Ah, hey Shamless, nice to see you again. Let's paste this link. pushcx https://lobste.rs/s/ukosa1
Paste this link over here in the chat for anybody who hasn't seen it. And I will bring it up. So this is a thing that has been running chamlis_ having my own personal tangles with UK government bureaucracy at the moment
for 16 days because it was delayed by me being sick over the holidays but the UK has created an online safety act a law that says they get to oh I'm so sorry I have been interacting with the UK government bureaucracy lately for this and it you know not so much fun So the Online Safety Act imposes impossible burdens on small forums like Lobster's. And I say small in the sense of it's running as a hobby project. And as I just talked about, I run it in my spare time. It is not a business. It is not the only thing I do with my life. But the Online Safety Act imposes huge burdens on websites where we are assumed to be causing harms, especially to children in 17 different areas and need to read impossible, impenetrable legalese and guidance. And I say impossible because, God, I try and be nice on these streams and I try not to be sarcastic or cutting, but the guidance from the United Kingdom government is not written for the small forums that it claims to cover. It is not written for humans to understand. It is written for the lawyers of enormous online services that it is targeted at. So like Facebook, Reddit, TikTok, I suppose GitHub, anywhere people can talk to each other. And it includes ruinous penalties and jail time.

01:32:22So let's talk about what's changed since the last stream.

...34pushcx https://www.ofcom.org.uk/online…
Ofcom is the name of... Even this survey is bad. I took it the other day. Ofcom is the name of the British regulator who is imposing the censorship rules on everyone. Because one big aspect of this is the UK has said that they have jurisdiction over the entire web, which is bizarre nonsense. I'll talk about that more later. This bulletin doesn't really say much of anything new, except if you would like to light three days on fire, you can go to their online event. I can't imagine it being useful given the way they write guidance. But the thing I wanted to highlight in this bulletin is this threat. We'll be regulating a broad range of sites and apps from the biggest tech firms to small voluntary community forums that have not been regulated before. We recognize service providers get service providers. Individuals, hobbyists, mom and pop, the people who run the London fixed gear single speed forum like bikers and it calls them service providers. We recognize service providers, particularly small and medium-sized businesses. Ignoring individuals, randos, they have no idea what they think they're regulating. We'll have lots of questions about how to comply and Ofcom's expectations. That's why we're committed to providing resources and support. Charlie Brown voice. I've run out of both in the large, because we've been looking at this for more than a month, and in the small of this paragraph, I have run out of patience and kindness for Ofcom. In part because of the interactions I've had with them over email the last couple of days. But the part to highlight, I'm so mad I'm getting distracted. The part to highlight is they are making a point of saying, yes, small independent forums, we are coming to destroy you. And I say it I am attributing malice to them because it is impossible to comply with their regulations. Anyone who says, oh, well, this is all about illegal harms. So first off, that's dumb bullshit because the illegal harms are the act making certain kinds of speech illegal. And so they are justifying making it illegal by saying it is illegal. It's tenditious and dishonest. And their guidance is garbage. It is not written for people. It is written by and for lawyers. And even the lawyers I have talked to are exasperated by how badly written it is. Because I've now talked to a few. And then they talk about having tools. Is this the new tool? I think it's my next link. Their tools are utter garbage. Yeah, this is the new one. So their old tool says things like, use this tool to figure out if you're a user to user service. Question one, are you a user to user service? pushcx https://www.ofcom.org.uk/os-too…
like if you can't say something nice and i can't say anything nice about those tools so they have this new tool and anyone who thinks that ofcom's guidance makes sense or it is reasonably sized i encourage you to try to use their tool that is I have been saying that these are needlessly burdensome, badly explained, vague regulations that claim incredible power and impose enormous burdens on small forums. And this new tool is the most convincing case for that judgment that I could make is if you were to actually try to follow these steps, you would be convinced of every critical thing people have been saying about the Online Safety Act.

01:37:45So then the third thing I wanted to say about Ofcom was that I talked to them. I sent them an email. I think I mentioned it down here. I've actually, so I had someone introduce me to Ofcom because, oh yeah, I actually got to do something that wasn't Ofcom crap.

01:38:17So I talked to someone who worked at a large site working on OSA compliance. They made an email introduction to me to current Ofcom employees so I could directly discuss concerns with the regulator. Nothing came of this. They completely blew me off. I sent a couple of follow-ups. Nothing. Zip. Second, I started another email conversation with Ofcom. So I talked with the owner of a US small non-commercial forum who asked Ofcom about the OSA. They received back a long demand letter that all services, no matter how small, have to undertake the enormous task of evaluating against their thousand pages of vague guidance. I say thousands because I've totaled it up and it's north of 3000 pages, including the Online Safety Act.

01:39:19And after some routing, I got a response from the online, from Ofcom. So I didn't bother asking, are you kidding me? This stuff is nonsense for small forums because I saw the garbage that they sent this other person. And I only focused on the jurisdictional question of why do you think that you can regulate an American site in America? I have seen that the law says that you have chamlis_ Tim Berners-Lee invented the web, so it belongs to the UK
power but I need you to explain the legal basis for thinking you actually have power over the entire world because the way the world works is country make countries make laws for themselves and they can choose to get involved with treaties but there hasn't been one to my knowledge that says the United Kingdom is in charge of the entire internet Oh yeah, Shamless. Well, he did, but wasn't he working in Switzerland at CERN at the time?

01:40:40I mean, the way the Online Safety Act is written is like, if you can imagine a person in the UK reading your website, we have power over you. So maybe because Tim Berners-Lee could see a book that was written in English, england owns the entire web i say england instead of uk sorry i'm trying to be careful about this but like i don't really know much about uk politics and structures i've talked to a bunch of people in the uk and like i got this weird mastodon response recently where someone was talking about working from home in relation to the law but it turns out they have some kind of part of the government that they call home office and so they're not talking about home offices they're talking about this government department it was very confusing so i got a reply from offcom and it was garbage i said i've seen what's in the law what allows the law this ridiculous scope And they said, well, it's in the law. And I'm being uncharitable and summarizing it because it was written by a PR flack who took 300 words to say nothing. So that's a false start. I mentioned it in this comment, but I have reached out to the US Embassy because, hey, we have diplomats for this sort of thing. No response from them yet. And then let's grab that link because I'm going to write an update.

01:42:47so let's yeah you know what i'm going to just write this over here and paste it in because my i don't want to drag in my personal browser on stream that never works out so that's that's story merging down there

01:43:47the osa names jurisdiction over the entire web or the law on the legal basis they had for a country's law being able to do so chamlis_ All the planning charts and demolition orders have been on display at your local planning department in Alpha Centauri for fifty of your Earth years
APR flag responded. Yeah, I have thought a lot about that bit. You know, beware of the leopard in Hitchhiker's Guide to the Galaxy. Yeah, that's come in my mind quite a bit. chamlis_ hehe
Because so I read Hitchhiker's Guide when I was maybe when I was 12 or 13. And I had thought those bits about talking to government bureaucracy were satire. I didn't realize it was a documentary.

01:47:03So the other thing is I've gotten pointed to some members of... Members is capitalized, right? Chambliss, they're capital M members? Like that's their title?

...30chamlis_ errr, possibly?
All right, well, we'll capitalize it. I was trying to figure out who and so I was talking to someone from the UK and I was like trying to say like who was involved in this legislation like who drafted it who introduced it, who should I talk to about the ridiculous jurisdictional claim. And I couldn't even ask that question because, like I can ask it, I mean I can answer it for American lawmakers and I know exactly how to find that person because i've done this well. You look up who the bills author and introducer are, and you read the bill, blah, blah. And then you go find their, the members office number and you call and you ask for the legislative aid. And then you talk to the, the lawyer that is on that Congress members team who drafts these sorts of things. And then you get direct questions and I couldn't even figure out like, well, They're obviously not called Congress members. They're probably not called legislative aides. I don't know how to figure out who introduced a bill. God, I tried to find it, and there was someone with an incredibly British name, and it was like Lord Birthington IV or something, and it was just... I don't know. You would think I was doing a gag with how British the name was. Look, I'm...

01:49:13chamlis_ Lord Parkinson of Whitley Bay?
lord parkinson of whitley bay i think that might actually have been it so if you go into the uk osa thread on lobsters somewhere down there towards the bottom someone linked to the legislative tracker and then there was a the only two names i could find were a woman and then like i want to say parkinson because i remember thinking of the disease actually so yeah it must have been this lord parkinson and it's I am trying not to have a big American chip on my shoulder about the whole thing, and I am only making the snarky jokes about this offline, but it is really weird to see someone with the title of Lord from an American chamlis_ and Michelle Donelan is no longer an MP, we've had an election since then
perspective we actually have a whole lot of cultural baggage around taking that seriously so that's part of why I'm asking members like oh yeah that's another thing speaking of having had an election we've had an inauguration

01:50:44dr3ig just one Lord for US of A
so in american politics there is sort of a half serious idea that hang on how do i put this better let me finish this sentence and i'll explain what i'm talking about this lord of the usa

01:51:27Speaking of trying to write things in dry ways, yes, of course I have opinions about US politics that we're not getting into on stream.

...58So how do I explain this clearly?

01:52:15That's weird location. So the US is broken up into 50 states. At the federal level, the national level, we have a two part legislature for complicated historical reasons. So each of these states is allowed to appoint two senators. And so there is one body that is 100 senators because we've got 50 states. And then there is another body, the House of Representatives, that has each state has a number of members based on how many people live there. And so a state like Wyoming that is very small. I want to say the population of Wyoming is still only like 300 or 400,000 people. They only have one representative. Big states like California and Texas have 30 or 40 representatives. However, there are a lot of American citizens who live everywhere in the world. And a political idea that people occasionally kick around is Hey, shouldn't these people have a representative in Congress? They have their own unique needs, like the complexities of international taxation and issues around visa and public policy, international travel. Should they be represented by someone? Because if you add up all of the American citizens who live outside of one of the 50 states, you get a total number that is more than, say, the population of Wyoming. More than the population of many of these states, actually. dr3ig especially since americans always pay taxes to US regardless of whether they live there
And they are largely unrepresented. Yeah. So America is one of the very few countries that taxes people taxes its citizens, even if they earn money outside of the country and are residing outside of the country. And there are international reciprocal tax treatments with other countries where you can sort of get a credit. So say just to pick a country where I know there's one, if you live and work in Japan as an American citizen, there is a tax treaty that says, well, you have to pay some income taxes to Japan. And the United States will credit you on your US taxes for the amount that you pay to Japan. And there are a ton of asterisks on every noun in that sentence, because there are a lot of complexities and it is not total. And I think in the treaties, the two countries kind of balance payments between each other in the back. It's very, very complicated. It might be helped by having a national representative for people like American citizens living in Japan. And then. Without making a value judgment also for complex historical reasons, a lot of US national policy is implemented via taxes. So. One. thing that comes up a lot lately is should you get should the government encourage the installation of home solar or the purchase of electronic or electronic electric cars instead of gas powered cars so setting aside opinions on the policies it is difficult for the government to say us well hey car dealership or solar panel salesman when you go give someone a bill for that thing just take a thousand dollars off of it and let us know and we'll give you a thousand dollars the way this policy is always implemented is the individual who does the purchase At the end of the year when they file their annual taxes, they are given a credit for that by the government. This adds a lot of complexity to things.

01:57:07Yeah. And then in the US national myth of Why was this country founded? Why did we split from the United Kingdom? In the national myth, it looms large that taxation was a big part of it. And the famous phrase is no taxation without representation. And the understanding of this phrase is that legitimate government derives authority from the consent of the governed.

...54This has all been knocking around in my head a lot the last month. Me waving a flag patriotically and going on about the Revolutionary War in 1776 would not actually add to any of this, and I do appreciate... A couple of weeks ago, I was griping about this, inventing a little on stream, and Daphne pointed out, no, that really would be a detriment to your rhetoric. So I've left it out, but that is about as spicy as I will get about that. So let's finish that comment, actually. Where'd that go?

01:58:41Ask.

...47Oh, you know what? Now I'm irked and I'm thinking about it. pushcx https://en.wikipedia.org/wiki/W…
The term, the broader term, the historical context about this is there is a Westphalian system that started in 1648 about countries make rules about themselves. There is actually an enormous amount of interesting history because governments and laws used to work very differently, especially with the history of religion in Western Europe. There is a lot to read there.

02:00:00chamlis_ you probably already know this, but for example Lord Parkinson is not an MP: he's a member of the House of Lords, not the House of Commons
Wait, don't shameless. Can you break that down? I thought member of parliament. So in the U S I mentioned that there are two parts. There's the Senate and the Congress, and we just say Congress member for someone who's in either. And then you say Senator or representative, if they're in the Senate or the house of representatives, I thought member of parliament is anybody who's in both is member of parliament only for house of comments. What's the term for someone who is in the Lords or the House of Comments, like both? I can break out my set theory.

02:01:13chamlis_ to my knowledge and wikipedia agrees
mjiig Parliament is the combination of both houses, but an MP is just a member of the house of commons
mjiig I'm not sure there's a widely used word for a member of either house
Parliament is the combination of both, but an MP is just a member of the House of Commons. Oh, weird. Well, thank you. I appreciate you pointing it out, because...

...28I don't know what to write here. Like, which...

...48I don't know what the inclusive term would be here. To the legislators. They're all legislators, right?

02:02:31know what's a better way to put this it's not that i stopped caring or anything it's just

02:03:46chamlis_ the collective term might be Members but I haven't seen that widely used
collective term might be members. Well, Shanlis and MJig, if legislators doesn't sound terrible to your ear, I'm going to go with that.

02:05:27chamlis_ fair
Is there a how to contact your MP page that I could link here?

...40Yeah, here. Thanks, Google. Here we go.

...54Oh, so this is doing the thing of MP means House of Commons, right? Surgeries, what? Some of these British isms.

02:06:10chamlis_ tee hee
I hope your MP doesn't do surgery on you. And then. So is this talking about them in a general way, because the table of contents up here said something about Lords, right? mjiig At a previous address my MP was a dentist, so...
chamlis_ so you don't vote for members of the Lords
mjiig Lords aren't elected, so generally the person you want to bother is an MP
contact are the lords like do you talk to them as individuals i don't really understand the split between the house of commons and the house of lords you don't vote for them are they appointed by someone lords aren't elected chamlis_ and they're not tied to a geographical constituency
So generally the person you want to bother is an MP.

02:07:05You know, I keep asking these questions because I keep getting nerd sniped, but I don't actually want to know how the UK government works. This is like I'm finding out about various Pokemon and I don't actually play that game. So I don't need to know that shadow type beats leaf type. mjiig How deep into the weirdness of the UKs upper house are you looking to get, because you could spend a whole stream on it
Like it's mildly fascinating in that nerd way, but yeah, I would rather not. Thank you. You're getting exactly what I'm saying.

02:08:35chamlis_ I have essentially zero comprehension of the US system
Frici best way to contact an MP is by Phone from what I understand cause other ways to contact them fall into "secondary ears" that are never really forwarded to the actual MP just filtered much like every other place 🤔 (but I am not british so might just be flat out wrong)
Yeah, you know... And I'm putting the date on this because on lobsters we do relative timestamps. And so I don't want to... I want people to be able to see what they are. Actually, I'm going to put it at the top level here. And I'm going to do it in a browser that I'm actually logged in on, right?

02:09:29oh and the other nice thing so i'm i'm off stream editing in my personal browser the other nice thing is i'm catching typos so let me hit post on that so anyways it is i think it's a non-american or a non any country i don't know there's only so much time in the world you can't have a understanding of every legal system in the world. pushcx https://lobste.rs/s/ukosa1/uk_u…
let alone be informed enough to have an opinion on them that is part of the frustration here with the UK claiming jurisdiction over the world is like I am starting from nothing I had not heard of I might have seen the word but I had not heard of Ofcom before the the thread we had about LFGSS I was like what is this Ofcom thing Daniel Katz- So I think they say yeah so they linked to this off COM site and I was like who are these people, what is this about. Daniel Katz- And then, why are they making guidance I didn't get it first I had to look them up on wikipedia and find out like oh okay so they're roughly the American FCC very roughly. Daniel Katz- And the law is doing that thing that our laws do where. The legislators write the law and leave some of the details to a regulator that can be more responsive in some ways that it's hard for a legislature to be. Or they can be the ones tasked with implementing something.

02:11:37Order is wrong. I've been tinkering with this comment HTML and CSS, so maybe I will fix that. I've wanted to move this over to CSS Grid and Flexbox for a while, because there are a couple of janky things that can happen with comments, but

02:12:03Over in IRC, and I noticed because I got the notification, Caius, who is the person who submits the what are you working on this week, are asked a question about whether the GOIP database contains v6 addresses. And I said, yeah, it does. And...

...44Every single nerd, self included, starts out by looking for one weird trick to duck the OSA. I think compilers have trained this instinct into us. Like if you run into a bug, you just have to fix that syntax. You just have to change that one thing. You add the one test, you add the counter cache, we can materialize the column and then this problem goes away. So I do appreciate that. I am saying nerds affectionately, not derogatorily. I do appreciate that nerds do this. Well, let's here's a problem. Let's find the solution to the problem. Most problems have pretty straightforward ways that you can address them. And so. But the actual effect of watching dozens of people come up with the same ideas of, well, let's just say X. Let's run the site over email. Let's point out that we're not in the UK. There is not one weird trick to get out of this. There is not a two-line diff that will fix this bug. This is a big slog.

02:14:19Look at my terrible grammar. See? Some of this is... I did sleep well, so this is just me being sloppy. Got that verb-subject agreement. I get worse when I'm on stream because I'm trying to talk while I do it. is that UK persons contact their MP to delay approval.

...56You know, it's not I'm running out of options.

02:15:17So I tweaked this little off thread. I changed the language to say more explicitly, it's not that I am running out. It's I have almost exhausted my options. And then there's a little verb subject agreement there that I fixed.

...39All right.

...51yeah so let's see so i'm at two hours 15 minutes of streaming good time to remind that this is lobster's office hours if you have questions about the site of the code base or you just want to well if you have a question about the site of the code base you can ask if you want to commiserate about the online safety act you know there are some excellent whiskey distilleries up in scotland and in Northern Ireland, so you could send me some of your wonderful UK whiskey. Actually, my go-to is Akentoshan. What is that, the 12? Oh, God, I can't spell it. Akentoshan. I got it. No, I didn't get it. Akentoshan. Oh, yeah, there we go. I did get it right. Okay, yes. This does not count as highly invasive age assurance. This guy. So if I'm going to have just a nice little sip of something. This is kind of my go-to. It's widely available in the States. It's affordable. I like it a bunch. I'm not any kind of deep whiskey connoisseur. chamlis_ hey guys the DB miscategorises my IP, come round every Friday and we'll do a Lobsters LAN party, catch up on the week's comments together
I have a friend who is. And so... Oh, did you check out the, what is it? I linked to it in that thread.

02:17:46Yeah, here we go.

...53pushcx https://db-ip.com/db/lite.php
So if anyone's wondering if you were going to get geo-blocked, so this person, secret developer, says that they are in Ireland, which is not in the UK. And you know, they say subject to the UK's OSA order. Bad news, buddy. The UK thinks it applies to you. Oh, I'm biting my tongue. Political history. So... chamlis_ I'm spinning a yarn, it has me dead to rights
yeah there are both false false positive errors where it thinks that people who are not in the uk are and there are false negative errors where it thinks that people who are in the uk are not i might have said that first one wrong sorry it's sometimes hard to get these things right when i'm talking off the top of my head for a couple hours straight

02:18:56that's why i keep saying this is the least bad plan this is i don't know you're welcome to throw your ip into the database and see what it says about you i don't know if that's the light version because they worry about it getting scraped or if it's the full version let's see where are we january 23rd so i have a reminder on my personal calendar on the first because that's when they refresh the free database. And then when I buy the commercial one, which is on my calendar for March 15, before the Online Safety Act takes effect, then I think that one updates weekly. And so I'll have to write a cron job. Alright, so I'm going to do that contribution stuff, and then we can do story merging if folks have questions. But before we get into that, I'm going to run to the restroom, and I'll come back in just a minute. chamlis_ I wonder how they make the free one worse on purpose. ignoring the last octet?
turn that on you can think of questions and comments and favorite whiskeys actually i do take recommendations be right back

02:22:41alrighty thanks for hanging out yeah i don't know how they make the the free one you know they have a note there that says it's reduced accuracy they could also like throw away any smaller entries and just lump them into their nearest folks or you know there is the thing what is it called I would bet trap street there we go. pushcx https://en.wikipedia.org/wiki/T…
I would bet that it has the IP equivalent of trap streets where. chamlis_ ooh
They a map may include fake streets, because if they see. In the US, at least you can't copyright facts, so the fact that there is a main street and first street crosses it no one can copyright that and own the idea of a map. But you can own this specific picture of a map, and so, if they add a fake street here. That is copyrightable because it's sort of the artistic expression of drawing a map. And I'm probably mangling this. Like, don't take legal advice from me. chamlis_ bio break notice is still up, by the way
But I know bits about this because I've gotten nerd sniped by it. Oh, shit. Thank you. I'll turn that off. Real slick production here, right? I can't remember to unmute. I can't remember to stop pissing. Not making that a stream title. So a trap street is a copyrighted a copyrightable feature that can be used to detect that somebody copied your map. chamlis_ it's still there lol
And so I would assume that like that light database and then even the free one, it's still there. Fuck, I didn't hit the... Look at me. Frici ain't a real stream until you forget a couple things. including the bio text source LUL
There's like a little crossfade button I have to hit. I definitely don't have a Elgato thing. The biotech source. Yeah, I need a... What are those called? So I actually know a Twitch streamer who streams video games. She has one of these things. I think she has a bigger version. And it has all these customizable buttons that you can make to control your stream. Maybe it's this one. Maybe I'm just thinking of the knobs. I remember I've seen it on her desk. I definitely do not have that. I don't even have hotkeys set up. Frici its the XL probably
Frici the 15 button one
There's an XL, the 15 button. Frici no 30 *
Okay. chamlis_ rebrand and sell it to emacs users
Yeah, that's, I will probably, you could also get the, what was that called?

02:26:00or no, not led LCD keyboard. Yeah. So someone made a keyboard where folks are cutesy about these dumb things.

...21Speaking of UK exports.

...30They made a keyboard where, I guess several keyboards now, where every single key is a little LCD screen. Frici its a decent investment tbf even outside of stream/media production
So this is the, I guess, the natural extension of the Steam Deck, where you can put all your shortcuts on there. Oh yeah, I'm not making fun of it. It's just a level of professionalism that I do not aspire to on stream. I'm happy on stream if I can avoid I don't know putting my bookmarks and my social security number on stream if I managed to not mess that up too badly i'm a happy camper.

02:27:17So let's let's look on let's do something positive. One of the things that's so frustrating about the online safety act is. chamlis_ have a trap SSN just in case
It has burned a ton of time and attention to do nothing like we don't get any features out of it, nothing is better for it it's just an incredible drain. So let's do something positive have a trap SSN yeah there we go i'll copyright them and then, when someone steals my identity i'll go after them for copyright infringement.

...58chamlis_ my SSN is the AACS key
So I said I put it in that issue. Which issue? Is AACS the one that's like for decrypting DVDs? Blu rays and stuff.

02:28:20chamlis_ blurays, that's the one
Yeah, so this was the comment. Yeah. yeah someone posted that as a comment on lobsters a while ago and i deleted it because i was like hey please tickle the dragon's tail on your own website with your own lawyer thanks and i had hoped to i remember there was a whole thing back in the dvd days where people were mad about copyright infringement so they decided to post it to dig a million times it was one of those really misguided teenage protests where they were like very mad about the general situation and so they took it out on the wrong person let's check that number because i want to say it's three two

02:29:31Yeah.

...38And it started, I was just looking at it, but I wasn't looking for it. Yeah, 2012.

02:32:30Thank you.

02:33:12What's the other part of this?

...25I think that's pretty good. Any other big philosophy things? I mean, if you're on this stream, you've been hearing me ramble, not just for these two hours, but probably for a while. Anything else I've said about code philosophy that seems important? Because I've said, you know,

02:34:26That's pretty good. dr3ig you also said that you don't mind if the site goes down for maintenance for bigger prs
I think I've said well. Oh, yeah.

02:35:21I wonder if I should make this a bulleted list. Ah, it's fine. I like paragraphs. All right.

...51There's a thing I sort of want to say that I don't know how to say well. So like what I want to say is we're willing, we're pragmatic around writing ugly code rather than trying to write code that is

02:36:22This feels like I'm picking a fight when I say something like this. So I'm trying to say... I'm trying to say I would rather use a Rails callback and all of the pain it entails than like add monads to the code base and add a type checker. And I love those things and they've improved my programming, but there's a lot of... This is not a big fancy project and we don't actually benefit from theoretical benefits. We have longevity, but we don't have scale sort of deliberately. And I mean scale in the sense of it being an enormous code base. And I don't know how to say that without coming off as a dick because I have seen so many people have the argument over engineering versus computer science to put it another way.

02:37:38I don't know.

02:38:04espartapalma Hi, good day, folks
I hate to phrase the fact that. Hey, it's Park Palma. Nice to see you again. It's the right way to say this. I'm trying to flip this as volunteer product.

...45Trying to say like our design philosophy is informed by the fact that that is so clunky, right?

02:39:03But like I've squeezed so much here that I can't put design philosophy at the end. My better half is such a better writer than I do. I am. This would already be done five minutes ago. if it wasn't me writing this.

02:40:03When I realized all of these sentences started with we,

...37I call it ops complexity. It is such a Bay Area phrase.

...48What's the right way to say this? I wanna say like, I'm not adding new production services to add. Yeah, I guess that's fine.

02:42:50that's not bad i think that's all the big stuff if anybody has thought of other stuff that i tend to talk about on stream of oh this is why we're doing something let me know and i'll add in a bullet point but this seems pretty good turn the spell checker on okay we're good

02:43:26And then, all right.

...49I want to send those to Rahul, because it's working with Rahul that has really prompted this. I have been thinking about all this stuff the last couple of months I've been streaming, because I've been forced to say these things out loud over and over. But it is especially his full requests about this, so I want to mention this.

02:45:42So that's some more. If you ordered an issue or opened a PR but don't want to continue with it, especially when a maintainer is asking for more and more changes, please do tell us instead of changes, revisions. Yeah, that seems pretty good. Just going to add that to the last commit. It doesn't need a separate one. All right, so I'll push that up. Cool.

02:46:33So there's that. Alright, so oh man already two hours 45 minutes in so I usually aim for about three hours. And I know there are a few people on so i'll talk a little more about story merging for anybody who wasn't here at the end of the last one. So for a couple of streams I audited every merge we've done on the site. and categorized them into these broad categories just to get an understanding of why do we merge stories because because it's a unique feature of lobsters that people aren't familiar with from reddit and hacker news we get a lot of questions and confusions about it and on the last stream i kicked around the database refactoring for story merging because it's a big source of bugs the way it's implemented now. And had the UI idea of instead of saying, well, we'll have a big list of links at the top of the page and then all the comments, that can be a little confusing of is someone replying to the original post or the rebuttal, that kind of thing. Well, let's have a list of headlines and topics up at the top, and then for each link, give it its own thread. So it would be a separator in the tree of threads that's on the comment pages.

02:48:12And I can make that UI change strictly without any database changes, but we're getting to database changes for reliability and improving features like users should be able to suggest merging and unmerging stories and When submitting a story, the submitter should be able to choose to merge into an existing story if they know it's there. And then also, if someone has written a top-level comment with a link to a story, it would be really cool if users could say, hey, that should be basically a story that gets merged in because you're adding more resources.

02:49:03And so the UI stuff, I could hack that out almost immediately. Yeah, I'd give Rahul a merge conflict around his work on the queries, but I could do that. I'm still not confident in what the database structure should look like. Because there's the question of, should there be a new model called Headline that's like the idea of one of the entries on our front page? So I'm talking through my thought process, and I would really appreciate kibitzing on this if folks have ideas. because I know you're all experienced developers. So if each one of these was a headline and then it had one or more links attached, that would clean up the database structure.

02:50:19And then there's the question of when comments are promoted to stories, Do we create a story object? What even is a story? It's just kind of a link. We look at the. So these things are kind of jumbled up where. Like the title would move over to headline, although I guess it also has to be on the individual stories. And then a lot of this stuff is either duplicated or moves up to the headline. But then, you know, I see things like, oh, user is author. Well, that's really easy to answer if it's a comment.

02:51:26I, Equivocate between story text so like this story has text in this description Only about 10% of stories do and Only 3% are like this one where there is not a link here and the difference is around 7% of them it's it's largely dominated by pdfs because the practice is to copy and paste the abstract in here because the diff bot can't extract a summary for those so when we have a headline with multiple stories this would kind of be the phrase the structure of if we did the minimal changes

02:52:29I guess when I'm not confident, I should just start nibbling at it where I know we could make the UI change and we can see what that looks like and how people like it. And I can slowly drive these other changes. I mean, introducing headline is going to break so much stuff. And so it's not clear of Obviously, the fourth would be...

02:53:11Sort of like go the other direction.

...20And honestly, this might be nicer. It's a lot less churn. Because if I introduce this headline thing... We have all this stuff around hidden stories and saved stories and upvoted stories and moderated stories. And like I'm touching every single foreign key in the database.

...46So again, what's the difference between a link and a comment?

...57Especially if we are extracting links.

02:54:28And I'm using the word link. It's one of the, it really wants to be a God object in the system, but it's sort of implicit. We have like, I think three or four custom ways that we parse URLs because they're so important and we treat them ever so slightly different. And I recently added this link model that is extracted from all comments and all stories. And it's mostly, I had mostly intended it to be for internal linking so that that's something. So if I move that down, like I want the feature, but I don't want the refactor in the Yagni sense. Because I can do that. I can do that. I can do that. None of that requires touching the database. I can cross-link stories in footer link.

02:55:53Footer with back rep links to Story to comments linking here.

02:56:08What else is there? Do I want to have them on the per comment? Not really. That one. So if it happens at the story level and it happens at the bottom of the page, it'll be fairly chill. But if I did it where like this random comment, just to scroll to a random comment, not that I know anything about it. If when someone six months from now linked to this comment internally, there was a little thing added at the bottom or added to the metadata, even if it's collapsed in a detail, that feels like a really prominent place to put something and it could potentially promote some really nasty behaviors where so like this is a totally anodyne comment there's like a mention of payware so maybe someone gets really mad about payware six months from now and they're like oh well everything is bad about payware as evidenced back here with this guy who has bad opinions about it because blah and Like right here, if there was a link to, well, here's someone calling this guy a jerk, that's really unpleasant.

02:57:42And it feels like if it's way down at the bottom of the page, there's a list of links into comments. You know what? I think I'm being too cautious. I keep saying this. And actually, the vibes are good. And that kind of thing is I can't design the whole feature around the half a percent or 10th of a percent because that's a likely number of abusive cross links. Yeah. So let's do

02:58:25Got to redesign the comment headers at some point.

...34And then this also needs some amount of don't cross link if source comment has a negative score or mod soft deletes the link. that i want to make more work for moderators but actually the good will outweigh the bad here i think quite a lot yeah so there's kind of a water line here that there's more available to do without solving this database stuff and you know as soon as i say keep story top level i think of how many

02:59:35So we have 63 mentions of story ID in the schema. And some of these are redundant where there's a column story ID and an index, but all of them have to get touched. What a lot of churn. If that's what it is, we'll do it. But if we kept story top-level, oh yeah. There's another one.

03:00:32So if you click hide on a new link that just came in five minutes ago, and it gets merged into another story, should that get hidden? And the answer in the code base right now is We have considered it a bug if it's not hidden. And boy, it adds a bunch of complexity. Because we're vague on whether a story is an individual link or multiple links together. And the code and the database schema equivocate between the two in a bunch of ways. And so this is really getting at that indecision in our got object.

03:01:43Feels very religious.

...53And since I don't look at this and feel a lot of confidence, maybe I want to just start nibbling on these UI kind of features. Cause that'll also refresh all the code in my head and force me to think about all of that. Okay. Yeah, that's a good plan. Yeah. So we're gonna, we're gonna call this merge plan and then over here. Yeah. Put that down there. I mean, write docs last, obviously, right? Maybe never write docs. I should put that in the design philosophy. Now I'm committed to writing documentation. All right. I'm feeling pretty good. Got a lot done this stream, and it was really, really nice to get to talk to a very active new contributor. I'm assuming Rahul is rolled out by now because he hasn't said anything for a while, but thanks again for dropping by. To Rahul and then, of course, to all of you. It's been fun talking through all of these projects. As always, happy to take questions and comments about the site. I'm going to wind up and then our next stream will be the usual Monday 2pm Chicago stream. So you can polish your questions and comments and pull requests and for those of you in the uk please do contact your your letter legislators your members your lords your ladies whatever you call them please go ask them to not pass i don't even know what to call this there's like a it's in the comments on the story but there's this whole thing about The regulations don't actually take effect and start destroying sites unless parliament approves of Ofcom's guidance before March 16. So you have, what is that? Just over three weeks, which is nothing in legislative time. But three weeks. Please do get involved. And then otherwise, thanks for being involved in the stream. I'm going to roll out. I hope you all have a good rest of your day or rest of your night if it's dark there. I'll see you on Monday. Take care.