There's a hole in my bucket

03:14How's it going? It is Monday, February 17, and this is Lobster's Office Hours. I'm Peter. I'm just getting started here, so let's see. I was out running errands, so I'm a little disorganized here, but this is Lobster's Office Hours. If this is your first time visiting, hello, welcome. I run this website, Lobsters, and if you have any questions about the site or my moderation choices or, boy, what else, the code base, just drop in chat anytime and feel free to ask anytime. And if I can't address it anytime, I will stick it in my notes and come up with a response in a minute. When folks don't have stuff to talk about, I work on the code base. Mostly. And one of the things that I was recently working on on the code base a couple weeks ago was this new icon for story merging. And on the last stream, I merged a fix from a stream viewer who noticed that the icon was not quite legible. And I made a couple other tiny CSS tweaks to this to improve the spacing between the icon and the number and make the number not bold. And my plan for this stream, unless folks have stuff to talk about, is that I was going to work more on the story merging UI, which has a bunch of, what do we call it? Sort of a support thing I'm doing. Talk about that in a minute. That's the cabinet we'll get to. Yeah. Anyways, if you want to sound off as you arrive, that's always nice. Let's take a look. And Fritchie, if you are around, I had something to mention to you. We'll see. I think Fritchie usually makes Mondays.

05:36Let's see now. So... saw something unusual a couple of months ago was a actual spam bot named pollies and we got these kinds of just incredibly spammy like so this is the text is very chat gpt and It linked to just garbage like a spam site that pretends to be Ring customer service. So that's just a straight up scam. This thing about the Miami Open is some kind of, I think it was a pirate stream pharmacy scam. I mean, they might actually send you pills. Who knows what's in them? Some kind of place to argue about

06:41Company is what done you wrong. We don't usually get this kind of garbage tier spam bot that doesn't care at all about the topic of things. And a lot of that is because of our invite system. So the invite system, I don't know, sometimes people think that it exists because we are the cool kids club or we think we're the cool kids club and that everybody on lobsters is the best of the best and that is not at all the case that is not at all the intention the intention is to spend less time dealing with this kind of bullshit because if you have open signups you get an infinite amount of this and this kind of thing is pretty easy to catch with basic spam controls, but it never ends and it always gets smarter. And if you grow, you become worth individual targeting. And so the invite system is not about being a secret tree house where only the best programmers hang out. It is about the million kinds of abuse we don't have to deal with when we have open signups.

08:11And so lately on stream, I've been working a lot on story merging. But I think maybe in a few weeks or months when that's done, I might turn more towards some things that are related to the new user experience, enculturating new users, and doing that with an eye towards reducing our barriers or making the invite system a little more permeable. So hopefully these spam bots don't show up. This one was on my mind because I just heard back from the user who invited, who was a totally normal user who got tricked by somebody in the IRC channel. And now that they called out when that happened, it helped me to go back and look. It's happened, I don't know, maybe three times, four times over the years that someone has actually gone to the lengths of going to the chat room and pretending to be a real person and trying to trick someone into inviting them. So that's three years in the, what is that? Six and a half, seven and a half. I'm losing track, that I've been running the site seven and a half. so as long as it stays a once every two or three years kind of thing, that's a manageable level of spam. All right. Fruity, if you're around, say hi. I'm just going to just say that every once in a while because I have something to mention to them. And then otherwise, I've got this cabinet view. Because a cabinet is a piece of furniture you put lots of little things in and they needed a view to see all of these story merge or story templates at the same time where i can look between them and see all the differences and so i wanted to actually make all of this a little more readable as the first thing on my stream here so if you have slight questions about the site feel free to pop in anytime Oh, actually, I should throw it in chat because I think if you connect to Twitch, you see chat like the last 10 lines. And as long as nobody's talking right now.

10:51pushcx Ask questions about https://losbte.rs anytime, or hang out and watch me work on the site.
twitchtd hi
There we go. We'll put something in there. It occurs to me that even though I can't get a Twitch API key for security. Oh, hey. i could probably still get something that authenticates to the twitch endpoint for the irc front end to chat i may tinker with that i actually spent the weekend doing big social stuff so no tinkering really so where was i here cabinet index and then i had the the helper right yeah So instead of this, I don't know what to call it. I'm trying to think of what to call this text to stick it in a variable and line break it. I guess it's a. What if I called args-inspect instead? I think that might get me line breaks. No.

12:24Didn't I have awesome print in the gem file? I don't want to rabbit hole there. Let's just call it the debug string equals split on whitespace. Let's just split on actual string. Someone has written... That's a lot more than I want. I was hoping I could just one-liner it to roughly split the strings into approximately 80 columns. is squiggly here doc yeah that's not what i'm looking for if it wasn't a code base i care about this is the kind of thing i would throw at an llm tool but especially with the court case that just got decided last week about copyright output from LLMs. Not looking so hot.

14:14Huh. twitchtd what copyright case and what was the result?
Rails comes with a word wrap. Great. Where is that? Let's look at zeal. Because I didn't know that function.

...35All right, that looks about right. Funny that it's a helper rather than monkey patching string. I would have expected Rails to monkey patch strings. So let's just put that back, and we'll call word wrap on that. Syntax error. Yeah, you'd need to put that on. Alright, that's reasonable. And then we want to have a… Oh, the… I think it was Reuters?

15:32pushcx https://www.findlaw.com/legalbl…
Yeah.

...38So this is the one i'm thinking of that just came down the last week or two.

...54svyatik60 Hello. I would like to know which Linux image would you recommend for a beginner developer?
So they basically said that there was copyright infringement in the scraping and training and. The interesting thing about it was...

16:15So in the US we have this concept of fair use. Yeah, does this... So there are conditions in the US under which you are allowed to reuse copyrighted material. For example, If I was criticizing this article or that shitty web design that you just saw where it kicked me back up to the top when I closed the footer, I could take a screenshot of that copyrighted material and say, let me explain why this is bad. And fair use is the section of the law that says, well, you're allowed to use copyrighted material in limited amounts if you're criticizing it or writing a review or otherwise not commercially competing with it. And there is... An enormous amount of like detail on that, but I had thought that, Civistec, I see your question. I'll come to it in a sec. I'll finish this point. I had expected actually that the most likely thing that was going to happen in one of these cases around the copyright output of LLMs was that the resulting material was going to be considered transformative use, which means changed so much from its origins and original meaning and original use that it was effectively a new work. And if this is something you're actually going to legally rely on, you better get a lawyer to explain it because, you know, I'm a non-lawyer butchering it. pushcx https://www.theverge.com/news/6…
But there was a very strong decision that none of the possible defenses for under fair use were appropriate. Actually, this is probably better than the one I already linked to. So Twitch, that was the one. So Svea Tech, you say you're a beginner developer. What kind of developer are you? Are you making video games, web stuff?

18:38My answer won't change too much, but actually if you're doing, I think there are one or two security focused distros that have a lot of tools around, you know, network security and penetration testing. And I don't actually know enough to suggest any of those. I would just. pushcx https://wiki.archlinux.org/titl…
tell you to use Arch Linux because it's got a great wiki. So anything you are trying to look up just about fixing your development environment is in here. Arch, I think, is a really nice balance of it pretty much is just going to work for you and you can tweak all kinds of things where Ubuntu is very popular but is trying to push people to be very hands-off. Arch is also very broadly used by developers including me and then if you are doing something like video game development the Steam Deck under the covers is Arch Linux and I think that's where a lot of people are going to be targeting stuff so good luck all right so got that wrapping otherwise i'm happy to take more questions let's see i have a content tag with a this is like the terrible version of flex where i have to write this kind of method call and method call.

20:47If I lined up the closing parents, that's probably enough. There we go.

21:03Something in my house is making a strange noise. Like a vibrating fan. I'm gonna go find out what the hell is making weird noise in my house.

...25Oh good, it's the fridge. The fridge is buzzing. Not like that's important and full of expensive groceries.

...41So this is nice. We've got a header. Get this. Let me maybe put this up in here. I think if I do that, it'll be... No, there's a way to indent it. I don't remember what our styling for this was.

22:12inside guess not Oh, it's singular. It expects an ID. All right. Well, we're not doing that.

23:08Let's just leave that off.

...16Megan's put the partial first. It all looks bad.

...43Oh, I didn't turn on the stream notice and turn that on i probably have an extra microphone unmuted yep there we go if my sound's been a little funny should be better now

24:22Let's just make this header the summary.

25:19Let's just get rid of debug render.

...39What's your error? Why are you printing? It's so weird to see Rails printing. CSS in here. Oh, you don't like that. I'll put a comma there.

26:34Let's see, these titles are three to 12 words. I thought I could strip that in.

27:08Here we go. Get a few different sizes.

...35Trying to figure out if it would be. Too noisy if I put in HR in these.

...48Yeah, I don't think that helps.

28:12So I think that's probably generally fine. And I can come back to this cabinet and add more renders. The end of the last stream just basically got this working.

29:28That's what it was. I was going to change the... So there's a global variable at user that we use throughout views. This is going to be a lot, isn't it? Yeah. How many instances are there? 75. And I would like to be able to render these stories with different viewers. But the viewer goes in, the partial goes and checks the global variable.

30:20So that's where I ended up.

...30As I was wondering about if I changed, let's do the simplest thing. So if I said,

31:03Let's just grab the moderators and then back up in the controller.

...31See what I worried about was that if I reassign this variable in the view, is that going to be lifted all the way back up to the controller? So now if I reload, I was not able to assign. This last one should have gained a mod edit link. It got a regular edit link that shouldn't be there. Hang on, something, I don't understand the flow here. Why is there an edit view or an edit link at all? So this is what the submitter should see, but we're not logged in as anybody. So it has a user, which is a random new user. So it must be comparing this user ID field is nil the way that rails association is works is that abstraction is leaking i bet so if we go look at list detail and we look for the word edit we get is editable by well that's in the merge stories

33:23Ah, if it's a new record, it's auditable. So it's assuming that if you are looking a story before it's saved, well, that only ever happens on the edit form. So we're kind of violating that assumption.

34:50That sure didn't work. Oh, this is the URL is editable. There's another general one.

35:31now we know it's all right that's better don't love it but it's better

36:05All right. I'm going to just not add behavior. There's something I don't care about right this second.

37:11Hmm. I don't know my data here that he's still listed as a moderator.

38:01Oh, this is used for the, this name is maybe a little misleading because it's not the active moderators, It's everyone who has ever moderated. So that's why it didn't add those other fields. And then did this output print what I expected? Because I had added that debugging to the controller.

...51Okay, so then the change to that instance variable didn't come back up to the controller okay so then i'm not going to have to stop and refactor the whole world to add something like a current user method that cabinet overrides. I do have the. This would call for a helper I do have the hassle that. snowg010398 What are you working on ?
I would have to know to put the user back, but I'm going to make a block helper for that, I think. Yeah. Hey, Snoji. pushcx https://lobste.rs/
I am working on Lobsters. Lobsters is a social news site. It is visible here. This is the office hours for when people have questions about the site or the code base. And when nobody is asking me questions about stuff, I tend to just work on the site, whether that's bugs or pull requests or new features or fixing features.

40:14Let's go and say, I don't actually need to capture the block there, right? So if I grab this and I do that, then we'll say here that we are at that voice.

...53It's going to be Kyle. I think there's a third. I'm trying to get it to have me because I want to see the mod editing stuff.

41:13Well, that's not right. He's never been a moderator.

...38So there's the mod edit link that I wanted to see. Okay, so as user works, so I can drop out this debugging. Condent this guy. It's always hard to see what's happening in ERB in the space of all of this stuff, but all right.

42:02So...

...11It's just to duplicate this one.

...46What's the column named? Is Moderator.

...58This is starting to tempt me into dragging Factory Bot into production, because I know I have traits for these. We'll see how many of these I get before I get that frustrated, though.

43:33I see the value of something like... There was that rails tool somebody mentioned for doing these kinds of things, and now that I'm building up all this state, I can see wanting to be able to toggle things through these states, but I'd rather just have many of them. Why is this not underlined? Oh, it's never underlined. It should just be moved to the left.

44:33Why break muscle memory for people?

...43All right. So what do I have next here? I can see things as a user.

45:27It's funny how this H1 has more space under it than above it. That's a really odd design decision.

...42We could probably ditch that at some point. There's not so many H1s in the code base. No rush, though.

46:05I think that's probably enough with the basics. Just say the user was created a while back, so they stopped being green.

...44Thank you.

47:22Just give a little explanation for when people see this in fraud. I'm not linking it anywhere, but people will be surprised if they run into it. I think that's probably enough that I can go ahead and bring that in. I'm just going to switch back.

...57So let's make sure master is up to date, because it may not be.

48:17It doesn't already have Vim up.

...39Epic_Ninja_Elephant Happy Monday.
Oh, hey, Epic Ninja Elephant. It is. I'm hacking along on this cabinet view.

49:08To the right place.

...53Go get that rich and delete it off the prod. Can I open up pull request for it? Yeah. Just go ahead and drop it.

50:42That merged. Did I make a branch? No, I made a stash. So let's see how many conflicts we get when I pop this. None. Not bad. So the bigger project I was doing was revamping this for, oh, there's an error.

51:11revamping this for story and merging UI. So what's my error? Oh God, this is a 360 line partial with 1 million ifs. And it's saying somewhere in here is a syntax error. So I have a def somewhere. No, I don't. Oh, I did get a conflict.

52:07So I thought I fixed a typo here. I guess. Also this addition. Okay. Yeah, it would have been really nice if Git had said something. Does it have conflicts in all of these? Yeah. That could have been more explicit. That, you know, it blew up and failed as opposed to Just oh, you both modified these. Great. Oh, my God, that's a huge. So this was the section I moved. Right.

53:15Yeah, so I'm just going to drop that out. There is no way I'm getting to maintain all the changes I made and correctly resolve these conflicts. This was a fix.

...59index all right so these are probably these two are heinous partials that incorporate list detail so i think they're probably okay but i'm going to bounce the real sort of just to make sure they get re-rendered and then i'll see in a second if i can load the cabinet view and get something reasonable out of it hopefully seeing these in enough different states tells me that Looks like it's actually rendering. It's just taking its time getting warmed up here.

54:40It's not taking its time rendering. It's rendering like every fucking user on the site. Some kind of preload went terribly wrong.

55:03So if we look at the server output, it found stories. This is probably it found every story. So list detail is not really expecting to work on a new record. And so it found and tried to preload every single story that's not merged into another story oh rails i did get roughly the right stuff out of it it just because of the way the view went back to the well.

56:15I'm getting some extra debugging here. It's going to take a minute to load. Whatever that extra domain is that's wrapping onto a new line, something's wrong there. I'll have to fix that. That's probably just something that got butchered by resolving merge conflicts and it got shoved onto the wrong line. So this is the part in list detail where to count the merge stories, it goes back to, all right, we'll find them instead of following the association.

57:05So now this should load promptly. Good. All right.

...27So this is failing to list those three because I think I just set the cabinet view of it to say that there were stories. Oh no, we added several. They should have been in the array. But adding this not deleted means that Rails went and tried to check in the database. And they're not persisted, so it failed.

58:08Active Record is such a bad design pattern. The library itself is reasonably good, but this is the kind of nonsense you get with Active Record, where every object is live from the database and has all these convenience methods for hitting the database.

...33Even if I took these and I wrapped them up in a scope, that would also break because Rails has to assume everything's in the database. God damn it. I suppose I could make a transaction in the controller. Save them. Render the cabinet. And then drop the transaction.

59:11I don't know any other way to do it. And then all the... But then, if I did that...

...25These fucking after commits would run. And these are harmless, but... with comment, comment?

...48Yeah, after commit, deliver notifications. How funny, so I just moved these from after save to after commit.

01:00:04And that accidentally fixed this. See, my worry is these active record callbacks are going to start firing and do real things with test data in the cabinet.

...24There's no fucking isolation in a Rails app. In every... I mean, it's not just the thing about there not being packages like Packwork solved. It's just everything is live talking to the database.

01:01:00So this is all pretty bad. Rails has kind of painted me into a corner here and I don't see a way out of it.

...26kaayfour Do you have subs enabled?
If I commit these stories That's fine for now, but there's no way to say that models shouldn't have after-save callbacks. So then I've just loaded Chekhov's gun and pointed it at my foot, and the next time something creates a callback, that's going to run in prod when somebody loads the cabinet.

01:02:00And I don't have any way that's expressive enough to say, nope, no subs. I am not a Twitch affiliate because they require that you give them a phone number that's not a VoIP phone number for that. And I don't have any that are not VoIP. So you can click follow. Sorry, I'm not trying to make money off of this. I'm just having fun working on this thing. for certain values of fun that involve sighing loudly at rails.

...46Epic_Ninja_Elephant Who is your preferred VoIP provider?
I don't know that I have a preferred VoIP provider. There are one or two I use, but I don't recommend them. It's one of those things where everyone is kind of bad at it for my needs. kaayfour I just have the free sub to gift is why I ask
Because my needs are unusual.

01:03:30Oh, that's neat. Is that a thing from Prime Gaming? kaayfour yeah prime thingy
Twitch shows me that you have a little crown badge. And I can see it when I click on your username, but I don't actually know what it means because I don't know a heck of a lot about Twitch. All right. Well, I appreciate you offering to gift it here or considering gifting it here, but there's no paid subscriptions to this channel. It's just me knocking around. twitchtd I'm running lobste.rs locally, what enables action/controller recorded on the sql query like: SELECT `keystores`.`value` FROM `keystores` WHERE `keystores`.`key` = 'traffic:intensity' LIMIT 1 /*action='newest',controller='home'*/
I think that would actually just make me sad. So right now streaming is this thing I do that's kind of fun and a good way to do office hours and maybe get a different path for people who are curious about the site to ask questions. But if I did this as a Twitch affiliate and I like made 90 cents, I would feel terrible in a way that doing it for nothing feels totally fine.

01:04:40So Twitch, your application controller, if you look for traffic, there's this method set traffic style that runs around every action. And it calls the traffic helper that has this cached current intensity. So that's what that query is that you're seeing. So the Lobster's logo changes color. So if we go look at production, the site is not especially busy right now. Zero. The traffic is bugged right now is what that is, because zero is not right. If folks are posting comments, and they are, it's sure not zero.

01:05:42twitchtd that query was an example, i was specifically interested in /*action='newest',controller='home'*/ :)
Oh, that. That's a... Oh, I see what you were asking. I misread your question. So that's over in... Where is it?

01:06:14twitchtd I never saw that so I'm wondering how it works cause it's really cool
I'm saying here we go. It's config application.

...50pushcx config.active_record.query_log_tags_enabled = true config.active_record.cache_query_log_tags = true
this query log tags enabled. So let me just grab both of these for you. Yeah, it's just Stock Rails feature. And it's really handy for figuring out where this stuff comes from. It's separate from this little indented thing that tells you the line number. But this little indented thing is not super accurate because of queries being lazily evaluated. So sometimes you will see where this happened in the view rather than the line of code that queued the controller. So it's pretty, it's a little bit hit or miss, but it beats nothing. But this part with the action controller, it's that log tags. twitchtd ah, i was wondering if it could do lines, but this is cool anyways
I want to say I was able to configure those tags.

01:08:11Ah, yeah, here you go. The other part of it is in this query log tags. pushcx Rails.application.config.active_record.query_log_tags = [:controller, :action, :job]
That's a one liner. I don't know why I didn't just put that in. I don't know why I put this in an initializer instead of just putting it as the next line in config application. That may not need to be separated.

...55So what am I going to do with this data that I don't want to persist?

01:09:07I can either set myself up a time bomb by having the controller wrap this in a transaction that it discards rather than committing. And then as soon as something adds another after save callback to story, comment, anything related to any of that data, that's going to start happening in prod. Or I can just say, this is why we can't have nice things and make cabinet a development only thing and let it just go ahead and pollute the working copy with extra records. I kind of liked the idea of having the cabinet be visible in Prod to demo. Oh no, when there's a Wow, there's these 20 different ways a comment can appear. I think this is just going to be Rails is why Rails can't have nice things.

01:10:41Oh, OK. And let's actually go back to the gem file. And Baker can go back into the test and development section, which is not sorted.

01:11:24I guess if it goes back and this only runs in dev and test, I do have factory bot.

...51So if I just, I don't know how to, I just say create story here. No method create.

01:12:43There's got to be a demo of calling this without all the fancy RSpec stuff in place. The fact that there is a book rather than a docs is a little disconcerting. I could just call it a class. Sure. Let's just call it a class, see if that gets us there.

01:13:35okay so that worked but the story factory didn't create a tag ah it assumed that there's some test data

01:14:09I tried to work around that.

...26Can't be now. All right, so let's build another record, because everything with Rails is like that. There's an old criticism of object orientation that what you wanted was the banana, but what you got was the jungle holding the tree, holding the monkey, holding the banana. And of course we're immediately into that kind of situation where now we have to create the world.

01:15:15Really? Are you just recreating that user? I wonder if the sequence is resetting somehow. Yeah, I'm just getting new usernames. That's fine.

01:16:01see something is resetting factory bot maybe it's oh I get it it's changes when I change the server something in factory bots sequences is getting reset when rails reloads code so anything where I've put one of these simple kind of placeholders is gonna fail in exactly that way until I have reloaded that many Username is invalid.

...46Generating these with periods in them.

...59It's got to be a way for me to say don't do that. No. Specifier. Separators. OK.

01:17:38all right now we're getting random usernames random users created

01:18:07And I figure out how to express this great. Slash limitation.

01:19:13So let's make a couple more stories.

...40It's disconcerting if I already had a collision on it generating usernames.

01:20:01So at this point, I think I can stop banging on the cabin and go back to banging on the actual partial.

...16Let's figure out why the, all right. So there's some kind of new line that got added there. Or like we want to be in that span. Something there changed between. Yeah, this is going to be just huge diffs. I almost wish I had It's almost enough to make me want to revert this and go do it from scratch.

01:21:48This needs to go back.

01:22:13And if it's taking that long to load, we're loading the world again. Yep. So straight back into the same hassle I had.

01:23:01So many of these. have these dumb placeholder attributes. Because I didn't want to drag in Fakoo for every last thing, we're going to start getting duplication errors.

01:24:00So I guess these count as improving. I don't know.

...16All right. Better. Now these need to say.

...30And then once everything has done dozens of queries, everything needs the tag specified because this is not smart.

01:25:13This has got to be a common hassle, right?

...59Override. Yeah. Same build strategy of their parent.

01:26:25Create list. Oh.

...38I guess that's what I'm gonna have to do so post user with posts yeah see all of these are this doesn't have any concept of an object a model has to have an association to be considered valid

01:27:35And there's the end of it. So one of these things is probably roughly what I want.

01:28:05So there's a whole hassle in our test setup where story has those tags, tag1 and tag2, because the test helper is set up to always create a tag1 and tag2, and then a bunch of stuff is coupled to it.

...31All right.

...41Yeah, so it automatically always creates a category with those two tags because so many things expect that. And then a bunch of these tests depend on those existing. As opposed to just using stuff that's in the database already. Rails makes all this stuff so hard. This is just. complexity for complexity's sake. I'm just, or incidental complexity is what it is. I'm just fighting against rails instead of writing the code I want.

01:29:44How did that one do it? Was it array.new? But again, it's not going to be persisted. I need it to actually be persisted. use inline associations that's not a trait it's a factory okay

01:31:03I'm going to not blow up every test by saying, well, what if I make a new freight?

...19It just finds them.

...52Now I think this is our validation failing.

01:32:09So we do a bunch of hassling around to get the form to let you edit things. Cannot add inactive tag, so we just gotta filter the tags.

...32Kind of lucky to get that on the first try.

...43So I burned a bunch of time trying to replace prosopite with The Rails eager loading stuff. The eager loading stuff has the minor, not eager loading, what is it called? Strict loading. It has the minor inconvenience of not actually working. And now Grospite has the hassle of working too well. I'm going to replace these with pluck. Because if it hasn't hit the database, what I just need is their names. Good enough. generated the same username twice. You know, seeing that username generation thing twice in two minutes makes me wonder if it has a really limited pool of usernames it generates.

01:33:56Hmm.

01:34:25I could disable press apply. 17. Yeah.

...57It's the actual query it's running.

01:35:14Because I expected I was going to see it say, select the names out of the tags.

...48So at least these are all different lines. OK, let's just disable ProSupply.

01:36:29Still let's move it up to the controller then get rid of that.

...59Hmm. Great. It doesn't care.

01:37:09It's still running anyways. I'm like, I don't know, what, four or five levels deep into yak shaving around a Rails misdesign, where everything in Rails is coupled to the database. I can't loop over the associated stories without touching the database because I have to prevent an N plus one rendering the merge stories. And then because Rails makes one plus N so easy to create, I'm adding prosopite to catch them later. But then because I'm creating multiple stories to put onto cabinet view the check is giving me a what's effectively a false positive like it's correct there is a one plus n but i don't care about it here because this is like i'm toggling into and out of caring about one plus n's five times to work around rails's design

01:38:31And now prosopite is failing anyways, even though it's paused. This is really frustrating and I must be short on sleep because I'm getting very frustrated.

01:39:24Let's make sure it's not just something with this block syntax that I've only used once.

...45No, it's not.

01:40:09So there's some... Something happening where... Oh, I bet I know. It's... I bet one of the... I bet I was just looking at it. So in story... We pause and resume. And that issue I just saw was someone saying, hey, if you have nested pause and resume, we don't track it. And the resume resumes all of them.

01:41:02And this check tags that's working around The fucking Rails way of associating records has to call press a pipe and pause because that code is so fucking painful already.

...41So I got to diagram this. So how's that story go? So there's an American folk song called There's a Hole in My Bucket, where there's just a series of escalating problems. And the hole in my bucket here is... Crosopi complains on creating, or I'm loading tags one plus N. Crosopi, well, yeah, can't be paused, because the story creation pauses and resumes.

01:42:48Because the story creation has a hassle with Rails associations. Because everything is coupled database in AR, which is not specific enough, but is the root cause here.

01:43:22And then what else is getting me here?

...30The template has to save records to the DB, paste everything, cause scopes, trigger lookups. I feel like I'm eight or nine levels deep here.

01:44:01Because AR constantly tries to introduce 1 plus n queries.

...38I don't have to use factory bot for test data. Just got into it because it's complicated. kaayfour burn it all, write the SQL yourself :)
Oh yeah, there's the whole side quest of at user, coupling of global mutable state.

01:45:06You know, K4, it's tempting. It's tempting. I've tried to avoid touching this check tags method. because it's very complicated. And there's this whole coupling of the story form has a method called tagsA, it's an array. That might honestly just be a, Oh, yeah, it's so that we can log these changes. Because there isn't a... Of course, all this runs on a fucking callback.

01:46:38ActiveRecord has now grown a set of convenience methods for every attribute so you can find the old values of it before you just persisted it. And there are like two levels of it where one is when this record was hydrated, what was the value? And then before you called save, what was the value? And after save, something like that. There are like two or three levels of it. And the big sin is that active record models are immutable. And I call it the big sin because it's the source of so many security problems in Rails and so much flaky code where, I mean, we've already seen a bunch of views and methods have to call dot new record to check if they're being passed partially valid data. Because instead of just creating a record and either you have it or you don't, either you're talking about the platonic idea of a story or a specific record in the database, what conflates those two. And so now your story model has to know whether it's persister or not.

01:48:29This is just not my day for coding.

01:49:06If I could rewrite this... Actually... Maybe this doesn't have to have a pause and resume around it, because... It could be that the issue is calling taggings.tag. If I call... Alright, hold on.

...44Okay.

01:50:01Did I typo? Yeah. No, I didn't. Why don't we... Oh, I can't say that and do.

...32All right, so there's the 1 plus n. Yeah, but then it caches, so I have to remember to reload the story between each.

...57So if I add that eager load tag, I think it'll go away. And I can just delete the PresaPipe.

01:51:22God, half the suite is going to fail because I touched a factory, right? And then I'm not going to know if I actually... All right, we're getting there.

...42Couple of failures.

...49This is the exact thing I just touched. All right, so... So who's failing? You are. Stories spec.

01:52:27Specifically this section 496.

...36Just down here. Oh, we're looping things again. Well, this could be a where.

...59I do actually want to. So it's this.

01:53:21So this marked for destruction is a virtual attribute. That's fair.

...37Guess not.

...48twitchtd I was curious about the activity being set to 0% on lobste.rs, any chance you can run the following debug query I just created? https://gist.github.com/thomasd…
Because as soon as I do this, I'm recreating the taggings So then the new taggings objects don't have that virtual attribute.

01:54:10Yeah, Twitch, I can run that. And when it comes back, it's an array.

...21Let's change gears a sec and look at your query.

01:55:01Error in your syntax. Oh, you've left in the Ruby here. So anytime it has this hash curly braces, that's Ruby variable interpolation into the string.

01:56:23Looking at this business logic, seeing if I can combine these two loops in some way.

...35twitchtd updated the link
I can't because of this. Actually, I probably just broke this mark for destruction, right? twitchtd though if you don't want to context switch i can wait
The eager load, the same way I did this attempted edit. Let's try it.

01:57:06I'll be honest, TD, I'm considering context switching away from the stream. I'm so frustrated by this. So let's try your gist again.

...48Is that a... Yeah, you've got an extra semicolon? What is act? Okay, so it's naming that. I think that's just a typo on line 15, right? Because I can see, I recognize this query. twitchtd ya
You're adapting it from Traffic Helper, right?

01:58:25Yeah.

...46blew up my clipboard tonight. Oh, there it goes.

01:59:12Might just be that semicolon, because that names the table. I pasted the wrong thing. Hold on. the right thing in the clipboard.

...51There we go. Alright, your query is running. So there's your answer. twitchtd ah low == high
That looks like a totally reasonable set of numbers for this time of day. So what's the bug? Low equals high. Huh.

02:00:21That's strange.

...32for all of these intervals. Really? There's no variation? I don't believe that.

...49Let's just grab this.

02:01:27Why can't I run this inner part?

...51twitchtd ah you copy pasted the ) act part
twitchtd i think the subquery ends right before ) act
twitchtd at the end
Copy pasted the Oh, you're right. Yeah, I was reading down to that one.

02:02:14Why does this have one row?

...33Okay, so the problem is this inner query.

...49It's been a long while since I've touched this.

02:03:18Yes, you filled this in to just say, check the last 15 minutes of data. How would low ever be different from high in this? Because if there's only one row, there's always going to be low and high.

02:04:15twitchtd doh, good catch, you need to fill that in with 90 days not 15 minutes :S
Yeah.

...46yolosiku_onegaisimasu Any reason this livestream happens on twitch and not Jitsi?
Hi, Yolo. The short answer is because people use Twitch. Jitsi is just not very popular. And as far as I know, it doesn't really have any setup for streaming to a couple thousand people. All right, so we have that in. Then we can run this whole section. That's the one that didn't work. I'm just going to edit.

02:05:55yolosiku_onegaisimasu It's something I've had surprisingly good experiences with for a user facing free software project
I also suspect if I use Jitsi, I'd have to recreate a whole bunch of infrastructure for myself. yolosiku_onegaisimasu BBut yeah valid
All right, so does it have like moderation tools? Does it have something so that it scales to dozens of live viewers at the same time? I mean, there's just so much stuff that Twitch does out of the box that even if Jitsi does something that looks superficially similar, there's a huge amount of functionality we don't have to fight with. So, TD, there's your answer that, no, low and high aren't the same. They only were because when you subbed in that 15 minutes...

02:06:59So we definitely shouldn't be seeing 0% in prod. What is the sum of these? 196 or just 96. We should be seeing something. Feels like I'm looking at a Fahrenheit temperature between 32 and 212, you know? So something around 40%.

02:07:33And we're seeing zero. So I wonder if this is not a query problem, but a cache problem.

02:08:07Yeah, there's a problem. The stored value for the low traffic is nil.

...16The stored value for the high traffic is nil. And so then the stored value for the intensity is also going to be nil. dzwdz the stream notification in the footer is a great touch, not sure when you added that
No, that's just zero.

...33So if I called I just, hey DZ, I just added it maybe a week ago. I also added the streams to the about page because they weren't really described anywhere. So if I call current activity, I get 99, which is, sure, it's one or two off from what I just added up, but This query is just going to be constantly changing a little bit. If I set current intensity, I have to pass low and high. No, that's this. So who calls cache traffic? Because somehow it ran and passed nils. Oh, I think a background job does. So it calls traffic underscore range. Let's call that manually.

02:09:52So that's valid. So if I call, all right, I'm going to fill the cache manually.

02:10:11So those are some, no, book, insert, null, null, zero. So where's the bug? Here's a good chance for somebody to become a channel VIP by spotting a bug before me. So if traffic range is returning valid data and it did it said low and high oh i bet i bet we're getting bit by a type error here that this got tweaked so it's returning a hash, low and high, and then this is assuming an array. And so that's why we're all right, so

02:11:27kaayfour delete the `.first` ?
kaayfour just get back the array?
If I call traffic range.

...42I don't want an array. I want these two values. So I could call dot values and I would get the right thing.

02:12:00I'm wondering if I could just.

...18So I'm going to get nil nil. Oh, no. Well. And it's coercion is. Helping. I don't think I can explode a thing like that. Yeah, that did not give me what I want. So I really do want to just throw in values there. So let's edit it live in prod first. This is fiddly. Whoa, man, for everybody who misses a color scheme. There you go. Some angry fruit salad.

02:13:1833, 293, exception.

...45So current activity Is this... This has been broken for two months.

02:14:06Yeah, it's when we replaced this execute with select all. But there's another one in here. Yep, yep.

...18There's no contract between these things.

...33I want two. I want to see these.

...44Don't you feel confident in code like this to array first first value like. What are we doing here.

02:15:09kaayfour oh yeah, primitive obsession ftw
No define method values for an instance of array. because I'm just getting back a single thing. I wanted to say first, first, first.

...48If I had cutesy Twitch integrations, I would have that gif of Bill O'Reilly screaming, we'll do it live. And I would play it whenever I edited code in prod.

02:16:20What's going to be really funny is I'm going to get this working in production, and I'm going to copy the code back to local. And then tests are going to fail. Why do you still say zero? Oh, so I'm logged out of lobsters. I'm seeing a cached page.

...51I slapped that on. Actually, maybe that gets ignored by the full page caching. I don't remember, even though I was just looking at it the other day.

02:17:09Still getting zero, even though we cached. Because we inserted zero.

...22All right, so why did we insert 0? We called current intensity with low and high.

...48This hassle with current activity, which is just the center of that loop.

02:18:06If SQL is more composable, I would have this as like a Nobody say view. We have so much pain from views. But I would grab these rows, and then I would know the current was the lightest. Instead, we have to define them twice.

...56OK, so the query is fine. It's either something dicking around here with the results.

02:20:06It wasn't first, first, first I wanted. It was first, first, second. That's great. That's really a wonderful API to couple to. Not at all brittle.

...30gtfrvz caddr
kaayfour could also do .rows
Yes, that's exactly what I'm doing here. People ask why you don't write stuff in Lisp. Like, we could do that forever. There we go. There's a reasonable value put into traffic intensity. Because I had said somewhere around 40%, just eyeballing it.

...55Yeah, I already got something that busted the cache. There we go. Oh yeah, the search page, even the form isn't cached. All right, so there's the bug. Let's get the heck out of prod.

02:21:17Save a stash.

...32rescue that traffic helper that we just edited and don't know the path to app helpers, traffic underscore helper. Oh, there's a extra directory in here. And then

02:22:05Were there any other queries touched by this commit? We already did the first one.

...25Wrong clipboard.

...37Well, TD, thanks for getting curious about that. Ooh, moderate. Exciting. Cat is laying on my arm, so now I can't click or type very well. Thanks, Kat.

02:23:17Pull up this Dependabot alert off screen and see if it's anything I care about.

...36If we This probably doesn't affect us.

02:24:11So there is a bug in 3.1.10 where someone could inject something into your logs. I don't think we're vulnerable to this one because they would have to include particular... They would have to use rack auth basic, which we don't use. But I tried to update RAC, and I guess 3.1.11 has not been released yet.

...56Or I have something else. Yeah, none of these have a constraint on RAC that it not update past 3.1.10.

02:25:16Yeah.

...45I'm not going to worry too much about a thing we're not actually vulnerable to.

...58And it can only be used to put nonsense in a log file that we're not actually looking at. So I'm deploying that fix.

02:26:40And as soon as that's done, I'm going to wind up this stream. I usually aim for about three hours, but I am too frustrated to continue. This whole thing with fighting to put records in the database, and I can't put records in the database because of the tool that prevents me from... It's the fact that Rails has a misdesign where everything is coupled to the database, and I have four layers of things that work around that misdesign, where there's factory bot, there's that global variable, there's its associations, there's tags underscore A, there's prosopite, and we're stacked up, and these five levels of things that work around this misdesign in Active Record are all conspiring to prevent me from adding a very simple, like, take this template, render it with this data.

02:27:49And I'm just too frustrated at wanting to do something very simple.

02:28:00That's going to be the title. There's a hole in my bucket. alanweiss rip
So if you have any last questions about the site of the code base or like TD there, you want to catch a bug, I would love it. But otherwise, I'm going to wrap up. Oh, hey, Alan. Yeah, I got to sleep on what to do there because that's just, oh, good. Now I got another conflict.

...40Wow, because I started copying the fix locally.

...53kaayfour you always told me to 'write the code you wish you could', then figure it out
Oh my god, now I'm fighting Git.

02:29:04K4, I don't know you under that handle. You could whisper whatever handle I would know you from or something else. But otherwise, you're using a handle I don't recognize, and it sounds like we've talked before. Because I do generally write the code I wish I could and then figure it out. But in this case, it's just all Rails plumbing.

...42Oh, I have an idea who you might be based on that K. But I don't have a sufficiently indirect way of saying it on stream that wouldn't involve saying your name or your initials.

02:30:06kaayfour just go by K now
Did we watch Gary Bernhardt over lunch? Is that who you are? Yeah. Okay. Yeah, okay. That's exactly who I was thinking of. Okay. Yeah. You know, the code I wish I could is probably like... Oh, no. You know, so I've looked at the last couple of streams. We had Joel Draper on, and he's got the Flex view component library. And so every time I touch views and I get annoyed at Rails, I think about Flex. He just released 2.0 on Friday. But this one, I would just end up straight back into it because there's the one part of the template that's going back to the database to find the merge stories and flatten things out. The hassle I'm having is still with Active Record. It's not actually a view thing. I'm just exercising it from the view layer.

02:31:38Well, I hope you're doing well, K4.

02:32:14Okay.

...44That's such terrible.

02:33:08If this fails, it succeeds. This change is sprawling where I keep having to fix a fix to a fix. Yeah. This one is probably failing because I touched the factory.

...40No? Ah, it's just already staged.

...49But I didn't touch the tags field, so who knows why that's failing. This change, I wonder if I could just extract this.

02:34:37So I'm trying to put all the cabinet list detail stuff away into the stash.

...53Just grabbed everything. I thought save only take, I can save patch. Oh, save dash S for staged.

02:35:11or I can say this.

...33So now all that is put away. Do the specs run?

...46not even a little so it's just something about trying to check it like this it's failing it it's not marked for destruction it's not media count it count as 0 given the exception.

02:36:31Let's back this up.

...46Make sure I didn't introduce stuff. Yeah, that got reintroduced, but that's fine. Just delete that. And this is going to give me a precipitate error, right, on that code on the bottom?

02:37:31Something about this, probably the eager load was causing it to refresh the tagging model. And then this virtual attribute marked for destruction was going away, which is why it failed up here. Yeah, so this one now will be the crucified error. about this second section at 495, looking things up.

02:38:14And this code is broken, but no test is failing, because it's doing the same thing.

...39This is probably wrong too.

...48Hmm.

02:40:05And then I'm going to get back an array, and then I'm going to have a m times n, you know, oh, m times n performance where I walk that array for every time. They can't have an ID. Tagging still have an ID, do they?

...40Yeah. How do they do? At least that much.

02:41:02Can't even type new. I'm writing some bad code.

02:42:03That's terrible. That's some terrible code. It's probably going to work, too. That's the worst part. Now I'm going to get tempted to commit terrible code. Oh, good. It failed. Why did it fail?

...29It shouldn't have. I guess I don't understand the failure, because the... The eager load there is refreshing the associated relation. And then that's what's blowing away those marked for destruction tags. And so I thought it was saying that that was always evaluating to false.

02:43:20Yeah, all of this code is painful because validation happens on the model layer instead of on the controller layer. And the big pain here is there's authentication happening here where there are certain tags that moderators and the admin are allowed to use, but users are not. And there are certain tags that new users are not allowed to. So the question of validation depends on who's doing the editing, not just on the model. So you have to know who that user is, which means it is a controller level concern, but validations happen down in the model. And so there's this question of, is the model valid? And that question is insufficiently expressive. What we need to be asking is, is this edit valid? but you can't really express that in active record. We would have to create like a service object, a story editing model, and then try and reify that into the concept of editing a story. And then all of this stuff would move up there and we'd still have this pain because we'd still have, we'd still have to walk the associations.

02:45:05Oh, use findEach. Oh yeah, because maybe I have more than a thousand taggings associated. We even have 50 tags. Ah, I do in the database.

...38All right, I'm going to quit banging my head on this and try and do something else entirely. Change gears. Thanks for the folks who hung out today. Pretty frustrating stream all around here, I bet. But this is the very exciting work of maintaining a website. Is occasionally you just end up fighting your tools for an hour or two. Anyways, thanks again to TD for starting that investigation into the broken traffic intensity I appreciate it. and Everybody have a good one take care the next office our stream is scheduled for Thursday morning 9am Chicago time a different time in other lesser parts of the world.

02:46:41And if you want to Oh, the thing with freaky they didn't drop by this stream but freaky notice that chat logs were missing from the stream archives.

02:47:01And I fixed them. Where they ask. NoGoodNick_ cheers
Speaking of potential stream related bugs are the archives meant to be no longer include chat nope that was just a bug I. had accidentally written it. There was a difference between the timestamp as an integer being is this a unix epoch timestamp or is this a. second since the start of the video timestamp and the two edited and slipped out of sync and ruby doesn't have types so didn't catch it actually i thought preachy had a was a channel vip hmm i'll have to dig back all right yeah that's all i got hope to see you on thursday take care folks