I Just Made Another Silly Mistake

01:24pushcx This is office hours, ask questions anytime!
graefchen Heya limesHi
Dig in. Yeah, when folks don't have questions, I just work on the website. Because there are always things to fix. Hey, Grave.

...41Let's see, we got polls, issues, I think actually things were quiet over the weekend.

...56Oh, there was one thing. Merged.

02:10Yes, I merged this one on stream, I think last time, and this was a nice little improvement for having a helper. And then I think this is just hanging out. let me double check on these two i'm not sure there's anything ready to merge but maybe this is the options oh right build failure ah that's that's part of the bugs i'm into let's let's put that on the to-do list so we've got the

...54my windows are funny all right tag list but before that we've got to do the fix main yeah and then this is the one that got reverted Well, it was a fixed code that got reverted. This one, I fixed this in prod, but in a hurry. So there's still a test to finish for it.

03:47Okay. And then That's all the changes since the last stream. So the other things I had, because I mentioned in my little pre-stream announcement that I had plenty of bugs, is that there was a error out of the Mastodon connector. And then there was maybe a bug or two still hanging out in the job queue. So I will look into those. And then, otherwise, we'll hack on stuff and talk about anything folks bring up. Pretty straightforward.

04:34Search itself is yielding. Yeah, this was why I was so puzzled.

...59pushcx https://github.com/lobsters/lob…
I'll get more into it when I fix the CI build to be green, but this one, it inserts a comment in the database and then it searches for a comment and it doesn't find it. So if anybody wants to sneak peek at that one, there's the GitHub commit with a pending test. And the weirdest thing is if, laryngidose checked out thing before it still fails then it really is doing what i thought it was doing i don't know what that one all right so bugs bugs bugs let's get a full list of the bugs

06:03Actually, I'm going to do the Mastodon one because while that's running, I happen to know pretty much exactly where the Mastodon bug is because we've seen a couple. When users can link their own mastodon accounts to their current accounts and i kind of hacked this up pretty fast and then have been slowly catching exceptions and exception in prod most of which are like if you link your account and your mastodon instance goes away how do we send an api request to unlink your account and the answer is We can't. We can only drop it from our side. We can't really do anything on their side.

07:13And this exception is saying that in the Settings Controller Mastodon Disconnect. 225. Let's put this on the left. We'll go left to right here. That calls app.revokeToken, which is here. And then that calls fetch. That calls sponge. And then line 148. Raises a DNSer. Ah, so it's wrapping... the exception that's why it didn't get caught so we wanted to catch it i think down here this does not look like i remember so dns So there's a fix, so we won't have an uncaught exception, but that's not what we want, because what we want is to delete their token, which will happen. Yes, okay, so that's fine. So this adds an exception, or adds the error message, and then this checks and says, oh, did I get returned an app? And the answer is no. And then, or actually no, it's in here. So there's no check on the return value. And then regardless of whether we were able to revoke the token, we go ahead and delete it from our end and save. Yeah. Okay. Let's note that.

09:34I want to note it here or there. Let's note it here.

10:47chamlis_ hey all
pushcx https://github.com/lobsters/lob…
new one hey chamlus if you wanna be nerd sniped you should check out that commit i put in the chat just ahead of you that has a test that is failing for no discernible reason there's a little discussion of it down at the end of 1809 here but I'm not ready to jump into it. So if you want to like beat me to it, I have been puzzled by it. And another contributor has looked at it and was like, well, you're not making an obvious mistake, which is nice to know, but now I'm either making an obvious mistake or something very strange is happening. I honestly, I wonder, I wonder if there's some kind of like search index I have to update. because the test is running in a transaction. That could be it. Oh, maybe. So off stream here, let's just write this.

12:16Mass.

...56So this is the kind of... Pete. Who's Pete? This is the kind of message I send folks when I fix bugs. So exactly one person hit this bug and got a 500 and I can see their... I can't see their username in the bug system. Why do I not see their username? I see their IP, and I see their cookie, and I see their master on name, but I don't see their user. Let's put that on the list. I thought I fixed this before. So there's that. There's a blogging username.

14:07bsandro hello cyberpals
So many clipboards in Linux. As soon as I'm streaming, I'm always getting the right one. Hey, Bessandra. All right, so let me go find this user's name in prod because I need to know it. So that's going to be a select username from users where settings like quote percent the name of their instance.

...58And then now I have their username. Kind of wondered if it was a single person instance, and the username kind of matches the DNS name. All right, so there's that message sent. Cool. So we're doing things out of order. That's fine. I really should fix main before I go on with this other stuff, though. It's bad to leave the build broken, especially for contributors. And then I can come back and try the Telebug stuff.

15:53Inbox mailbox spec.

16:02So these, okay. I fixed a bug or two here in the inbox mailbox last stream or the stream before. I included a bug just to keep things from getting boring where I think maybe here I had pulled this from the old email parser and I left... Actually, I bet if I hit U... Yeah. There were a couple of these variables like this email and the str was actually string where the variable name in the new code didn't match the old code and I didn't catch it because this is lightly tested. And then I fixed one more, one more, and then got to this place where I think we're working in prod, but the tests aren't working. So let's just get that caught up. So it's this first test. Let's do the top posting one. Spec mailbox. Inbox spec dash E post.

17:4166. 66. OK. So this is running. But it's failing to create the comment. So the point of this test is that if someone replies and they do a top quote like this, we want to strip that out. And if they have a signature line, we want to strip that out too. So let's quits mail body, because Chambliss on the last stream already caught one possible bug there with indentation. So now I just want to look at it. That looks correct. OK. Can we see here the processed mail? Status bounced. My kingdom for algebraic data types. Instead of having record of the bug i just have and like i would like to know which of these bounced methods were called but we can't know that shouldn't be multi-part because you know i just set the body to be a string So we did not see multi-parts. It's not this one. If the sending user is nil, if the parent is nil. All right, so let's multi-predicate. I wish it was easier to debug these things. I've been thinking a lot about language design this last couple of days. OK, so yes, we are in here. Which one of these is failing?

20:18So it is blank and we found the story. All right, so that's a bug in decoding, which could be one bug for all three of these.

...53See what we get for these.

21:05Content type. Content type, more like content typo.

...23Type is text plane. And the body raises an exception. This should probably be male body. And we are saying body and then pass that over

...51Hey, look, I added debugging and the test started passing. Let's look at the diff. So I guess this was silently failing the mail body. No, it was throwing a 500. No, but I got down here and I saw this error message. There's no rescues here. All right. I'm suspicious, but I would also prefer to take the win. Let's see if that fixed any of the other specs. That did. OK. So what bug ID was that? I don't think I have a bug file for it.

22:56WhiskyFueled42 hey guys, hope you are doing well
All right, so let's go ahead and push that and deploy it so that anybody who tries to post a comment via the mailing list can do so. Cool. So that's two bugs down in 20 minutes. Oh, hey, WhiskeyFueled. All right, so is that the only reason this test failed? Yes. There's my three errors. Let me give this one more skim and then I think I can just hit merge on this PR.

23:38Tags title. Got the tag link. It takes an option now. Yeah, it's the old positional argument, but that's fine.

...59Should this be options merge or options reverse merge? This is going to override the options. Yeah, that wants to be reversed, merge.

24:55Frici Greetings!
OK, free cheat, good to see again. All right. I'm just going to hit commit. And that will probably start the build again. That's fine. So then these have the new divider tag. Looks good. Alrighty.

25:28OK, we'll just give that a minute, and then I'll get to merge. All right, so let me pull this up. It's just the one PR. There's just this one issue. Let's look at telebugs for a second. Wrong terminal. I want to keep the scratch up. Frici Finally catching you live again! work has been so over the place i've been consistently missing your office hours. I heard today is very productive as I was tuning in.
So this code, the sentry, so telebugs is our bug recording system and it uses the sentry gem. A bunch of these competitor bug reporting systems use this same gem.

26:25And I feel like what's happening is we're getting an initial call to user. And then the second one is ignoring things, but no, we have the IP address. So off screen, I'm looking at the person who had the mastodon and their thing was only hydrated with an IP address, which I think is the low or location app controller app controller.

...59Yeah, so there is telebugs.

27:09See, we call it once without this stuff, and then we call it a second time. And I feel like the second one is getting ignored.

...48Ah, yeah, well, I fixed a couple of bugs just before you joined the stream, and now I'm chasing another. This is not what I want. I want the actual code. Authenticatable spec. Helper. No, this is also a spec. Authentic. There we go. The concern. See, so down here... Once the user is loaded, we set all of this stuff. And I don't think there's any other place that at user is set. Come here, let's just look in app. Yeah, so these mailers are fine. Authenticatable is the one we're looking at.

28:54And the rest of these are internal to these classes. So this line of code has to be run for the user to be logged in.

29:08Since that also calls this sentry.setUser, I guess it's ignoring the later call. Let's see if I can recreate that. Can I look inside the sentry?

...27So there's a set user. Is there also a get user?

30:04So we have a scope or a syndicate, right? So, oh yeah, I don't know. Let it nest.

...23So it expects it to be a hash and then it overrides. And then over here, if we call set user, return unless initialized, and then get current scope set user. So whatever this scope is, let's try that again.

...59Get current scope, it's promising, nil. And you know what? I didn't even spell address right.

31:21I'm being indecisive about variables versus values.

...29Where did you put it?

32:01I wonder if this is being different because it's probably just going to error about database not defined or something. Yeah. Let's go look at prod. There's just too many potential differences. Let's just go look at prod.

...26So we'll say. Sentry dot set user. Let's try version colon one.

...43What was it? Get current scope. OK. This looks like a bigger object. We've got some data in it. We've got a scope, fingerprint. We got the user. Yes. Okay. So it did have the user.

34:22All right, responded to a message off stream there. All right, so now if I said version two, scope says version two. And I wanna say there's, this might be in light mode. No, it's not, good.

...54I want to say there's a description of how to log a test message. Yeah, sending your first error report. You add the SDK. You do all these things. We've done all those things. I don't want to do team management. I thought there was something about sending a message.

35:31These docs have been updated since I first read them. And for the better, there's a lot more here, but... Yeah, we're looped around. There's a way to select .log.

36:01Sentry is down. Oh, I wonder if their detector is telling them. Or maybe they just changed. No, they look down, down. Or the ad blocker is blocking them. That's possible.

37:10All right, so that's going to sit for a while. I did see Internet Archives say something about a recent outage on Blue Sky, so maybe they're down. It sure doesn't look like that call to set user needs to be in one of these configure scope blocks.

...46breadcrumb get added? I don't remember what that is. Yes, it does. It's showing things like the query that ran. We do have

38:17Oh, they put set user inside configure scope.

...29I wonder over on prod, then how did it get the IP address the first time? So what if I call telebugs user with the IP address This is not type checked, three. So I'm going to ID one username, one email, one at the lowest number. But then if I go peek inside the sentry scope, I see all of that. So my telebugs method worked.

39:29Hmm. Why isn't it running in prod, though?

40:19chamlis_ for the search issue, if it's an innodb index, I think it's because the fulltext index doesn't include new rows from the current transaction, which rspec begins for each test
Does all this context come in?

...27Requested. Yes, so I have a context block in teleblogs that has the basic OS info and a hash and then like a request And then it has this hash.

41:20This is one more of those I'm missing something obvious bugs.

...30Message is the method I want to call. All right, so.

...42All right, it's not on center, it's on.

...55Okay, so there's my user enrichment in there. If I go look in the bug tracker, do I have a bug as of two seconds ago with my user on it?

42:29chamlis_ just tried nesting and that didn't work for me
yeah chamlus that was kind of the thing i was mumbling about it as a possibility if we nest a transaction explicitly like call comment.transaction create the comment and then outside of that transaction do it or do we have to close the whole okay respect i wonder if there's a way to close that transaction and not

43:43Cool. What are you doing? So much JavaScript.

...57Well, this at least is in the last couple of years.

44:42Oh, that's interesting. A before all and after all run outside the transaction. So I could shove the comment creation up. I don't love it, but that would do it, right?

45:36I definitely don't want to turn off that transaction wrapping for the whole suite.

46:02chamlis_ could that break other tests that assert on the total number of comments?
yes that would certainly break other tests that assert on the total number of comments unless cleaned up after if this one test has to clean up after itself that's not the end of the world the other thing we could do is just like execute the raw SQL query commit to close the transaction and then do the search and then open a new one so that when our spec abandons the transaction at the end, it doesn't. Right, so like we're kind of breaking the encapsulation. but we know that RSpec can't really monitor all the queries. chamlis_ neat
So we could just sneak our query in to end the transaction, commit it, then go do our thing and then delete the comment. Like it's ugly, but we would have a working test of the search. And you know, if you put a one-liner comment over the top of it, does that make sense?

47:42Oh, what if prepare exception notifier is running after authenticate user? oh that would do it that would do it all right let's drop out of this come here ah they're not called filters anymore what are they callbacks how do i inspect the

48:51i'm still i'm in that nested vim aren't i yes so or it's just then okay so there's a callback chain there's an

49:21callbacks method okay there is a callbacks let's let's get a new terminal in here so i can actually see what i'm doing the the vim terminal is nice but every once in a while gets these kind of confused things. So let's say application controller callbacks. Oh, no, it's the pager.

50:13There we go. Fix that in a minute. Put that on my to-do list.

...30It's going to process action. I expected this to be an array.

...52so there's prepare exception notifier where's authenticate

51:22You say? All right, robot, let's try you. I don't know how to read line.

...55It was underscore process. This is the same thing I was just inspecting. Well, there's a chain. I guess this is an array. This opens here, closes here. All right, so wait. There's an authenticate user. And then later there's prepare exception notifier. So these are running in the wrong order.

53:00Is there a prepend? Because what I want is for the repair to happen first, no matter what I have that context. And it's prepend.

...20Prepend, yes. See insert callbacks. an array hash of options you want to tell me what the available options are some filters like only an accept okay so we'll just try prepend

55:09I want to write something longer.

...50Is it spilled with an iAuthenticate table?

57:04it feels real good all right so that's been a pain for me for like couple of months since we started using telebugs so nice to have that fixed oh the tag list i bet the green is build is green right yep what's left i think i can just squash and merge this because it's small

58:54And i'm stream but that. Telebug's test message I sent did get enriched with the correct affected user. So, things are getting better.

59:40Make sure I resolve that.

...50All right, so we can close out some tabs, which is the best part of bug hunting. back to this test. Channels, it sounds like you pulled it up and tinkered with it. You wouldn't happen to have finished solving that while I was dealing with this, would you? What is it? The search spec? Yeah.

01:00:28Let's see that failure. chamlis_ I've tried the COMMIT, it looks like there are magic SAVEPOINTs as well that I haven't managed to mimic correctly
so this big exception is saying yeah so what the test does is it creates a comment that says hello world and then it sign says let's search for hello and the page should say world and it does not you've tried commit it looks like there are magic save points as well that i haven't managed to mimic correctly i don't know what a save point is in this context

01:01:10oh you can do a like a partial transaction rollback that's interesting i it's a named marker without cancelling the entire so it's kind of like a a nested sub transaction but it looks more like goto. Sort of. chamlis_ I'm on a laptop over ssh atm so feel free to take it on as you'll probably be quicker
All right, so, respect docs transaction per test. Okay, so,

01:02:15Database cleaner strategy is transaction. Let's find out how database cleaner.

...30Google, it's not me scrolling down to the bottom of the page over and over. It's just Google being JavaScripty. Oh, bright.

...47config before suite well can i just say let's find the docs for this

01:03:24Transaction needs to know, are we running the test inside a blocked cleaning?

01:04:00So by the time this test runs, it's in the transaction. So probably reaching out and tinkering is not going to do what I want.

...32Is there a, okay, so there are strategies.

...43Strategy, that probably even spells them right. That's not what I wanted. I just wanted to inspect.

01:05:23there is an object named health strategy strategy equals okay what if that's a yeah you're not actually using the object

01:06:05The empty does not exist. OK. That's progress. But that's an exception about saving a story. So changing the strategy must have closed the transaction and dropped the tags out of the database because there's sort of a fixture happening here somewhere.

01:07:01What is it, tag1?

...07rails helper it's in rails holder yeah that's an odd bit of config the strategy is transaction but we clean with truncation feels bad

...47That still did not work.

01:08:15Yeah, so then I see your save pointer there, Chamlus.

...30chamlis_ :(
All right, so I don't think I'm going to be able to fix it from inside here. I think I should configure a new

01:09:00that let's grab this out and we don't want that we don't want that and we'll say before fall database cleaner dot strategy equals deletion I think

...35Still think there should be a safe point to open. So this is separate from the cleaner. ArcSpec is putting a transaction around things.

01:10:05Can I introspect the current transaction? Yes, I can look at this. All right, let's try this robot.

...39I just got back, it returns an integer representing the nesting level of the current transaction. Or I can say dot current underscore transaction.

01:11:18Well, there's an object and it's a save point transaction.

...28What's a save point transaction?

...54Looks like I could maybe close state commit. Release save point, save point name.

01:12:16Go back to save point.

...28or new.

...35So of course it works as this object mutates itself.

01:13:18So when I say, look at the top one, and then we get two. Oh, the error is coming at the end of the spec though, isn't it? Yeah.

...37I was thinking these were reversed. Let's get this back in here. Make sure that we're seeing what we expect.

01:14:45So I can instantiate a new one of these with the same name, right? Will that automatically actually do it?

01:15:08So it wants the connection. and it wants the save point name which will give it the existing one and then the parent transaction can we look at that let's get it out

...46no we cannot all right let's just say nil right who doesn't love a good new one ruby

01:16:37That was a weird edit. I must have typo'd something and standard army got confused. Maybe I only had one colon or something. I'm going to find method new.

...53For an instance.

01:17:02So rather than call that, I will say active record connection adapters save my transaction yeah so that's two fields all right Let's go the other way. Instead of commit, is there an alternate command I can give MariaDB that says, no, in the middle of the transaction, I want you to rebuild your index?

01:18:07for innerDB only committed rows appear, modifications to the current transaction do not apply, and nothing else.

...53I really want to force you to re-index. I just want to have one fucking end to end test.

01:20:43Oh, so useTransactionalFictures is not actually transactional fixtures. It's the whole thing.

01:21:02Be rolled back at the end of the example. Data created in a before context. are not rolled back.

...31All right, so let's leave all that spelunking out and add a or context to Add comment.

...57You know what? I'm going to do the simple thing. I'm going to be ignoring, and I'm going to do the simple thing.

01:23:10So the other part of this that I want to test is that it's upvoted.

...26So here's the comment and its folder. And the upvoter, that sure doesn't look like it's checked.

01:24:10So I need to look at this.

...20So I'm logged in. Let's find a comment. Consequitor. I have uploaded this comment. I see that it's uploaded. I search for Consequitor. find a lot of these comments. All right, hang on, I don't actually have to, let's upvote that one. Okay, it is hydrated. How do I see that in the source off the top of my head?

...59The comment has the class upvoted.

01:25:09It does not have the class upvoted here. So it works, but it doesn't test. So that is, what a stubborn test this is for the second time it works, but it doesn't want to test. All right. I wonder if this is because sign in stubbed chamlis_ the user was created after the comment, so not sure how they'd have upvoted it?
Specification helper.

01:26:02Oh, you know, you're right. I forgot to put that vote in the database.

...27I don't love this method signature. You know, it's a test. I could just go make the data. Now let's do it. on one on hello world comment story id hello no user id is

01:27:40of course this side effectful method doesn't have an exclamation point on it number of arguments expected five to six so you got a user id new vote that was the one Wait, how did I? I was reading this line instead of this line. That would have been painful to debug. One of these days, I'll fix that method signature. All right, so it goes comment ID. No, it goes story ID, comment ID, user ID, positional arguments. Hey, it passed. Hey, the glasses says upvoted. OK. Let's move this up to the top because it's global. Well. chamlis_ nice
It's not the global context, it's the described context, so it's a describal, not a global. chamlis_ do you want an after to destroy the comment?
All right. What was the bug ID? Do I want an after? Yes, I absolutely do. That's a really good idea. Thank you.

01:29:44will it i'm not sure if this will work because i actually have tried to make it so we can't delete comments all right so i should probably remove puts from here. Okay, that actually works. That's a little scary, but okay. Because usually we soft delete these comments everywhere. And if I don't do it, I do have a comment, don't I? Yeah. Alright, so it has to happen.

01:31:51I don't love it, but I can accept it. Let's look at that diff. All right. What was my ID number? 1809. Is there anything else here? No. All right.

01:32:336, 1809. I don't, where's

01:33:23Chambliss, this is you, right? I'm bad at matching up usernames and human names. But I want to give you the co-authored by credit.

...38chamlis_ yeah that's me, don't worry about credit though
I've got to double check GitHub. Maybe she's not watching anymore.

...48Ah, it's too late. I already wrote it. chamlis_ you had the idea already
But you figured out the transaction part. All right. So I had the idea, but I didn't know it. It was just one hypothesis. And after I already beat my head on it for a half an hour. All right. Oh, because I merged. So we're going to have to do a quick rebase.

01:34:26Amen.

...54You know, it doesn't matter if I deploy a test, but it feels good, you know, to bump the bookmark. All right. Yeah, so that, two, three. Man, everything's getting fixed today. If this was a horror movie, the monster would pop out behind me for saying that out loud.

01:35:30chamlis_ gem.coop reverted?
Oh, you know, Olivia, I just realized your GitHub avatar is, was the little guy's name Totem from Monument Valley? That was a really cute couple of games. And yeah, I reverted that because I have to be downstream of Ruby Central. I don't yet for the other folks, and I don't trust them yet. chamlis_ yeah it's totem crudely edited into bloxorz (flash game)
I'm trying not to open the whole can of worms in part because moderating is harder if I get into the endless Ruby drama. Of course I have opinions on it because I've been programming for a while and following. Oh, I don't know Bloxors. But yeah, the tale of that thought is... Yeah, I don't know.

01:36:42Let's look up block source. That sounds more fun.

...51Oh, it's like Steven's sausage roll, huh? Okay, I've seen a couple of these, probably even this individual game.

01:37:26Yeah, I've played this game before. Wee.

01:38:04Do I teleport even if I... Oh, okay. It's just a switch.

...14chamlis_ good to branch out the streaming career
All right. I am not actually a Twitch video game streamer.

...30Next, I'll start doing speedruns of some obscure game. chamlis_ there's a monument valley 3 now by the by, it's more of the same but still pleasant
And then I can alternate speedrunning the game and writing a tool-assisted speedrunning tool on streams until Carl Jobs catches me and does an explainer video about how I'm a cheater. Oh, great. Yeah, I think I heard about that. You know, for classic games, there was a great game called Triple Town, and the studio has moved on. And so it's in that, like, abandonware limbo where you can think it's available on Steam and works. Who knows for how long. i think it's available on ios because they haven't had a breaking change yet but it stopped being available on android because breaking change and so i have a friend who plays the heck out of triple town which it really is a nice game can we find a it's sort of a match three game but it's its own thing i was just thinking about this because i was chatting with her okay so yeah here's here's some gameplay we're gonna mute that oh don't need a such a fast all right here just regular gameplay so you put down these objects and you have to put down angry panda bears to get in your way when you put together three grasses they make a bush when you yeah this person did not do that well when you put down three bushes make a i think a house and then three houses or no they make a tree and then three trees make a house three houses make a church and it goes on and on and it's just a very chill low stakes game and i have this fear that someday the android version is going to break and i am going to have to hassle these game manufacturers to let me update the apk just so i can get it back in the store for my friend who just you know sometimes people like fix on a game and they're like that's it i'm just gonna play 13 000 hours of candy crush over the next 10 years she's been playing a lot of triple town over the last 10 years so You can't replace a touchstone game like that. chamlis_ was just gonna say that's how people get to level 10000 or whatever on candy crush
They're like kids with the stuffed animal toys, the plushy stuffed animal toys. Well, you cannot replace the rabbit toy. Yeah, Velveteen Rabbit. ChaelCodes Hello.
All right.

01:41:51Oh, hey, Chael.

01:42:00Yeah, I think Candy Crush is that same kind of game where it's not super hard and it's just very relaxing to play. And I think there's a place for that kind of game. You know, it's the worry stone of games.

...24So in the job queue, there were a handful of errors still. And it looks like there were three kinds of errors. I'm kind of skimming off screen because I don't remember if the job queue might have user IDs and IP addresses and such in a job. And I try and be deliberate about not throwing user info up on screen unnecessarily. And then I'm also pulling up telebugs off screen because all the bugs that I've resolved in the last couple of days can also be resolved.

01:43:40And Telebugs isn't loading, so let's look at Mission Control.

...57ChaelCodes Not sure how the game talk got started, but the creators of Monument Valley made a game - Assemble with Care
So Chael, I had someone message me.

01:44:04Creators of Monarch, assemble with care. I don't think I know this one. we got on to monument valley because chamlus has a clever avatar on github and i recognized half of it oh is this one of those packing games i have seen a couple of these games where the whole point of the game is you're organizing objects or repacking boxes

...48ChaelCodes Yeah! But it's repairing objects.
chamlis_ ooh nice, thanks for the heads up
ChaelCodes It's on mobile, and really well made.
ChaelCodes I loved monument valley.
these things had to start with gameplay oh so you're like tinkering with objects interesting i did enough of that when i was younger yeah you know i i can see how this sort of ChaelCodes Great game devs.
pushcx https://store.steampowered.com/…
scratches the edge of putting things together how fun i'll share the link here in the chat for anybody us two games do they just have the these couple then alba a wildlife adventure okay

01:45:50Oh, the developer is apparently streaming Monument Valley right now. Huh. Oh, because they're coming out with an expansion to Monument Valley 3. Hey, there you go, Chambliss, so you get even more levels. ChaelCodes But you were saying that Miles DM'd you.
We learned something today.

01:46:20ChaelCodes He told me.
chamlis_ wowee that's opportune
yeah i was i was saying that it was very funny because i got a message that said i just ran into two people who work on lobsters here and i was like oh that's interesting and he was like yeah one of them she was really friendly and i was like oh you mean rachel yeah obviously that's rachel and he's like i don't know i don't remember who the second one was and i was like Yeah, I don't know either. And he described the second person. He was like, Oh, yeah, they're involved with lobsters. And I was like, involved, huh? That's funny. Like, either they mean that this person contributed, or I don't think any of the moderators are at SF Ruby. And then it turns out that he thought I was in the room there because of ChaelCodes Irina pulled me up on stage for Hack Day and I said "I promised to build modmail for Peter" and he thought I was looking at someone
the way you had phrased something about the pull request for ModMail that you're working on, and he just assumed I was around, and so he was telling me that I was at the conference, and I was not in fact at the conference. Took us a minute to work that out. It was a little close to being a sitcom. All right, so I'm looking at this exception in the log, and there's a few of these. A whole oh wow. Well, good thing you have that experience speaking from the stage that you. could handle it. Man, I cannot. I got the impression that SF Ruby was a fairly big conference, like 500 people. I cannot imagine with no warning getting pulled up on the stage to be like, here, talk to 500 people. That would be no fun. Notify comment job line 22. ChaelCodes 403, but hack day was more like..
So this tries to create a notification. ChaelCodes 20-30?
And then the exception says, you already created a notification. What else creates it? 403, oh. Did they do the old RailsConf thing where there's one day of presentations, a hack day, and then one day of presentations?

01:49:23ChaelCodes Kind of? Opening and closing chats and demos
would have created opening and closing chats and demos. Okay. So this Delivers mentioned notifications, but it's trying to mention or it's trying to create a notification for a user who already got one. So yeah, this shouldn't be an exception because this is fine. So the bug is that somehow Someone has mentioned, but it didn't recognize that they already got a notify. So I really do have to look at the individual comment to figure out what's going on here. So I'm pulling up, yeah, you know what? This is a public comment. And learning the idea of one of these is not the end of the world. Let's go look at prod.

01:51:10OK, so this one does app mention a user. I wonder if they mentioned the person they're replying to. They did. Aha.

...41So they already got. Yeah, so the error is they already got a reply. And you can't get two notifications of a single comment, which is good. So who calls it comments controller?

01:52:17What creates a reply mailer? What tells someone that they were replied to you? Email mailer, reply mailer dot reply. So this job does both despite the name and I'm not seeing it. So this should have taken the list of notified users and passed it on, but it did not.

01:53:14So something happened here where this function did not return. The username.

01:54:05See, here is a spec of this exact situation. Sends only the reply notification on reply with a mention.

...21And there's a sender.

...29Don't need that.

01:55:03just improving these wonder if

...38wonder if it's something about the rejects for mentions it's not noticing the mention because we have an issue filed about that or this

01:56:27What is plain text commenting? Linkify, then strip tags and convert entities back. Why does this exist? I'm thinking of the Gandalf meme. I have no memory of this place.

01:57:16It's from 2014. It's older.

...29It's from 2012. Email, okay, so this is dead code. This shouldn't exist.

...44Let's get rid of plain text comment. Just use comment, because he was clearly thinking of converting the comment into a text body, but instead, We just went with just send the markdown. So let's do that. Any remaining?

01:59:13Why is this split opening the vim diff?

...36I don't think about it right now.

02:00:30Let's pull this back to spec jobs.

...44Where was I? Let's put that in there.

02:01:19There's the, but somewhere along the way, some kind of bad edit happened. It's not comment, it's parent.

02:02:07Wait a minute. So this, okay, should be calling both.

...37That's right. It's got the star.

...56It's just.

02:03:13Already notified. All right.

...30You want to run.

02:04:26So it does recognize the username one was already mentioned. Because it's in the notified array and the list of people to notify is empty, so it's done. Except why does this say empty?

...55This should still list that recipient. So something is wrong with the regular expression.

02:06:09chamlis_ you're scanning the parent, not the reply
chamlis_ not sure if that's intentional?
scanning the you're right it is non-intentional no and i improved the variable name to avoid exactly that kind of foot gun i did it anyways to notify

...40So it sees the mention of user one, notified user one, nobody to notify. And then this one's dead code, but it should still have found...

02:07:03It should still have said username one. Don't like that.

...20So I have a test of exactly this situation that threw an exception in prod. Meili left a comment. I don't know, MixUribe left a response and they mentioned Meili. And it tried to create a notification that had already been created. So either this logic is correct, well, either this logic is bugged in some real subtle way, or something else is creating a notification.

02:08:16chamlis_ does any of this run on edit and could behave differently?
No, this is the notify job that doesn't, happen until after the thing has been persisted.

02:09:10Just let me look from the controller down.

...20So if the comment is valid and we save the comment, put the vote on it, touch the story.

...41Touch the parent and queue the job. You don't have some kind of after save, do you?

02:10:02After commit, we have log hat use, mark the submitter, record the upvote, recreate links, update associated caches. None of those things sound like create notification. And I'm reasonable about naming. And the word notif only appears once for that association. which how does a comment have one notification if multiple users can be notified of it?

02:11:03it's got to be a has many because the notifications has the notifiable type and id so that'll be comment comment and then comments okay

...33yeah don't have anything pointing so yes this should be a has many we just weren't using it as notifiable yeah that's just naming the other side of the relationship

02:12:01i think that was accidentally copied from message since a message can only have one recipient it can only have one notification so that part is fine

...56What if the job ran twice?

02:13:03What was the line of the exception? It doesn't say there, but I had it off stream. It was line 22. So if I remove that.

...21Yeah, it shouldn't even have been in there for that comment. So no, it's not running a second time. Right, because deliver reply notifications.

...39This would have thrown the exact same exception.

02:14:14I see it. I see it. There's still the prod console.

...28Let's grab that user and then chamlis_ good spot
Yeah, so we were only sticking them in the array if they get an email or a pushover, but people can opt out of that. When we create the username, they have to go in the array.

02:15:09All right, so wait, let me put it back. Let me say, instead of this, we will say false. And now the spec should reproduce that bug.

...31No, green dot. Oh, because it's email replies is the one I wanted to turn off. Haha, there's our exact exception on our exact line of code. I mean, you know, it's off by one because I added one line. That's fine. All right, so bug successfully reproduced. Let's go ahead and reimplement that fix. You know, how many times have I been burned by I think I get it and then I forget to run the test and I didn't get it? Roughly one million times I have been burned by that. So why do I have two loops? Because I didn't want to make this a hash or a set. Yeah. You know what, there's not going to be that many notifications. That's fine. We can loop it twice.

02:16:51Now our test is green again when it should be. Yep.

02:17:02Green again when it should be. expected zero to equal one. Oh, cause there's an comment notification sends only reply notification on reply with mention because they still were added to the array And so they didn't get an email about the mention. That feels not right.

...53If you turn off email replies, you may still want the mention. On the other hand, if you turned off the reply, this is only the case where someone is replying to you to mention you. And if you turned off replies, you probably don't want that mention. I don't know, this is one of those ambiguous cases, like I could break out the big Slack notification chart

02:18:33So here, in the scenario where Meili has email replies off but email mentions on, should she get an email about this? I lean towards yes.

02:19:03could see it either way.

...14And you know, I guess maybe no, because if someone has disabled a reply,

...32Yeah, so like the the universal is, well, I turned on mentions. So I want all mentions. But the other universal is I turned off replies. So I don't want replies. And I don't know which one should trump the other. I guess I don't want to encourage at mentioning people needlessly.

02:20:04So I'm going to say that's OK.

02:22:56Feels pretty good.

02:23:05Let me look at that. So this should be.

...28Notifications.

...38That's fine. But yeah, that's the correct model. A single notification just doesn't make sense. So I did catch an error there.

02:24:26That is a dense message. Fix duplicate notification exception when reply at mentions parent commenter.

...42Yeah, I don't know how to say that much better. Let's run this back. I haven't seen them run.

02:25:03I must have left a put somewhere.

...23Did I already push it? Yep. That was... This one. So we'll just roll it in here. That's fine. OK. All right, so.

02:26:00How many of those do I have in the mission control?

...11And yes, I should discard those because there's nothing to be done about those. There are 1, 2, 3, 4, 5, 6, 7, 8 previous pages. 9, 10, 11. I should double check that these are all failing on the same line number. Looks like this is a fairly regular thing that people are replying and at mentioning the person they are replying to. Because there are like 25 of these. I don't know. I'll count. But actually, can I show these? Yes, there's nothing sensitive here. It's just light mode. So.

02:27:20One. I'm skimming for the line number.

...33It's two.

...48Or let's just, I hate that Firefox considers the search term to be tab specific. All right, so we're here, two screens. Two screens. Ooh, that was a different line number. All right, I'll come back to that one then. That's four, five.

02:28:52seven, eight, nine, 10, 11,

02:29:2713. And you can see, I mentioned this on the other stream, but this queue was not, the failed job queue was not correctly wired up to Telebugs. So these have been building up. There's another one. Building up for a couple of months. Another one. Another one. Whatever this line 18 is. Got a whole run of them. I triggered rack attack.

02:30:13All right, so that's about half of them. And if it was alternating, like, you know, 2218, 2218, I would be checking to see that The job wasn't getting re-enqueued and triggering a second time, but let's go look at that. There's all the codes in my head. So line 18 just got pushed down. Was this line, what's the exception? The same exception.

02:31:00Huh. Let's go look at the exact line of code on prod. I don't want to play around with what line number is what, because I think it's the one that creates the to notify in the old version, but I'd rather know then guess. And the release has been garbage collected. 2025, 8, 18. All right, let's go look at the,

02:32:01This says notify comment job, just making sure it doesn't say message because I've been streaming for two and a half hours, which means I'm starting getting a little tired, which means I make mistakes. All right, so let's blame so that we can easily jump back.

...37Jump. September 4. Line 18 was a comment. Wait. So this is August 18. All right. So there is some other that used to be on line 18.

02:33:10It's the same bug. The line number just changed. That's all. These are just the older ones. Let's double check. The common ID is this.

...51that much of it i yeah there's an at here i remember this thread and if we go look at the parent username oh it's parent comment oh yeah there's the mention this was a direct reply because i remember this thread Cool. So there's, how did you click retry? I want to discard.

02:34:43What are the odds that just sent an email?

...54Well, the hidden story one would prevent the user who hid that from getting one, because I remember that fix.

02:35:11Yeah, that might have just sent a reply notification.

...22No, because it has an is active. Does the reply section have it?

...39Yes. All right. So someone might have gotten a duplicate email. No, because the user they were replying to is not an active user. All right. Let's click some discards. I lost count, but that's about 20 of these.

02:36:11Alrighty, and as long as I am making silly mistakes, that's a good time to throw on the banner.

...28Look, I just made another silly mistake. There was a different exception type in there, and I just discarded it. I should not be logged into prod if I'm going to make silly errors. So now is a great time to ask any last questions, because I'm going to wind down the stream and definitely get the heck off of prod. Where's this? All right. Yeah. Nothing else open in the browser there. Well, the only other exception I remember seeing in that list is the rare deadlock we have.

02:37:23pushcx https://github.com/lobsters/lob…
And I didn't think there was a second kind, but that one Is this deep cut issue that's been kicking around for. Six and a half years. That very rarely. We see a couple of production deadlocks. And it's like once every. Month ish.

...56And it's something about the order that stories, comments, and their votes are created at. Because all of these things want to update each other with all of the various caches. So there is some, execution flow i haven't nailed down yet where it's like if you upvote it could be as something as simple as if you upvote a comment at the same time someone upvotes the same story you take a lock on vote to insert you update the comment score comment tries to update the story count of the story's score because comment votes do influence story ranking and at the same time someone else upvotes the story so they also want to lock well they also want to lock vote but then they want to lock story so it's got to be something else

02:39:30Maybe it's if you reply at the same time someone else is voting because that would want to lock comment and then vote because you vote on your own comments. That's possible. These deadlocks could be happening on reply.

02:40:04Oh, but then, you know, it's also some of them, some of the exceptions have mentioned that they're updating the story.

...26That one's going to be a mystery. I guess I will just have to wait another week or month for it to happen again. All righty. ChaelCodes Bye!! Thanks for the stream!
Well, this has been the Lobster's Office Hour stream. chamlis_ thanks for the stream!
The next stream will not be on Thursday as scheduled because it will be American Thanksgiving. And so my fellow Americans, ChaelCodes Happy Thanksgiving!! 🦃🍽️
Frici Thanks for the stream, take care and enjoy thanksgiving, don't spend an entire week with pie for all meals again LUL
quote the politicians i hope you have a nice thanksgiving but i definitely will not be streaming i will be eating every kind of pie and then the next stream will be the following monday and let me double check because i've been doing thanksgiving prep yes i will be streaming next monday same lobster time same lobster channel look, Fritchie, you're not my real mom or my real dad, and they both want me to eat pie. And you know, the definition of pie is pretty broad because it can include savory pies, like a meat pie. It can include a pizza pie, although them's fighting words. So to leave you on a very programmerly, taxonomic fight. Enjoy your pie. Take care.