Where did it come from, unstructured control flow

04:38jangomandalorian Morning all!
chamlis_ hi! twitch finally saw fit to let me follow without adding a phone number
chamlis_ are you intentionally quiet/muted?
jangomandalorian am I the only one not listening to anything?
jangomandalorian oh! lol
No, good morning. I am not intentionally quiet and muted. I was unintentionally quiet and muted. Literally, the thing I said... So thank you for popping up, Django Mandalorian. And Chamelis, I appreciate you pointing it out. I was literally just saying that I'm getting kind of a slow start this morning and not firing on all cylinders. And then there I am rambling for a couple of minutes, totally muted. Not so great. So getting a slow start this morning, basically. Let me just tidy up here.

05:24Oops. Yeah. Yeah, there we go. I got the whip here. If I now load the domain, we have all this familiar code. So let's talk about where we're at here. Aside from waking up slow and starting the stream a little slow. Oh man, I made a, checklist of stuff I have to do before I stream. And yes, unmute your microphone is one of the last ones. one of the last steps. jangomandalorian 🤣
And it's funny. I got into making checklists in part from making and using run books at Stripe and. They're just so dang handy. There's also, as long as this is, you know, I'm recommending a book or two every stream. pushcx https://bookshop.org/p/books/th…
There's an excellent book called The Checklist Manifesto by Atul Gawande, I want to say. Yeah. This one is great. It's about the humble to-do list, basically, and how... using it in surgical settings greatly reduced errors. And that's things like we accidentally left a sponge in this guy, or we didn't correctly follow all of the steps for sanitizing an IV, and so we accidentally gave someone a nosocomic infection. So that's a major killer. And I believe specifically, or maybe it wasn't IVs, I think it was catheters now that I recall. And So this is a book about what Gawande learned from implementing them at different hospitals. And it feels like for them, one of the biggest hurdles for implementing these things was experienced professionals. And they had this situation where if you asked a senior nurse practitioner or a physician, do you know how to run a catheter without risk of infection? They would say, of course, you know, I've done one of these every day for the last 20 years. I'm an experienced professional. How dare you impugn my skills, my credibility by asking if I can. And then they watched what happened in practice and jangomandalorian To do lists / check list are the way to go (humble plug to todoist, the product I work for)
very experienced people get in bad habits and they cut corners and they just sometimes make mistakes. Humans are actually pretty terrible at lists of fiddly little tasks that you have to do perfectly in the right order the same way every single time. It's just not something the human mind is well suited to. So they learned a lot about how to both how to make good checklists for their setting, but also how to build a culture and a workplace that wanted to use those checklists, that was willing to not just trust them, but sort of humble themselves to the checklist a little. At least that's how I think of it. I don't think he says a phrase like that. But when I have my streaming checklist that is literally, you know, Open this window. Check that box. Go over to this tab. Oh, man, I just realized over here I didn't uncheck some other things. There we go. I don't know if the stream just showed my checklist for a second, but that's okay. Getting the value out of it requires that people actually follow it. And it's so easy to get bored by it. And... To stop trusting it. So there's a lot of... This was one of the best books I read that year. When did this come out? Maybe 2012? Somewhere in there? 2011. Okay. Oh, Django Mandalorian. So you work on Todoist. That's a pretty darn neat app. jangomandalorian Yes, and yes
I'm not... Todoist is a Mac app, isn't it?

10:00Yes and yes, yeah. Yeah. Being over on a Linux desktop, I have... I have that situation where like every six months to a year, I'm yak shaving weird personal productivity stuff. And so I've never really settled on a nice single solution for years, which is where you get a lot of benefit out of things like Todoist. So... I've heard lots of very nice things about this app, and I know a couple of people who attribute a lot of their professional success to it. I don't get anything from Todoist for that. That's not a plug or an endorsement. That's just because I feel like it. I also don't get anything from bookshop.org or Amazon or anybody. I think there's a lot of value to being able to say, jangomandalorian Yeah, i like your approach, more "honest"
no i don't have any endorsements no i don't have any involvements that's part of why i discourage donations for lobsters is i don't want to be in any way beholden to anybody to the level of i don't want to have to explain it you know even if people give donations with supposedly no strings attached it's so easy to start making decisions that benefit the people who've donated to you it's just again one of those things about how people work. And the only way to avoid that is to just never do it. Yeah. So some of it is honesty. jangomandalorian but understand that content creators might need a way to earn money, so...
It's also the easiest way to exercise personal integrity is to not need to exercise personal integrity. So if lobsters had a job board and people could pay to post job ads that one you know gets suggested occasionally and would be a convenience for the community but all of a sudden it's a well when company foo that advertised a lot on the job board the last two years when they have an outage and they post their post-mortem are the comments being moderated or voted in a way that's slightly more positive to them and moderators could exercise a huge amount of like personal integrity and be very careful about treating company foo exactly the same or you could just not have to do all that by don't ever take money from company food don't run the paid job board and i'm it's a very let's throw that into the throw that into the rambling topics the the value there for staying away from that stuff is only available if you can afford it there's a saying I always liked don't get hungry it's a very aspirational but you can make much better decisions if you take care of all of the other stuff in your life so that you're never desperate in tempting ways. And it's acting out of a position of privilege to use the jargon, but if you can set yourself up for it, boy, so many things come easier downstream.

13:51So anyways, speaking of, personal integrity and responsibilities. We're addressing self-promotion on lobsters, which is when folks show up to the site and they post a ton of links to their own stuff and they don't really interact beyond promoting their websites and they treat lobsters as a traffic source. And we especially have issues where we recognize when people are posting links from brand new domains where is it hey this is an unseen domain from a new user we try not to do too much of that to give you time to learn about topicality and then i guess it's not spelled out here explicitly but also don't just try and take from our community you can participate and share with our community but don't just take from us And we have hassles with sites like Medium, GitHub, Dev2 that the domain model is not a good match for because everybody who publishes on github.com has the same domain name. I mean, they have that or github.io, but the domain code kicks in for subdomains. So if you're bob.github.io, you do run into this limitation. So let's review the diff. Kind of what we've done over the last two streams has been to build up the, let's get the rails running in the background so I can show stuff, has been to improve the domain editing form. That is where we can see lot of this work for okay so for github let's have this idea of writing a regular expression to select out an identifier for the origin or the author i kind of went back and forth of should it be author should it be origin but i want to hang on to author as a name for a model probably for or authorship i don't want to have author and authorship that's a little confusing to me but to be able to attribute and say, oh, github.com slash Bob, that's our user tilde Bob. And then in doing this on the last stream, we ran into two big hassles, and one was we're not quite following the Rails way on this form, and it's doing a bunch of things, and we dove into a thicket of all this Parum stuff. And I don't want to recap that. It's on the previous stream, which is up on my stream archive. And it has the nice transcript if you want to skim. And you don't mind reading around a million transcription errors. At some point, I have to look at how to pass off a dictionary of every noun from our code base and every noun from Ruby or web development in general just to get more specific things and also scrape out every every username from the twitch chat because as i talk to folks it it gets every username wrong speaking of a username it's going to get wrong chamlus if you're still here the you asked at the beginning and i explained when i was muted you said that you were able to follow without a phone number that was a chamlis_ ahh nice, thanks
setting in the twitch dashboard apparently people get abused by having new followers i don't totally understand the attack case and so they can restrict how long people have had to had twitch accounts for before they can follow you or that they have to have a phone number and which they make that frustrating choice of assuming that warp numbers are bad chamlis_ I guess people could have abusive names?
And I don't know if that was a default or something I set without thinking too hard about it, but especially because I've been annoyed at how Twitch handles phone numbers. I turned down that setting. If we get covered in spam, I may have to tweak it back, but we'll see how it goes. You guess people could have abusive names? Yeah, I don't have the clever, like, Bob Smith has started following you banner that appears over the theme. We have a pretty low-key thing going on. And it's funny that's been forced on me by all of those clever Twitch overlays work based on a Chrome browser engine plugin for OBS so that you can, cause you know, who wants to write their own layout engine, their own text rendering engine, or even just call those libraries. That's a lot of fiddly code. And so the community is kind of all the weight is gathered around let's just embed a browser in a plugin and the browser can overlay basically another visual layer on top of things and then just everything is css and javascript and that's that's fine as far as it goes i understand that as an app lawyer it's not there's something going on with packaging it for arch and we had one of the arch people on an earlier stream who explained it a little bit so if you dig back for cbe or chrome or chromium browser engine in the stream archives you'll find it but basically i can't install the package right now or there isn't a nice convenient work package to install so I don't have any of that kind of stuff and I've thought about coding a lot of it myself because you know basically all of the stream Vim is up and I can write Vim script or script it with Ruby and it would be kind of fun to have you know a line here where people could you know chat would scroll by or something but unfortunately I can't get a Twitch API key to do it I did reach out to their What are they? I guess they're a developer relations team. And I said, hey, can you take a look at this? And I did get back a response, but it was basically like, lol, no. They politely said they can't or won't make any exception to their broken policy on phone numbers. So that keeps the stream kind of Very low-key. But, Shamless, I think you're right there, that it would be people with abusive usernames or... The other one, and it's already happened twice on this stream, is folks who come by spamming for viewers. I don't think it's been visible to you, the two of them, because both of them had the little red box that said, like, likely Banavator or probably spam. All right, so where are we now? So this is the code we've added to domain for selecting out the origin. This origin method is probably going to move over to the origin model here. It all got committed. I don't usually commit this kind of whip. I'm happy to let it build up. I wanted to jump over to another branch and do some coding on lobsters. I fixed a bug around escaping posts on Mastodon. That one's kind of funny, actually. Where is it? It's an issue tracker. I think there's a screenshot here. No. The screenshot was very funny, but all of the spaces and everything in the post, all of the punctuation was getting double encoded when it was sent to Mastodon. And so a title would be like the percent 20 Mastodon percent 20 bot, and just totally unreadable. So I, that was a fun little bug. It was especially fun to fix by removing 20 lines of code. All right, so a big chunk of the end of the stream was talking through these issues with corner cases around when you submit a domain with a slash on the end or no slash on the end, we don't want to consider those different origins. And so it's like the regular expression we ended up with which is kind of the template for a good one, is getting fairly fiddly, but I guess we're okay with that. Especially because I can just list that example one here. And realistically, anybody who's using it is going to

23:54Actually, I don't need to edit that there, because there's a second part. Realistically, anybody is going to pick up the two that are printed on the screen and use them. And the other part of it was, you know, if the regular expression doesn't match, let's just fall back to an origin that is the bare domain. And that seems fine as a catch-all, especially with a suggestion from, I believe, Nagwe said, hey, if this edit form gave you fast feedback by loading a bunch of URLs out of the database, that would be really nice. So let's add that to the to-do list. Ah, yeah, it's already on the to-do list. chamlis_ it's probably not relevant to this approach and is less flexible, but the public suffix list is for a kind of related thing (marking eg github.io sites as separate for cookies etc) and might be of interest
All right, so where we're at, this all has, I think we added one test, but it has basically no tests. Yeah, a couple of small tests around the selector.

25:02Yeah, ShamList, the public suffix list keeps coming up, but it doesn't handle, and it is very closely related, but it's not quite what we want because what we want is to be able to handle public affixes or chamlis_ right, yeah
know understanding that medium.com alice bob and carol are effectively three different sites in a in some sense of the word sites it's all medium but but they have very different authors and we want to be able to have some idea when people are are different authors so we wanted to list stories by origin And I believe I got started on that. Or finished with one of these exceptions. Yeah.

26:09So the home controller in lobsters has all of the code related to

...24showing stories rather than it being broken out by oh there's a domains controller that understands how to show stories because they all share a template they all just live under the home controller which almost could be called the story list controller that might be a better name for it because it doesn't really do anything it's i think it's just named because it's the the home route of slash so Let's make sure that we have a, yeah, we have no routes for origins here. And it's tempting, it's tempting to put the origin under slash domain, but there's a unfortunate clash here where if we have, yeah, so we are just diving right back into corner cases. So the new one is... Right, it'd be nice to say that. I guess we can do that because there's going to be the extra separator. And as long as we have a separator, We're fine. So let's go ahead and say we'll go ahead and redirect the singular. Yeah, let's put this at the bottom here.

28:16So this will be the domain and this will be the origin. I like to try to be A little more specific than just ID, which is the rails default and I kind of vacillate between are we using the specific rails ID or are we using the. Are we using the better term.

...50And I guess we'll copy pagination into here.

29:00and we never published under that old one, so let's just do, so if we have the origin, and we have multiple pages, we're all in good places, and origins, we're eventually gonna come back, I think, and add Add them to resources. Oh, this wants to live under here, doesn't it?

...42I'm going to have to double check that this generates the right route. I never remember this syntax. That's basically what I wanted. Wow. Right on the first try, when do I ever get that right? And I'm a little cautious about nesting resources in Rails. The rule of thumb for nesting resources in Rails is don't do it. But origins are very explicitly refinements of domains. And this doesn't, nest the controller, this just nests the route, which is okay because I can still support this route. This is me kind of working through the decision of not following my rule of thumb. Yeah, even if we had an origin and we wanted to handle it separately in its own controller, it's going to have to include the domain to filter it out, right? Because the identifier is going to be We should call it identifier, not origin, because that's what it's called on the model.

31:10And we'll just put this with the other one.

...22And I guess I'll duplicate the page thing. It's going to be good when I come back and finish pagination and I get to delete all these extra routes.

...45It's funny, as a developer, there are a few things as satisfying as deleting code, especially when it's special casing. So if we're selecting out the identifier, the identifier only makes sense in the context of a domain. So you could have medium.com slash Alice and github.com slash Alice. They would have the same identifier of Alice. Well, the identifier, the result is going to be, isn't quite matching what the code is doing. because the selector is pulling out the identifier, but then with the replacement, where is it? With the idea of replacement, that's becoming say github.com slash Alice. And we've especially talked through having the selector for github.io replace with, let's see if it's still in the database here. So you have github.com here. Not sure if the data is still here between having to run and unrun migrations all the time. Yeah, so the idea was that the selector for github.io was gonna look like, this where instead of being after the slash it's whatever's in here before github.io and if we replaced with github.com slash so if there's this it's encoded the idea that these two different domains are owned by the same author. They are effectively the same origin, which means that the identifier produced is not just going to be Alice or Bob. It's going to be this whole section, github.com slash whatever.

34:17if it's going to be all of that hmm well then we're not going to end up with right because we're not producing just alice we are producing

...42And that looks cockamamie. I don't have a better word for it. It's just like I understand what's happening, but that's not clear at all. So speaking of not nesting resources, it's going to have to be this. And all of this, including the slash, is going to be the identifier. So that's probably going to be some custom rails routing because by default its section selectors don't include splash all right so did i accidentally close my yeah i closed the scratch all right

35:51This one's getting fiddly and only will get more so. So I want to do two things. Well, I want to do this in a test-driven fashion because it's going to be so easy to break with the Rails routing to DSL. Do I have an example here? Let's grab a... an existing spec and save it as spec routing origin spec.rb.

36:35To do this, I believe I have a domain factory. Yep. So let's also have a origin factory. I don't think I have one of those. Nope.

...54So let's... And a factory has... Let's real quick look at the associate. Okay, we say association. Speaking of remembering a DSL. See, all of this... One of the things that happens on stream is like every 10 minutes, I'm like, oh, right, that's another DSL. And If I can opine for a sec, this is why Lisp never made it is because in Lisp, you can make a DSL all of the time. And that's cool. Like that's the real ultimate power of Lisp. But learning and remembering a DSL for every different subdirectory you go into is kind of hideously painful.

37:51You know what I really like is instead of doing this, I believe I have faker syntax or somewhere. What are we mad about the factory already registered domain helps if I save these files, especially if I'm going to do this thing where I copy and paste files. Do we have a name? No, we don't have a username.

38:32What's the... Let's jump over here. Has a list of the generators, right?

...50Mm-hmm.

...58And then where's there for a complete list, see what I want is a, a username generator and there's gotta be one, right? Otherwise I'll just take first names, but what was funny name?

39:31Surely not. Oh, these are terrible puns. Oh, that's not fun. Can't punish people like that. I do appreciate that whimsy, though. All right, so we will just say dot first name.

...54All right, good.

40:08Oh, that's not the Ruby method, is it? It's downcase, downcase, because it wants to be a verb. This one, downcase and upcase, they always struck me as not quite English. chamlis_ I think there's Faker::Internet.username
I wonder if it's a symptom of, not symptom, a quirk of Ruby being started in Japanese where they kind of invented their own verb up case and down case, so they you know it's not a term you think there's faker Internet username Oh, that would be nice.

...55Oh, there are. let's do that well, thank you channels that's a nice improvement.

41:09There we go. So we have a factory for making origins. We have a route. We have a cat laying on the mouse. So if you see that whipping around, or if you hear me complaining about nipped, getting nipped, then you'll know. So what I want most is

...41something like this to be recognized as home controller or origin with a domain no with an identifier yeah You know in I think the first stream of working on origins I kind of went back and forth on whether identifier would be just Alice or whether it would include the site name and. Another rule of thumb, I have is to just not have compound primary keys it's almost always better to repeat data, a little bit then.

42:27require that you load two different models from two tables or load them in the context it's just not a great fit for rails route to single origins we'll save this let's. let's also say.

43:02So one thing I can see people doing is seeing domains and seeing origins and adding something like this to the end of a domains URL. People do edit URLs on the site all the time. And so I want to say that it recognizes and what is it?

...32Is there a way to just say, oh, thank you for giving the mouse back just in time, cat. I think there is a way to say recognize path, but there's no controller, so you'd have to do it up in another, that's... I can't put it here, okay. So let's say that this one is a request spec and I guess it's not going to look like this at all it's going to look like. A get to. This URL.

44:41Guess I didn't end up needing Faker. That's all right. I'm sure it'll come in useful at some point. Better to code that up while all of it is fresh in my mind instead of coming back months later.

...59You know, it's funny. I glance over at chat, and so I sometimes get reminded of things that have gone by a minute ago. The way Faker puts all of their lists in details is what thwarted Control-F. That's also why Discourse drives me up the wall. thevibingguy plain vim what a chad
Discourse, the Rails forum software, is it does this lazy loading thing because they have performance issues with rendering, say, a page with 10 comments on it, which that's really something. Hi, the vibing guy. You know, it's not totally plain Vim. Look at this italic font. Do you see italic fonts in plain Vim? That's called syntax highlighting. thevibingguy ohh
So we got that. Let's run this test file. I don't have the, I didn't make this redirect. So the second one's definitely going to fail, but the first one probably will pass. Didn't I code that up? No, I was actually doing this one test first. So both of these will fail. That's fine. All right. So let's get the spec on the right. I don't know what it is. It's such a knee-jerk response that the spec has to be on the right. I am occasionally okay with the spec on the bottom, but the spec can never be on the left or the top. I don't know what that is. So we will say... Let's get rid of this. We're not going to nest under domains. We're going to say, just grab all this because it's going to be very similar, including slash dot plus. I want to call it identifier rather than ID. so we will say origin let's just not do the singular i think domain for a hot minute when i introduced it i put it under the singular and then i gave in and followed the rails convention of putting plural in urls but that means that there may be links to the old ones out and so you can kind of never get rid of the old redirects or realistically i could after a couple of years but It's so frustrating. Maybe I'll make my own rails with my own conventions. And then DHH and I can have a dance-off, like in West Side Story.

47:56It's kind of... I almost want to write a... a slightly heinous regular expression to specify that the identifier can have at most one slash in it because I don't want to consume this slash page section. Yeah, maybe let's just leave that out because I still have been working on pagination. I dropped it off on stream because it turned into just me reading docs on stream, which is no fun, but knowing that I want to fix pagination. Just leave this commented out for now. Let's leave a note about that.

49:03So with that, do any of these specs pass? Well, that should pass the first one, right? Yes. Good. Now let's pass the second one, where if you guess at a domain slash, this isn't quite the identifier. Let's put it after page. And we will just say to redirect.

...50Oh, I've got to.

50:00All right, not author first, domain first. Author second. There we go. I guess I'm old enough now that the Gen Z slang like Chad still clangs in my ear when I hear it.

...28All right, come on. No, no route matches. Where did it land?

...46So we made this slash domains, domain author. To redirect to origins, domain, domain author. Boy, that looks correct. Let's see that test failure again. Anybody spot the bug? That's how you get to be a channel VIP, is if you spot the bug before I do. On any of these, especially like writing specs or me breaking things, no route matches slash domains. So if we do a get to slash domains, I bet it's the dot. It's either going to be the dot or the slash, like the identifier, because this ID should permit a dot. No, OK, so it's not that. So we get domains. I didn't typo domains. Domain slash author. What if I drop off format?

52:12If I drop off format, it adds it back for me. Thanks, Rails. Man, I'm slowly accumulating the list of things I would do different if I was in charge of Rails. And the big list is like, the most visible one that comes up every freaking stream is singular verse plural. And then the one I've given conference talks about is, active record, and then the other one is the way Rails handles formatting, which is just constant foot guns to me. Hmm.

53:03No route matches. Why though? What if I do it here? Actually, let's do it on curl because it's easier to see the output. Is that just 404s? Well, now I am going to bring this back to Firefox, because I wonder about... Yeah, I was wondering if this was going to say there was an active record record not found for this.

54:09See, doesn't this look like this route? Domains, domain, author. Rails. Rails, why are you doing this to me? Is it the, oh, I get it. It's the constraint. so this constraint around id that it's allowed to include periods is effectively what this regular expression is doing so github.com has a period in it by default the default one that rails creates for id does not permit periods because it's trying to split out formats and so by changing away from id to domain i stopped using that

55:06Yep, there's the 301. So now the spec passes. So either I could rename all of those to domain, which realistically is just churn that I'm not going to do, or I can just leave that as ID. So there we go. That's about as close as I ever get to test-driven development, writing the test beforehand. So we have this origins idea.

...40All right. I believe there's origins. All right. And now we want to list stories by origin. It's not stories controller. It's home controller. Home controller. Thank you. And so here we're going to say origin equals origin.findByIdentifier.

56:32Paginate. Yeah, we're going to leave out pagination for now.

...45We'll just say like limit.

...53We'll just limit to that. Yeah.

57:02And I'll have to make a new. All right. So what else is in the to do list? Let's open up some tabs because I'm going to get lost in all this. There's a. The origin model is going to need this scope.

...28And so stories is going to need an origin.

...36Okay, I already added that.

...50So it's optional to belong to an origin. And this has many stories, but they're not optional. Or actually, they are. You could have an origin with no stories if it gets defined and accessed. That's fine.

58:13And then the story repository, which is what looks up stories.

...25Oh, boy. If I don't stop misspelling identifier in various ways,

...55This one just copies the options over to the Rails cache. OK, fine. So say origin stories base for the user limit. Good, good. Now let's make that partial.

59:35So as long as folks are... One sec, excuse me, let me clear my throat.

...53So as long as... Oh man, still froggy. Guess I shouldn't have coffee with cream in it.

01:00:05as long as folks are hanging out.

...14The big purpose of doing this stream is office hours. So if you have any questions about Lobster's the site, Lobster's the code base, Lobster's the community, you can throw them in chat anytime here. Otherwise, when it is not those office hours, it's me coding. Banning is going to happen, but none of this code is there yet. And probably should be. So where's the migration for? At some point, I'm going to extract a bannable moderate module.

01:01:18So this is going to go in here and becomes, yeah. And because object orientation, everything gets switched around.

...39This kind of mechanical transformation is one of my least favorite things about how object orientation breaks down. And you have to just schlep stuff around to convert between an imperative model and an object-oriented model, because this stuff like add column isn't on a block the same way this create table is. even though you're kind of doing the same operation. Maybe I'm griping about API design. Maybe I'm not.

01:02:35So that goes on the to-do list somewhere here.

...55Okay. So we've got the origin.

01:03:28And then there's that inSubmitters thing.

01:04:15Ooh, that's singular. That's a bug. That should be plural.

...33All right, this all seems pretty reasonable. Let's see if it loads.

...45Nope. All right. So I still have that browser. Let's see what the exception is. It's me typoing.

01:05:06And the new one. Can't find origin where origins identifier. That's fine. I asked for one that doesn't yet exist. I'm almost okay with that. Although, you can't really go from an origin back to a domain. So yeah, that's just going to have to be a 404.

...41And I talked about how this migration needs to go through all the domains and go through all the stories and add all these origins. So it's time to figure out what that's gonna look like, I think. So this seems fine. Oh, I have to link it from Let's finish that thought, and then... You know, it's funny. I've been working through stuff, and the to-do list is longer than I started an hour ago. It's like that programming song, 99 Bugs on the Wall. You take one down, pass it around, 103 bugs on the wall. Let's go over to the view. Let's finish that thought about.

01:07:02So it's going to look like that little string at the top of the domain page. So let's go to, so up at the top of the domain page for github.io has like, I'll come fix that in a second. Like how many stores from zero submitters I have to include the domain here somewhere, but it's going to be, yeah, we'll just put it in parentheses, I guess.

...55Yeah, that works. And then that n plus 1 error that keeps coming up, it just clicked with me where that is. That's in model story. The presentation tries to preload things like the domain. Well, it needs to preload this new origin.

01:08:28And then the list detail was on the to-do list. Hello, sir. Big cat is stretching out. I'm definitely getting nipped in a minute. So we're going to say if the story has an origin, then Link to the. Let's link to the. Identifier and say origin path. Is it origin path or origins rails? Rails has decided that it is origin singular. Who can predict this?

01:10:07I don't, that was my, my edit, which surely it looked like that, right? Let's go back to domains, github.com for somebody with lots of data. OK, better. This doesn't need to be something changed in here in the styling where this link didn't used to be blue and underlined. So what did I change?

01:11:03It had a class of domain on it.

...22which is not quite an accurate name anymore. But yeah, I don't see a value to painting that bike shed this second. If somebody sees a better CSS class name, feel free to suggest it. There we are.

...43All right. So with that, we have a link up to the domain page. We don't have a link. down to origins that's a new one that's a so it is the third of the month and in chicago it is moving weekend every time especially like June, July, August, September. This one's a little late, but lots of moving trucks in our alley this weekend. There is one belated guy, and he's either mad at a... Must be mad at a trash truck that's the only thing he wouldn't be able to get by. So if you hear giant horn honking, well, I'm in a very fancy studio, right?

01:12:52I think I have to do the migration first if I want to have any good data.

01:13:10All right, so having done all of this stuff, let's roll back the migration.

...24Can't roll back the migration because I've edited it.

...41just_bandit__ hello, new viewer here. I think you're using nvim. Can I check your config ? I am trying to switch from vscode to nvim, but I have hard times opening new files and cycle trough them :D
Hey, JustBandit, welcome. I am in fact using OldVim, not NeoVim, because I am old coder. I have kind of bounced off of NeoVim maybe earlier this year. Yeah, this spring I played with moving over to it, and I was mostly able to get my config over, but I couldn't get any plugins working. The docs for plugins and the installers just didn't make any sense to me. So my big suggestion for if you have hard times opening new files and cycling through them, I would strongly suggest you install...

01:14:33So this is OldVim, is FZF, and it has a feature. Where's my mappings?

...49yeah and then i have it bound to semicolon yeah so i bound semicolon to this fcf function and if you just call it it finds files so as i'm over here i can just say semicolon model origin and it auto completes that's my biggest suggestion the rest of it is just getting comfortable with vim's shortcuts for windows and if you use them tabs just_bandit__ cool! Thank you! How you deal with the tabs? You have a particular plugin for i t?
And there's nothing to do for that but practice. Practice a little bit at a time, like learn the window commands one week, spend a week with them, come back, add the tabs for a week. How do I deal with the tabs? Do I have a particular plugin for it? No, I just have a bunch of config for it. Let's see. I have... i'm i'm scrolling something off screen i'm scrolling my vim rc off screen because it has all kinds of notes in here there we go so i do have some i do have some tab customization so i i use vim as my terminal multiplexer the way i used to use just_bandit__ cool, I am using tmux too
tmux and shell before it and so a lot of these are shortcuts copied over from my tmux config but moved from control a over to control t vim has this concept called tag stack that is Yeah, I don't think you really need to use Tmux with Vim anymore unless you're SSH-ing into another server and you need that connection to stay up. Or you need that Vim process to keep running when your connection drops. I think Tmux is overkill for local development at this point with Vim having all of this nice tab management stuff. And so I made a bunch of shortcuts for... creating tabs and swapping left and right and pushing them left and right. And these various shortcuts are also basically the shortcuts I use in the awesome window manager in Linux for navigating between windows. So when you see me do stuff like bring extra windows on screen and shove them off or bring them on and resize them, that's all just just_bandit__ awesome! thank you sir!
basically the same keystrokes and having the same keystrokes at multiple levels of abstraction works out really well for me so i would suggest you don't start with making 20 shortcuts at first start with one or two because i have found that i can learn shortcuts at a rate of about One every couple of days or you know, maybe five a week if they're really closely related, but otherwise just two or three a week so just do it slow do it real slow take your time. there's a you know if you work through. What is the command I think it's just been tutor. I assume neo vim has their own version of him tutor. it'll teach you the basic stuff just go through it slow and if i were starting new between bim and neo vim if i understood the plug-in stuff i probably would go with neo vim it's hard to say but i do like being able to script in lua bim script I have cut my teeth on it, but it's not something I would suggest someone spend a significant amount of time learning. Lua is so much nicer as a language. And I understand, you know, vim script predates Lua by, what, a decade, two decades? New index found on stories.

01:19:19Maybe I should make a vim plugin so that when I start editing a migration, just_bandit__ indeed! thank you! I liked nvim because it's fairly easy to use lua to load plugins. I started playing with lazyvim as plugin manager
get a pop-up that screams at me and says stop it you have to run rollback first man i don't even know which in moderations so this and this yeah it's okay it's just standard oh my god

...56just bend it. So it's funny you say you find easy to use loa to load plugins i also started with was it lazy vim it had a i think it had a different name maybe this was it yeah with this clever pop-up thing

01:20:32just_bandit__ yes this one :)
just_bandit__ but from folke
Maybe it was somebody else. Hold on, let me peek off screen at my... Oh, you know what I looked at?

...53just_bandit__ this one github.com/folke/lazy.nvim.git
There's a... Maybe I was just confused. Maybe I bounced off of... So I'm looking in my browser history and my personal browser off stream. Oh, you started with this folk one. Yeah. So I tried with this folk one and I bounced off it. There was something about the getting started docs that I didn't quite understand. Like, so it says, you know, look at the documentation and it was like, yeah, here's how you install it. And I was like, okay, sure. just_bandit__ structured setup is the one
I don't know why I would decide between these two things, the structured setup and the single file setup. You can also use, if you prefer to keep everything, okay, well, why? Why would I do this? Why would I not do this? You have to tell me these things. And then... I was trying to figure out how to add more things and the docs just kind of jump into writing specifications. There isn't like a step two. You know, like I'm not packaging things. just_bandit__ so i have something like this: require("lazy").setup({ -- Load plugins from lua/neo/plugins spec = "neo.plugins", change_detection = { notify = false }
I'm trying to use packages and all of the docs just went straight to how to make packages. I just wanted to use packages. So I don't know. So you have something like require lazy setup. just_bandit__ neo is how I called my folder
What is Neo in this context?

01:22:40There was just like when I was getting started, Neo is how you called your folder. It's like your local username, maybe.

...56Oh, no, you must be saying Neo for NeoVim. just_bandit__ sry, too big text sec
Yeah, I don't know. I spend a lot of time from working at startups, I spend just an enormous amount of time thinking about first run experience of how do things start, like how do people understand The concepts, what concepts do you introduce them to and do you teach them the concepts? just_bandit__ ├── init.lua ├── lazy-lock.json ├── lua │ └── neo │ ├── init.lua │ ├── lazy_init.lua │ ├── plugins │ │ ├── gitsigns.lua │ │ ├── icons.lua
Do you ask as little as possible of the user at first when they're starting? just_bandit__ ok not good :D
And I kind of bounced off of NeoVim for stuff like, your ASCII art did not make it through Twitch chat. just_bandit__ haha yeah
I think I see what you're doing, like you're pasting the output of tree, but no, it's okay. You don't have to worry about formatting it. just_bandit__ basically i have lua.neo.plugins
And, you know, the only way you could format it on one line is Lisp style of nested parentheses. But I see what you're saying of you had a top level init and then a couple of directories under it. just_bandit__ i can give you if you want what I have so far :D
Yeah, it was the syntax for what to put in there that I didn't quite get. I don't know. If it works for you, I'm very happy for you. I don't... I really dislike fights of, oh, who's the better Vim? Vim or NeoVim or Vim mode in VS Code or is Vim better than Emacs? Is NeoVim better than VS Code? Like, I hate those fights. They're not interesting. So...

01:24:47So let's loop over all our domains and find the origins for them. Yeah. And instantiate stuff in our database. What are you mad about, standard? just_bandit__ good luck with the dev part! I will lurk for a bit! Thanks for sharing few tips about vim!
Oh, because I haven't run the migration now. That's fine.

01:25:14Ah, you're welcome, Bandit. Thanks for hanging out. Happy to talk about config. You asked something that's not on the FAQ, which I appreciate, as opposed to folks who come by and are like, no colors. Well, there are. So for each one, really, this is for each story.

...45And you can just skip it unless there's a domain for it. And then if there is, it becomes story.domain.

01:26:19This is one of those. This is one of those lines of code that makes me suspicious about object orientation in general because it's just a bunch of nouns slammed together and it doesn't feel especially compelling as to whether it should be on story or on domain or on origin. Realistically, maybe this wants to be on origin. Maybe this just wants to go away. I think this wants to persist the record.

01:27:26Probably want that. Because I don't think I want the caller to have to know that it might need to persist the record. It should just be there. So we'll say if there is an origin. espartapalma hi, hello
And for some of these, I'm going to have to make Domain selectors for GitHub. Oh, hey. Welcome back. GitHub. Who else? GitHub, Medium, Dev2. And what was that painful example? Dataswamp. Dataswamp was one of those places where I bit the bullet on getting some really gross stuff off of the site. But most of data swamp.org is really excellent technical blogs. And so it kind of hurt that I only had a very blunt tool. There was a just some bizarre trolling from one author on that shared site. So if origin equals this, well, then let's go ahead and it's automatically saved, right? So then we'll say story dot origin equals

01:29:19So I'm calling update attribute because I think that's the one that bypasses validation and I want to bypass validation because I know there are some validations on the story model that only make sense in the context of new records.

...43And I have been, Yeah, update attribute, name comma value, not a hash. I've been working on that recheck gem I mentioned that is finding lots of validations could be written better than they are so for example there's lots of comments in the database that no longer pass their validation because the author of the comment can no longer use the hat that they used so this concept of you can apply a hat to a comment to say oh i'm especially speaking on half of the ruby core team or lobsters itself as the moderator well if you're no longer a person Speaking for that team you doff the hat and the comment then says like oh it's invalid you're trying to use a hat you're not entitled to. that's not great. don't need a special save. Okay.

01:31:11I think I can just pass a block for this, right? Let's check in the... Because I want to do this if the domain is here, and if it's not, it's probably off on some sister site.

...41It's got to be kind of specify the column. Yeah, that does what I want. Thank you.

01:32:24So this one is the one I've seen. It's the one that's in every freaking code example. So let's just grab that out of the view and then add more backslashes because that's the theme of this coding is adding and removing backslashes.

01:33:03Alright. That looks accurate. We'll see if it blows up.

...32Okay, I have to escape the dot there. Good. And then dev2 is the same basic strategy as the other. Yeah. And you know, as I'm writing these, a couple of times in developing this i've talked about how similar almost all of these are and it feels like overkill to expose the full regular expression but being able to do this one with github.io seems worth it

01:34:59You know, I wonder if any of these have a www available. It's not quite the style to require that anymore.

01:35:18Yeah. So you strip it, right? So let's just look at where's like the latest post. So here's some guy. Okay. And if I say, good. If I go to medium.com versus, and they strip it as well. All right. Small mercies. How about GitHub? You know, I don't.

...57They strip it as well. All right. Thank you. Thank you, nerds, for being really consistent about your URLs.

01:36:07And this is the code that wants to be over in setting the origin or setting the selector. Yeah. So I've basically written it here.

...37And I'm going to get rid of this if because it's also possible for the stories to lose their origin. So if we deleted the selector off of a domain, all the stories should get updated with basically a nil.

01:37:09So is it possible for this to return nil? No. It's returning domain, which is wrong.

...22Get code. So if we find it, there we go. And so, then the migration should run this new code, I know there are some folks who say that migrations shouldn't run code out of your models. They are wrong migrations are not code to maintain indefinitely their code to maintain for a week or two until they have run. In production and on most all developers machines and anybody else can schema load. I guess that's one more for the list of things Peter would change about Rails. Key origin, is it? Who's throwing the exception? Line 30. Do I have to say origin ID?

01:38:36Now we're doing the thing with MariaDB where half of the migration has run. And I'm going to have to yank out half of this migration, right?

01:39:01So that section ran.

...11So the index didn't get created. But the origins table did. That's so strange.

...53Oh, it's because I dropped the... No. There is that uniqueness constraint.

01:40:31migrations, moderations. And I guess it got dropped because I dropped the column. I hate having to do this manually.

01:41:06And I'm just like mentally running the code. All right, that should be it. Should have collected that in a script because you know I'm going to have to do it again. And here I am having to do it again. That index didn't get dropped.

...44Yeah? Did I paste it wrong somehow? All right, now I get to hit up a lot to try and rerun all of those commands while the cat plays with the mouse.

01:42:30Is that everybody?

...58I think that's everybody. I'm going to grab this over to one of my many Linux clipboards so that when I find the next typo and have to do it again, I can at least paste. No, it's partial. Oh, I wish MariaDB supported DDL transactions. Here's one more for the why are we not using Postgres files. And, you know, the answer is obviously it's just so painful to have to migrate.

01:44:02failed.

...28If I were, you know, the other approach here would be

...40if i didn't have data in the database that i cared a little about i would just blow away the dev database and reload from schema the whole point of this is i'm trying to fix a hairy situation in real world data so i have all that loaded in dev so i can't just drop and recreate the database because that takes i don't know 10 minutes

01:45:10Please work. Please just migrate so I don't have to do that again. I can't drop index stories on origin ID, duplicate key name, index stories. Why are you like this, Rails?

01:46:10Now let's do it in order.

...20Oh, god. Even the Control-R isn't working.

...48Let's make a, this is getting ridiculous. Let's make a scratch file here.

01:47:05There's replacement. There's selector. There's the index. Glamorous life of a programmer copying and pasting partial fixes around.

...54And it's faster to just recreate stuff than yank it out of the history. Man. All right, so that's the one that we had this error about, right?

01:48:30Okay, I don't see origin, so I shouldn't get an error about it, even though that was the exact error I just saw. I don't think this reference automatically made an origin, or I would have already had a, right? Watch it fail. Table origin already exists. Okay. Grab all that. All right. Try again. Adding the reference must be creating the index for me.

01:49:45I'm going to go look for that index if this runs. All right. So that's interesting. Oh, well, I didn't insert any prints. So let's look and see. So now stories, yeah, okay. So the references automatically added the index. So that's why I was getting that frustrating error. So maybe if I had said add reference here, I would have realized that I didn't need this.

01:50:22But that's Rails as you can operate at many levels of abstraction. Okay, so with that, did we get any origins created? No. Why not?

...52I'm going to keep this around.

01:51:02But I'm going to close it because I don't need to look at it. So when you set a selector, you set a what's calling update origins. Perhaps selector previously changed or replacement previously changed in fire. It's one of those.

...27So we'll roll back. Although I could have just done it in the console. Well, that's all right. I fixed the migration, right? espartapalma you can also use if_not_exists in their create_table and add_index to avoid that sneaky error of already created
Maybe it's the previously. Let's look at that real quick. So if I ask a domain. Yeah, it's this previously. So there's a distinction in Rails for exactly when in the lifecycle in this mutable object did it change? Because you can have validations that run on... Validations are, I'm sorry, callbacks in Rails. The whole model lifecycle callbacks. They're sort of... There's a joke about goto programming. Someone made a joke called come from. So goto is like, let's just give up on structured control flow and jump off to another place. And I'm aware that that is not the history of it. It's just how assembly works. So there was this idea someone made as a joke called come from. where it was the inverse of goto. easeout intercal
easeout boy oh boy
So you would have a statement that said, come from X, which might be a line number, intercal, yeah. And you would be able to say, oh, when you reach this other line of code, jump over here to my code. And it's sort of hilariously bad, like that's the joke is you can't even tell reading the code to come from that this control flow is going to change is there an example yeah so come from 40. and so it's really weird when you get to line 40 here there is nothing here that indicates the control flow is about to leap away and in this the python example maybe it's a little clearer because Basic is passing out of notice. So there is a label here, but there's nothing to show that it's going to come from, and the control flow is going to leap away. And that's basically what active record callbacks are, is they come from four models. So there is nothing in calling domain.save that says this stuff is going to run. And since it can run in all of these different situations on these mutable records, there's stuff like previously changed and changed to differentiate the several points in a model object's life, an individual instance's life, where it can change since it came out of the database. And the code didn't run here. because I picked the wrong one.

01:55:01I'm going to put in a little debugging.

...20Because I would like just that little extra feedback when the migration is running. Because if it actually runs on stories, well, when there's, oh, I'm sad. It didn't run on anybody. When there are stories, why didn't this run? Let's run it in the console. Boy, oh boy, are you laughing at my migration that refuses to run correctly or my snarky take on? Does this find by block not work the way I expected? Yeah, you didn't run the block. So this is a Ruby gotcha. Things that don't take blocks don't raise errors. If they get a block they didn't expect, the block is just silently ignored. I had this happen a couple of months ago when I was calling some innumerable method for select or filter, and it There was one that took a block and one that didn't, and I got the wrong one. And then it was very confusing that the code was there and then basically ignored. easeout just thinking about intercal. have humans always had the nature of trolling
All right. Come here.

01:57:10Have humans always had the nature of trolling? Yeah, probably.

...21All right, so let's see if I recorded a Vim macro correctly. Nope. espartapalma it got me confused why using blocks in find_by ...as in: "I didn't know you can do that! it that Rails 7.x thing?"
There we go.

...35Oh, no, lost my dues.

...45Yeah, doesn't it feel... Why are we like this? I guess I didn't edit correctly.

01:58:01Let's just do it manually. Try to be clever. RIP me. So... It would be... I think it's if you return an active... No, you'd have to call each. Yeah. Yeah. chamlis_ where did it come from, where did it go? where did it come from, unstructured control flow
I guess I'm just making up API in my head of it would have been nice if findby did that. So let's roll back and roll forward. Where did it come from? Where did it go? Where did it come from? Instructed control flow. That's excellent. Thank you. Yeah. That's That right there, chamless, usually I take some of the goofy stuff I say along the way, but definitely the title for this stream is where did it come from on structured control flow. Thank you. chamlis_ lol
I like having these fun titles for the stream archive pages. Why did migrate not even, did I get a syntax error?

01:59:26You can use change table. Oh, real standard RB. I don't want to do those things.

...44I just want to know why my code didn't run. Did I not roll back correctly? So add origin, revert.

02:00:03That's acting like I don't have a migration.

...15espartapalma I got you about invented API, because there's a find_or_create_by(...) {} and that may take a block, but that block only runs in the "create" branch... which is also confusing
Okay, so it thinks this migration is already run.

...24Yeah, there's finder. Is it finder create by that takes a block? But that block only runs in the create branch. Oh, wow. That's a subtlety I didn't realize about that. Now I wonder if I have any mistakes about that.

...43Looks like we don't do that anywhere in the code base, which is good, because I understand why find or create does that. Oh, that's such a pretty subtle API.

02:01:08Oh, I'm equivocating between domain and d. That's what's wrong here.

...43Yeah. Do you actually believe that it loaded 72,000 stories out of the database that fast? I don't believe it. Not in Rails. Stories.findEachUpdateActor. That looks accurate. Let's go look in the database.

02:02:10Yeah, zero. So, did I get the selector wrong? Or did this not fire? I bet this didn't fire.

...55Okay. Survey says false, false. Oh, come on. So when is this running at the wrong time? So after save, well, do we want to do it before save? Rails callbacks. I thought this was the right place.

02:03:43And we get false false.

...57Don't love doing this before save, because if the data isn't persisted because the save fails,

02:04:28If saved, change to published.

...37How have I never... That is a totally unfamiliar... So this becomes...

...58And this becomes change to replacement. False, false. But I did change it.

02:05:31What are we doing here? Oh, I didn't replace this instance. And I bet the instance is running on outdated code. False, true. There we go. So it was just me foot gunning myself. OK.

...57So let's roll back the migration. Don't need this puts anymore right let's be confident let's say this time it's going to work this time surely I won't have some weird. explosion that forces me to manually revert part of the. validation failed domain must exist.

02:06:29These domains do exist. What are we mad about? Let's go step by step through the trace back. So we got line 36 called update origins, which is correct, into the block called origin. This looks like an origin was created by this method, but it didn't. So there's find or create by. Yeah. Needs to be find or initialize. There we go. This one too.

02:07:34I don't know, can be created by.

...42Origin requires that this model exists. Or this association exists, but I wasn't passing it in because I was thinking of identifier as globally unique. And so I was trying to look it up by just one field. Yeah, that seems good. All right. So we are going to have to run, can I just run a script now? Can I run, what did I call that script? I called it reverse origin.

02:08:35But I have to do it in the terminal, or I have to do it by copying and pasting because I wrote this so that it would just fire off all these commands. And if each individual command fails, fine. But the way that script runs is it exits if any of the statements fail. There we go. This is promising now. This is the kind of long pause I would expect if we're loading 7,249 got objects out of the database and creating associated records for all of them. That's really positive. That almost makes me wish that in testing I had done like a limit 10 or something in there just so I could see that it ran on some data. But maybe, maybe we can be allowed to hope and think that this is gonna run clean and the migration is gonna be done. Or we can be like one of those movie characters where we're like, don't give me hope because I'm so sad inside that I can't hope. I've gotten a lot out of stoic philosophy over the years and it kind of makes me laugh at all those angsty characters who are like, don't give me hope.

02:10:13But this is pretty positive. Oh, this selector on GitHub.io is going to fail because all of these GitHub.io sites are considered different subdomains. This model is wrong for this use case.

...41Unless domain shifts to be like the base level domain. And that's where like the public suffix list is going to finally come in. People keep bringing it up, but now it's lurching in. Oh alright well let's look at the output of this before we get too frustrated, let me throw it on scratch but.

02:11:26All right, we've got 6,800 origins in the database. Let's grab a couple.

...40That looks nice. Look at those identifiers. And they're all on the same domain. That's really clear. I like that.

...53chamlis_ nice
So I'm going to grab one of those. Oh, JCS is one of the earliest. All right, uninitialized constant stories per page. Let's go find that. It is over on the stories paginator, not the stories model.

02:12:29Undefined method count for nil. Oh, this needs to be origin.stories. Or it needs to be at stories, right? At stories.

...54How do I do that in for domain that I didn't get an exception? locals thing. Yeah, it's a partial. Is stories not getting passed in correctly?

02:13:23So this says render above. equals the partial for origin with the stories.

...42Is this not running? Let's take a look. So eight. So it's an empty list. So that's a bug right off because the migration is not correctly updating where the domain model is not correctly updating the story, right?

02:14:35Yep, that's wrong.

...43So we're finding the stories. We're updating the attribute. Since I'm definitely going to be running it a couple more times. Let's go ahead and put that limit in I joked about.

02:15:27So if I grab all this, wrong grab. Add a little debugging, we'll roll back, we'll roll forward. A lot of rolling around on the ground this stream.

...52That's what migration bugs make me want to do is roll around on the ground. So that all looks good. Why are these not saved?

02:16:16Is this update attribute not doing what I think it's doing? Is it not persisting?

...33Let's call.

...40Yeah, see, that's actually, so it finds the origin. It updates the stories to set it to 48. Those caches are updating. There's some callbacks.

...56And then that's filled in.

02:17:07And yet the only record that that worked on is the one I just manually did. So even though it printed, this data didn't get persisted. Here's a great opportunity to spot the bug.

...37So we set these things, we save, and we find the stories. We update the attribute. An update attribute, I mean, it could be a, it almost should have a bang on it because it does go save. Maybe this is throwing an exception? It doesn't feel like it. You know, failing to save, but then, I'm going to pull up the update attribute. This method raises if the attribute is marked as right only, but it skips all this stuff. Well, it does do that. That's fine. Well, it's less fine, but we could also do update column, which skips all of that stuff. We called on new objects.

02:19:08If something's happening, then I'm not getting here. I put the .id column in the wrong place. But this looks very good. Well, there we go. Now it worked. Something is different between update attribute and update columns.

...56So that gets us an origin.

02:20:06And that gets us one story. That's fine. One is plenty. And now this wants to load. Undefined method count for an instance of story. So instead of passing stories, we're passing stories singular. Clean up that. Clean up that.

...36I think the home controller is maybe supposed to be calling paginate here. And it's breaking some contract with the cache. Let's see if that does us. That's better. Undefined method, edit origin path. Cause there's nothing, there's no form or there's no route for that yet.

02:21:30So that'll be for moderators, but I don't have to implement that just yet.

...46Let's just grab this stuff.

02:22:09Do I want to set that? So moderation has this very multiple foreign key thing that SQL is especially bad at, where it can associate to multiple records, and so there's no good story for how do you associate of well we have a user column and a domain column and a story column and a comments column and i don't think i want to set the domain column yeah well look at that that's that's a success So let's go up to the origin view to fix that typo.

02:23:31There we go. Missing a parentheses. This should be a link to. Hey. That looks really nice. Alright, so having taken the limit out of the migration, I'm going to roll it one more time. Run it one more time here. I'm rolling, I'm rolling, I'm rolling. So for those of you who are not similarly finely aged, I'm singing a song by Soul Coughing. pushcx https://www.youtube.com/watch?v…
arh68 i just hear the limp bizkit song SeemsGood
a wonderful 90s band that broke up in i don't know 99 or 2000 and then got back together eight months ago god that was incredible that they did because the the lead singer wrote this book called the book of drugs i think i've mentioned this on stream before about how Things that happened to him while they were...

02:25:12Things that happened when they were together as a band. And it was not a happy story. It was not. Boy, it's not the kind of book you would have written if you ever wanted to get back together with those guys. So I don't know how they all buried the hatchet, but it was nice to see them live a couple weeks ago. It's funny how much faster this runs if we call update column and don't touch any of the validations, migrations, etc. I know some of what's happening on story, but It's our got object. So this looks nice. So this should still load and now it should link all of these. Yep, all of these different origins. And the origin links up to the domain. Look at that. Let's go to the scratch file and cross some things off. Boy, let's commit, right?

02:26:29So there's that, there's that, because we have the migration and we have the list of all, or no, we have the list of stories by origin that's already checked off. Oh, so I don't get to check it off again because I checked it too early.

...53I am going to add, let's come in. Gosh, I push so much code around. Push so much code around. Yeah. Let's go ahead and make a commit for this.

02:27:29All right. And then. In for domain, we can say... Let's bring up what one of those looks like. We can say... So here's a Railsism... don't have a great answer for. What I want is to iterate over all the origins, but only if there are some. Actually, I guess here I only care if there are some.

02:28:25No, here I care if there's a count.

02:29:11So now this guy ought to say, that's fine. Let's add that route.

...22And we'll have a route for just specifically if you say slash origins. And this one can go to domains. Should go to domains, origins, or should it go to origin? Let's place the rest routing kind of breaks down. Well, I'm listing origins.

02:30:13And then up in the, get rid of that.

...46can be pretty short and then we need a view well that should be enough to load this right because now i have a route that looks good yeah let's just squeeze it up in that top line that would be better

02:31:314,800. It calls for pagination, right? No, absolutely not. Although origin is probably going to want to have a cache of the number of stories from that origin, right? If it does that, then I don't have to hit the database with a giant count query here for this page. So maybe I'm going to run the migration one more time. Oh, ARH, I saw your comment, but I didn't ask. What Flint Biscuit song includes the word rolling? Or does it include the word migrate? Did they do a road song?

02:32:23arh68 uh "rollin'" i think
I don't even have a directory.

...34So let's make the directory singular or plural plural.

...59Excuse me. Limp Bizkit has a song just named Rowan. Oh, okay. Shows what I know. I'm going to have a p, and we're just going to say, where's the closest to this? So let's just snag that. So we can kind of match that output, right? I don't love this preposition part of. It's sort of not part of, it's like a division. Maybe origin isn't even the right word and I've got to do a big search and replace.

02:34:10No, this is wrong. It should just be.

...22But this should link up.

...47I think I mentioned on stream I've been playing with Flex as a Ruby template. It's been really neat. It's kind of funny. The way I jump into and out of Ruby and HTML, it's sort of pleasant in Flex to never think about that because you're always writing Ruby.

02:35:19And there's none of this instance variable.

...33See, this is the thing that's going to hammer it. And it's probably going to trip. Actually, I know it's going to trip the n plus 1 detector. So let me just leave that out.

...51We'll just put an end there for now.

02:36:03So now this guy should move up here. And origins github.com slash org. No, this one wants to be Got the route wrong, didn't I?

...52So it's slash domains slash ID slash organs. And this should have been the domains origins path. What does it say here? It said domain slash whatever. Oh, it's the clever helping you out if you make stuff up route. Every time I do something clever, it's this colon author thing. Got to put origins first, because otherwise it's not specific enough as a route. I don't know why I'm reloading. There we go. And I say, there we go, a syntax error. But now the URL didn't get redirected. We're clearly rendering the right template, or at least attempting to. The domain has changed. What's the syntax error here?

02:38:11chamlis_ no end loop?
Oh, there's no end. And this doesn't need an equals. Shamless, yep. You saw it just about the same second I did. chamlis_ darn!
So you've got to beat me if you want to be a VIP around here. Identifier for nil. Why would origin be? Well, here's another bug if you want to erase. Oh, I called it origins. And then it's not the instance variable. It's just a local variable. So I got away with it again. There we go. OK. That's just one big list. And a couple of these are saying that my regex is not quite right. You know what would be nice? For that fast feedback on the domain edit form, instead of being like the latest 10, it should be the longest 10 because that's going to surface this kind of error. So where's the...

02:39:45that in my to-do. This really isn't an ordered list, or at least it doesn't really make sense to count them.

02:40:11Yeah, it's a little wild for GitHub, but I'm not thinking this is much of a page for people, especially not the GitHub one. But for other sites, smaller ones like Dataswap specifically, this would be kind of handy.

...37chamlis_ I think you're missing "origins" at the end of the intro text
hmm all right so do i want to do the stories count yeah or what else is there to do it would be nice to have a list of if maybe it's a little spicy to include whether they're banned on this page. I mean, that's why you're looking at this page, right? That's one of the most interesting things to see. And then it would be on the individual. origin page can have that detail. That's fine. Let's look at the domain.

02:41:54Where do I have a counter cache? It's got to be somewhere, right? No.

02:42:08Date counters, user has counters.

...30See the real interesting thing is pages where this is going to say and stories from and submitters and it's going to be like 20 stories from one submitter, which is the. The real standout for self promo.

...47Alright, since I haven't used it let's try and lean on the rails.

...59counter cache, which I don't think about much. But this isn't quite what I want. I want the count of associated records specifically.

02:43:30yeah and they for the post decrement but i don't want to have to have like stories go and update its domains i want to just say declaratively this is what the counter cache is and i thought there was a nice rails feature for this, but maybe it's something I have to hand roll.

02:44:09Yeah, this is just generic.

...17Count the number. Oh, there it is. It is called counter cache. It's just a separate. Okay. So let's roll back one more time. One more time. So we would like for origin to count the number of stories.

...55Put it up here.

02:45:07And then we would like domains to count its number of stories, because in for a penny, right?

...26And do we want to count the number of submitters? Yes, but

...36But there is I'm going to have to manually do that. So I'm going to commit to running this migration again, but I want to move incrementally.

02:46:13So post that belongs to its stories.

...43And then up in my view, up in my domain view, instead of saying stories.count, I want to say stories underscore count.

02:47:14and then up in the view for origins of origins i want to say origin stories underscore account does that seem right

...43Yeah, they both have a field stories count. Let's see if that wants to run. One more time. No. Wrong number of arguments. Since the migration failed halfway, I'm going to get to run that script again to fix this stuff.

02:48:07Given two, expected one. What are you even mad about?

...23You think I'm missing origins at the end of the intro text. Sorry, it took me a second to see your message. Which intro text? Back in one of these? chamlis_ for the list of origins for a domain
It's here. It's part of the link. If that's where you were looking for the list of origins for a domain.

...57arh68 it was like `github: 4789 _origins_` at thet op of the giant list
Yep. Yep, that's correct.

02:49:07Yeah, that's a missing word. All right. So why is the migration mad about this? We're saying to domains, we're going to add a stories count integer integer that's missing, right? This is the inverse of that string munging I was doing before. Good. Looking positive.

02:50:00So let's look at the partial update. Still running. Won't let me peek. But yeah, that page just before I hit reload had exactly the text. You were right that there was a word missing. I'm going to be a little capricious and say that a missing word doesn't count as a bug.

...39Bring you back up.

...45All right. Come on. How is it taking longer to load now that I added that cache? chamlis_ I feel nitpicky even bringing it up and costing time tbh
Maybe I should be running MariaDB optimized in the background. Oh, everybody says 0, because none of these counter caches Got refreshed because I did stories update column on all of these yeah so it's the migration needs to fill the cash.

02:51:32I call it. I just say. Yeah, see, I want to add the where. What is this going to look like?

02:52:07So I want to run one update query. Is it update all?

...32Yeah. So domain.joins stories date all stories count equals count star. Is it going to join? Let's see what 2SQL says about this. Probably it says you're terrible. Update domains. Stories. Inner join. Set stories count equals count star. Yeah, it's not from stories. It's just count star, right? Invalid use of group function.

02:53:30I just... Just slop it.

...54Right, with an update all, it just runs. OK. So let's grab these two. I say two because I'm going to do this for domain and for origin.

02:54:33You want to blow up on me? We're origin underscore, origin dot ID. Bad edit.

...50Actually, really bad edit. We're origin underscore ID.

...59Can't spell. All right, it's correct here. That'll teach me to copy and paste better. So now we load the big page. There we go. Kind of tempting to sort by number of stories, isn't it? As opposed to... This sort of arbitrary, when was the first origin, first URL for each origin submitted? arh68 so are you so far in the plural camp that `1 stories` is fine lol or wat
I don't need to make a leaderboard though, right?

02:55:47Am I so far in the plural camp? No, I'm so far in the singular camp.

02:56:25arh68 oh my mistake
Just editing it off screen, because it's a monkey patched on a string. It's not a helper. That's one more level of escaping here, where one stories. Why are you? Oh, because I wrote stories plural instead of story and I didn't change it. Getting tired enough of making dumb mistakes.

02:57:10What am I missing?

...29So it's whatever. Oh, I have to pass the.

...38And as long as I'm doing it there, I should probably do it up here, right? Because you might only have one origin.

02:58:10There we go. It's a little noisy having that thing repeated, but it doesn't make sense to try and strip it out. That would be too much clever code.

...30OK. Let's go, do I want to go most recent first? Oh no, then that's really misleading because it's origin by the date by the most recent first story. God, I can't even say that, let alone explain it. So I will not throw a .reverse in there. What's that part of? easeout alphabetical would help you locate an item you had in mind
Don't want to just say from domain. Yeah, alphabetical would help actually.

02:59:44Thank you, ease out. I think that's a order by. I think that would be an improvement. arh68 i always loved the phrase "sort by random" HahaThink
Oh, and one of the nice things here about that is it would bump any submissions of the root domain to the top, which is generally gonna be quite nice. What do we have here? easeout oh that's nice too
This one's gonna say, huh, these look like, Places where it has failed to find the right origin. Oh, look, there was an extra slash in one of these. Yeah, hyperapp went slash slash. What went wrong with these other ones? Cydrobel. arh68 is it `http://` only ?
This feels like... Why didn't this get parsed correctly? Is it HTTP only? Oh, maybe. Oh, that's probably it. arh68 like `//` or `https://` are gettin htru
It's possible that people typed their own. Let's check on one of these. That's a good bet. So this story is right here. easeout or bare domain yeah
Let's just grab the ID out of the URL.

03:01:30So the HTTP, you can't submit, you have to have a protocol on the front is what I was meaning to say. Actually, let's do that different.

...57Let's look at all of them.

03:02:05All right, I made this association for a reason. Hey, ease out guessed correctly. So it's between that extra slash that I saw and HTTP singular. Excuse me, that's my little reminder. Didn't think that alarm was on audio. So, you know, I wish I could have a... a Twitch poll, although actually, see, one of the really fun things for an integration with the Twitch API, I would like to have a betting pool on how many times am I going to have to roll back and run this migration? And everybody could just pick a number and, you know, whoever is closest wins.

03:03:00easeout I think arh gets that credit :)
So. I'm going to put these question marks in here. Oh, yes, Ezhel, thank you for correcting me that ARH guessed correctly about it being the singular work there.

...34This wants to be a plus, not a star.

...44Run that rollback for rollback is fast. It's the roll forward that takes extra time.

...57I mean, it looks like it's been taken about 55 seconds, which is not bad. And then I want to add it to the example. because that's the one that's going to be the template for all of these. So it's here and here.

03:04:25So now if I migrate, well, it doesn't immediately blow up. Oh, it's not yet to updating the stories. All right, there it goes. All right, so we got about 50 seconds and I'm not going to keep singing stories that include the word rolling. What's that one that's rolling on a freight train? Some classic American rock song.

03:05:05And then once this is done, I'm going to reload this. And I mean, looking at all these, it's going to 404 because no one has submitted the base URL of github.com. Yep, 404. Good. Now here, that base URL is going to go away because it doesn't have any. Yep. That's funny. I almost expected that we would have had There's a corner case. So I was thinking, there was a couple years ago when Microsoft acquired GitHub, people submitted so many links about that, so many hot takes, so many alternate URLs, that it was one of our biggest merged stories in site history. And so I was thinking about that as Well, how is that going to show up? So all of those story objects, all of those stories, the ones that were merged are going to be listed here, but then you'll click in. You'll click in and it'll be... Yeah, so...

03:06:28Hmm. Let's look in the database. I want to see how this behaves. Let's grab the identifier from... Yeah. Identifier from origins join stories on stories. Origin ID is the origins ID where stories.mergeStoryID is not null. And just give me a couple. Hey, look, github.com slash Microsoft. Exactly the people I was just talking about. So if we said for the origin, oh, look at that. Doesn't that replacement want to smash capitalization? Because I'm going to get capital M Microsoft lowercase. All right. So I can only juggle so many things at once. Let's go in here.

03:08:04So what's my bug?

...11I'm getting a 404 because I think the lookup is calling downcase, isn't it?

...32No, nobody's calling downcase. But we're not finding it.

...55So we only have the one.

03:09:05capital m i missed the the r i typoed and i said microsoft so this capital m yeah i think i just want the domain model to smash that case

...38Although I would actually rather do that in the origin model.

03:10:01And I could add a validation on the format, but that seems like overkill. A little bit redundant. All right. So is there anything about requires? Nope. All right. So I think that handles downcase. Now let's go back to... handling merged stories. And I'm going to pick one where it'll hopefully be a little easier to pick out which story because there are fewer of them. So which one of you guys is a merged story? Not you. Not you either. Okay, I'm puzzled. So let's grab not just the identifier, but...

03:11:26So this one, ZLFEJV. is neither of these. I bet it's just not showing up in the list because the stories patogenator is tossing it.

03:12:12yeah so instead of seeing anything so it says two stories from three submitters but actually there are three hang on as long as i'm like doing an enormous yak shave i don't want to remodel stories but this is one more place where the data model for stories for story merging is biting the site it is kind of an endless source of these

03:13:05So it is probably model story or space. Yeah. So on the idea of, pulling stories, we have this base, which respects deletion moderation and does some other smart stuff. And the base includes this term unmerged. Actually, I'm genuinely puzzled that this two stories is correct. Did I not change that to use the cache? Yeah, I failed. So that should have used the cache. So the number would be wrong. That would have alerted me a little earlier. Not that it would have been wrong, because what is wrong in this context? Yeah. So it makes sense to me to list merged stories here, but I don't want to list the merge target. So that instance of whatever ZLEF1 is, ZLFEJV, I want to list this second one for GitHub, not this top one, because it would be weird to say that this link from Jasper is from that origin. Like that doesn't that just doesn't make sense. So I don't I don't want to include merged stories here, or rather, I do want to include them. See, it's the difference between are you a story or are you a link? I have an open issue about fixing this model. Where is it?

03:15:40I'm going to say it's one of these that's got a title that's a little different.

...57pushcx https://github.com/lobsters/lob…
Yeah, so I left this comment here and this is the one that's where it just these bugs never quite stop where there are always these design issues. And you can see, look, so the tempting thing on this, and the reason my cursor is where it is, because real tempting to say, well, let's just add another Boolean here.

03:16:37And then the Boolean, like this user could say,

03:17:13Right? So now the scope has one more option. It does one more thing. And the knowledge about story merging leaks one more place.

...34I actually kind of like that display where you can see the icons. You get what's about to happen, why. When you click into this one, the top line isn't going to exactly match what you have. So that part is okay. But this, this makes me a little sad. Especially because it's not convenient to express with the ternary or I could just say, if this was slapping a string together, I would just say, you know, the inline question mark foo, colon, whatever to split the two. And instead I have to make this temporary variable.

03:18:20Yeah. All right. Well,

...51know it's funny i've gotten to check off what one two three seven things today but the list only got shorter by about two and then this one i've been putting it off but handling github.io It just literally doesn't work with this model that has had all this sweat put into it, because each of these is considered different domains. And if we want to change that, where subdomains are all considered part of the... That's a really big design question. And it unfortunately calls into question a lot of the code I've written on these last two sessions.

03:20:08going to have to think about that one i think i'm gonna end the stream here on this big hairy question of how to handle subdomains because it feels like the right thing is to use the public suffix list to figure out that github.io is a public suffix there's a bunch of moving parts of okay so then do we refresh that every week is that a cron job arh68 how many other .. suffixes are there ? other than `.github.io` i wonder
stops being data at rest in our database and becomes a process that's a lot all right so i'm gonna end there feel free to ask any last minute questions as i'm winding down here let me review this diff and get it committed how many other suffixes are there other than github.io i wonder yeah let's well let's get this committed

03:21:23What is this query going to look like? I'm going to... All right, so we're on the... Where's OBS?

...39I'm going to BRB. ARH, if you want to write that query, I'm going to just step away for two seconds and then we'll figure that out.

03:23:19Okay. Puzzle out the query. arh68 uh no I'm still thinking
I suppose you might actually have, you know, a job.

...47I have a cheesier version of it.

...57It's gonna look like... which direction is it gonna be? It's gonna say...

03:24:41Never remember the arguments to this.

...49I think it's one indexed, which is what keeps throwing me off. String, pause. Okay. No, it's not substring. It's substring index. That's it.

03:25:50substring index. No, it's not substring index.

03:26:03Instr. There it is.

...15We're getting partial matches.

...33arh68 wow that public suffix list is kinda huge
I want to know where all the ones where the top domain is the end.

...52The length of.

03:27:07That's not it.

...19easeout prepend a . to top.domain in the where?
It's got to be. So if we have bob.github.io, prependa.to.

...35easeout I guess that only clears one class of partial match
yeah yeah that's probably part of it and then i don't think i need this i was trying to get at yeah we don't i don't think ease out i think you're on the right path because we don't have to check every corner case we're just trying to get a rough idea why is in string let's call this let's swap these around oops

03:28:35So why would. In string.

03:29:05I think I'm just getting the join wrong. I'm not saying on. This returns zero if something isn't in.

...33That's so weird for... yeah that's what it is it's i assumed that it was going to return null because sql is so fond of null so i was just getting everything there

03:30:10Clean those out. What are we thinking here?

...23I don't think Insta is behaving.

...35So does it matter if I have the leading dump?

03:31:01There is something going wrong here. We're just finding every match or every pairing in the database.

...18Where instr and the subdomain is not the same.

...35Am I reading the Insta docs wrong? Is it doing the wrong thing? It's the same as the to argument, except the... It's just the opposite of Insta.

...57So ARH, I'm... I'm flailing at your query here. I would like to be able to answer your question, but I'm not spotting what I'm doing wrong here. This is a good place for somebody to get VIP by spotting a bug.

03:32:18arh68 lol ya i'd move on idk how to cleanly handle this
We're trying to say where. Oh, this is backwards. I'm doing the opposite. So I do want locate.

...32There we are. I believe that's working.

...40It's at least thinking about it for a second, which given the number of rows in our database seems pretty fair. And ARH, you say you'd move on, but actually this is kind of the last thing I want to do before I end the stream. So if anybody has any last Office Hours-y kinds of questions about the site, the code base, Now is the right time to throw them in. We're going to see if this query exits. Maybe I should have said limit one. I want to test this query real quick. Oh, read line. Read line, why are you being so troublesome?

03:33:38I tried to say limited to github.com. Let's restart the red line here. Maybe it wants to behave. Yeah.

03:34:24Why does it think I'm doing addition? Am I concatenating wrong?

...59Yeah.

03:35:21So it should be possible, but we're just not behaving.

...38arh68 i'm used to SQL Server so i'm not good for catching maria syntax i'm afraid
Yeah, this feels like it's some kind of fiddly syntax thing. I wonder if I just drop this.

03:36:03Oh, there we go. OK, so it's something about that concatenation. So ARH, you specifically said you wanted those where it wasn't github.io? Kat, Kat, don't type.

...38arh68 ya like is there a 5-long suffix list we can get away with, is what I wondered
Cat is in a playful mood, so let's see how long before the microphone gets knocked off the desk.

...54arh68 loading the whole suffix list into the DB is .. an option but idk
I see where you're going there of the whole suffixes. Yeah, I don't know that we would even have to load it into the database. It could almost just be like a JSON file on disk that we load, throw away for that process or for that story submission. Well, here's that issue with wanting the period.

03:37:37It's at single plus.

...53Let's try and put that back.

03:38:02See if this still blows up, because all of these something that match x.org or x.com are not so helpful but like z.cache.foundation i must be doing the these are all backwards aren't they ah too tired to write fiddly sequel

...38Yeah, this loading the whole public suffix list, there was a blog post just recently by a guy I know. Let's see if I can find it.

...59So here's your They're not in the order I wanted. I wanted the... Oh, no, they are. There's the sub. Here's the top level. Yeah. So we have ignoring github.io. easeout ok this is a good ballpark answer
We have 7,200. A lot of WordPress. Subdomains for Twitter. Substack. Stack exchange. arh68 1 github-io 2 wordpress-com cool cool 3 ... substack? tumblr... I see
Yeah, so these are all going to be on the suffix list. Quorum. Huh, didn't know they had subdomains. I don't know how we have a just straight org in there, but this is some kind of bug that we have that. Whoa, maybe 7200 isn't right. arh68 ohh ya can you submit to dot-less URLs ? those are so strange
That was a lot of nonsense that I scrolled by. So here's our github.com. Oh, there's clearly some way to say username.github.com. And I had always assumed they were at github.io. So there's something else there. Oh, it must be the GitHub pages, right? GitHub pages are at io. Let's look at one of these.

03:40:24Yeah, GitHub pages. And then a lot of these are smaller.

...37arh68 rubygems . pkg . github . com ... so they DO go deeper than just 1 level, interesting
I've looped.

...51That's some kind of bug.

03:41:09Oh, we strip off what looks like www, which sometimes has a number after it. So that's where that's coming from. What an interesting little bug. Have to file that one.

...41easeout ha interesting
All right. Yeah. And air H air H when you say, can you submit to dotless URLs? I'm not sure what you're looking for there.

03:42:05arh68 like `//org` or `//ai`
Ruby gems package, GitHub com. They do go deeper than one level. Yes. Yes, they do. Oh, like slash slash or gray eye.

...24You should, I don't know that you can submit dotless URLs. I think our URL parser, where is it? arh68 there are so few, it's just unusual to see any w/ sites
Oh, it's over in utils now. So we have HTTPS.

...48Well, the domain allows but does not require that you have a dot. So yes, you should be able to submit just a bare top-level domain like org, which is how it took www10.org almost. It would have been normalized. Hmm. Alrighty. I wonder if we have any, I don't want to try and write a regular expression of SQL syntax for regular expressions to find me any bare domain with, you know, domain with no dot. arh68 anyhow good stream PopGhost i'll let ya go no more SQL string manipulations
That's just not going to go well. Yeah. If you want to write it, I'll, I'll run it, but I'm going to wind up here calling into the stream. Yeah. Well, thanks very much for the contributions and the kibitzing. easeout gg wp
Thanks for hanging out with me today and coding on origins as we found even more corner cases and had to rerun the migration, what, 20 times, 40 times, somewhere in that neighborhood to get it to work. That was a little much. All right. I will be back on Monday afternoon per usual. See you then, folks. Take care.