I am unreasonably attached to ellipses

02:57pushcx https://lobste.rs
Mostly it's office hours talking about any kind of site norms, moderation stuff, the code base. I'm happy to jump into stuff. rebelelder afternoon 👋
You can pipe up with any kind of questions, or if you want queries to run against the production database, we can edit and tweak those live because doing those offline is incredibly painful, even if you set up a dev-env. And then when folks don't have questions about the site. Oh, hey, revelator, welcome back, or elder, excuse me. So last couple of streams, I've been working on this origin thing for dealing with self promotion. And if we don't jump into more office hours, he stuff, I will keep working on it. Because I ended off with a Hey, here's a sneak peek of where I want to go with this. And I realized that if I shipped what we'd actually coded, it would probably be a little more painful than needed. Let's ship it in a partially working state. Actually, so there's something else I did with that too.

04:22So there's another feature that I coded maybe a month or two ago for a link model. And it might be time to come and use that. One of the things that I do sometimes, not super often, but sometimes, is code features and come back to them a couple of days or a couple of weeks later. Because as we've seen a whole bunch with the origin model, there isn't the site Doesn't have a ton of tests always. It has a pretty appropriate level for its maturity and its needs, but... Oh, another good one would be right where this stream started, changing how voting on stories worked did not work out in practice. And maybe the second or third stream, I did a deep dive on how the code worked fine, but the social norm and expectation did not work fine. So I was thinking about this origin and I realized if I shipped the code we wrote last stream that would enable it, it would just be a little too much too fast. So I wanted to enable it and kind of watch it for a minute and add some UI around it rather than immediately touching another social rule without a way to To make something more out of it, I guess. So let me get specific here. The. Let's see the last thing I added. Well, there were some tests. There were some tweaks because there was always more was fixing things and I would really like to fix this fixed ship. This fix excuse me. Was. A validation.

06:28Where's my validation? I bet I'm not on the right branch. I'm not on the right branch. Where am I? Okay. Once again, you know, it's a nicer... We have these kind of like running jokes where I have issues with audio. The other running joke is I'm always on the recheck branch at the start of the stream because I tend to hack on it into the evening. But recheck is the personal project side product that I'm working on for database integrity. And I tested against the actually I can actually give you a little demo. Yeah, we'll do one little demo. So it's bundle exec recheck. And I know it works with the message one.

07:26So a thing that happens in Rails, in every Rails app, is that production database ends up invalid. So this went and pulled all of the messages. So these are private messages in lobsters, pulled all of them out of the database and check to see, you know, can I even just call, can I call dot valid on this method or on this record? And currently in the database, 13,808 are fine and 99 are not. And what's going on with these Xs is every X is a thousand records. So if it sees, you know, that chunk of a thousand is passing, you get a dot. If any of them fail, you get an X. And this, what is this? Divided by 13, 808. Yeah. This like a 10th of a percent are bad. This is pretty normal for Rails app that like, What is this, 1 in 1,000, 7 in 1,000? That's pretty normal. I've been a consultant to a lot of Rails apps. And seeing this one, I happen to know exactly what's going on here. The validator for messages checks if, I'll just show you. I don't have to talk through it. Call me out on that, by the way. If I start talking and talking about code that I could just show you, just tell me. So it's this validation here. rebelelder is a hat like a role?
A lot of active record validations get written in this way where what it's saying is if you want to use a hat, and a hat is a way of saying that you are affiliated with a project, and you can say, Yes, I'm a Ruby core developer. I'm a Git developer. rebelelder ah, makes sense
I run this programming language. You can use those in private messages to say, yeah, a hat is like a role. Actually, it's very close to a role. Yeah. Here, I'll show you the in production. Here's a list of current hats. I think they're alphabetical. And I think the word action shouldn't be on the end that. That's a bug. Let's put that in the post stream scratch.

10:23So, for example, Fox Boron here is an Arch Linux developer and If they are talking about issues related to Arch Linux, they can put that hat on and say like, no, I'm officially speaking on behalf of the project. So when I tell you we don't support X or Y is buggy, you know it's a pretty authoritative statement. Hats can also be used in private messages. And there's this really common pattern in active record validations. Anti-pattern, I suppose, is a better term. design issue where people write them for for the moment where at the time a message is being posted it validates that they are allowed to wear that hat that's fine however that's not really It's only thinking about the form for writing a message, only thinking about record creation and not thinking about the life of it. This validation really should check, was the hat wearable at the time the author wrote the message? Or it should check, am I a new record?

11:56And either of those would be fine. I would prefer the atemporal one that says at the time. And the frustrating thing about that is if I have to load a message out of the database and tweak something about it, maybe it has some other metadata field like is read or is deleted, and I have to manually go touch it, when I go to save it, I get a spurious... failure of validation. Oh, and the reason that's spurious is people can take hats off. So I gave the example of Fox Boron as an Arch Linux developer. If they step down from Arch, they can say, hey, I just no longer work on this. So they can't use the hat anymore. So 99 messages become invalid, right? You can see it. this is not a big deal on messages like we they are kind of well they're almost right once it's more complicated with like stories where oh yeah stories are so heavy this is going to take a long while to run but they have a lot of validations and a lot of associated records and so like Seeing the god object fail validations is not too surprising. It is a little concerning to see like a whole row of like, hey, wait a minute, that's 10,000 stories. And in every 10,000, one of them is, at least one is bad. Yeah, that's a problem. I'm just going to cancel that. Oh, and I guess I'm going to have to fix these stats. It should still get partial stats. Like I said, work in progress. All right.

13:47Let's get back to Lobster's main. There we go. Extra topic. At some point, and maybe it'll be Monday's stream, especially if I can get the origin stuff squared away. At some point, I'll give a fuller demo of what using recheck looks like. I don't want to be on master. I want to be on origin because that's where we've been working. rebelelder it looks useful so far
Yeah, with these. Or if there ever was a candidate for squishing down migrations. Yeah, actually. I haven't yet released recheck, but it is already, like I caught that bug in messages where, oh, that validation is just not defined super well. And that one's a pretty minor one. The other place where recheck is really useful, and this is what some of the failures on story are, is rebelelder I’m only just coming back to both ruby and rails after over a decade away, falling back in love with ruby.
in production rails applications you will see errors that are like can't happen bugs and what i mean is some combination of fields that should be impossible has happened so an example might be if you are an e-commerce store you might see an order that has No shipping record no tracking label, but then is marked as delivered and the customer even has a positive review and that should be impossible. But you'll see stuff like that in databases so there's I gave a talk on this seven years ago six. Time flies and coven made it real weird. So. The organizers of KCDC were looking specifically for a Rails talk, so this is very Rails specific, and I talked about... I thought I put the slides on here. pushcx https://push.cx/talks/kcdc2016
Let's put that on my... So I talked about it specifically about Rails, and all of the code examples here are Ruby, this is everything that works on the active record design pattern has at least four of these issues one or two of them are very active record the gem specific but the rest are about active record the design pattern that especially if it does partial updates in the database you cannot have database integrity and and for the very pedantic person who is saying ah what about sql integrity constraints the answer is those are so painful that basically nobody uses them and some of that is questionable early decisions by framework creators but some of that is just check constraints in databases are sort of They're painful to use, there isn't a good workflow for revising them, and they're hard enough to write that basically nobody does it because they don't get a positive ROI. That's why recheck is like, all right, if I can't fix the library, let's detect and fix these things properly. So this was long intro here, but this is the code I wanted to start off with was I added this validation to stories that said, hey, if you're a new origin from a new user, let's give you an error message similar to the one about new domains from new users. And I realized that one is going to be a big hassle. It's going to be a bigger hassle than I want to ship immediately. Especially because what I really like to do is nudge people into commenting first and talking first. And the reason for that is People who want to exploit lobsters as a way of driving traffic to their thing really don't want to write comments because it's site-specific and it is expensive on their time, speaking of ROI, in a way that posting a link and moving on is basically free. Writing a comment on a story that's not even theirs is an expensive investment. that's the difference between are you participating in this community or are you just here to exploit this community so this check new origin it's trying to get at that but it's not good enough yet so i think it would just inconvenience there's a false positive rate to these things as people submit new stories some of them are genuinely like they're not self-promotional they're interesting they're just a domain we haven't seen before they're an origin we haven't seen before and that's fine and i think the false positive rate is high enough this isn't worth it so rebel elder i did see your chat message here you said you're coming back to ruby and rails after a decade away and falling back in love with ruby i think that's pretty great the I think we're at a really interesting time. pushcx https://lobste.rs/s/sb6nmj/what…
There was just a story here about Rails 8, and I'll share the link. I've been saying to folks, let's put this in the scratch too, Rails is in a really interesting place with so rails rails world last year dhh got up and announced a bunch of vaporware poor drama reasons and then Even now, folks are saying, yeah, like ActiveJob that was announced last year is still not quite fully baked. We've discussed it on stream, and folks have popped up in the comments to say, like, it doesn't really do what you want. So the announcement for Rails 8, I watched the talk as well. I thought it was pretty good. It acknowledged that last year's stuff was kind of Aprilware. And it feels like a whole new wave of Rails saying, we want to be batteries included. there's this list of problems you're going to solve. Basically, every site is going to have to fetch stuff out of the database. Let's just set a convention for you name your tables like this. You name your foreign keys like this. And it feels very silly, but in 2004 or 5, when Rails got started, there were debates over this and it was an active thing you had to think about at the start of the project. And Rails kind of said, oh, come on, that's not a good use of your time. Let's have a convention for it. It's not even a config file. Let's just pick one and go with it. That's also the same kind of vibe behind standard RB. And I really enjoy that quite a lot more than just using RoboCop directly. So Rails 8. manuelfcr Standardrb is indeed much better
also has a whole bunch of, well, look, you're going to deploy your thing until you get to the place where you are hiring multiple... Welcome back, Manuel. Manuel, excuse me. Yeah, the defaults out of standard RB are very nice. And I think some of that is... Justin Searls has opinions I generally agree with. And then also Justin Searls is running a process for getting at defaults that has resulted in defaults that I find much less objectionable than the ones that RuboCop seems to. So I don't know what's different in those processes. So Rails 8 is going like, look, if you're going to deploy, you're going to go to one of these platforms of service guys like, well, Heroku is the classic one, but Render or Railsway are the modern ones. or you're going to do something like lobsters does where you're going to take a VPS and you're going to set it up. So let's just bring that thing that everyone does until they have a couple of production SREs employed. And let's just make some options for that. And then also lots of people are going to have web socket. Great. Let's import it. Lots of people are going to, cache html fragments let's make that whole process smoother and it wasn't it wasn't actually bad rails does almost none of this partial caching for tedious reasons but one of the reasons is that setting up redis or memcached is is adding a moving part that i didn't want to add and then solidq It's getting into Sherlocking a little bit where it's like, well, for the first couple of years, Rails had a new popular background job runner every two years or so. And then Mike Parham came out with Sidekick where he had implemented and had used all of those in anger and then said, all right, well, what if I made one that basically fixed all of the issues of the previous ones? And he knocked it out of the park on that. So he's done very well with that. And now Rails is like, all right, look, everybody wants background jobs. It's at least a little funny that this is a separate project that is run by an external person and then also a commercial thing. This really should just be a feature of the framework. Honestly, the only odd thing about this is that it's named solid instead of active or action. I don't know what solid means. manuelfcr It's because it's DB based
If you haven't seen it, in Rails, active is a backend thing. And action is a frontend thing. Oh, solid because it's a database thing. Oh, I see the pattern. I think we'd have to say it's a new database thing, right? So it's active record, but it's action controller. then it's solid q yeah i kind of see it that one i guess that makes sense for for solid cable see i've been thinking about this back-end verse front-end distinction which i only learned a year or two ago kind of blew my mind i always just thought it was arbitrary and i had to memorize them and it's it's ish you cross your eyes a little bit because like action controller is the ultimate combination of back-end and front-end concerns. So it didn't... It didn't get a lot of love here. Oh, man. Asset pipeline. That's another one where I just know it's going to break again. And actually, that's maybe a place to talk about my one hesitation about Rails 8. which is also, I guess, the one that I have in 7 with last year's Rails World, is this stuff is going in and it's very new. And what often happens with these big new features in Rails is they go in a little bit half-baked, where they work, but then... In practice, most everybody bumps up against an edge or two, a sharp corner or a, we need this feature and it doesn't support this feature. And sometimes that's really positive where if Rails is moving fast enough and firing on all cylinders, people start making contributions to these libraries and they solve real world problems. And it's a wonderful example of Yagni where We're developing features just as they're actually needed in response to real world programs and the resulting APIs are even better. Mostly that's what happens with Rails. Every once in a while, and a concern here is the replacement of the asset pipeline. Every once in a while, we end up with the thing where the fundamental abstraction or design of that doesn't cover use cases that didn't get recognized when the thing was first shipped. And it's really a hard breaking change to try and support them. So the asset pipeline has kind of been unreliable for years. Like I have griped that 7.1 is the first, maybe 7.0 and 7.1 are the first minor or major releases where I haven't had the asset pipeline break in production. But I remember it was like five, it broke. Five one, it broke. Five two, it broke. And then I think we jumped to six and it broke. It was very frustrating. So I'm very, you know, one of my phrases is cautiously optimistic. I am very, very much more cautious than optimistic about, hey, here is a totally new asset pipeline. Hopefully we have learned from all the previous asset pipeline stuff for the last 15 years. We'll see how that goes. I hope it doesn't break in production is really the point of that. And we barely use the asset pipeline because we have one JavaScript file and I think maybe two vendor JavaScript files at this point, one or maybe two. We have one CSS file. We don't even use SAS or any of the other things on top. And still, it's been kind of frustrating. So we'll see how this Rails 8 goes. It's, you know, I'm cautiously optimistic. There's a lot of features here that are positive. So, oh, I started saying there's a SQLite feature here that was in the talk that didn't get attention and doesn't show up even here in the blog post that I'm really enthusiastic about. Let's see if I can scrub the video for finding it real fast. Did they link the keynote? And it was back here in the last quarter or so of the video. I'm going to make sure it's muted. I don't know if you get my desktop audio, but You're talking about Kamal. It was such a small feature, but I loved it. So let's slide this out of the way. Yeah, there's probably not going to be a big spike of YouTube viewers on it. But somewhere in the back here, he showed a little screenshot of what database.yaml is going to become. And that really caught my eye. Maybe somebody can scrub and find the timecode that I'm not finding, because I can't imagine. I don't want to become like, Gen Xer reacts to YouTube keynote. That would be a horrific use of the channel here. Somewhere back here, he talks about these queues and these databases and configuring the production databases. No, I think I've missed it.

31:21Yeah, it's somewhere here before the thruster stuff then. See, it was such a blink and you miss it thing. so here's cron jobs recur there it is yes no no this is it this is exactly what i wanted all right what's our time code 48 19. i'm going to grab that put that in the scratch so let me talk about why i'm hunting for this

32:02And it's what Manuel said because it's database backed. So for a long time, we've had this config database YAML, and it lists, oh, we have production, staging, test, development. The thing that's changing here is this idea that more than just a username and password, which is not super interesting in config database yaml i mean the most interesting thing is figuring out how to safely inject secrets into it in production is production is going to list multiple kinds of databases so he's coming out of that solid cache stuff so he talks about these oh we're going to have cache 1 cache 2 cache 3. but what catches my eye is Especially knowing that he's been talking about doing more stuff with SQLite databases, this is better multi-database support in Rails is the bottom line for me. We understand that in production, you're going to go talk to Postgres or MySQL or MariaDB probably, but then also you're going to have more databases like pushcx https://wafris.org/blog/rearchi…
a solid cache back end or a solid queue back end or you know a solid whatever and i think there's a lot of potential there there was who was these guys so this blog post and disclosure i know michael and i gave him some editing tips of basically like how to write this stuff better not code tips on they have this web application firewall so it's a way of saying like oh people probably shouldn't be posting random garbage like people run vulnerability scanners against rails ever or i'm sorry against lobsters every day every month or so somebody gets a 500 out of the site and i'm like okay these are all bs but let's make sure the site doesn't throw 500s so a web application firewall is a way of blocking these things and they need data on ip addresses and ranges of oh this is a tor ip you can probably assume the traffic out of here is a little bit hinkier and then Previously, they have used Redis for storing all of this kind of data, but then that required people set up Redis in production, which is a chore. Well, if Rails is going to ship with really nice multi-database support like SQLite, well, let's just ship that IP database as a SQLite database. And this post is all about performance and how much faster it is The other thing is, I feel like it's a missed opportunity here, but it's a whole moving part. You don't have to set up. It's Redis. It's one fewer production service that you have to install, pick hosts for. Do I want to run this on the web server, on another server? Do I have lifecycle management for those? Do I have config file management? Do I look at the logs? Do I have alerting? There are 10,000 things you don't have to do even if these lines were swapped, SQLite would be nicer just on the human toil side. And so when I see DHH waving his arms in front of YAML that makes multi-database support, oh, this is pretty interesting. What if we did this practice of shove more things into SQLite databases? I think that could be really nice for like a local first APM application performance monitoring. The way Scout and what's that other popular one? There are a couple of these and they run as SASSs where it instruments production hits and it says, how long did it take? How much time did it spend waiting for the database or rendering views or doing miscellaneous? And then it ships that off to a SaaS that has, boy, they know exactly what business customers are willing to pay. They are dialed in on their pricing. But that means that hobbyist projects like Lobster's just flatly cannot afford something like Scout in production because they charge serious business prices. Well, if an APM wanted to write to SQLite and then ship those logs occasionally, Well, then there could be a really nice offline kind of APM. Or, you know, WAF. All these sorts of things that they need data, but it doesn't have to be particularly fresh data. It's sort of in an in-between between OLTP and analytics. I think this could be really nice for. So I'm really... You know, with those caveats of, you know, there's a little bit of vaporware, there's a little bit of half-baked, I'm pretty optimistic for Rails 8. I really think it's adding a couple of, it's a whole other wave of, hey, everybody has to solve these problems. Let's be batteries included about it. So I'm looking forward to it. I really am.

38:17long answer maybe nobody asked getting back to origin i've got let's see i want to leave a comment of

39:00Yeah, approvals is what I was talking about at the end of the last stream of... Well, on story submission, rather than block, let's ask an inviter to approve and say, hey, does this look good? One of the things... Lobsters has its own culture that is pretty chill, that is collaborative, that is creative. And one of the benefits of not having open signups is we can maintain this culture. Because if we had open signups, we would have been overwhelmed the first time the site was linked off of Hacker News or off of Reddit. Because if we have 250 comments a day that come from 200 commenters and maybe 2,500 people a day are logged in and voting for the quarter million people who visit, Well, the first time a subreddit links 20,000 people to the site, our culture is whatever that subreddit said so the invitation speed bump lets us maintain our culture like it's it's the wall around the walled garden and along that line we have to teach people, well, what's normal here? What's on topic? What's not? Because we look a lot like Hacker News. We look a lot like Reddit. So people assume we have their same very broad scope or their same rules or their same cultural norms around rhetoric. Nope. And acculturation is largely just a function of time. You hang out and you get it. But there can be these nudges for let's show you how we do things at at the point where you're about to step out of line and kind of bring you in so like one is titles cannot contain graphic code points that means like emoji like please don't put emoji in story titles that one is trivial to enforce you can see it's you know two lines of code here and the regular expression Other norms like don't flame people. I can't write a story validation or a comment validation that catches that. But hopefully, people start engaging with the site by reading it and interacting. And they realize like, oh, this is actually really chill. I don't see any of that bad behavior here. So I'm not going to start it. I wish. So the nudge of what if when you make a story submission, we kind of package it up and rather than throw it in the stories table, I think it's going to have to be a serialize it to JSON and hang on to it in kind of a general purpose table. Because if I put it in the stories table, I'm going to have to touch every fucking query on the site. is going to have to know about these partial unfinished objects that are hanging around that's just too ugly these things aren't really staying stories they are proposals or submissions. And if I make a story submission table well, then it has to say perfectly in sync with. the stories table and the stories model. So there's a whole bunch of pain. So if I serialize it to JSON, well, that's pretty cheap. I can deserialize it, apply it to a form. Yeah, I think that design would work. So let's look at origin, because I think maybe I can check this off. Because if we start recording origins and don't do anything with it, where's my little build command? I want to see. I think I have a spec that'll start failing because I commented out that validation. And once that's done, I think I can actually merge down the origin branch. it would be nice to start browsing origins and get a feel for what they look like in production especially if i'm going to have to do some shotgun surgery to domain yeah so here's that there is one spec for that validation i think I think the RSpec syntax is I can just mark it as pending like that, right? Let's just run the one spec.

44:44Yeah. And is there...

...50I'm wondering if there's a way I can have that pending message include some more info. Yeah, I can't just say pending as a function. So let's do that instead.

45:24is going to, let's just say, wood. And as you can guess by the fact I have to look it up, I don't use this pending syntax much. So let's see the output. OK. Failure. Oh, that's interesting. So it still ran, but then it... That's fine.

46:01So it counts as a... Yeah, it's not a failure, although it printed a failure. All right, I follow. And it's orange instead of red. Okay.

...33I'm going to say comment out. All right. So if I look at all of this, I think I'm in a pretty good place. And I am happy to kind of want to skim this real quick.

47:00touched a lot of stuff. One of the reasons I want to deploy this is the new origin thing. I'd like to see that in production some. But then also, it just touches so much code. And some of it, like the routing, was incredibly fiddly. So I'd like to deploy that. If nothing else, so I can deal with the technical issues before the social issues separately from,

...37All right.

...46So let's merge in the branch. Let's make sure we're up to date. I'm going to hit Merge on PR.

48:01There we go. Big merge. Merge made by the Ort strategy. I have never heard of the Ort strategy before. It's a new one. No octopus. And if we look at this, see pretty much everything of, yeah, it branched off. It ran on its own in parallel. Good.

...38So I've merged the origin branch. And one of those places where I did this strategy of writing code, deploying it and not actually using it much was this link model. I didn't do this one on stream, but I did it around the time I started streaming. The We have a lot of stories and comments submitted with links, as you would expect. I wanted to extract links from comments and from stories and then also recognize them incoming. So let's go look at production.

49:33Let's go look at production. We have... Let's... do that in that order.

50:18Now that'll handle the null case. So let's just grab literally the most recent link in a comment. So Here is a short URL. Almost nobody uses this but me. So here's someone who left a comment. Oh, that's literally an old one. Oh, I didn't order. It's nice to look at recent data. So I wanted the database to model linking off to other stuff to say hey there are more interesting links that are almost certainly related to whatever this first topic is and this is a great example from literally a minute ago also from a new user hey welcome that says here's some documentation here's a i guess it's a link to the talk and this is just a write-up That's kind of wonderful. I would like to surface those better. There's also the potential for, say, to comment ID. Actually, I can just say where.

52:06I wanted to track internal links. So this comment, and I'm not spotting it. This comment is a bit of a wall. Ah, so in this comment, it links to another Rust thread here on lobsters. It is probably expanding on or improving the understanding posted in this. comment a few weeks ago. It would be nice if this comment, or maybe at the bottom of this page, there was a few more hyperlinks that said, hey, we are talking about ourselves, we are relating to ourselves, you can see more of this discussion somewhere. So that's why I added the link model. There's no UI for it, but all of this is in here. It would also be nice to catch specifically to recognize duplicate links. So specifically what I'm getting at here is, oh yeah, the other thing for origin to do would be

53:41I think that probably has to happen fairly soon.

...56So on the story submission page, I'm going to add a warning, I guess. that's gonna look at links posted in the last couple of days and say, hey, if this is already linked from an existing discussion, here you go, contribute to that discussion.

54:28So over on the stories controller, we use this is resubmit, and that's basically a way of saying, If you're resubmitting a link, there is an extra flow where you see a little warning that says, hey, you're resubmitting a link, we'd like you to add a comment. So I want something that's kind of parallel to that.

55:02So what I'd like is,

...28Let's see, it's going to be, I'm going to have to double check. So this range syntax is still fairly new to me, so I don't have it memorized. So we want to find links from comments that are recent that have the same normalized URL.

56:09And that's an attribute on story, so I can just directly reference it. And that's why I did all that story normalization cleanup a couple of months ago.

...27I guess what I want to say is, because then I can just say dot any on it. This is sort of a scope. Actually, this really is a scope. Let's move it over to link.

57:35So it's utils.normalize. And since I'm not pulling it from the field, I'm just going to run that normalization myself. It's idempotent. And if you see a very unusual amount of syntax highlighting and red stuff in the gutter, it's because I started adding sorbet to the recheck gem. My editor has picked up that I haven't thought about Sorbet, and so now I can't seem to turn it off. I haven't actually tried hard. That pull request is still just hanging out. I've fixed a couple of... I did a deep dive. I'm not going to repeat it, but did a deep dive on how I've been trying to contribute to Sorbet by... fixing a small bug but boy is it a big code base to jump into i i just submitted a patch to some of their c plus plus and i don't know the last time i wrote c plus plus i can't really claim to have written c plus plus a day or two ago because one of the maintainers pointed out oh you could just delete this function call so i deleted that function call and i didn't touch the rest of it i can sort of understand what it's doing but boy is my c plus plus rusty And of course, I don't know the structure of their code at all.

59:43What did I just call that scope? Recently linked from comments.

01:00:04You basically just want a little warning that you have to click through. It doesn't even have to be an error. Just a, hey, you sure about this? In which case, maybe I can move it up to the check URL dupe. Let's do that.

...35So this grabs form errors. It has JSON. Yeah.

01:01:36that a little and then here that's probably fine right so when we check for duplicates we're just getting what is this json is this maybe just one of the Oh yeah, we don't actually use this JSON. This is dead code already.

01:02:37This is the form errors.

...44I don't know that I want this to be an actual error. So let's think about this. So yeah, here we just say we have a notice.

01:03:08I don't want to literally fire off a query from here. It's a little tacky.

...30Okay, okay. I know. Shouldn't say with syntax errors. That's okay. All right. So... We're going to say, if there are linking comments, let's pop up a flash and we'll say,

01:04:49I think I'm going to start with a notice. And if this works out, come back and maybe make it an error. But I want to see where this goes for reducing how often I have to do story merging. So one big place that lobsters differs from HN and Reddit, which I've name checked already, is we can put multiple URLs into a given story. and keep the discussion together instead of you know something happens and then we get i like that rails 8 announcement we might on reddit you might see a link to the youtube a link to the blog post a link to the app signal blog post a link to the mobile youtube version and that'll be three or four or as people respond eight or ten stories and reddit is huge staggeringly huge so splitting the conversation among four to eight threads for every mention of a beta release, it's like, okay, it's fine. It's not the end of the world. If you read the comments, especially the lower ranked comments, there's a lot of churn where people rehash stuff that's in other threads, or they ask questions that are answered by the other links. We get to keep it together. And I think that's a wonderful feature for the site. I have some gripes about how that database is structured. And it's working its way up the priority list. Maybe it'll even happen this year. That's not a joke, actually. I do think in terms of like, what are we doing this quarter? What are we doing next year? As opposed to, oh, everything has to happen this week. Because, you know, it's a volunteer project. We put a couple hours in every week. We have a couple of code contributors, but nobody's getting paid to work on this. So I think I'm gonna start this out as a little notice, as a warning, and then maybe it can become an error, but... Yes, we'll list the...

01:07:22This is the link model. So that I can already tell I'm going to get an n plus 1 error here. This returns links when what people want.

...40So we will say include or includes.

...52Story has the best example of it.

01:08:01Includes, plural.

...18And then doesn't I'm going to display anything about comment. It has a whole lot of fields to preload. So I think I think I'll just grab. tumdum3 this seems to be unrelated to origin changes - have you already completed that last stream?
So we don't really have a way of a pattern for displaying comments succinctly like experts. Oh, hey, TomDumb. Yes. Earlier, I had the... Wrong, sir. Let's not leave that open. Earlier on the stream here, I merged in the origin branch. Because it has so much code churn, I don't want to deploy it live because if I break something in production, I have to log into production and frantically fix stuff while 500s come in. That does not seem like a great use of streaming time. But I commented out that validation because I thought it was going to have high false positives. And this is conceptually related in that it is working on kind of acculturation features of making sure people understand how the site works. All right. So we don't have any short way of displaying comments is where I was going with that.

01:10:06I could list stuff like who's the author, what's the story, how long ago. It would be really nice to have an excerpt of a couple of words around where it was linked. However, any exerting is going to be a pain in the butt because what if I grab half of a pair of HTML tags. So if somebody said in their comment, they said something like, this link is great. Well, if I excerpt, I might grab, you know, the opening on. Sometimes people use that. And then half of my page is italic. And I I don't think I want that. So do I want to link to the comments or I mean, I have to link to the individual comments. tumdum3 can you maybe strip the html tags?
But maybe I want to group by story because then we could just do the list detail. No, that doesn't link to the individual comments. And so if there's like an active topic with an 80 comment thread, could I strip the HTML tags? Yeah, probably. As long as I'm...

01:11:45So I think what it is is as long as I strip HTML tags but leave in the one that the commenter made that we recognized, it would be okay. So let's say, where does this code want to live? In a helper on the comment model? Probably in a helper.

01:12:34Yeah, a helper.

01:13:06Yeah. So we're going to have a comment and a URL. I'm putting this off in a helper so I can test it because this is going to have to be some TDD. So here, let's close out some of these extra tabs. This is probably okay for now. This is a false start.

...38Let's get that out. Not a lot of tests on these helpers.

01:14:43There's a couple of ideas of things I want to test for behavior. Let's set that up. Yeah, I think I'm okay sharing these.

01:15:08Create a comment with some text. So there's levels of translation here where this helper is going to work on the rendered HTML output, but we're going to input the markdown just to try and keep it closer to what the users do. Come here.

...46Let's make multiple, I don't wanna share data. And then I'm gonna just, isn't there a, Faker. I want to say Faker will give me some lorem ipsum. Got word, sentence, sentences. That's what I want. I just want this comment to be longer. And I don't care what's in it.

01:16:51So let's call

01:17:14And I realize I'm already going to be passing around normalized URLs, but this is such a cheap bit of bulletproofing that I'm happy to do it redundant, especially because string munging this is cheap.

...35So what do I want? I want...

...42I think I have to... For structure, I have to strip all of the tags out except the link and then limit. So an example of what I don't want to deal with is if I said give me three words around it, well, it might be,

01:18:13is like i can't type so in this one if i grabbed three words beforehand i would grab like is and then that this chunk of a tag and so then if I ran strip tags this would not be recognized as a complete tag right so that's that's no bueno I have to strip the tags and then recognize the words so really first thing I want to do is a comment where

01:19:14Check this. Let's just say, come on, test data. I love, oops.

01:20:01Let's just say.

...07So this is me just checking that the underscores are going to get you marked down comment. That's the HTML output to include strong.

...29And then if we excerpt it, we should not have a strong, that's fine.

...39So, all right, tum-dum. Did you have a strategy for stripping HTML tags or is this my problem? Because I think we can get there with nocogiri, because what I really want is to loop over the tags, and for anything that is not a link to our URL, strip it. Oh, we could still end up in this situation where if there's multiple, so I'll just say if it's not the first link to our thing, strip it. Yeah. So, loop HTML.

01:21:55chamlis_ is this not the markdown I'm used to or would single _underscores_ give italic <em> rather than bold <strong>?
If it is the first link, and I'm just going to say like, I'm just kind of making up syntax and stuff here. I want to kind of sketch this out as a comment.

01:22:36That's an end. That gets an end.

...44Keep it. Else, strip the tag. Right? So that's the first chunk of it. And then it's split by word. Grab up to, I don't know, what, five on either side of the link? Ten? Let's pick a round number.

01:23:18And then we'll join it back with a single space that also handles new lines. It's all right. That's, you know, enough of an excerpt. And I'm keeping this little warning here.

...36I'll delete that before I commit, but I just want it on the screen of if it were three. Is this not the markdown you're used to or would single underscores give italic M rather than bold strong? Yeah, Shamless, that's why. Oh, and Shamless, I don't think you're on the last stream, but I saw your email about improving the transcripts. I haven't had a chance to implement it. I have not forgotten it. chamlis_ oh nice, no worries
I appreciate you thinking about it and contributing. I don't remember offhand what we get.

01:24:36We get EM tags. Okay. Well, I would really like strong tags because EM as a character sequence is going to show up in lorem ipsum. So what is it? chamlis_ it should be double * or _
Two stars? What gets me strong? Two stars. All right. So thank you for that catch.

01:25:05Well, you say it should be underscore. That's not what it is right here in that example I just ran. chamlis_ sorry, __double__
And I don't think we're doing anything special with that markdown. We have some customization in the markdown processor. Oh, double underscore.

...31Yep, you're right. There it is. rebelelder :s/markedown/markdown in the spec ?
rebelelder markeddown even
remember markdown better than i do i see how you were trying to say that man no marked down rebel elder chamlis_ english needs parentheses for unambiguous parsing
rebelelder ah okay
all of our place well most all of our places where we take markdown we render that once when the record is created to a column named marked down underscore whatever yeah shameless you got there exactly where i was so we also have like a regenerate markdown in case we ever touch the markdown processor and we want to regen for all comments yeah see this validate false there's where recheck would come in handy because That's only there because the validations have an issue or two. So this is me saying I want to run on the markdown output. Oh.

01:26:57All right.

01:27:07So yes, we do have no go Geary. Oh, just the HTML entities. No, that, that doesn't do what I want. Yeah. I want to grab this with no code gear, I guess. Do we have any other places? Web mentions parsed mark downer. Yeah. Let's look at, I think the story use is pretty simple.

...42So we'll say...

01:28:06It's a helper. Do I want it to know about the structure of comment? I don't think so. I think if I just said, yeah, I think if I just took the HTML, I would be in better shape because then I could do it on story descriptions too, which is a thing I'm going to want.

...38So let's say words around link. And then this I'm going to pass in. This I'm also going to pass in. Yeah. Make that a little more general purpose. All right. And then snooker, you already have a, So I'm pulling up this stuff because I don't immediately remember the Nokogiri API off the top of my head. It has a lot of API, actually, which is great. But yes, I'm not searching. Let's grab the Nokogiri docs.

01:29:44What am I supposed to do with this? Man, people don't test their crappy cookie banners. Oh. So I was thinking it was broken just because it's narrow. No. is to consent that doesn't count as consent man like i'm not a big fan of these gdpr cookie banners i understand they were trying to annoy people into not having them at all but the number of non-compliant banners is especially irksome

01:30:35All right, coding.

...45Parsing a fragment. You know, it might actually, it is going to be a fragment.

...58rebelelder it's annoying when you click on "manage settings" to deny, then 1000+ partners are declared with "Legitimate interest" yeah sure
This is not going to have an open HTML, a close HTML.

01:31:10Oh, yeah, Elder, that's also very true. That's nonsense. We don't need errors. Honestly, if something is coming out of our markdown that has an error, that's a problem for us. Just show me the structure.

...42tumdum3 maybe traverse ?
So this has searching. I don't actually want to search unless I can just search for star traverse. I'm lazy and don't want to. No, this is all very XML. Here we go. Yield self and all children to a block recursively. But it's not enough to traverse because I need to mutate as I go. Which, you know, I say that. And as soon as I say that, I think we do that here in the markdowner.

01:32:45tumdum3 there is 'replace'
because it loops all right save it loops tags to tweak them yeah so it has this walk text nodes and it allows it looks at things and replaces them and then node.string-content. Yeah. And then how does it become? OK, so it's not re-outputting. The thing that's going to get returned at the end is nokogiri.node. html5.

01:34:05So the thing returned is going to loop the parse tree, edit that, and then have Nokogiri render it back.

...20So I want to say, this must have been early code. This could have been a one-liner instead of doing it like this. So we'll say node. Let's see what we get. So then this post process here, we're going to say.

01:35:22I'm going to bring up the, the node might be an element. So we're going to say, we're going to loop all of those. We're going to say, Oh yeah. We're into this stuff now. So we'll say if... Actually, let me just edit what I got. So I have a little off screen, but I have the... Man, not made for a smaller window. Not made for older eyes either. So here is where I've grabbed this example of href. And there is a way to ask it what a node's tag is. You linked me back to where I was.

01:36:50Add class, append class, attribute. But I want to know what the type of it is.

01:37:14Really? None of them are telling me?

...22Now I'm skimming the code samples to see if any of them. So it transforms into s. Name? No, that's.

01:38:01There's a replace. Oh, let's look at that in a second. If I can find out what the heck an element is, I'm going to have to pull up the console here. So what's this replace? Replace this node with node or tags. It can be a node, a document fragment, or a string containing markup. Returns the reparented node. That's very promising.

...41Seeing here if there's a node.name. That's so not useful.

01:39:46OK, it is name. All right, so we'll say.

01:40:12I guess as long as we're here, text, yeah. I'm honestly, so this while is going to return, is effectively a map? I'm a little suspicious that I don't understand this API. And so maybe this is going to want to be replaced. Let's play with it.

01:41:30Well, I'm not getting the API I wanted.

...55Oh, because walk text nodes is... So the thing I'm stealing out of Markdown or walk text nodes is specifically for just find all the bits that are text. So I guess I just want to call node.each.

01:42:21Or not. So I'm getting back a document fragment. It has a numerable, but then when I called each, I got nothing.

...56Yeah, that's not doing what I want. So the object that Nokogiri is returning is mutable.

01:43:36Now I'm doing what I should have done five minutes ago and just googling for does somebody have a way to quickly replace tags with no go here. And here's a whole bunch of Doc search. Yeah, but what I want to do is search for

01:44:07I guess if I search for star, right? Let's bring that back up.

...23So then I get the strong tag. And then I say each do tag.

...35Yeah, let's use the single line now. Tag, and then, so he's doing a mapping, and then he's saying tag replace with, right?

...59Yeah, I'm still getting, I'm still getting a strong accept. So fun with mutable objects. Let's go with doc equals. Setting up my doc.

01:45:53Yeah, so it's mutated doc. So I'm going to be reinitializing doc a lot here. Oh, OK. I see. So. I think maybe that tag text worked, and I just wasn't printing the debugging correctly.

01:46:37Yeah, OK. So that's what I want. I want to use search, and then I can call dot HTML in the end.

...59So I'm going to say, first, search for all tags for each one. For each tag, if.

01:47:23Yeah, so that's this line. If the tag is an A and the tag's href is our URL, if the normalized href is our URL and it's the first link, then leave it alone. Otherwise, replace with that. And then we're going to return. So then our output is going to become artist.toHTML. And here's that split. It's the default. The default for split is any white space, right? Can't remember if it's any white space or new lines. Any white space. Split. And then I want to find the section around the link again. That's a little fugly.

01:48:42Especially because this thing is not going to know the structure. So let's just, one step at a time, return the whole thing.

01:49:04Let's see if that wants to work. Let's say Swift Fragment.

...42So let's see if this first test wants to run. That's basically it. So what line was I on? 17, 7.

01:50:09Initializing all of Rails to replace one string. Let is not available from within an example. Oh yeah, because I should have said, because I moved that in. That's a really useful error message. Be like, we know what you tried to do. You probably made a bad edit. Go ahead and replace this. I love that kind of message.

...48I renamed it to words. Words or fragment? I kind of like fragment because it implies you're still getting back HTML. Yeah.

01:51:24All right, so that actually ran. Great.

...36So now the tricky bit is how do we grab words around the word we want? I kind of have to find the link again.

01:52:02Let me see the output. So one of the nice things is we're downstream of the markdowner, which is really regular about its output. Oh, it stripped the link. Okay, small bug. So before we do that, before we do that, Let's fix that bug.

01:53:22So either the tag name is coming back as a symbol or something. P strong A href. So let's take a look at that class. I bet it's just a symbol. Feels like it. I get why a parser would turn that into a symbol. But it's not. It's a string. Okay, hang on.

...58Let's see.

01:54:23I wonder if it's something about normalize. So I'm expecting this to not print that it recognized. It did recognize. Then why did the a get stripped out? Because I didn't call replace.

...51something replacing all of my this is an interesting bug for anybody who wants to be a vip in the channel it doesn't replace the link but the link is It doesn't replace the tag, but the tag is still getting deleted. What am I missing here? This if isn't firing for anything else.

01:56:04So it replaced, oh, the whole thing is wrapped in a P tag. And so it's flattening out everything at the end, I suppose. yeah so because we're mutating as we go we're still seeing all of those tags but the very first thing that happens is it finds the p and it smashes it so we're going to see that same kind of hassle with with really any tag around so like if the link is surrounded by italics let's make that another

01:57:09So I think you could say that that's line 37. No, it's line 19.

...32And I'm just double checking that the markdowner will let you put strong around a link. And it does. So there is another test case. So how do we handle if the tag, if it's a wrapper tag, we just want to include whatever its contents are, not just its text. Text is flattening out. the tags. So there must be a method on node, whether it's children, it's probably children. Alias for element children.

01:58:42Okay, that doesn't include text. So hopefully I can just replace with a node set.

01:59:01And I don't love this because the first thing that's going to do is try and replace the P with its children while I'm still iterating over the old version. So I'm not sure if this is going to work. Because then I'm iterating over something that's already been replaced. We'll see how much pass-by reference is happening inside of nocogiri. Oh, the right amount. So this threw away the p. And then what we got back was exactly what we wanted. Nice. Now let's try the next one. This might actually work, where that strong around it gets stripped off. Yeah. Yeah, that's nice. That's pretty painless. So I'm going to comment out this debugging because it's getting a little noisy now. I understand why it wasn't working. Yeah, we'll just comment it out. may need it again in a second so i was saying that and we can see it here the output of these is really regular where markdowner always says href and then i believe we have it set to always have this ugc tag this is a It's sort of an SEO spam thing where we're saying this link came from user generated content. And on the SEO side, it's a hint to Google that says, well, hey, don't wait this too highly. This is any user could have submitted this. It's also an anti-spam thing of, hey, if a user is linking off to a spammy site, don't penalize me too hard for it. At this point, I don't know how much Google cares about that, but I really don't want to tweak our markup.

02:01:06Oh, I get it. I know how to fix it. We're going to parse it again.

...17So if we loop it again, actually, it might already be. So we just went and threw away all of the tags, which means if we loop over that again, we know literally the only tag is the thing we want. And on that, yeah, so now we have a document fragment that just has a text node, our attribute node, children, and then another text node following.

...57Okay, so...

02:02:37Might have to special case that.

02:03:06So now we say. First, search for an A. Do you want to search for an A or do we want to search for the text?

...32Yeah, these texts are split up into multiple text elements. because of how that replacement happened.

...45So I think I want to just grab the... If I knew the index of it in children,

02:04:18Yeah, so here's the children. So if I could find the A and know that it's index 2,

02:05:12That index took a block. That would have been nice. Index by. There we go. But it found me. Index width. Here we go. So it's looping them. And for each one is getting that. See, what I really would like is the actual position where the block is true. I really don't want to call each with index.

02:06:31tumdum3 searching for 'a'? but there is none
Oh, yeah, it should be strong. But what I'm not getting, yeah, so that's probably, that might be part of why index by didn't do what I was wanting. Yeah, I don't want to find the element. I want to find, it's such an un-Ruby thing, like I want to find its numeric index so that I can numerically find a few things around it.

02:07:07And I know there's a, what is it like each slice where I could kind of cons through this thing and it yields to the block each call it five wide element.

...29tumdum3 find_index?
I'm still gonna have, so one of the things I'm hassling about is I know this document Like a text might be multiple words. Find underscore index. That's probably it.

...55Nope. Didn't return what I want. I got a nil. So I know I'm...

02:08:12getting the wrong thing, not getting a integer back. And then there's this hassle where the text that comes after, depending on the preexisting HTML structure, might be one text per word or multiple. All right.

...40All right, I'm gonna step away to use the restroom and then we're gonna keep hacking at this. So let me turn on that little note. I'll be back in a second if anybody wants to spot this.

02:10:56tumdum3 find_index should work -maybe instead of puts do a return in the block
Alrighty. Maybe instead of puts, do a return. Okay.

02:11:10Yeah, can't return out of a block like that. What if I just say, oh, because puts always returns nil. So yeah, you got it, TumDum. Thank you. So find index is what I want. However, these text nodes might be one or more words. So what I'd like is either to merge all of those text nodes, which might be a feature.

02:12:00I'm googling real quick to see if it's a feature. Doesn't look like it.

...15Unless I ran it out and through? Yeah. easeout would merging text nodes equal stripping tags?
So let's just do that again. Would merging text nodes equal stripping tags? No, it's this code that strips all the tags. But you can see in the output here for if I have this string before strong hello afterwards, a text node might be one word, it might be multiple words. And one of the things we saw in the text output was If afterwards had had an HTML tag removed in the middle, it would be split up into multiple text nodes. So probably the most robust way would be run it back together. And now I should have... Yeah, it really has to be... one text node the link one text node or the link and then a text node on the other side i mean it could be just the but this this would slam all of them together so

02:14:10So if I search this for something that's not here, like, I don't know, a meta tag, I get an empty array. And if I search for something that is here, yeah. So if I search for if search.search a, This can be, this can be up here.

02:15:33then

02:16:20So what i'm thinking of is we're going to have three scenarios. The common case is that we're going to have. Three nodes where it's going to go text a text but it's also possible, we have a text or text a.

...44So what I want to say is. If there's anything to the left, I feel like a C programmer, just knowing that the structure of this array. And it's parse.children.

02:17:22All right, so if the index is not zero, then we could say our children first place with first children first. So that. Text. Yeah. Let's grab that. That's so long now. I'm going to mutate it.

02:18:42And then the same thing on the right side, right? If the index does not equal builder and count minus one,

02:19:35It's a lot of work.

...49So here's a good place for a poll. Who thinks this code's actually going to do what I want? There's a lot of dancing in here. in and out of HTML and in and out of structure, are these tests going to run? I feel like I've got pretty good odds.

02:20:19So let's try that first one.

...30Oh, a green dot. Look at that. But it stripped out too much text. It was supposed to include... Yeah, so we have to... We do still expect to see the word love. We did not. So too much got pulled out.

02:21:00Mm hmm.

...52when in doubt, puts it out.

...59So it does, oh, that join is wrong, but it actually is fine.

02:22:21So I'm going to simplify this and just do stuff to the left for the moment.

...30I wonder if maybe the node thing sees that it got a string. Why did this pass? Wait, it failed before I had a space and then it passed after? No, it must be that this section I commented out was breaking things.

02:23:50Instead of first, it should have been last. And then this needs the same space and the same space extra before it. That actually might be right.

02:24:21Look at that, an excerpt. What a lovely excerpt. What line is this, 17? Let's see if the other spec wants to run.

...39-huh.

...54tumdum3 I find it surprising that you are not checking the whole excerpt in test
So this one also should run. Doesn't have an expectation in it. I just want to see the output and kind of eyeball it, and then I'll figure out what expectation. Yeah, I'm going to do that here, I think. Check the whole excerpt in test. Wrong number of arguments. Who's mad? Because I passed the comment and not the... No, that is actually correct. What line are we mad at? 29? That is not the right line number. Who's mad?

02:25:39Help respect line 29. I didn't say... So I was calling some random function named excerpt.

02:26:04Why do I have a random function named excerpt? Where is that coming from? Good question.

...16Hmm.

...27Whoa, why did this return an array? So I output the...

...39But it's not an array. tumdum3 it's the faker
Is it just it generated something with a bracket in it? It's test data, so we'll see something different the next time, right? It's the faker. tumdum3 yup
Oh, the faker is returning an array of strings, isn't it?

02:27:10Yeah, so a faker is returning... Yeah, alright, I'm done. We got there at the same time. Yeah, that... That counts as finding a bug. Thank you. You got it before I did. Thank you. So.

...57I guess the real reason I'm not checking the whole excerpt is that rel equals UGC. I don't care about it. And I don't want to explain that it's there. And I don't want the test to depend on that thing that I don't care about. But then I have to know about it. So even if I regular expressioned it out, it's just.

02:28:39So what if I just say end with? Pretty sure that's part of the API. Going the wrong direction. Let's run all these specs.

02:29:10Why is it not less than 20? If I take eight on either side and I know there's like two words on the left. One, two, three.

...27That's only 17. Maybe it's splitting on... Oh, if I don't pass single space, it's not.

...53Did I write the...

02:30:03tumdum3 I don't really know ruby but it looks more like like a syntax error
This is a little goofy. I'm not spotting this one. tumdum3 in the tests dsl
I can count to 20. 17 is less than 20. It's... Yeah, the test DSL is...

...29I look for... This has come up before on the stream.

...46Yeah, oh, it's to be, because it has to become a match or object. The RSpec really does make its own DSL. It's times like that it's not super useful. I want to say when I took over maintaining the code base, I think we had a couple of mini test tests, but mostly used RSpec. And I ordered those over to RSpec so I could remove them. I think in the last couple of years, I've slowly gotten a little more skeptical of RSpec. Because this kind of thing, do I really benefit from not saying expect excerpt dot for not just phrasing it as a Boolean? The error messages are a little better, but then I don't know. Am I losing more time to the error messages in the DSL or knowing the DSL? I don't know.

02:32:22I guess I'm still operating on HTML, so.

...42btd_12 whats rails
Rails is a web development framework for making websites. You can visit RubyOnRails.org for more. Lots more. So if I said I want this to look pretty good, so I'm going to say I'm knowingly passing bad HTML into this guy. and I should still... Yeah, I think I want to grab these, that it should still include the link, should not include the strong. That's good enough, right?

02:33:50I don't care if Nokogiri recognizes the unpaired tag and discards it, or if my code throws it away. Like, I literally, I don't know what level of abstraction it's happening at, and I'm fine with that. I just wanted to be a little paranoid about that, because, come on, I'm parsing HTML. And even if I'm only parsing the output of CommonMarker, I just get so paranoid around that. Let's see. So we've got the HTML.

02:34:52Grab this. tumdum3 maybe it would be good to check (un)ordered lists - I think it also generates nested tags
Put a couple of sentences worth of stuff in.

02:35:06Maybe it would be good to check on order lists. You think it generates nested tags. Oh, that's reasonable. Yeah. So, I mean, we have one here, but with With nested tags, it might be nice to know that we can go a couple of levels. Yeah.

...48Let me see this spec pass first, and then we'll check on that.

...56Did I not call it HTML? All right.

02:36:13It didn't join it back together.

...27Why did I get back an array? Oh, it's the, if it's not present, that's what it is. The one code path where we break out early, we broke our type signature. Oh, I got three short sentences. All right, let's... Got to be less lucky than that or more lucky. So let's say 30 sentences. I think the minimum for Faker's lorem ipsum is three words for a sentence. There we go. And then I guess as long as I've had to edit that one, let's do it here too for a realistic amount of data.

02:37:39So I'm gonna put this one after the wrapping one.

02:38:41How does that look to you, Tom Tom? tumdum3 yes
Is that the test you were thinking of? Let's give that a run. 390. tumdum3 but there are ** at the end that are not needed
The other question is, do you think it'll pass? I think it would if I didn't typo anybody. So we... Oh, I didn't include a... Yeah, true, there's extra at the end, but I didn't include a backslash n. I think I might actually need two. And then at that point, it'll become an unordered list with one element. Yes. tumdum3 nice :)
Very nice. You know, as long as we're nesting, though, let's put those back. Let's, you know, it'll be, or another guy, we'll put the underlines around it. So then we're saying, even though we have a whole tree of stuff, we include that, we include m. Yeah. tumdum3 not include em also
Yeah, I think that's a good call to get at the case of being nested to say not just we're doing the first level, we're doing multiple, especially given that the way we have to express this is a single, well, we don't have to, we could end up walking the tree, but I really don't want to walk the tree. And we sort of implicitly do it because we know, I mean, we saw it in the output but comments get wrapped in P tags, sort of, like the text does. And we've seen it in the previous examples where the overall thing was wrapped in P tags, but it's nice to be really explicit about what we're testing. Yeah, good call. Yeah, we'll just say there's no M. I would almost say there isn't a P tag, except I don't want to take the P out of example. It's a little too clever and makes the test a little brittle. All right. So now we have a method where, all right, let's toss all this debugging.

02:41:23It excerpts a fragment around a link. Lovely.

...41I move this closer because it's sort of part of the loop. In a way, this other stuff is just generic setup. I mean, not generic, but you follow. That is a lot more lines of code than I thought it was going to be.

02:42:01But that's OK. It's pretty robust looking. Let's look at the diff.

...17OK, so it's just the plumbing to get the linking comments in. And that's where I was in the form error of displaying it.

...40This feels bad, though. I'm going to say. I'm going to have to call raw or unsafe, and I don't love doing that.

02:43:03Yeah, I'm just going to have to do it. Going to have to bite the bullet. So this becomes raw. Excerpt fragment around link. Why did it autocomplete? I left it in the test.

...48All right. And then I think I'm okay. Not printing metadata, like the story about like the story title, the author, how long ago it was. All of these are going to be in the last week, which is the merge window.

02:44:19yeah yeah i think that's okay we'll start simple so let's this one is the hard part you know all the code was all pretty straightforward and mechanical but the The hard part is explaining the human side of what's the cultural norm at work here. And the cultural norm is we don't want to split up discussions.

02:45:01We don't want to make merge work for mods. That's me being selfish, lazy. We don't want to use links as super comments. Nobody knows what I mean when I say super comments, but I don't have a better term for it. But a thing that happens is someone submits a link of here's example.com. And then in the comments, there might be an argument, there might not. But someone says, oh, well, I want to link to rebuttal.net because I think it proves that you're wrong. But I don't just want to link to it. I want a whole new story and a whole new thread where it's all about how you're wrong or how I'm right. So I will submit rebuttal.net as a new link. You know what? Knock yourself out on Reddit and HN, but that's not how it works on Lobsters. We want to keep the discussion together. So... with the caveat that I can't use super comments because it only is a term in my head. I've got to think of a way. And then generally, I like to phrase these things as a positive rather than a negative. So instead of saying don't, don't, don't, what's the value? Well, the value at work is rather than resubmit, easeout ha that does happen a lot. "A is true" and "A is false btw" as distinct stories on the front page
Please add to the existing discussion.

02:47:06yeah so ease out yeah it happens I catch well I would like to say I catch most of them, but if you think it happens a lot on our homepage i'm i'm missing some of them, it is constant on. Those other social sites and it's a little bit of a pet peeve. People actually I think what the pet peeve is. is I merge these or I remove them. And sometimes I get people who are like, no, that's not what I was doing. And I'm like, that's, I can literally see you in a comment in the first link. easeout I didn't know how bad it was so you must be doing a lot
And you had that link there and you said, this is why rebuttal.net is why I'm right. And now here you are submitting rebuttal.net as a top level story. Please do not BS me. That's not endearing. It's okay. don't know how bad it was so i must be doing a lot yeah if we could dig through the mod log if you want to write that example query it's still office hours it's not super common anymore although i think i've seen two in the last week yeah i don't remember what the topics were but i remember i've done two of them in the last week where someone you know there's also the It's not just the super comment thing where it happens. So a fairly mild version of it happened. Well, not mild, ambiguous version of it happened just in the last day or so where there was this link submitted, why gov UKs exit this component. I should take the caps off of that. Exit this page component doesn't use the escape key. And it's a link to a, I don't know, this beeps.website. They must have worked for GovUK because it's about, like it's written as an employee from an employee's perspective. Just the other day, I linked to a GovUK page Oh, 11 days ago, man, time flies when you're having fun. That was about some of their design standards. So the thing that jumped out was, I was like, Oh, like, I've seen this same structure. I don't know why he, she they put strike through their entire table of contents. And I wondered, If this link was submitted, as a a response or you know if browsing around gov uk they found this subpage so that's like a harmless version of it that happens sometimes where someone submits oh here's new compiler and they submit the home page and the whole thing is released and everyone is interested and then we get another link that's you know and sometimes there's a day or two in the middle where someone links to like a detail page five levels deep into new compiler's manual. It's not really a separate discussion. You got there through the first thing, but it's harmless. Like maybe you're not even splitting up the discussion and it's worth having a focused discussion on that thing. Sometimes it means that the person tried to start a discussion in the main thread about that. that feature and nobody commented, so they go, oh, well, let me just wait a little bit and then I'll submit a story that is that specific feature that I really want to talk about so it doesn't get lost in an announcement that has 50 comments on it. That one's actually not just harmless, but totally OK, because the value there is hey, let's have an interesting discussion about this meaty topic that I feel got missed. And I think, oh my God, if I had the BRB on, no, I turned it off. It has the effect of generating new thoughtful discussions. And the important bit is it's not... hey, let me split out the fight so we can have a discussion where I'm the only one who gets to have an opinion, or my opinion is privileged above others because it's the one that's in the link, or it's the one that's in the story text, or the one where it's, hey, here's an underappreciated detail. Let's appreciate it. That's pretty great. So is there a way I can say like these negatives but say them politely?

02:52:34I really don't want to try and like squeeze in a hundred word description of the super comment thing. I think it might be enough to just say What if I said, please grow the existing discussion? Adding a comment there will bump it to the top of the active discussion list.

02:53:30So there's this page up here of slash active. We don't get like bump comments. That would be so irritating. I would filter those out with a comment validation. We have a couple of those, but I added this page, especially for people who are irritated by story merging. Because part of the irritation at story merging is someone feels like the link they have to submit, even if it's a different aspect of something we already have, or it's a response to this, they want to have more attention on their thing. And if they are responding to a story that has already slipped off the homepage, it feels like the only way to get that attention is to put a top level story in rather than put in a comment. But then this very first thing that I kind of consider this an alternate homepage, because it's so important, is what stories are getting active discussions. So if you leave a comment on a story, straight to the top, like number one on this page. And I put it next to the logo so that people would go right into it. So it's an attempt to say like the best thing on this site is the discussions with other people. The magic of lobsters is the discussions. So let's promote those and encourage those and hopefully encourage the very healthy ones. So that's what's going on with story merging. That's what's going on with don't have super comments. tumdum3 minor rails question - why link to /active using raw html instead of link_to ?
It's all in service of growing healthy conversations.

02:55:36You know, no real reason. It's easier to just kind of hammer out the link. I'm kind of torn on how much value there is. easeout s/herf/href
Link to is more valuable when there's a model. It's also more valuable when you don't typo. Good catch. Well, there's a value to link to. Link to doesn't typo href.

02:56:18Is it active path?

...25It might be active story path. No, it's just active path.

...41Yeah. So when I'm just like, realistically, the active path is not going to change. and I don't really mind the chore of, well, search for slash active. That's not a bad chore, it's pretty mechanical. I will almost always, actually, I will say I always reach for the link helpers if there is a record involved, because I don't wanna spread out the code that is how do we turn a record into a URL. I would like to keep that knowledge centralized in routes, but it feels pretty harmless for these static routes that have no dynamic component.

02:57:39So bump is forum jargon. And I like it because you see the word bump and if you've ever been on a forum, you know what that is. But I'm a little like I don't want to accidentally introduce the practice to lobsters comments where people are like bump for more attention. That's so irritating. Don't do that. You have to write something substantive.

02:58:22So let's say, so this is one of the, I kind of actually, I kind of love this little section of the comment model. There is speaking of acculturation and little nudges. And some of this is just straight up JCS's opinions and Simpsons references. So if a comment just says this with a period or an exclamation point, you can't post that comment. It just says back, nope. And weirdly, it matches your capitalization. Okay. If you say TLDR as your whole comment, it just says, you know, wow, a blue car. That's a Simpsons reference where Homer is amazed at something that is unremarkable. So if someone is demanding a TLDR or posting a comment that just says TLDR, like, oh, that's so rude and dismissive. Don't do that. Or if someone says, I think I added this one where there was a meme for a little while where people were posting words all spelled out with spaces in between. So he said, like, don't do that. If you write me too or nice, like that's what an upvote is. And then this one's like actually a bug. But in the middle, in the middle, Something really opinionated happening.

03:00:21arh68 can I ask an office hours query
So we're going to say, do we want to say something a little bit snarky about don't write bump comments? Doing bumps is very 80s. Oh, hey, ARH. Yeah, please. Let me finish the thought of this message. And then I was about to commit. So that's actually a great time to come in with an office hours query.

...54So it starts with bump.

03:01:08kind of like see i don't want to just have a snarky error message i would like to nudge the the behavior of

...37That's really redundant.

...51Just say don't bump threads. So if it starts out with the word bump,

03:02:08Yeah, that's good enough. All right. So here's... If you want to start typing your question, I'm going to commit in about a second. I just want to run this suite one more time.

...28arh68 ok so there's this .. nameless imageboard where ya can't post any comment that's not unique
Oh, I didn't actually wire up the... Yeah, I did. Didn't actually see it work. arh68 I was wondering how many non-distinct comment texts have been posted, what #1 is
Like I've tinkered with it, but I haven't seen it work. I'll tinker after.

...53So you have a nameless image board where you can't post a comment that's not unique. Sure, that's even a mode on the Twitch chat where you can't post a comment that's not unique. I had it enabled for a minute, but it was super irritating when I was having all those microphone issues and we were debugging it. I should probably turn that back on at some point. We're wondering how many non-distinct comment texts have been posted. I see what you're getting at. That's kind of an interesting one. Let's run that query.

03:03:56And I'm pulling up production because I know my local database is weeks out of date. So what are we looking for? We're looking for short ID. And well, let's just get the ID first. Comment C1, join. comments C2 on C1.comment equals C2.comment and C1.id is less than C2.id. Does that look like roughly what you're thinking? We might as well grab both, right? arh68 I was thinkin like group by content, order by count desc limit
No, you want... you wanna actually have a count, right? So let's group by content, order by count. Yeah, okay. Yeah, so I got there.

03:05:29At this point, I feel like they're going to be short. They have to be, right?

...43Let's see what we get. That's going to take a hot second to run. Maybe I should have just run it locally. Oh, that's actually really, really sweet. What is that? Thanks. Thank you. arh68 `.` is weird but interesting
and thanks are five of the top 10, one, two, three, four, yeah. And then there's some yes and no. That's not bad. All right, let's look at the top 40. The dot is, so the dot is a couple of things. It has been a way of saying really don't get what you're saying or it has been a moment of silence so like in condolences or it has been a i deleted my comment let me just put a placeholder in so i want to see more lol yeah hey we got more variations on thanks More variations on congrats. This is actually, like, a surprisingly wholesome little thing. arh68 +1 is perhaps my least favorite but ya not bad
Like, I mean, even why is the only, like, or why not, like, out of these top 30, how so? Like, they could be slightly accusatory questions. Plus one. Yeah, that's... Plus one and less than three are maybe good candidates for those opinionated little... Actually, that's... Yeah. That one, plus one, actually.

03:07:43It's... easeout there's an upvote for +1
Literally the me too or nice or... Plus one. yeah there's an upvote for plus one and that literally like one of the reasons i did that is we already have that message and it's not the end of the world that you know in over 12 years we have gotten one a year this is not exactly a burning issue but it would be a slight improvement to say like come on just hit up vote you don't need to say plus one and bump things This nice, does this not? Case insensitive? It must have been added after the nice comments. So I don't think we've had. Yeah, so me too. I want to say has been, yeah, there's been exactly three me too comments. I'm sure I could make some more variations, but there have been a handful of them.

03:09:09Whereas I guess nice came along a little later in the same way that plus one came along a little later.

...29That's a fun one. Great query, ARH. Thank you. Yeah, really wholesome to see all the congrats. So I'm around three hours. arh68 PopGhost thx for entertainin my curiosity
I want to see that story working. Oh, yeah. Man, all this work with text, with tests. And I didn't log it. Oh, yeah. I am happy to entertain your curiosity. So I'm already logged into localhost. So let's leave a comment on one of these. Just says, I love this site. because it is full of interesting tech i'm trying to get more than eight words okay whatever i said something so now if i go to submit If you caught the little pause before I clicked Submit, that was me looking up to make sure I still saw a local host and not actual lobsters.

03:11:06Ooh. So we just dumped JSON instead of rendering correctly. Man, good thing I tested, right? no the so this is an error page form errors raised unknown column links.created at oh yeah because links don't have a created at who asked for that so the rails tries to render error pages and if we are having that kind of code where the js is like yeah you give me a page and i'll just slap it in

...56Who mentioned it's created at? We're created at seven days ago. What I want to say is we got to join over to from comment.

03:12:10And we've got to say where comment. So we're going to be injecting, rebasing in a fix up. All right. Let's see if that wants to do it. We got a title, but we didn't replace that. Let me reload this page real fast.

...44Unknown column comment. It's comments plural, even though It's not comments, plural. It's from underscore. So in here, I had comment singular. It needs to be from comment, the name of the association. And then Rails will transform that correctly into whatever it is alias the table to. OK, I've got a bigger error, undefined, because I didn't pass in a comment. Where are we? Form errors line 38. Let's take a look at that.

03:13:35This one wants to be link.comment.markdown.

...51Did I not? Undefined method comment. It's from underscore comment. The exact same edit I just made. Hey. Okay. I love this site because it's so full of... So, all that hassle with the parse tree and look what's missing. My link is missing. Why, why, why, why? No bueno. Let's log it. Well, let's debug it.

03:14:53I expect it's still going to say. Oh, now I don't even get my... I've seen this. This bug just got reported recently. So there's one section up here for these flash notices. And when there is some text and a validation error, in this case that I don't have a title, I don't have tags, They kind of fight over who gets to be there. And as you change focus on elements, it can trigger them to swap in. So that's not. I don't love that.

03:15:50Let's figure out what layer this is happening at.

03:16:08What is it called? Extract fragment right on link. Proper singular. Never remember how to call these. Excerpt.

...22Okay, good. And then comment last. Marked down comment.

...39I see that one worked. So that implies that the link is not getting properly passed in whatever URL the person is searching for. Because if I had said search for example dot net, then I would get what I'm seeing here. So what I would bet is somehow we're accidentally getting like empty string or nil passed down. Because I want to say normalize URL eats nils, yeah, and turns them into empty strings. This is going to be up in the controller. And we're going to come back here because this display is not useful. I think I am going to need to add story title or something or a link to the comment. We'll think about that in a moment.

03:17:35So grab the link in comments.

...42And we use story.url correctly. I already had four mirrors open. Oh, I'm passing the link object here instead of the, yeah, it's a local variable. So that ought to do it.

03:18:15And then I can put back that raw that I find slightly unsettling, but it's not re-rendering.

...31Nice. All right. So what makes this display useful? The list is useful, but we need to say What, the story title? What do you think? Story titles can be quite long.

03:19:00So this is going to get me an n plus 1 error in a second. I'll fix it. But I guess I could just link to the comment, right?

...31So we link to individual comments rarely enough that I don't remember the exact path. It's going to have the anchor redirect from short ID. Yeah. Cause the. It's kind of a limitation of the Rails router that you...

03:20:14It's how it treats anchor text, where I'm not sure it can generate anchor URLs, but it can't recognize them because the browser doesn't send them. When the browser asks for like, slash story, one, two, three, hashtag, and then the comment ID, it actually doesn't send the comment ID up to the server because it was considered a client side thing. And now all the, now, as of 18 years ago, all of the client side JS things have their own routers for firing when a page loads and recognizing the URL. easeout I think you intended, and first wrote, "top of the Active stories" as opposed to "story"
Or the anchor fragment. Why am I not loading? Oh, it's that darn cache thing I haven't fixed.

03:21:10Top of the... You're right. Good catch. Ease out.

...37Once again, it's from comment. And it's URL. Yeah. Thank you for catching my errors before they go into production. So this is the comment ID. And just throw in the short ID, like, I know we're all developers and people have looked at the, if I said like, like kind of made it explicitly an ID by putting the word comment in front, like it's repetitive, but then you get what we're saying. Is the story title more useful? It's not so much that I want to avoid joining through from comment to from comment story, although I wouldn't mind leaving that out, but it's also story titles. The median story title has like three words, but the max there is like 18. You can have quite a few words. And so it just feels not great. And do I want to say a go? What is it, time ago in words? It's in the comment view.

03:23:30Yeah.

...54This is a weird little bit of code, because if you had just passed updated at, updated at is initialized to the same value as created at when a record is inserted, rather than it'll go in with the current timestamp and updated at would be null. No, updated at would be the, yeah, there we go. That feels pretty good. Do I want to include like a dot dot dot to make it clear that this was excerpted? I mean, I know as a Gen Xer, I am unreasonably attached to ellipses. Yeah, and that combines funny with if there's a period here, let's just leave that out. I think it's clear that it's an excerpt from, it's clear from context that it's an excerpt. That's a little meta.

03:25:08This is just a fix up. Yeah.

...26So let's rebase it in.

...34This is Fugitive, the Vim integration. There we go. That's nice. All right, so let's go back to the scratch.

03:26:09There we go. That's a really nice little improvement. And then it's getting the origin code into production. So I'm right around 3 and 1 half hours. So I think I'm going to call it unless anybody else has any more office hours questions like, what's the median number of words in a historic title? I don't even know how to write that query offhand. because we don't have an array type. I could count the number of spaces in a title. Yeah. Someone is going to have to write that offline. I'm not going to struggle through that for 10 minutes.

...55All right. It's a pretty good place to end. Thanks for joining me, folks. Nice to code. And we're going to see how folks like the new origin branches. I'll see if I can deploy that in the next couple of minutes. easeout gg thanks for the stream
Yeah, I have an appointment later, but if I deploy now, I'll have an hour or two before I have to leave, which is enough time to hassle with any 500s that come up. And then on Monday, we can either add and change more on the origin if there isn't anything that's so urgent or i could do a deep dive into actually using recheck because it's it's come up on the stream and i've kind of teased it a bit but i could do a proper demo of it on monday especially if folks are interested let me know everybody's got my email it's linked off of the streaming homepage under the video there you can also grab me on blue sky that's about that all right take care folks