I'm doing a lot of sighing on this stream

02:39frici it looks like the starting screen? 🤔
If folks have questions about the site or the code base or anything else, like running queries against the database, I am happy to... Firefox. I am happy to get mad at Firefox. Run queries against production database. frici I've been in the wrong lobste.rs then XD
It just looks like the starting stream. Oh, no. I didn't click over to the scene. Thank you. frici Ahoy !
At least I remembered to open the volume slider so I don't sound like I'm... What's that? frici you sound fine yeah
So I don't sound like I'm a robot. Yeah. Welcome back, Freachy. Yeah, the wrong officers. I was still thinking about that from the last stream when that irritating self-help guy came up. Still nothing to do with them. Alright, so... Where am I? If nobody has questions about lobsters, the site, lobsters, the code base, lobsters, the community, lobsters, the lunchbox, I am happy to spend my time coding on the site. And offhand, one of the things that just came up, this is one of those rare cases where whining on the internet works out. And I griped about how I wish GitHub would allow me to enforce a policy that comments claiming an issue are deleted if a PR isn't opened in a week. And then the second gripe of any updates on this is I was following a GitHub issue for a feature request for Blue Sky. And with the giant influx that happened in the last couple of days, I think like eight people came by and were like, can we have this feature soon? I would really like this feature. And it's like, you're not adding anything. That's just a lot of noise. I had to unsubscribe. So I was just whiny and grapey. But Sumana Hari Harswara, dang it, Sumana Hari Harswara, excuse me, offered a suggestion. chamlis_ hi-ho!
And I'm skipping over the discussion, but the gist of it was that chamlis_ oubliette is such a great word
really well first off the the behavior is called cookie licking and I totally had forgotten that name for it because it's been ages oh hey Shamless you are here on my to-do list but I'll pick you up right after we talk about cookie looking the the annoying thing about cookie looking is when sometimes people post the like hey I'll take this comments and then never follow through It's fine. Everybody's a volunteer. No one is strictly obligated to follow through, although it would be very nice etiquette to at least post a comment saying, hey, I can't take this on, or it was more complicated, or I just don't want to. And so then in the future, as a maintainer, it feels really frustrating when people come look at issues, and then maybe they think somebody else is working on it. And Sumana... who has written a lot of great stuff on open source maintaining. frici absolutely love their blog
If you haven't seen her blog, which is just herlastname.net, Hari Hairswar, you can get a lot of great stuff there. pushcx https://harihareswara.net
Why didn't I just link that?

06:27And I believe Prithvi Dasumna, gone you said they thought sumana was she yeah she okay trying to be polite has written a lot of nice stuff on open source maintaining and even politely listened to me when i griped about how painful it is to try and keep a python dev environment working So maybe at some point here, I should install this Zulip bot. It looks actually really straightforward to have, especially just she literally wrote out instructions here in this blog post. And Firefox didn't jump to the right spot. But basically, she explains what cookie looking is, that it's this kind of situation that's been frustrating for me. And that you can use Zulipbot to allow people to claim issues and very explicitly be in the assigned user field, and then also come back a couple of weeks later with their tracking inactive claimed issues. And if they aren't finishing or making progress, Well, go ahead and post a note saying hey thanks we're going to assume you've wandered off please reopen or please reassign if you're still interested, and I think a really nice thing there. Is it doesn't feel like a human has judged you and thrown you out. I'm saying really good for the potential contributor. A bot saying, hey, it's been a week, is a pretty low emotion sort of thing, where me, the maintainer, coming by and saying, hey, you didn't do what you said you were going to do, that kind of feels like crap, right? So I think I'm going to have to wire that up at some point pretty soon.

08:47So Shamless, I mentioned, if you're still present, I mentioned that you were on my to do list. chamlis_ I'm here
I don't see you in the users in chat. I was really hoping they would. Oh, no, I do. OK. They were just in a different section because I didn't know if you saw Shamless but right at the end of the stream when you were saying goodbye I was coming back from some active storage stuff and I marked you as a channel VIP and previously I've always done it for people who spot bugs but I wanted to say that I really appreciated all of the corner cases and figuring stuff out that you've done on this avatar feature and the active storage setup and the Let's not foot gun ourselves. So I don't know. I don't really have any like hard and fast. This is what it takes to be a channel VIP, but you certainly have been one. So thank you.

09:58chamlis_ thanks, happy to do what I can
And with that.

10:04I was going to jump back into the pull requests. Well, no, it's me thanking you. You don't have to say thanks when I say thanks, unless you are Canadian.

...18Have a few more pull requests. frici don't think too much about the userlist. its absolutely unreliable when it comes to who shows or doesn't
We've seen this one on stream on the very last stream, and I had a couple of fiddly little changes. The user list is unreliable. That's good to know. I have noticed that up at the top of the stream, it says there is a number of viewers. And at the top of the stream, it says eight. And there are actually like 15 people if I look at the users in chat. So I don't know what the distinction there is, but I imagine it's some enormous distributed system that's syncing inconsistently. frici users in chat might just have an IRC connection but not be watching the actual stream that is playing
I don't know it's not going to keep me up at night i'm not doing this because number go up mostly doing this because it's fun and to try and. break down the any kind of barrier around lobsters development. yeah freaky I don't know it might be I don't know that. When someone talks here, I see a little icon on their name for whether they. are not watching the video but it doesn't list that on users in chat actually my one big gripe is i wish i could get this the my chat list that's in the stream manager to include pronouns because otherwise i just have to memorize them and i'm not great at it and i'm i care about being polite about that And it doesn't even show on somebody's Twitch's profile. So even if I click on someone, it doesn't tell me, and I can't make a note on their profile, like a private note about them. I don't know. It feels like there is a lot of opportunity for a very small feature to just help me remember these things. The other thing is lots of people frici yeah that would be nice to be included in actual native twitch, but there is a community extension about that.
They have one username here, one username on the site, one username on GitHub, one username on IRC, and I cannot possibly keep track of all of them. gmem_ howdy
That's real hard. Yeah, I saw the community extension about it. frici that would be it indeed
There was someone who was named something like Alejo, A-L-E-J-O, something in that neighborhood. Hey, GMam, welcome back. There was like a Chrome extension, but I got the impression it was only for plugging in for viewers and there didn't seem to be a Firefox extension. And honestly, I just ended up sad that I can't get a Twitch API key again. So we've seen this pull request once before. And I think it's in a pretty good state now, where it's clear what's happening. Everything is very nicely tidied. Yeah. So this is, for anyone who missed the beginning, this is yet another pull request from Eduardo, who has contributed a whole bunch of pull requests to lobsters in the last two weeks, which has been really great. And I think this is ready to go. Yeah, this one was, we had a off by one and he wasn't sure if we were doing something clever. We are in fact not clever. No, I just try and be really deliberate and explicit and like include comments when I think I'm being clever. All right, so if that all matches and then we have the big migration, this is all wonderful. And there's none of the, Character set stuff that keeps trying to sneak in. That's over in one of the other pull requests. And I noticed they responded, but I didn't see what it was. Why did size go away? Is this the default? Okay, unsigned can arrive. Why are all these size mediums disappearing? I'm not getting it. Did they touch it here? So there's one for foreign keys.

15:01I don't see why they...

...10I got to ask about it. I was hoping to merge this PR. So that's me being reluctant to push it back for another round of review. But it's really odd to me to remove the database restriction. I liked that it was. Which were the, so it was 95 lines between, that's not the categories table. So on hat requests, memo is on what? Invitation requests. Yeah. No, it's not invitations.

16:08Yeah.

17:20All right. Let's just add a comment. But that's really the only thing that's jumping out at me. It's really nice to add those unsigns. I remember I've done... This is one of those odd Rails places where it's very easy to go wrong. I've added or tweaked several of these column definitions to bring foreign keys into type matching. But Rails is real... relaxed about types on these things and so they just easily slip out of sync so i'm really glad that active record doctor is going to start watching those all right so i'm going to jump back to the nowhere conversation make sure i wasn't these are all sorted that's correct that's all set That's great. Alright. Alright, so I'm going to wait for a response on that to merge it. This one is honestly, it's just the if hat request didn't include this, I would say, oh, there must be a purpose to removing this explicit medium size, but I would you keep it in one place and not the others? especially a very low traffic table like hat requests. So I'm really, yeah, I don't get that one. All right. So I'm gonna, actually, I'm gonna grab that URL.

19:15And I'll plug again as I'm between PRs. If you have site questions or code-based questions or anything, I'm happy to jump in.

...30So normalize URL is 1350. So this one was a fiddly string processing that we do in a couple of places. And I also am hoping this one is ready to go because we've done one round of review on the last stream. Oh, this one. This one was. not just unrelated, it was...

20:41I figure I'd better be happy to handle the merge conflict myself, given that it's my harebrained feature. Oh, this is one we've seen a couple of times. So I mentioned character set. There's one of those fiddly things that keeps trying to sneak back into the DB schema, because whenever you run a migration, the schema is rewritten from scratch. And so if your database is very slightly different than the one Well, frankly than mine, which is hopefully copied faithfully from prod you can end up with these spurious little metadata changes around tables. And I don't know why so. They have. This really looks like the version we use let's go to prod.

21:43So we have 115.1 and a slightly different number. We have 10.11. They have 10.3. We save for Debian. They save for Debian. Well, I'm almost surprised that anything is a little bit more recent than production because production is still running Ubuntu long-term support. And so the packages tend to be a bit out of date there, but that's a weird one. So I'm wondering if this is, a default collation, because it's possible that we started the database a while ago and we had whatever that default was at the time. Presumably, from this diff, we had Unicode CI. And then they maybe changed the default. And so when Ryzen 114 created their database, perhaps they got the new default.

23:06So it's possible to change the default. I don't see. Oh, here we go. So we have a bunch of additions. I don't see anything about. So I'm searching for Unicode CI to see if there's anything about. No. General CI. Oh, man. A million spurious hits. I'm trying to find outside of the table, do they have any text that says, oh, you're going to see this because. The answer is no, they only appear in the text right let's do one more search.

24:05yeah that stack overflow chatting about it. They suggest a third char set that i've never heard of.

...34chamlis_ is adding "go" to the start of a google query a trick? I noticed it a couple of times last stream too
is adding go to the start of a query a trick no so it's not a trick it's me my default hang on i'm trying to figure out where to even start this explanation so on stream i use a different firefox profile so that i don't have like history items popping up or auto-completing or other just personal stuff I don't care to share or being logged into websites that I don't expect to be. And on my personal browser profile, I mostly use DuckDuckGo. And then when I want Google, I can prefix with Go because I have a keyboard shortcut. We actually just had a, where was it? This came up over here. And I want to say it was a keyword shortcut. Let's just say shortcut. What did I do? I know it was me.

25:59pushcx https://lobste.rs/s/npvdod/fire…
so there we go bookmark keywords there was this and so i have a bookmark keyword that is set up as go to take me to google i don't remember why i didn't make it g presumably my i already had something but it's deep in muscle memory because you know you use the shortcuts for long enough and you just stop seeing them and then I have slowly learned that Google has a little more depth than DuckDuckGo. So if I'm searching for, well, for fiddly programming topics, I typically start with Google. And so I have the muscle memory. What I should really do is just make a keyword shortcut over here so that I get the same behavior out of the streaming browser. nogweii Hello! How's it going?
Anyways, it's a really nice feature. chamlis_ makes sense
David Price- I in that comment I just linked I talked about how to search lobsters but have it for all kinds of sites. David Price- hey now great welcome back. David Price- So this general ci is actually maybe the older one.

27:28So I was hoping for someone saying, oh yeah, one of these is older. One of these is newer. nogweii it does say general should not be used any more
And the answer is there are very small differences between them. It was for performance. That's not quite what my question was. I'm really curious. Like when did a database change? Oh, someone asking it of Drupal. It's that why, why is my question.

28:09And everybody is talking about the change for when MySQL fixed the char set stuff around UTF-8. All right. I'm not going to keep worrying about it.

29:29So we had a string change. And they are. They are a student who is assigned a improvement.

...58And I had mentioned that I don't know, I was reviewing this on the last stream, that when I see split, I suspect somebody is coming to Ruby from JavaScript, and I suspect that they're a more junior developer, and so seeing this person say that they are a French student learning software development is like, ah, yeah, okay, my one little quirky flag is on there. nogweii hides his calls to .split LUL
yeah it says general shouldn't be used anymore but i think that was saying utf-8 general not utf-8 mb4 yeah there's nothing wrong with split it's and maybe it's just me being an old fart of like micro-optimizing code by seeing an object allocation. Okay, so they did take my change here. And they took it here and then they added the two tests I wanted. And they added examples here. This looks really solid actually. All right, if the.

31:35What I didn't realize was they already kept it.

...54So that's all resolved, that's resolved. okay oh found a bug in my regular expression so instead of saying sub bang hmm all right well you know honestly rather if the tests are green i'm happy with this I'm going to run the tests just to see the build go. And I think I can clear my review. Come on, GitHub. So my review was submitted. So we'll give that just a minute to come back. Of course, it will come back much faster because by root split up this build into several steps. So let's go back to the pull requests. There was one more open, also Eduardo. And I didn't look to see yet what state this is in. He may not have had a chance to touch it since I was last here.

33:34this dynamically inject that check URL do. That's also kind of a junior development flag where rather than say, let's figure out how to get to fix the problem at the earliest spot of we have some errors, but not all of the ones we want. When a developer says, oh, well, let's come in and we'll later fix up the thing in this ad hoc way, especially This kind of implies string manipulation. I only see junior developers reach for that kind of tool. Everybody else wants to go and fix it in the earliest possible way.

34:21OK. So I want to see the current state of their code. Yeah, it's just the one change that we reviewed last time on stream. See, he must be waiting for me with these questions. Which maybe I can answer. Come here. Stories controller. So this calls valid. And that's calling the...

35:11So these checks should all fire. So I was wondering if any of the checks were set to interrupt validation in some way. So I'm really puzzled why.

...49It's running sort of in stages. I don't even know how we got to this UI. I wonder if this is a local edit to the code that's causing the errors to render twice, because you should only be able to get the errors to render once, even with JavaScript. And then the similar stories partial is rendering twice.

37:00You know, as soon as I say that, I looked at the model. Did I look enough at the controller? So this calls this, and it calls form errors. I think this is just API, and we use this.

...42what's this so if there are any errors for the story we print them and if it's not a resubmit i wrote up i showed it last stream but i wrote up a big long feature request to split this up and now every time i see a related code i'm like oh wouldn't this be easier if it wasn't doing six things at once if it was all split up. Hmm. All right. I almost wonder if this if similar stories dot any should be up in the story errors. Yeah, whatever. However, Eduardo produced that screenshot, it really doesn't look like this code.

38:54So I'll just kick it back with a question because I don't quite get it. And Google is binding some key that, or GitHub is binding some key that I'm expecting. to scroll me up. All right, so there is all of the open pull requests looked at.

39:31I guess I'm feeling a little nitpicky that I wasn't able to merge any of them. no this isn't about me or my mood it's they're actually just small things that need to get looked at on these and i wonder if their authors will come back promptly even before the end of the stream so before i end the stream today i should come back to the pull request list and reload it, see if any of them have said anything. There's a possibility. That would be nice. And coming back to the tab, the PR for normalizing the URL looks like they had a small error. So hopefully they catch that when they

40:45Actually, there wasn't. This is already resolved. I was thinking there was something else to do here, but it was actually just the thing I hadn't realized was done. Okay.

41:54chamlis_ it seems upstream MariaDB defaults to latin1_swedish_ci, but Debian/Ubuntu patch it to default to utf8mb4_general_ci and Arch patches it to default to utf8mb4_unicode_ci
Hopefully that's useful to them. Keep jumping over here. Upstream MariaDB defaults to Latin Swedish CI. That I recognize as a very old default. chamlis_ so I imagine the existing schema was generated on Arch, and the PR on Debian/Ubuntu
Debian Ubuntu patch it to default to general and Arch patches it to Unicode. Oh, so it's me. It's me who has the odd database set up. nogweii nice research
Yeah, the existing schema. So I run Arch on my development machine, and then we run Debian, or I'm sorry, Ubuntu in prod. Yeah, Shamless, you've, well, that is why Shamless is a VIP. chamlis_ cheers
Oh, so the whole issue, hold on, I'm going to grab your, your whole comment from off screen here. Which one was it? Was it this one? Yeah. chamlis_ I have sources but that message was long enough
Shamless, do you have a GitHub username you'd like me to add here to link you? chamlis_ it's just chamlis on GH
Yeah, you don't need to cite sources, I believe you here. If you'd like me to include your GitHub username, let me know. It's just shameless.

44:06nogweii @chamlis_ I'm curious, can you send sources?\
All right, well, let's set it, fix it on master. No time like the present, right?

...15So here, yeah, and so there are general CI, and then where do I have UTF-8 MV4 unit code? And it's active storage, which is a migration I just ran last week. origins, which is two weeks ago. Yeah, so the only tables that are wrong are the ones I have run migrations on recently. chamlis_ I need to verify a phone number to whisper to you; I don't suppose you're on Libera?
So there's more confirmation that we understand what's happening. Okay.

45:02nogweii I'm not yet but let me get on there
nogweii I've been meaning to
So I'm going to go ahead and change this to say nogweii so no time like the present to fix that :D
General. General. chamlis_ haha
And then.

...39jangomandalorian Hey everyone! 👋🏼
So this is fine. It's showing me what I have on my local. I wanted to see it in prod just to make sure I'm not going to edit this to something that doesn't match prod and doesn't match my local.

...57Yeah, so this did get general in production. And the other one was active storage attachments.

46:14Let me active underscore storage.

...23Also says general. Active storage. Blobs and variant records. Oh, and that's exactly where we're going to pick up with the Avatar code in a second. Hey, Django, welcome back. OK. So everything in its right place.

...57Shamless, what's your username on lobsters? Because it's not Shamless, if you don't mind sharing. I'll thank you in the commit message. If not, no worries.

47:14chamlis_ I don't have a lobsters account, I've just lurked for years
Actually, I want to write this commit message so I can link to that comment so I can explain, because otherwise it's not clear. chamlis_ stumbled across this stream clicking around on mastodon
So if I grab this link. You don't have a Lobster's account? I am going to really have to invite you if you're fixing so much stuff. Oh, well, thanks for dropping in. so let's add this commit huh i wonder like this code is fine but i'm a little puzzled how i haven't introduced this like 30 times over and over. jangomandalorian I thought I was the only one here with no lobster account 🤣
I wonder if it's only touching, oh, it's only touching the collation field on new tables. And we only rarely introduce new tables. nogweii do you want an account? LUL
So I've been like editing column types and it doesn't change the collation. But if I've added a table from scratch, Arch is gonna give me one thing and production will give me another. Yeah, okay. And Nagwe, thank you for handling that. I would invite you Shamless, but I don't know your email. I don't know what the whisper rule is. Actually, I don't know that I've ever checked Whispers on Twitch. jangomandalorian @Nogweii sure! why not?
I hope it's not an inbox that's piling up full of people I have accidentally ignored. nogweii chat bubble lookin' thing at the top of the Twitch screen
Doesn't look like it. All right. All right. So that's real nice. So speaking of active storage blob and active storage variant records, the last stream ended with me tired and struggling to iterate over the variants because I couldn't find a reflection or introspection API. So let's hop over to that branch.

50:11So on the user model, we have this attachment field called avatar. Yeah, Nagwe, I found it. I have one from what I presume is a spammer in February when I don't even know that I was on Twitch. And one from four years ago from a college buddy. So either I have Whisper locked down or just people aren't whispering at me. So the feature here, this thing happening with the replace username is when I call an image tag to link to one of these variants, it's leaking the original file name that the user uploaded with. And what I would really like to do is in the settings controller, be able to override the file name. And I couldn't see a way, like I can loop the variant records, but then I don't actually know the name of the variant. And this was one of those things where I get very slightly different behavior out of the Rails console as a view, which is maddening.

51:51Yeah, so I said, give me the first one of these. jangomandalorian @Nogweii sent you a whisper
I have a variant record. See, the darn page is stripping this off. But I can't actually introspect on it and say, like, what's your name? What's your variant name? Like, who do you think you are? How do I get back at this symbol? And after thinking about it, I'm just going to slop it.

52:33let's just make a list of symbols. Like i'm going to throw it to do on this, but i'm not going to keep beating my head on trying to introspect on it. Especially when I can define it right here. I don't know why i'm not using tab complete all i'm going to do is add typos that will make this matter harder.

53:28I don't have to include a reference because there's only going to be one mention of this constant.

...45And if I have the object, if I have the avatar, I believe I can say variance

...59no what is it is it like get variant no it was variant singular there's something cutesy about it okay it's not a hash it's a function there we go so i will say you don't have a yeah Edit user avatar variant. Name. Now I'm getting to the line of code I want.

54:53So that feels like it should do the thing I want, which is let me smash these file names.

55:10Okay. So I'm going to test this by re-uploading my avatar. which I will start the Rails server tab for. All right, getting there.

...38So let's delete the one that's there, reload.

...49Browse, and I will grab my lovely cat photo, which also allowed us to test that we were correctly stripping exif, which I believe is one more of those things that Shamless caught. I don't call save on the variant with record. All right. Do I call variant.lob.save? That's plausible. I got logged out for that? Really? That's a weird one. Why would triggering an exception in the settings controller log me out?

56:40I don't get that one. That feels like six months from now, I'm going to see some weird fucking bug and be like, oh, I was getting a clue and I didn't realize I was getting a clue. Hmm. Very suspicious.

57:04More of this.

...26You know, if I it's odd, because it does this thing with well, whatever that class was in the previous exception, where it was a some odd object instead of an active record collection i have to go through it logged out again huh you get an exception from settings controller you're logged out that's i wonder if it's oh i wonder if it's just something related to this form because this form allows you to change your password and i believe it rolls your token when it does yeah so it sets your session token if there's a password i'm not sending one yeah very suspicious All right. Let's delete those cats. Reload. Put them back. In case you're interested in seeing a larger image of these two cats, I did post it to Blue Sky yesterday. My mouse was getting squeaky because the wheel was full of cat hair, and I Just did a little tear down and clean up. Not like a hard electronics repair, but a really satisfying one. Because for the longest time, I owned just really, really crappy electronics tools that were just whatever I had put together. Hey, look at that. Put together over the years. And so I got a fancy toolkit from iFixit. And it's just... Every time I use it, I'm so satisfied. I'm just like, ah, it fits in my hand nice. Somebody thought about what all the tasks are for spudgers. And then my code didn't run and it's still named Let's try and do it manually.

01:00:28Wrong. And if I query that back out, I see it. If I say variant blob, I see it again. If I save. You sure as hell look like you're doing an update that I wanted. Did I just not save the settings controller properly before I did that test? No, because I had to wrap that prosopite around.

01:01:32Hmm.

...57Can't grab it fast enough, let's just say.

01:02:20So if I do that, I have nice debugging, at least for whatever this weirdness is.

...43Let's grab that. Let's put it in two cats.

...55Save. It thinks about it a second. And now that it has thought about it, if I go look at server output, I should see that nice output. It's all squished in because Vim's terminal is a little bit janky. So there's my post. But then there's nothing about all those methods. There's some selects. There's the upload, right?

01:03:45There's no update happening for my variants.

01:04:01And then we get down to updating some template stuff. So if I jump from here to the database, these are definitely going to still have the wrong name. Yep. All right, so what's happening that this code isn't running? There isn't an exception to break the control flow. I don't see that in the log.

...43nogweii is avatar not truthy?
Constant should be getting code reloaded just fine. Is avatar not truthy? Oh yeah, it was going to be, this wanted to call for like dot attached, something like that. What was it called? It was in the text. Attached question mark, yeah. So there's that, but we know this part is running because I could see that file name replaced. Oh, wait, this is just out of date. Hang on. All right, so I've got my settings page. And if I reload it, it knows I don't have an avatar. Come here. And if I reload my public profile, well, I get an error, that's fine. Because I only started replacing that code, I didn't actually finish. So now if I go look in the database, there should be nothing. And there is nothing, correct. So just to keep the Vim terminal the full width, I'm loading the settings page. I have the settings page loaded off stream. I am attaching two cats and hitting save. Oh, here we go. I saw the update go by. I saw the file name. And then it comes back and does something after. I wonder if this is Rails is stomping on my edit somehow. Because I definitely saw what I wanted. So if I drop down and search. Yeah, so here, look at this. There's the insert I wanted.

01:07:25Oh yeah, here's my, even better, here's my debugging. So it's saying, yeah, I know it's inline 3x, I know it's profile 1x. Let's go look in the database. Watch, it's not going to have my file names, even though I explicitly said it. They all say 3x. Now I'm going to turn on the webcam and pull out my hair in clumps. Why would they all change to 3x? Just because that's the last one? See, these are all... Why are you saying you're renaming from 2x to 3x? And then ID is 97, ID is 97, ID is 97. That variant method is giving me the same object over and over. Ain't that something?

01:08:43But then they have different IDs. So where are they getting their names from? I wonder if active storage is doing something clever. Well, see, I was wondering, like, if I set the base avatar file name, which we do in the controller, is it updating the variants? But it's not doing that because I'm not seeing push CX JPEG. I'm seeing specifically the last thing I set. and I'm iterating the same variant each time. Did I not call the write method? I did. So this is so strange. So if I ask it for, Any variant, I get the same one. Something really weird is happening here.

01:10:00so here's 1x gives me id 97 and 2x also gives me id 97 and 3x also gives me id 97. so this dot variant method is not doing what it looks like it should be doing

...33nogweii is there an error with `variant(:random_junk)`?
Is there an error with variant random junk? Oh, that's a good question. Yes. Yes, there is. It complains that it cannot find that variant name. nogweii so it does know about the registered variants...
Well, I mean, it's not actually, this feels like a really fundamental thing for this API to be getting wrong for me. So it yeah it does know about the registered variants I can't introspect on them or reflect on them, depending on which terminology, you want to use and boy is that a hair I don't want to split right now. But.

01:11:30know what the weirdest thing is hey i get it hang on all right hold on so this so i said give me i said give me the variant and i got back a variant with record the blob is the original the variation is the one I want to be tweaking. See this? This guy has the correct size. I'm not getting the same. All right. So all right. Come here. So if I say inline3x.variation, that's actually the object I wanted. But it doesn't have a file name.

01:12:30So how do we get to the blob of the variation? Because when I go in the database, there it is. So somehow, Active Storage just does something really clever with When it says blob, it means the top level object, but then also these variants have blogs. I wonder if this is some kind of is a verse has a inheritance where it's in the same table, so it must be is a.

01:13:40So hopefully it's setting that pager will let me LS one of these things without looking at ASCII. Yeah, well, this one's small. So the variation knows these things. And it knows it has to go look in the... Well, no, this is finding the parent blob. It's not finding the individual record even.

01:14:30So the variant with record also has a file name. nogweii there is a `model.file.with_all_variant_records` method
Model file with all variant records. Yeah, I think that's it's even in my notes. With all variant records, I think that's a. nogweii ah
preloader to avoid one plus n errors well not errors but wasted queries yeah so i can access a file name but i can't set it it has this concept of a active storage file name but it won't let me replace it. This is the strangest API and it is not documented.

01:15:45So, figure let's go look at the, Version 3. How about version 7.2.1? Let's go look at the Active Storage API. We've dug around in this before, but it's not actually explaining how to get at these things, and they're really thin. So if we have a blob ahead of the file,

01:16:32turns an active storage file name instance of the file name that can be queried. Right, but this is all what the user gives me. What do you know about your variants?

...54Nothing in the blob. So either it's a mix in

01:17:03Yeah, definitely seen this doc before. Now see, this is just BS.

...38If I ask for the file name on the variant, I should get the blob, but with the extension of this. And that's not actually what we saw in production. So on my profile page, it actually just leaked the file name that was uploaded, right? No, this is the thing that it got smashed to. Alright. One more time.

01:18:21So if I don't do any of this, attempting to replace the file name...

...37And off stream, I'm grabbing the photo on the settings page and attaching two cats. You know, this setup is almost bad enough to make me want to actually add a proper test for it. But this feels like I would just be testing the active storage interface, which is probably not buggy. But so you say, So it is using the file name from that I overwrote onto the parent object.

01:19:31So the reason I'm sighing is there's this giant digest here, this hash. And I want to have these files visible to Nginx to serve directly without a Rails app server in the middle, because I don't need any kind of ACL on it. guess as long as the rails server knows the digest that's fine yeah so there's my bunch of file names

01:20:28And here it's just keeping those file names in the table and not actually using them. That's so weird and feels dangerous.

01:21:09As long as the file name keeps coming back without saying two cats, I guess that's fine.

...22This feels like the setup for a bug where I'm going to do this and I'm going to trust this. And then two years from now, Active Storage is going to say oh well, we have this file name field in the database, why don't we make that visible and it's going to sneak into the code and we're going to leak all these file names. The best way to not leak them is just to overwrite them in the database, but I can't actually do that.

01:22:05nogweii how are you attaching the image to the object? supposedly you can do `.attach(blah, filename: "foo.bar")`
How am I attaching the image to the object? You can say dot attach, but it's just handled by this attributes.

...22nogweii ah. hrm
And to come in off of a form, I think I have to use this. So the,

...42have a file field here and if i didn't do it like this where it goes into the objects under or it doesn't go into the params under the avatar object you know rails does its whole jiggery-pokery to put fields onto records and allow you to imagine that what you're getting back from the client is a nested hash. And so instead, like I could pull it off on its own. It's so flaky. yeah so i'm considering replacing with a file field tag but roughly every time i've done one of those replacements it ends up biting me in a year or two where now it's not in the hash and you don't have strong params running on it properly and yeah see i'm a a bad enough person that if i could just get at the primary key for the variant, I would just go hit the table myself. But I can't.

01:24:39nogweii I've seen a couple of references to `model.file.blob.update(filename: "foo.ext")`
Digest format. What's your key?

...46Yeah, your key is that whole giant freaking digest. You've seen a couple of references to model file blob update. You know, that's not how I set the file name. So that's pretty interesting. So let's say admin avatar. Let's get a consistent object here.

01:25:22okay so i said update and i think active record blob is an active record or i'm sorry i think blob here is an active record object so this is just the standard active record api and we can see that it just wanted to update one record in the blobs table so if i went and said

01:26:00nogweii hrm
yeah so doing that dropped in test one on one record but it didn't do it for any of the variants it doesn't propagate this this way it equivocates between whether a variant is a blob or is not a blob is pretty frustrating

...31Because what I would really like to do is have the blob for all of the variants. So I know the parent, there isn't like a parent ID in that table, was there? They have an ID, they have a key, they have a byte size, they have a created at. You know, so I'm willing to slop it, but not quite to the point of saying, hey, you know, if they have the same created at, nogweii I was seeing it in a before_save hook, so perhaps that timing means variants will copy the new name at creation time?
we'll just go and overwrite their file name. Like I'm not quite that willing to be awful. You're seeing it in a before save hooks and perhaps that timing means variants will copy the new name at creation time.

01:27:22Hmm. Okay. So instead of doing it after the record has persisted, let's do it up here. How about that? So here's where I set from the user, and if I just immediately say, hey, if they've got an avatar, smash the file name, and then I'll call the generic save method, and I will just comment out this that was trying to come in and do it after.

01:28:03yeah that feels like i could test it in the console but i don't want to i want to see this round trip from a browser come on settings page load on up oh it's doing that damn cache bust

...36I should delete some data just so it goes faster. Actually, speaking of deleting, or speaking of data, I would like to try to finish up the avatar stuff. This file name thing just burning an hour is not really pleasant. But if I can finish up this avatar stuff, I would really like to make the next stream a demo of recheck. and just dig into that because I know right off the top of my head that there are plenty of little code issues it'll raise. All right, so I've saved that. Let's drop back here. chamlis_ that'd be cool to see
Hey, Nagwe, look at that. nogweii ah hah, neat
You got it. chamlis_ nice
If I override the avatar's file name before active storage persists, it slaps that file name onto the variant records. And I would really like them all to have individual names, but that is a level of fiddly that I just don't want to deal with. Because it's doing that whatever. Yeah, that's going to be a... A comment and a move on with my life. All right. Good enough. The programmer's credo. Actually, the one for this file name, avoiding the file name linking, is we do these things not because they're hard, or not because they're easy, but because we thought they would be easy. Something like that.

01:31:00nogweii what's the plan for prod file storage? local disk?
Yeah, for production storage, I'm just going to use local disk. I'm trying to keep down the number of dependencies. bruxisma As a wise man once said "we do these things along with this, that, and the other thing and then our code is on the moon."
We only have one web server, and it is comfortably over-provisioned. So I'm going to do that and punt for a while. Oh, Bruxisma has an even better slogan. We do these things along with this, that, and the other thing, and then our code is on the moon. Yeah, that's legit. so div storage duplicates this file name down to the variants it'd be nice to name them username underscore variant dot pxt

01:32:09Yeah, I guess that's the nicest I can put that. All right, bam. Okay. So this one is, you cannot DOS by uploading a huge image because Nginx will prevent that. So I don't even have to do that one. I'm just gonna delete that to do. And what happens if you upload a non-image? Let's go find out. Active Storage looks like such a general API. The reason I wanted to test this one was, it just doesn't seem like I can, I didn't write any validation that said, oh yeah, that's blah, blah. I am expecting an image. So I'm going to upload, you can't see the file picker, but I'm uploading my current version of the scratch notes. So if I save, I bet this, ew, it's saved. Oh, I didn't want that at all.

01:33:25Okay, so it's saved. nogweii wuh-oh
identified and analyzed so whatever that distinction is it did both of those things but then it didn't produce any variance because it at least knew yeah lost the extension because my scratch doesn't have an extension all right rails how do i teach you to do the thing i want you to do to limit the allowed types. I bet that's just the generic validation, right? Attachments aren't sent to the storage service until it's validation. So I'm just supposed to write a generic validation. So let's figure out what the API is in the console.

01:34:19So if I say,

...30I can grab that. And I would really like... Why am I getting back nil? It did persist this.

01:35:05I know it persisted this because I pulled it out of the database. So it's ID 118.

...20But you say it's not attached. You fibber. So it stored a record and then forgot about it? That seems... real bad let's go look at its database so there's nothing in attachments not blogs now there's nothing in blobs did i delete that when i wasn't looking yes i clicked delete on it and then forgot man i thought i slept well last night apparently not let's try that again so i'm going to reattach pmp scratch so i've got it attached i'm going to hit save all right so let's investigate

01:36:37I must have just hit delete without thinking about it, of, oh, that's bad data. But I would like to look at the bad data. Good.

01:37:01So if I go look at the blob, I should be able to... introspect to figure out that rails doesn't want to make the variance or and whether it's that or whether rails thinks the content type is an image so what am i doing with this thing

...55Well, it has a dot image method.

01:38:04How was that implemented? Because if that dot image method on the blob turns true if the content type of this blob is in the image range, like JPEG, right? But do you include... So that's going to include like SVGs and I don't want people uploading SVGs. And the reason for that is there's a whole nother class of denial service attacks there where whatever that limit was, a meg or 10 megs of SVG can include some wild stuff. nogweii laughing entity bomb is a fun kaboom
Not only to say at that point, No, I don't think somebody could craft a SVG that crashes things, but I wonder if it's just getting this content type from the file name.

01:39:24nogweii it'd crash the user's browser, not the server (most likely)
Yeah, crashing the user's browser is a thing I'm concerned about there. The NGD bomb, yeah. If I said you don't have time. You liar. So Rails does the dangerous, unsafe thing of just assuming if you call it a JPEG, sure, that's an image. Sure.

01:40:17That's so dangerous. At this point, I want to ping some security researchers and go throw them at base camp because i assume base camp has a bug bounty and maybe if they eat all the bugs they'll come and fix this api

01:41:06nogweii I gotta assume Rails knows this is risky
It even created variations for my not a JPEG.

...16What am I supposed to do with that, Rails? How am I supposed to introspect this and know that you did nonsense? So if I look at the blob.

...42Nor does content type for serving and force disposition for serving. What are these?

01:42:06Okay.

...19So it has these attributes identified. And then I end up with these. Weird-ass errors that don't appear in prod. Unable to load vips. Cue for formatted images. What are you doing? Downloaded file from key.

01:43:08File not found. What does Rails think it means to identify a file if it's not They call it type inference.

...35Yeah, so it's interesting to see it called inference because in a security context, if you just ask user provided data, like what's your file type? That doesn't count as inferring when you're like, hey client, what kind of file is this? It's called blindly trusting. So the client comes back and says JPEG when it was a text file. And it's like, oh yeah, I inferred, I identified that that's a JPEG. No, you didn't. You just believed it. You wouldn't say that you

01:44:22determined a file's content type if you just copied it? Would you say you determined if a user is authorized to do something based on whether the client said they could? I would hope not.

...57Identifiable.

01:45:04You are in a twisty maze of small modules, all identical.

...21Where is identify content type? I swear. This is the most... Princesses in another castle thing I have seen on a rails for a minute. That one's just not documented.

01:46:02What on earth is Marcel?

...27We're calling the MIME type for a chunk with a name and a declared type.

...42And there were no magic bytes. And so then it went to the declared content type and then it went to the file name extension.

...56The way Rails use this though, makes a joke out of say fallback. Because if you trust the user-supplied extension before you fall back, you are not in fact doing a safe fallback.

01:47:26nogweii can you drop the extension temporarily?
Can I drop the extension temporarily? nogweii it's a hack for sure LUL
oh yeah yeah i feel like so i feel like i just keep seeing another dangerous weird thing and i'm kind of re-evaluating whether i want to trust this at all because when i see an api that does a really unsafe thing, my question is not, well, can I dance around the API enough that in this instance I can do the safe thing? What I really want is, is there a way, is there a setting or something so that it always does the right thing?

01:48:29Oh, Marcel was extracted from Basecamp. So that really sounds like to Basecamp, if you upload a random thing as .jpg, it'll start treating it as .jpg. chamlis_ if you can error and rollback when vips can't convert, does not being able to trust the content-type matter?
As long as it doesn't start with another MIME type that it immediately recognizes. Or, I'm sorry, Magic Bytes that it immediately recognizes. So, you know, if you gave it an .exe, If you can error and roll back when vips can't convert, does not being able to trust the content type matter? Yes, but I can't tell that vips doesn't convert. nogweii it's not an exception to not create variants
chamlis_ blimey
It's just... So this is the...

01:49:24so i gave it scratch and it overwrote oh this is the one i did from the browser i didn't persist the one where i called file.read and just passed in an io object so here it has a file name with no extension because the thing i uploaded didn't have an extension what was it First avatar with all varying records.

01:50:07Guess not.

...21Invariable error. What in the... Sam Hill is an invariable error. Is that the thing I want? That if vips doesn't think it's a file, chamlis_ so then even if you could trust the content-type, somebody could upload a file with a valid JPG header but undecodeable contents and there's no way of telling?
raised when it's called on a blob that isn't variable use hash variable to determine whether a blob is variable so how does variable work like how can it know these things until it passes off to libvips because clearly Like libvips is the adult in the room here. And so valid JPEG header, but undecodable contents, and there's no way of telling. Yeah. nogweii seemingly yeah @chamlis_
So what I really want is a thinner wrapper around libvips, because I trust it a lot more than all of this right now, where if libvips says it's okay, I believe you, but I sure don't believe Rails.

01:51:40And this one, the idea that I can say, yeah, I have this attached file and we want to generate six variants for it. And then Rails takes a file and goes, huh, I can't generate any variants for that. Okay, that's all fine. Like, why did you not raise an exception? It just doesn't care that I can't have that variant?

01:52:11I want to be above this.

...49This seems like a really dangerous API. nogweii carrierwave has docs that suggest it's much more sane
Maybe I have found a mildly safe way to use it, but not really. And I don't have a lot of confidence that it's going to do the right thing. So that looks like it actually still attached because it went into the yes, there's an avatar attached mode.

01:53:32chamlis_ similar things discussed at https://github.com/rails/rails/…
Still persisted.

...43Similar things discussed at this bug. Let's go look at this bug. Is the bug full of people pulling out their hair that they can't use it safely? Hey, that's sort of my bug. And it's closed. That's...

01:54:35They think it's a bug over on some third-party plugin. Looks for the magic.

...49Yep. And then here's somebody pointing out, like, they don't quite say it explicitly, but they say that this path is wrong, but they don't say the screamy bit that I have said of you just can't trust user data like that. And then it was marked as stale. And then it was reopened? Well, not reopened, but hanging around.

01:55:24The vips foreign load. Also a problem for invalid images dragged from Microsoft Teams.

...41Oh, this one is also closed on this third party guy. Solved as mentioned in the thread to add a processable image validator. nogweii MORE DEPENDENCIES
Well, that's exactly what I want, but... I have to use some guy's third party validator. nogweii cause rails just didn't think of this or what?
to avoid... Like, I realize at this point that browsers are incredibly hardened against, if not hardened, battle-scarred by bad data in images or things that claim they are images. But... chamlis_ make sure it doesn't run it through imagemagick to validate it or we've gone full-circle on that
This feels... scary.

01:56:43to just take what the user gives and believe their file type extension.

01:57:14Content type spoofing protection. Here's the thing Peter is worried about. By default, we don't do that. Why? Why? I get that it's a breaking change, but yes, break. Yes, absolutely break right now. chamlis_ it runs file to validate the type, huh
You are doing something unsafe. And I feel like I've seen an exception out of the Linux file command before. chamlis_ I think file has had CVEs, yeah
Hasn't there been a CV8 about that? Like, I can't Google the word file. I'll never get anything, right? Who owns that file command? Is there a package? Yeah, file has had CVEs. I'm doing a lot of sighing on this stream. nogweii it has but also been battle hardened/scared
nogweii it's used in a lot of validation pipelines
nogweii `file`, yeah
oh there's a title yeah it's been battle hardness card file i'm sure you mean file i feel like i've seen a CVE out of file in the last three years, which I guess is around the time where it's like, yeah, okay, if you can make it three years without anything, you're probably in pretty good shape.

01:58:59So one of the neat things about Rails is that it's extracted, supposedly, from a real project called Basecamp, originally. and you get all of these real world things that have been used in real projects but also you get weird quirky things that Basecamp doesn't realize it's weird about like Basecamp is paid for by businesses. chamlis_ I guess you could switch to manually using the image_processing gem and handling any exceptions yourself?
Users who are paying, I don't know, $20 or $8 per seat per month to work with coworkers in small groups are especially unlikely to upload weird broken files that are security attacks because they are paying money to share files. with their friends or coworkers. You could switch to manually using the image processing gem and handle exceptions myself.

02:00:27I see where you're going there, because like I said, Libvips is the piece of software in this tower that I trust most. Because I sure don't trust Marcel, because clearly it does the wrong thing for this use case.

...59This is just a whole pile of tools that I can use badly, and I want things that default to spoofing protection and default... I mean, even seeing this validator, it's like, yeah, you can opt in to not doing a ridiculously dangerous thing. That's just not where... The last couple of decades of designing secure software is gone stuff has to be secure by default, because if you can misuse it if you can leave off the I installed some third party gem and remember to key named scooping spoofing underscore protection.

02:01:58nogweii adding the gem & `processable_image: true` validator seems like it would indeed cover it
All right.

02:02:40nogweii and will use Vips as image processor
is it called spoof spoofing detector spoofing protection they call it both things

02:03:43So do I trust this at all?

...52Boy, I don't. I really don't. There's been... Look, we've just run straight into five different bad things. And... seeing things like the spoofing protection is optional and the image processing to validate is not the default like i guess what i'm saying is seeing all these core security concerns as options instead of defaults makes me worried that there is a similar lack of rigor in other parts of the design that I just haven't dug into yet. Maybe I am cynical and untrusting, but I have seen stuff blow up.

02:05:04nogweii ugh and there's a bug that was reported today on the spoofing detection depending on how you perform updates before it's saved to db :(
There was a bug today. Oh no. Let's go find it.

...43Oh, no. The spoofing detection is new? That's also concerning.

...55nogweii 2 weeks old new
I don't want to use new spoofing detection. I want to use 10-year-old. We used it on Shopify spoofing detection. God how does Shopify handle this? Do I know anybody at Shopify?

02:06:16Hmm.

...28This is one of those, like, it doesn't use Sorbet, and so you get bugs, like Nils.

...46So they added it a couple of months ago. Touched 106 files? How does this whole gem have 106 files?

02:07:05Ah, translations. Oh, that's a big lift. Oh, and then they have a gem file lock for every version. OK. Man.

...29I don't know about this. This is really off putting. It is. So the thing I want is the default path should be safe. And I will give up functionality out of the box to have it safe by default. nogweii CarrierWave is much older and seems sane from a scan through the README
Because if you give up safe by default, you have to be perfect. And boy, if there is one lesson from all of these streams put together, it is I am not a perfect coder. I cannot always remember every optional arg. CarrierWave, I haven't heard that name for a minute.

02:08:25chamlis_ just remembered mastodon is a RoR app; I'll investigate how they do this
God, the scary thing is our pull request to bring in active storage That originally came, I think was that GMEM, started that when adding Action Mailbox. And the way Action Mailbox works is it gets hooked from the mail server and it grabs the email and it dumps it into active storage and probably kicks off a job or queues a job to deal with it later. Okay, fine. But if action mailbox is just going to drop files into active storage, then I can't just have Nginx serve the active storage store. If it does, I mean, you'd have to guess that whatever, 30 digit digest hash. But then people can just read files out. I guess that's enough. I guess that's enough entropy. Oh, Shamless, that's really nice of you. Thank you for looking at how Mastodon does it. I mean, first thing, we could just look at the gem file.

02:10:00I'm almost tempted to say, because I'm so... There's no mention of security and everything seems to do the wrong thing by default. I'm tempted to say that if the avatar service we use fails, I would just turn off avatars and not have them. Because, you know, if I can't do it in a safe way, that's just not going to do it. What's in your gem file?

...48Well, I don't see carrier wave or active storage directly in their gem file.

...57So either it's hiding behind some other gem I guess active storage is part of Rails.

02:11:54Yeah.

02:12:14I am. Pretty concerned by this, and I think this might be the end of the avatars branch. Because... And I'm trying to figure out, like... Do I want to try and contribute back to Rails? Do I think Rails would be receptive to a, hey, you're doing it wrong? Like, oh man, I load those kind of comments, so I don't want to write something that's just... Hey, here's a bunch of issues. at the same time you know if i was less experienced i would say like oh i'm just missing something and i'm not understanding how to use this correctly and at this point i'm like no i don't know that it can be used correctly I feel like Principal Skinner, you know? No, no, the Rails devs are wrong.

02:13:58yeah all right well speaking of gumption traps although it's been a couple of streams since that came up this kind of thing where we just fall down a weirdly complicated api and an obvious thing turns out to be really hard is the kind of thing that just saps willpower So right now I'm just kind of, you know, as opposed to, oh boy, let's figure it out. Let's jump in, let's solve all of this. And it's really tempting to try to continue because the remaining items are pretty small. Like, okay, load the avatars, write a script to fetch the production avatars. Placeholders, that might not be small. Trigger rails generating various files. Oh, that's already done. Nginx config for viewing avatars. Why would I want to rename the user avatar class? I don't remember when I added that, and I don't know what that means. Man, this one. Is there a code way to prohibit using the original image? It's such a... bellwether i because i had this one fairly early in the avatars branch and i thought oh that's weird that i can't say don't use the original image anywhere

02:15:49And instead that seems to be an early indicator of some fairly serious design problems. That I can't get any kind of security guarantees out of this thing. Not that I trust. I don't mean to at all talk down active storage validations because it looks like it's really trying to fill gaps in functionality in active storage. But at the same time, it's not designed with an API that makes me think, yes, I want to use this in security sensitive situations. I think this is the end of this branch. Oh, that's painful. I spent so much time on this.

02:17:09And I guess the other part of this is there's this processable image. I want to see what that code looks like. You know, if that code in active storage validations is short and sweet, I would almost extract it.

...50Because if there's like five or 10 lines of code that I can add right at the place where we define the avatar and say, you know, validate only an image. Great. Then I have some kind of trustable thing. This is for testing. Where is this actually implemented?

02:18:40This is going to be another one of those everything happens somewhere else, isn't it?

02:19:06Oh, it's more tests. God damn it.

...21All right, must be one of these missing matches up here at the top where GitHub says, oh, there's more matches.

...36Don't want to match her. See, this really looks like it's supposed to be it. We include these things. Validate changed files from metadata. That sound like a workhorse kind of name?

02:20:04No, that just calls isValid on each, so this is just more metaprogramming. Dimension validator and aspect ratio.

...23So when we call isValid, we say initializeErrorOptions on the attachable. And then, oh, this custom add error. I bet it doesn't do anything if it gets an empty return if they're added.

02:21:10nogweii ActiveModel error magic, ugh
active mental. Yeah, a little bit. Like the the magic isn't so bad, but I'm literally not seeing where it tries to call out to live vips. So image underscore processing. So what if I go the other direction and I search for an invocation of the image processing gem? Nothing. Zero? Really zero. How do you work?

02:22:10nogweii so it triggers something in AcitveStorage and asserts on the presence of the side effect?
triggers something in active storage and asserts on the presence of the side effect. That's my best guess for what's happening here. And that was my initial instinct was, well, we can ask what the variants are. And if we don't see the variants, we'll just throw it away. But that didn't work right out of the box. And then it became clear that it's got this Marcel thing

...49So do we have any validations?

02:23:10There really is just this one file.

...19So we initialize error options, and then we pass it on.

...27Let's go the other direction. What did you think in the beginning? Oh, here we go. Vips handle 0 byte file sizes. Something in the metadata now calls read image.

...53And we have a image underscore processor. So you require that one is present.

02:24:08Here's the heart of it.

...19so if the if vips wants the suffix fine and we can initialize an object this is the kind of part of if i can get to something fairly well wrapped up that just says hey vips is this legit then I would feel pretty good about it.

...59And preferably with very little metaprogramming so that I can trust that it's actually running. All right. Let's look at,

02:26:04Oh, vips, you are so generous. You know lots of file types. I only want to support two, but okay.

...37Now isn't that nice? That's the kind of exception I want, where it says, hey guy, that's not an image. You can't do that. And I don't get back an object that says, oh yeah, I'm an image, but you have to call the are you really an image method on me to know if it's an image.

02:27:17pretty nice.

...26Alright.

02:28:46even know how to get the the file data to pass to vips now we'll go back into the active storage reflection hell right

02:29:14All right, so we got admin avatar, blob, file name. Yeah, what I want is the actual path on disk.

...43Mm-hmm.

...49But because active storage abstracts over the various storage methods, which are mostly remote services, It's not going to expose a file name API, is it?

02:30:35So there isn't going to be. Way to ask the disk. The disk storage. Or service disk service.

02:31:04So I was looking at this like, well, maybe the disk service will mix something in so that you can know it's a disk service object. No. Maybe you can get the data off it. No, nothing. Looks like it's called up data. Let's download.

...27Oh, that gave me back the actual data. OK. So.

...40What do you think that is, a buffer, a memory copy?

...49New from memory?

02:32:02What's your API?

...12New from buffer almost worked.

...34So it takes a string of options. string of options.

...50Buffer is not in a known format. That's correct. It's a bunch of text. That's good. Okay.

02:33:13Also gives me an error. All right.

...37It's just called vipsError. Yeah.

02:34:11This is progress. Because I don't have anything that limits the file type. But this is because I really only want to support JPEG and PNG. But this is short. As long as do I trust that new from buffer has this condition that it always raises on invalid data? I kind of don't now. Only enough of the image is loaded to be able to fill out the header. Pixels will only be decompressed when they are needed.

02:35:18chamlis_ good spot
ah so on their image they call image.valid i was going to say we do actually have to process the whole thing yeah it was just it's called new and i didn't i didn't automatically from the file or from the method name go boy yeah that sounds like it actually validates everything that i care about all right back to here to the shell Let's give it an actual file.

02:36:01I don't want to give it the two cats, do I? No. Where's that other image of mine?

...17Got it here.

...24So this is the avatar image I use a bunch of places.

...36You don't have a valid method. chamlis_ I think they use .avg for vips
So what was image processing calling? What would that be the average of like the average file size? Yeah. Oh yeah, I think that's what's happening here is, so instead of subclassing, they've, this thing is mixed up and knows both the vips API and the mini magic API and they diverge here. And so it says, if it's vips and you got an image, then call AVG. And then I guess this is the mini magic path. chamlis_ looks like it averages all Rs, Gs, and Bs
This is. crying to be split in two, I would say, except because my concern is security, I would really rather see no support for mini-magic. All right.

02:37:51I can see how this AVG would be forced to process the whole image. But it's very implicit. And so I'm wondering if the API provided actually has some method of validating.

02:38:18Yeah, something about revalidating.

...34Grabs the width. OK, what is revalidate supposed to do?

02:39:00Where would you be caching things?

...13Is there a doc mentioned for revalidate? There's a code. Update revalidate example. What does it do? Let's go look in the underline.

...38Something about an operator cache. I have no idea what this is supposed to do.

02:40:00thecoppinger 22 raiders from TheCoppinger have joined!
caches recent operations aha you can force vips to load a file and ignore any cache value by selling revalidate which means offhand this code is wrong because it doesn't set that okay

...31thecoppinger nailed it
Hello, the Coppinger. thecoppinger first shot
I don't think I know you. Hello to your 22 viewers. thecoppinger i've gotta run sorry, have a gr8 stream thecop2Salute
thecoppinger ooooo
So I am beating my head with a brick because I am trying to add direct avatar uploading to Lobsters. thecoppinger lobsters is dope!
thecoppinger is this yours?
Lobsters is a discussion site. Use a third-party service for avatars. But of course, if you are taking user data like files, thecoppinger love that
pushcx https://lobste.rs/
nogweii aye puschx is the admin
pourzied domain cost of this?
dreadscotr Hello
thecoppinger i've been trying to get into lobsters forever
yeah i guess it's mine in as much as a community can be owned by anybody so if you go to here i'll throw the link over here it sounds like you haven't seen the stream before but welcome i do appreciate you bringing a pile of people by so i'm happy to catch you up you've been trying to get into lobsters forever i don't thecoppinger it's invite only right?
I don't know. Just ask somebody for an invite. Preferably not me because I am trying to present and can't figure out or can't maintain three conversations at once. nogweii @TheCoppinger yeah, anyone can invite
Did you just randomly pick me out of the developer streams? Yeah, it's invite only because it helps a lot with spam. thecoppinger liked your vibe TearGlove
pourzied how does the inv syst4 mwork
Oh, it helps so, so much with spam. thecoppinger nogweii size me up
Nagwe, you were inviting people earlier. Can you look at the Coppinger and see if they seem like a reasonable person as opposed to a flame war? nogweii lol absolutely
thecoppinger i'm very unhinged and dangerous to know
And yeah, we'll see. Thank you for letting me put you on the spot like that. thecoppinger but very kind like a flower
Oh, well, unhinged and dangerous is bad because if you invite someone and they are a big jerk, and I have to ban them, then I do go back and look at who invited and say, hey, you want to exercise better judgment? thecoppinger thecop2Jig
notagilbert TheCoppinger is cool
And very rarely, once in a while, like, hey, you keep exercising bad judgment, or I think you did that on purpose. thecoppinger thecop2Dance
So it is possible to, you know, as you sow, so shall you reap? No. Cool. All right.

02:42:54So I've been using, so lobsters is written in Rails. Rails has active storage, which has been getting more mature the last couple of years. I'm trying to use its disk service to persist avatars and its persistence is incredibly unsafe. It just takes a user uploaded image and slaps it into the file system and just says, you know, if it ends in .jpg, we'll consider that an image. And that's not a level of security I want to operate on. thecoppinger @Nogweii i can dm ya some links to my persona if that helps
I want to actually look at the file and say, is it really? And in the last couple of years, ImageMagick, which was also not always great at this kind of question, has been really supplanted in production Rails installs by, yeah, the Copinger, that actually helps enormously. nogweii yes please
A lot of people want to just go look at a GitHub or a personal blog or something that says, you know, you have a reputation you care about. So if... Okay, so if... Vips is the software I trust here, and I believe it's maintained by Google now, and it is something I trust. then I'm not going to use this third party gem that kind of bolts on security validations to active storage in a really optional way. I want to see an API that is sort of unforgiving, that doesn't say, let me just take your file and persist it and start trusting it. pourzied best way to protect against DDOS on public APIs?
And then also we'll have a race condition to see if anybody loads it before the validation runs. Yeah. And then I want to see that it's actually an image.

02:45:02Porzit, it sounds like you've got a whole lot of questions that are not super related to coding or the coding I'm doing. The answer to DDoS last couple of years has just been Cloudflare. But otherwise, let's try and Well, I'm just going to say that I can't answer all your questions because I'm already dealing with incredibly fiddly stuff.

...45pourzied what chose you to code in this font lol
And then we'll revalidate, so we'll cache post. The what chose me to code in this font. Oh, I don't know bad taste. Under Twitch, there's a link to the FAQ. But I can tell you off the top of my head, the primary font is in Consolata. And the italics are coming from operator sans mono. And there's links to both of those in the FAQ. so we want to revalidate you know seeing this the revalidate cache busting thing makes me want to read the vips docs for any other options because if there's one surprising option there's going to be like 10 surprising options right okay full decompression yeah where are your options not a top-level documentation. This is the doc I was already in, isn't it? Yes. All right. So it's going to be just straight into the C code. Or go, isn't it?

02:47:21File name, doing a set of options. Just the options part. Yes. Give me a list of all of the options. Here we go. Name value pairs. All right. So it doesn't want that colon like I gave it. It wants an equals.

...53Many loaders add extra options. That's fine. I'm not concerned about like the odd namespaces and rotations of JPEGs. I just want a list of all of your options.

02:48:17Oh, and there's an ISA to check if the loader will actually like something, but that loader we already saw, It's sort of like it checks the header to make sure the header is very valid rather than immediately processing the whole file. And that's one of those things I see a lot from low level libraries like Vips where they know they might get instantiated with a very large file. And if you were to throw a, I don't know, like a one gig image at this, you might be very surprised when your early function stops and does a gig of IO. All right. So there doesn't seem to be more there. Is it equivocating between options and operations? No. Well, we know one is, yes, it is, actually. We know that Revalidate is the name of one of these. All right, so we have non-sequential cache, a compatibility thing. All right.

02:49:50All right, looks like none of these operations are relevant to us. So Shamless, you were saying that you think AVG is just a method that's going to force VIPs to eagerly evaluate the image to use the wrong term from functional programming. And I think I agree with you that in the absence of a really explicit validate this image method, we're going to have to call something like that. chamlis_ I think so, the image_processing dev made an issue asking this and ended up with https://github.com/janko/image_…
And on a image object, we can ask these things of it. You think image processing made an issue? Oh, you think there's something else related? I really appreciate you doing this kind of side research. Thank you.

02:50:56How could I? This is the exact thing I wanted to turn up earlier, where it was the, you know, what is the one liner check a file? How funny that I couldn't find it at all. Cause I was looking at this image processing RB and I didn't look in the directory.

02:51:25And then we say access sequential. So what does sequential do? I think true for any of the images. We're up in sequential mode to optimize memory behavior. chamlis_ I'm completely lost with all the twists and turns of this stream with what we've already gone over
nogweii heading out, good luck with the validation
I wonder if they're just doing that because active storage mostly is for remote files, and you don't want your image processing library jumping around in random access mode if you're going to take chamlis_ see you around
nogweii @chamlis_ thanks for being an awesome researcher
nogweii @chamlis_ you sure you don't want an invite? LUL
network latency over to s3 you're completely lost all right nagwe good to see you shameless so the gist of it is where we're at is we have the avatar you can attach it i trust vips i don't trust active storage

02:52:25And if VIPS doesn't think a storage, an image is valid, then I don't think so either.

02:53:31chamlis_ oh sure, thanks, I'll msg you an email on IRC if that's alright
nogweii yup
nogweii uh, give me a minute to restore my irc session heh
So this is hooked up as a model validation to the user, and I can metaprogram it to say, hook it into hasOneAttached. I could make my own meta method called hasOneAttachedImage, and that could say, yeah, it'll just pass everything on to hasOneAttached, and it'll tag on this validation. I want to see this actually work here, but that should be good.

02:54:14i didn't give my i should have thought of it earlier when that streamer dropped by but if anybody is new from the raid this is office hours for lobsters if you have questions about the site or the code base i am happy to answer them he dropped in while i was in the middle of i don't know two and a half hours into a Coding binge, and so I code when folks don't have questions. But if you have any questions about the site, the code base, the community, you can pipe up. All right. So let's see if this wants to work, right?

02:55:04I don't think I need to add some debugging. I think it all is going to be fine. chamlis_ oh I don't know if this will fail at runtime but the symbol in validate is typod
So I'm going to toss the one that's here and reload this page. And then grab it. Let's start with just a random text file. Let's see if we get the error. chamlis_ it did haha
Oh, it helps to spell the names of my methods correctly. Well, we know it tried to call it. Oh, yep, Shamless, you caught it. Good eye. Oh, and then it's not admin. That was me pasting from the console. Can't say download file not found. I wonder if it's doing this because of the previous exceptions. So Firefox would resubmit with the post variables, but I don't know if it would actually form in code. Yeah, all right, it doesn't like download. All right, active storage.

02:56:29So this lets me,

...36It's important to know the file is not yet available on the active. Oh, I bet download is failing. So because active storage talks to all of those third-party services and my validation is running before it's persisted, download has to be missing. Because I am trying to check it before it might have been persisted. And active storage has support for streaming directly to a third party network file storage service. So it kind of can't have this during validation. File is not yet available on the active create callback, but in the after create commit. But I need it a lot earlier than that.

02:57:39Well, let's go look at the active storage validations. Because how is it getting the image before it's persisted? Right?

02:58:00So when they handle it, they say, is a string, where does read image come from?

...22It's an image processor, has exception, just looped around. All right, what is this file object?

...44Let's see where it's coming from.

...55Oh, I don't want to schlep around in this gem for a term as generic as file, but I think I'm going to have to.

02:59:12So we have this. We have this context of attach. You can read the file path. This is promising. This must be where that method comes from. But it gets initialized with the file, and then it just saves it as an instance variable. Fit as an active storage blob. Then it calls find signed. Otherwise return the blob. See, why is this to call blob.download if i'm not allowed to call blob.download this should be firing at the same time right

03:00:38I could do it on an existing one.

...45Because there's nothing attached. Yeah. Invalid message strength. Can I just give you a file name? No.

03:01:24I haven't set it from the console at all, so I don't have it in the history.

...37Does it want a file object instead of a?

...51Yes, it wants a file object, not the contents of a file. OK, so now it's attached. finntechnz I think you either need to call `save` on your user
It's not attached. Did you not like it, or did I not type the file name correctly? I got the file correct.

03:02:15Yep. But it didn't want to attach it.

...30I think I'm recreating the admin object, maybe.

...47yeah so the that admin method is just some quick hack in my herb rc to grab my user object because i need it all the time yeah i'm not fintech i appreciate you catching i'm not even trying to persist this yet i'm just trying to say once i have it in memory can i ask the blob for its contents And the answer is no, that I just get this not-so-useful error before it's persisted. So if I said, you avatar... All right, if I call save, well, it's going to fail because of that validation I just added, right? No, but it did stick the thing... Whoa there. Why did that take so long and run in the background? That's a little weird. Because it took a second to make all the variants off of that. So yes, it's attached. And if I said download, and if I spelled it right, Yeah, so now it's willing to download the file. That was a weird amount of time to get two megs or three megs of data.

03:04:34So in the Rails-iest tradition, the active storage object is in a weird, partially validated state. And I can't tell that it's not persisted. And I can't have the validation grab the file because download tries to pull it out of active storage instead of trying to pull it from memory. Like this blob object has got to have it, right? One of these million methods has got to have the byte data in there when I'm in the process of uploading. So hold on. Let's first replace that.

03:05:37So I want to replace the avatar so I get back into the state where it's attached. but not persisted. Man, can't type and talk at the same time. So it's there, but I can't download. All right, I get this not so helpful error because the image I attached is not there. And if I say blob, so this is a new record and I can tell because it has no ID on it. It does have a byte size. So if it has a byte size, it's in memory. And I don't know where, I don't know what the API is, but it's here. It's not download. See if I can checksummit.

03:06:41Byte size came from user in the database.

...49finntechnz In my experience I've had to do inspections, transformation, and virus scanning on the uploaded file (e.g. Tempfile) object directly, before handing it over to ActiveStorage. The ActiveStorage::Blob records are just wrappers to remember where/what the file is in your filesystem (e.g. Local, S3)
So one of these guys is going to be the actual byte data and I can't see it. In your experience, you've had to do inspections, transformation virus scanning on the uploaded file for handing it over to active storage. The blob records are records from where that is in your file system. So. Ben Tech, I appreciate it. We've talked about, you saw me flip by the settings page in the browser. So we have the edit user, that's the user model, and it's attached by saying attributes equals user params. So you're saying that instead of doing the Rails-y method of saying on the form object, have a file field, you do just a raw file field tag, and then you attach it.

03:07:54I've been trying lately to lean into ActiveRecords model stuff. I don't want to like, I don't know, it just feels clunky to save it on disk and then try to attach it. On the other hand, it doesn't work that way at all.

03:08:22So if I put it over there, I could be like, oh yeah, do we have a params avatar? Then check that it's valid. Yeah. If it's valid, We'll do edit user dot avatar equals well.

03:09:01All right, so I'm just kind of plopping this in here to sketch this out.

...13finntechnz Yeah I haven't done it with Rails forms - only via JSON/APIs. I think the ActiveStorage paradigm is to put it on disk and then work with it later. I guess this is mostly thinking about sevices e.g. S3 where a malicious file isn't such a direct threat to the system.
Like, I guess you haven't done with Rails forms only via JSON APIs.

...26Yeah, it's less that it's a malicious direct threat to the system, and it's more if I persist the file first, I literally can't tell it's an image before it's persisted. And then Rails is going to turn around and start serving that. So this is the avatar image on someone's profile page. And if I tell Active Record to persist it, sure. It doesn't matter if it's disk storage or S3 or Google but even before it knows it's not a zero byte file or it knows it's not an exe that happens to be named dot jpeg active storage is like oh yeah i've got a file it's attached let me start serving that and i can see how basecamp kind of got there if everyone is paying to be a user of the service but that's no bueno for a public site like i cannot immediately take a file and start serving it.

03:10:43So this kind of flow is wordy as hell but fine as long as I validate.

03:11:03I don't love that it can't, it spreads out the logic a lot. And what I would really like, I mean, my ideal API would say has one attached avatar that is an image or real and rails would prevent me from ever attaching something that's not a valid image and it wouldn't even try to persist it because it has it in memory at the upload so i guess that's a question about active storage if finntechnz Here is a stackoverflow: https://stackoverflow.com/quest…
If activeStorage has a byte size, that byte size is going to tell me the name of the place it's storing it, whatever that attribute is going to be. Can I filter this down?

03:12:43Oh, that's great. Thank you. Someone who is explaining exactly the thing I want to do. Dot attachable. This is an undocumented internal API. Oh, lucky me.

03:13:18finntechnz Just fun Rails things!
Yeah, this guy proposes a thing that won't work. Just fun for Rails things. No kidding. All right, so let's try this in the console here. Because I'm in the situation where I've attached this avatar, but it's not persisted.

...45Let's do it on the user object directly, I guess. OK. It's got a file object. All right. Well, that's the thing I want. Yeah. Thank you, Finn. So now the question is, do I want to do it up in the controller or in the model? And I would really prefer to do it in the model so that I can extract this to a concern of some kind.

03:14:29So many buffers in PIM. All right. So we will just say, attachment changes for the avatar, attachable. And that goes in here, in here, and all right. So if that's all there, Now I should be able to get the error message I want in the form. So let's toss what's there. Let's reload the form. Yeah, my big gripe about Rails is how much it loves persisting invalid objects. Guess that's not quite the right file object. Action dispatch HTTP uploaded file. Must be uploading to a temp path or something, right? So I did it in the browser or no, I did it in the console. And if I pass in a file object, I'm getting back the file object. So it's acting different in a browser console.

03:16:07finntechnz You might need to add a `.tempfile` or something for VIPS
Let's debug it.

...14Yeah. Temp file or doc file. That's why I'm like, all right, I don't want to debugger debugger use it so infrequently. All right, here we go. So now. You are the thing I expected. Oh, I can't call ls. It's not a full IRB. Do we have a file name? No. Do we have a hemp file? Yes. Good guess there, Finn. Or good remembering, I guess. Better close the debugger. All right, let's reload. Please, form error. One form error is all I ask. Reopen the debugger off screen. Undefined method, byte size.

03:17:34All right, so...

...40Lost my place. This temp file. Oh, new from buffer expects a buffer. I can say new from file. I don't remember if that wanted a path or if that wanted a file object. Oh, there's me hacking out pseudocode in the controller.

03:18:15Well, it's promising. It made it past that validation to give me a syntax error from the controller. What are we thinking about here, guy? Well, the debugger opened. Uninitialize constant bits. Let's go back into this. And I guess I need a require at the top. That's not so common. that out for the moment.

...54New from file, given two, expected one. This is going to be, I guess it doesn't take the same options hash or options string. No implicit conversion of temp file into string. So Wish I had a sorbet or something to just tell me what's the actual duck type that vips is wanting out of its file object or file argument.

03:19:36What if I could say temp file read?

...52Isn't that beautiful? Oh, Fintech, you helped get this. Some great progress. chamlis_ 🎉
This was the exact kind of exception I wanted to see of you can't save your user because you didn't give me a valid. And this, the form object went into an inconsistent state. where it's both asking me to attach or it's saying there's an object attached and browse for it. So I wonder if in the database I actually have one or not. Because I shouldn't be able to see both of those things at once.

03:20:40finntechnz Very nice 👏
No, I don't have one. That's good. So it's just the view being funky. It must have kept it on the... Form object. So I'm just going to make sure. Active. Oh, man, I can't spell. All right, yes, there's nothing in the database, which is what I want. So progress there. All right, so let's tidy up a little. So I don't need all this pseudocode.

03:21:19Oh yeah, Finn, you might get a kick out of this big grumpy comment too. So because this is a public site, I don't want the file name of the uploaded image to be stored and reflected back to other users. Because if I name the file like, I don't know, not 2cats.jpg, but 2cats at 123 Main Street.jpg, that leaked some personal info there. And so I was doing some of that up here in the controller, but maybe it wants to move down to the model, right? Yeah, let's move it down to the model.

03:22:12It's not strictly a validation, but I guess I'm OK with just doing it in place. The other thing is, so it's doing the Rails-y thing where it just kept the image attached. And what I really would like is that this validation says, hey, if you have an invalid image, well, I'm just going to throw that away. So how do I get it to, can I just say avatar equals nil? Because if I can say that, I pulled that line of code over and I didn't update the variable. It's not edit user. It's just this object username and the extension.

03:23:33Okay, so that didn't throw any errors, but the form does still seem to think it's attached. So let's go look at how I'm accessing that. Yeah, so I just say file field. So that's telling me really, so if I said...

03:24:05Am I going to get what? It's not going to be just.

...17Yeah, it's that. So we have one. And is there a? I see it's interesting. There isn't a blob object here. And I kind of expected there was going to be one. full of invalid data. There's a blob ID. So the puzzling thing here is we're getting this if the user avatar is attached, tell them so so they can delete it.

03:25:14finntechnz Do you need to do avatar.purge? It's the activestorage method
It's like a true attached one. Do I need to do avatar purge? I thought purge was just if synchronously destroy. So the way this talks about destroy the avatar and the actual resource files. Yeah, and then this one says associated models, but there isn't an associated model I mean, I'll try it, right? Can't hurt. But there isn't an uploaded file and there isn't a model persisted to purge. But I guess that was it. So even though I set avatar equal to nil, I guess because it's an association sort of that like has an attached one, It doesn't use that same equals. All right. All right. So that was me uploading a scratch file. So that was just like a, a hundred byte text file. Let me try and upload a, a zero byte file because we saw from that. third-party thing that it had to treat them differently or add code to treat them differently. Good. Now let's see if it still works with a valid one, right?

03:27:10So I'll attach two cats. Ooh, promising. should appear on my profile i mean it should stay the same really good two cats still there all right so this is promising this this is small enough that i actually feel pretty comfortable with it i'm not pulling in a third-party gem that is thousands of lines of code and clever metaprogramming. I don't super love this average instead of an explicit method that validates that the image is valid, but I think I'm okay with it. That's fine enough. And then I guess Yeah.

03:28:21I guess what I want after that is for it to be a JPEG or a PNG. That's the other thing I care about. So we've got to take the vips object, not herb.

03:29:11Let's just use the same method I've been using now. Just read the thing into RAM.

...23OK, so we have auto-persistent.

...41So what do we have to ask this image its format? Because I'm not seeing anything about format, file type, content type.

03:30:00Somebody's spawning it in these methods?

...07No. Those are the XF metadata.

...23Ban, join, coding. It's coding.

...33No coding. Oh, there it is. Format. Uchar. That's not the kind of format I meant. There's alpha.

03:31:08I'm going to pull this off screen before I call inspect because it's not interesting. I know this image might have some exif metadata in it.

...26There's a set type, get type of.

...36All right, well, when all else fails, we'll ask Google first.

...52It's in the metadata.

03:32:01Aha. So we can see which loader was used. So if it loaded with JPEG.

...19All right. Huh. That's interesting. It's very slightly different. Instead of I get JPEG load underscore buffer, it gets JPEG load because he did new from file. And I just said buffer. So it has multiple loaders that might be JPEG.

...47Can I look and ask you for a list of all your loaders?

03:33:06Doesn't look like I can. Let's bring you up.

...40Yeah, I'm not seeing a nice list. So I'm just going to go ahead and say, if it starts JPEG load, it can be a JPEG.

03:34:14What did they call it? Image.

...39Let's just grab it. I'm going to pull it a bunch of times.

03:35:08And then I guess we'll find out if that's the name of the PNG loader real quick, right? I just got to find a PNG here.

...22Forgot our code base does not actually have a lot of images in it. All right.

...36So grab vips in and then image file. And then that just takes a path according to that nice stack overflow issue or GitHub issue.

03:36:09png load okay all right i think that's my my file format enforced do i have a tiff file anywhere i mean maybe i have one that is invalid I'm just trying to think of, do I have a BMP somewhere I can use?

...51I'm peeking at these off screen because heaven knows what these are. All right, we've got a test image. So let's just grab this guy and try to upload it.

03:37:13It's a small valid BMP.

...34All right, so grab this test palette.

...41There we go. So now, even though VIPS is very happy to read BMPs, you can't use them. That's good. And I just, I don't want to get into transcoding and I feel it's fine to force people to give a JPEG or PNG. So where am I at? Let's get rid of that scratch. Let's get rid of that zero. Don't need a debugger.

03:38:21And these are kind of two separate things.

...32Man, that was painful. All right, so all of this is handled. It's probably about time for a commit, and then there's a little more functionality. Do I wanna, all right, where am I at?

...52The settings controller actually doesn't need that whole list of variants anymore, because now it doesn't care. So I don't need this anymore. It's just as well that I can't introspect on all the variants of an avatar.

03:39:57So this can just come up here. it'll do a has one attached let's say has one attached image i never remember with these what the proper way to pass all these arguments on is All right, do I have to put the block at the end? This is just Ruby syntax tripping me up.

03:40:40I'll check that in a second.

03:41:14So I'm going to make a, I think I have to pass a symbol. I think I can just say.

...48And then move this stuff up. See, I've been talking about being suspicious of metaprogramming, but I don't actually want to drop 20 lines of image processing code into my user model. I just want to say, has one attached image.

03:42:25What are we mad about? Ah. All right. And then put bits up here. It's not in the user model anymore. There we go. Now, let's see.

...58Because I know I didn't remember that syntax. Hey, Odin project, that's been a minute. Don't want to yield. I just want to pass it on to the next method, which is going to be what?

03:43:40So if I say args block and args, let's see what terrible metaprogramming error I get out of this. Well, if standard likes it enough to just say, do this, maybe I even got the right syntax. So let's load the settings page.

03:44:08And think about it. so many extra tabs at this point we're coming in on the home stretch of this stream by the way so i'm going to get this metaprogramming work and then commit it so if you have any last questions about the site or the code base now is a great time to ask them because i'm going to go pretty fast from i see the metaprogramming work to oh thank heavens commit this to end of stream

...46And in case anyone's wondering, this is an odd caching issue that pops up occasionally in dev. It will load in a minute. I will just be mildly embarrassed until it does. In fact, I probably don't even need to just yet.

03:45:30Actually, I could probably just cancel this and attach an image. So let's grab the cats.

...45Syntax error. Unexpected left bracket. Wait, I got a syntax error out of model.rb? It must be because I'm not. So I just added this star args. I was saying args. There we go. Wow. Wow, that worked. To have metaprogramming work after a one-character fix, let me grab another. So I'm grabbing that text file, and I should get the validation. I feel like such a grown up programmer today. I met a program or something, and it worked on basically the first try. I'm so happy about that.

03:46:49Fintech, thank you for your help. Shamless, thank you very much for your many researchers and helps. All right.

03:47:03So that's a whole bunch. finntechnz You're welcome, it looks like it's working great
chamlis_ no worries, thanks for the stream
Like this stuff, this is on the downslope of it. I've got to go back through my stream archive to remember why I wrote that. Yeah, it really is working great. This has been such a hassle, these avatars. But at this point, see, I'm happy with it because it's going to take a trusted piece of code and make sure it's really an image. in my preferred formats and actually evaluate it before it persists it. Rails is so chill about, yeah, let's just, let's put this up here. Cause it's a long comment.

...56Great. Let's look through this diff. Oh, definitely don't do not, I thought I deleted those. All right. So what do we have? We have a blank line. That's fine. Come here. So this was the stuff that I moved down into the... Oh, this was the stuff that... I think it was Shamless that you figured out that if I reset the file name on the parent blob, Even though they stay wrong in the database, it applies on all the variants so we don't leak file names. So all of that is good. chamlis_ that wasn't me, I think it was Nogweii?
Application record. This is my new metaprogramming that pulls off the name and says it's attached. OK, then, yes, it was Nagwe because they were helping earlier. Yeah. Boy, this API has a lot of sharp edges. And then, see, isn't that nice and clean? I don't have to put a ton of stuff in the model. I'm not gonna make a gem out of that, but I'm happy with it. All right.

03:49:24Lovely.

...32all right so looking back at the scratch that's all of this stuff this this node on preload that i think shameless you made on the first avatar stream that's gonna go for when i rewrite the avatar image helper i guess on the next stream this stuff we're past the watch peter struggle and cry tears of blood about the active storage api section of the coding this is straightforward enough i might just knock it off so i can get into doing recheck demo on thursday because i really do want to get into that and this stuff feels pretty straightforward i mean download avatars obviously i'm going to fall into a never-ending hell pit of making sure that nginx is only reading the files it should out of the active storage but i can't really stream that because if there's anything that's going to involve a ton of splashing up server config i shouldn't yeah that'd be it so that one wants to get left off stream anyways okay what a great place to end i feel really lucky that What was his name? The Coppinger? Coppinger? It's almost a Cooper, like he's a barrel maker. I'm really glad he dropped on in. chamlis_ nice job getting through it
I was getting to the point where I was going to give up on avatars. Because this security stuff was pretty painful. All right. finntechnz Coppinger, he and I hack on a bunch of things together and he streams
Yeah, it's funny how much of programming is just managing feelings, you know? That... That frustration of not being able to figure out an API and feeling like you're missing something. I will have to try and drop by a stream and kibitz. If he codes Rails, I'll actually be useful. If he codes anything else, I'll be misleading. Very helpful, you know? all right folks so my next stream is thursday morning that is 9 a.m u.s central time the twitch schedule has it you can follow me on blue sky or mastodon if you want the announcement because i'm getting pretty good about remembering to make those and then otherwise i will see you around the interwebs which are a series of tubes take care