I've been fighting with getting a database moved from #MySQL to #Postgres for about a week now because of a dumb issue with Directus, and I finally fucking got it. The trick was, I think, to just make pgloader run slower.

I tried using the default settings that the pgloader docs suggest, only to have it shit the bed over and over at different spots, and throw errors about not being able to connect to either the PG server or the MySQL server, or simply hanging until I ctrl-c (or kill -9 sometimes -- it's a bastard) the fucking thing.

Anyway, here's the load file I ended up using.

LOAD DATABASE
  from mysql://username:password@host:port/database
  into postgresql://username:password@host:post/database

 WITH include drop, create tables, create indexes, reset sequences,
   workers = 1, concurrency = 1,
   multiple readers per thread, rows per range = 1

  SET PostgreSQL PARAMETERS
    maintenance_work_mem to '128MB',
    work_mem to '12MB',
    search_path to 'database, public, "$user"'

  SET MySQL PARAMETERS
    net_read_timeout  = '120',
    net_write_timeout = '120'

 CAST type char when (= precision 36) to uuid drop typemod

Slap that shit in a file (obviously replacing all the necessary bits with your own stuff), and the fire it up with pgloader like:

$ pgloader data-loader.load

Then sit back and wait for your database to migrate :-)

I hope this helps someone in the future, because the documentation, and the --debug output of pgloader didn't do me a damn bit of good. It was by pure luck and chance that I thought "let's try cranking that workers and concurrency down, and that rows per range, too, and see what happens. Well la-dee-fucking-dah, it worked.

Fun side note: Since this was for Directus, of course it presented another set of challenges once the database was migrated. Starting the server resulted in a Database doesn't have Directus tables installed error with zero other details. Turning the LOG_LEVEL to trace shed a little more light, which led me to this comment on GitHub. A tweak to my .env, like so:

DB_SCHEMA="my database name"
DB_CUSTOM_SCHEMA="my database name"
DB_SEARCH_PATH="array:my database name,public"

And 💥 the server runs!!

Fuck you, computers. Fuck you to death.

There's no way this hasn't been documented somewhere already, but it just dawned on me last week, and I thought it was pretty neat!

With the fun power of destructing arrays in #javascript, you can set multiple variables in one go using a ternary expression. Check it.

const [var1, var2, var3] = someVar
  ? ['true value 1', 'true value 2', 'true value 3']
  : ['false value 1', 'false value 2', 'false value 3'];

Not too bad, eh?

Color-code your terminal by server. Fantastic.

I've been using Downcast for a while, but it's got some really annoying bugs that I just can't deal with any more. It's basically just a podcatcher - a very slow podcatcher - at this point, and it constantly eats up about 50% of two CPUs when it's doing nothing, and even the podcatching is painfully slow.

So I wrote some javascript (with a little help from ChatGPT, because why not) and now I can download podcasts when I want without eating a bunch of CPU and being dumb. Yay!

Some high quality UI design right here.

Some very slick wood working. HT Jason Kottke

VIM in the browser. Fun stuff!

Hard agree.

I spent about 30 minutes looking for this autoripper.sh script tonight so I can rip all these Bob's Burgers DVDs. I swore I'd saved it somewhere, but I must have thought "I'll be able to find this again!" and deleted it. FWIW, I did star it on GitHub... But I was looking for it in my gists 🤦‍♂️

This looks handy!

Today while throwing the ball around with the big dog, I had the thought that I should completely rewrite this thing in #NextJS, just because. I haven't followed through on it... yet.

It could be fun, I suppose. But I also have other things that I know will be fun and will not, at any point, make me question every decision that led me here. I could spend an entire day playing #guitar, getting blisters on my fingers, and making exactly zero #music, and still probably have just as much fun - if not more - than sitting here, banging away on this fucking keyboard writing code that'll be used by zero people. Not counting me, of course :-)

I'm not sure why I even bothered since I'm completely positive I'll be the only person that ever uses it (and even me, I'm not sure how much longer, because who doesn't like reinventing stuff over and over again for absolutely no reason?) but also, why the hell not, I guess? Doesn't the world need more bad #opensource #php code? Do you want a dumb website like this? Go grab the code, and when it doesn't work, make an issue (or better yet, a pull request!) over on Github :-)

This just saved my bacon.

https://jay.gooby.org/2022/11/01/re-enable-older-rsa-ssh-keys-after-upgrading-to-macos-ventura

Got #boosted yesterday at 11AM. First half of the day was fine, but around 6PM, things went downhill fast. Spent the rest of the evening in and out of sleep on the couch until 10:30, then went to bed. I was freezing. Woke up every couple hours for some reason - sweating, freezing, rolled over on my arm, you name it. It's all done now, though.

I slept for nine-and-a-half-hours according to SleepCycle, and got 100% "sleep quality," and I'm feeling much better today. Not back to 100%, but probably somewhere around 85%. I can't imagine what actually catching #covid is like, especially without being vaccinated. If it's anything like what I've gone through (for every fucking shot), then I'm happy to deal with it for half of a day and be done with it. And hopefully I can continue to avoid catching it. 🤞

Listen up, meat puppets.

I don't see how this could go terribly wrong.

Do you want to make love to a sad old man?

We took a trip down to La Junta to see dinosaur tracks, old west ruins, and tarantulas. Mission accomplished.

We went on a #hike in Garden of the Gods after business pizza. That was fun. We saw at least 6 deer, maybe 8. Missed a pretty great sunset by about 30 minutes, but it was still a good hike. Lauren got to break in her new hiking shoes a bit, and I got my "exercise" in without having to walk on the fucking treadmill.

Work has been #work. Nothing fun, nothing terribly exciting. Day job has turned into just doing billing stuff - new prices, new plans, new subscription types, accepting payments from other countries, implementing 3D secure... Billing, billing, billing. I've been working on a desktop app (in #Electron, of course) for the night job. Not super fun, but it's different at least, which is kinda fun. Though I question the reason for its existence... I have a feeling that, with a bit of #CLI expertise, the person requesting this application wouldn't need it, but who am I to deny someone paying us to write code for them?

Storm's coming. I've been watching the lightning for the last half hour. I can finally hear the thunder. I hope it's a good one. It'd be nice to go to bed to some big ol' fat rain drops hitting the window.

Authenticate sudo commands in the terminal with your Apple Watch. It took me a few tries to get right, and I had to edit the Makefile so the target was ARM (arm64-apple-darwin21.6.0), and I even had to boot into recovery mode once to fix my /etc/pam.d/sudo file, but now it works, and it is glorious.

"Hyundai predictably fails in attempting to secure their car infotainment system with a default key lifted from programming examples," writes Slashdot reader sinij.

Old Mr. Bernard, Old Mr. Bernard, who have you silenced today?

Jack shit going on. Back to work this week after a week off. My nephew came to visit and I took some time off so we could do fun stuff. We went up to the top of Pikes Peak, walked around some downtown spots, ate out a bunch, saw Nope, played video games at home and in arcades, and just had a good time. The best part though, was sleeping in. Good fucking christ do I miss sleeping in. Not until like, noon - which I could definitely do - but like, just 10am. There's nothing stopping me from doing that now, other than the fact that I'd only have an hour between jobs, which is a pretty big deterrent. Ugh. Maybe one day I'll be able to just work one job 🤞

Makes me cry laughing.

Day job, night job. Food between. Nothing special happened today. But night job lately... Oh boy.

I've been tasked with building something to let people see when there have been changes made to a website. A new page, a change to a page, whatever. That sounds easy, right? Just scrape some pages, save the text, and the next time you scrape the page, compare the new text with the last bit of text we saved, and if there are differences, save those. Easy peasy, right? Fucking wrong!!

First off, scraping a site - by which I mean using some code to "look at" the homepage, grab the content and save it, extract any links in that content, figure out if we should follow any of those links, then follow them and start the whole "extract content, find links, follow" process until there's nothing else to "look at," is no easy task. It sounds simple, but it's really not. One way we can do this is to grab a page and try to extract links with regular expressions. That's a bad idea for a number of reasons that need no explanation. Another better option we have is to use a library, like Cheerio, that does all that hard work for us and let's us do something like $('a[href]') to pull all the links out of a page, but we still need to figure out if they're links we should follow. And if humans had anything to with the composition of a page, there's a good chance you'll encounter some dumb shit that makes no sense and needs a special exception in your code. Broken HTML? Guaranteed. Links with href values that make no sense, but a browser can somehow understand? Sure. Links with no href when the HTML loads, but they get populated by some Javascript after the fact. That wouldn't surprise me one fucking bit. All this is to say absolutely nothing of the fact that a lot of sites, especially larger ones, have protections in place to prevent bots from doing exactly the thing I'm trying to do.

Okay, so "scraping" a site is out of the question. Well, you know what most sites (though, not the bigger ones, strangely enough) have? Sitemaps! Boom!! We'll just grab the sitemaps, which tell us exactly which pages exist on the site, and grab the content on those pages. Easy enough. Oh, except some sitemaps contain URLs that aren't valid. https://somedomain.com /a-page.html is not a valid URL, unless I missed the announcement from ICANN that "dot-com-space" is a new TLD. I guess technically https://somedomain.com/./a-page.html is a valid URL, but like, what? Slash-dot-slash? Was this website built by Fatboy Slim? Oh, and these sites will also block us if we scrape pages too quickly. OH! Some sites have thousands - or hundreds of thousands - of pages. So even with a one second delay between scraping pages, it'll still take a stupid amount of time to scrape an entire site.

Oh, you know what?! robots.txt tells us pages we're not allowed to scrape, and also (I learned during all of this) how long we should wait between scraping pages. Let's look at the robots.txt and make sure (a) we're not scraping stuff we shouldn't and (b) we're not scraping a site too quickly. Of course, any jackass would realize that robots.txt probably isn't telling them what not to scrape or how long to wait 100% of the time, but I'm a special kind of jackass, and that did not occur to me. Not every robots tells you how long to wait, or what you're not allowed to scan. Cool, cool, cool, cool.

So here I am. Waiting sometimes 4 days for a site to finish being scraped (I'm not sure how long it actually took because I killed the process after 4 days) and learning about all the ways people can supremely fuck up something as simple as a sitemap. Seriously?! SLASH-DOT-SLASH?! WHAT THE FUCKING FUCK?!. I'm pretty sure the only way to get this thing to run in any sort of timely fashion is to completely rewrite it so it can be distributed across a fuckton of servers and processes... And a lot more infrastructure.

I guess I’ll have to learn how to hack my car now?

Nothing happened today. Saturday errands, then hanging out writing #code. Code writing went well, but Github is still being a fucking asshole.

I'm not sure what the problem is, but I've got a test that consistently fails at this one spot in their "Actions," but it runs just fine on my machine. Ugh. It's gotta be something with how I've got the action configured, or maybe the code is legitimately broken, or maybe it's something else, but I'm having a hell of a time figuring out which of those things it is.

One very interesting thing - during my last session of test debugging, I realized my local Redis version didn't match the version in my workflow config, so I updated the workflow config to match... And the fucking tests PASSED! Literally all I did was change the Redis version in my workflow config, commit that, and boom, I got the green check. "No fucking way" I thought, and I was right! Because on the next commit where I just removed some caveman debugging, the goddamn tests failed. Ugh.

No clue what the problem is, but I've decided I'm not too bothered by it. The tests pass for me (classic "it works on my machine 🤷‍♂️") so I'm not going to lose any time trying to figure out why Github hates me so much. I can still write code and run tests, so suck it, Github... Suck it dry.

Lauren and I were out running our Saturday morning errands, and we pulled into Ace to discover this beast.

The owner wasn't around so I couldn't find out anything about it, but just look at this thing! It looks completely custom, and completely bonkers. I wish I could've talked to the person that built it and taken a ride... It is probably one of the scariest and most fun things in the world.

Yuuuuup.

Nothing gruesome, no blood or anything (not even sound)... Just people running towards something, a moment of no activity, and then people jumping into a car and rushing off. Crazy shit.

Found this today after thinking "So many song lyrics are just nonsense, I bet I could use GPT3 to generate lyrics" and then thinking "I bet someone else has already done this." Not great, but also, not the worst. https://www.jarvis-lyrics.com/

Finally finished helping my friend get some servers moved around. The database was the worst. Some bullshit with needing permissions just right because computers are stupid... Anyway, it's done now. Hopefully I won't have to fuck with it ever again.

Yet another #nodejs #javascript #typescript runtime. Looks kinda neat. HT daringfireball

Meh, more nothing. I added hashtags to this stupid thing (#php, #programming, #stupid), so there's that, I guess.

Actually, not nothing. I'm in the process of helping a friend move some servers off of aging hardware onto some snazzy new VMs. Not really exciting, but also, it's something 🤷‍♂️

Cake - No Phone

Nothing today. I'm exhausted. Went to bed too late last night. But yesterday, holy shit.

A dude across the street blew up his hand! Like, people-looking-in-the-street-for-parts-of-it blew it up. Of course I helped look, who could resist?! I saw what people said was part of a thumb. A girl pushed it around on the pavement with her foot. I assume it was a part of something because someone quickly came and collected it in a bag of ice and rushed it off to the hospital. And then there was a fist fight at the same house. Nobody called the cops (good). I'm pretty sure the sun hadn't even set.

After that crazy shit, nothing eventful. Friends, fireworks, beer, food (so. much. food! ribs, chicken, coleslaw, mac-n-cheese, potato salad, baked beans, potato wedges, candied bacon, and more stuff that I'm forgetting - Lauren fired up the grill at 10AM) music, lasers -- one of our friends is a DJ and he setup some speakers and a laser projector on our house -- it was pretty rad.

The neighbors seemed to enjoy it, and one guy even came over and thanked us for the entertainment. The fireworks in the street, that have happened the past two years, pretty much stopped after that dude fucked himself up, but there was still a great show in the park and literally all around us in the neighborhoods. Overall not a bad night. For me, anyway. I'm sure that dude with the messed up hand has different feelings.

Monday. Nothing happened. I picked up my rental suit for a wedding on Thursday. Fought with some dumb shit at my day job, and more of the same at the night job. Meh.

The monster is only interested in killing people who are "suffering" -- they don't talk about their problems with their friends, family, counselors, whatever, and they suffer alone and in silence. They're depressed and they don't seek out help, so the monster kills them, and the monster is just suicide.

Right?

Doing this early today because instead of spending my whole damn day in my office, I'm going to setup the HTPC and hang out in the living room.

Today was good. Talked to both parents for a little while and got some another Indieweb feature - Webmentions - implemented on this dumb thing.

My new keyboard (Keychron K8 with hot-swappable keys) arrived yesterday, and I'm quite pleased with it. It feels a bit quieter than my previous keyboard, though that could be due to the desk mat that I also purchased. Regardless, it's nice to have function keys and number keys and to have the arrow keys back in a place that makes more sense... Now I just need to learn some new muscle memory for those things since I got so used to where they were on my last one. This week will have more typos than usual, I'm guessing.

We also got 3 new pieces of furniture - a credenza for the dining room, an entertainment/tv stand thing for the living room, and the bonus piece, a VHS/CD cabinet that Lauren is using to store her video stuff - for the price of 2! When we were picking up one piece, the seller offered us another one for free. Not too bad at all.

Back to the grind tomorrow. And then only a week until we get a fireworks show on our street.

Well, this happened. Can't say I'm surprised after the leak a while back, but it's still pretty amazing.

Nobody was stopping people from not getting abortions, so I don't understand what the problem was. What is the deal with these fucking people? Why do they want to impose their beliefs on everyone? Why do they think that making someone have a kid they don't want (for whatever reason) is going to be a good thing for the kid or the person that's gotta raise the thing? Why do they want to give everybody guns? Why do they get elected in the first place? What the fucking fuck is going on?!

In the car today I heard the president on the radio saying "Roe is on the ballot in November. You've gotta elect people that will..." blah blah blah. Dude, we elected you!! What are you doing? There's no shortage of democrats in power, and there's a democrat in the fucking white house! Executive order that shit or something, I don't know. The last dude didn't seem to have a problem doing that, why can't you?

Anyway, nothing happened for me today. Errands and yard work tomorrow. Might be picking up a couple bits of furniture I found on Facebook... We're at least gonna go take a look at 'em.

Another (almost) week of work, another week of nothing really exciting. We went to a Rockies game on Sunday for Fathers Day with Lauren's uncle, and that was pretty fun. Expensive as all fuck, but fun.

I'm beginning to regret this whole "daily recap" thing, since most of the time it's just me talking about what I had for dinner and absolutely nothing else... Whatever.

I love fireflies, or as they're called in south, lightning bugs. And these are some next-level fireflies. Via BoingBoing

Day work and night work, all the same. Got the carpets cleaned today and Dexter hasn't thrown up on them yet. I wonder if we'll make it to 24 hours. I think last time we made it to like, 16 hours before he puked on the carpet. I'd say more than 50% of our house is tile or hard wood, and he goes to the end of a carpeted hallway with no exit (in other words, he never goes outside that way) and pukes, shits, or makes some godawful mess that is super fun to clean out of the carpet. He's fucking cute, though.

As a result of getting the carpet cleaned, we spent the afternoon outside and the evenings in our offices, because we can't put the furniture back yet. Fun.

Update on my whining last night about Apple making shitty software. I discovered today that Podcasts.app (yes, I use it -- the app I purchased started acting wonky, so I had to do something!) also can't seem to remember the window size or position. Fucking ridiculous.

I got my site updated tonight! Running the newest PHP and CakePHP. I'm pretty happy. Wasn't as bad as I thought it'd be.

If the code you wrote 10 years ago is still in use, then you have a lot to crow about.

There is code that I wrote 10 years ago that's still in use... Though it's barely working and to the point where it's impossible to support. I just wish the folks using would stop using it.

I decided to investigate why the hell News.app on my desktop can't seem to remember the window size. It's a brand new computer that I've not had a chance to fuck up that bad, right?

Turns out it's not me, it's just Apple making shitty software (I said to no ones surprise). I found this post on the Apple forums and the "solution" is to uncheck a setting in the system prefs, but that only works if you quit the app. If you close the window without quitting it, no good - it resets back to a small, perfectly centered window. Apple, seriously, what the fuck?

This reminds me of a post by Benjamin Mayo about how shitty Apple (News, Music, etc) services are.

They are built to a passing grade, but nothing more.

That pretty much sums it up. Apple makes wonderful hardware, but more and more, their software and services are less impressive. News is a fucking catalyst app, which probably doesn't help anything, but isn't Messages also a catalyst app? It can remember where I place, and how I size, the window, so why can't News? Music is such garbage, that I try not to use it, and I fucking pay for it! Podcasts, while not a service, is still some in-house Apple software, is so shitty that I spent money on an app.

But like I said, their hardware is top notch, and while they can't seem to figure out how to make a decent RSS reader (which is all News.app really needs to be, right?), they do a pretty good job on the OS, too. Certainly the most attractive and easy-to-use *nixish desktop I've ever used.

Nothing doing today. Work in the day, work in the night, dinner in between. Last night was fun... Kinda. I spent some time upgrading a CakePHP 3.8 app to 4.3 and what a fucking nightmare that was. But now it's done and I just need to make a new server and deploy it. Yeah, make a new server because that's probably easier than upgrading the current one. I've had enough upgrading bullshit for a while.

It's been a long time since I've had to deal with the headaches of IE, but it's nice to see that it is finally dead. As long as you don't count the IE mode they're leaving in edge until at least 2029!! Whatever. Still good to see it go.

Bon voyage, you filthy bastard.

Boring day. Exhausting day. I was up until three in the fucking morning yesterday (today) for some dumb reason. We had people over for some drinks and grilling out, and it was fun. So much fun that I didn't realize it was almost 3AM when I cracked my last beer. I was wide awake! Not so much this morning when the alarm went off, though. Jesus, it was awful. I think I'm gonna turn in early tonight, try to catch up on some sleep.

You've somehow convinced me that I should spend my life toiling away with you, typing into you and trying to make you do things, and at every turn, you shit in my fucking face and make my life more difficult. Fuck you.

Monday. Nothing fun. Work was uneventful, day and night. Finally got a side client project off my plate, and that made me happy.

It was kinda rainy today, kinda sunny, too. It was weird. We thought it was gonna hail, it didn’t

I missed the WWDC keynote because I didn’t configure some shit right. It’s on my “watch later” list on YouTube.

I put a bid on a smart sensor for the EcoBee on the electronic bay because EcoBee won’t sell you just one. I hope I win. It’ll be nice to have more accurate temperature readings for the HVAC.

I had an idea for a social network (I know) and jotted it down in my notes. Probably gonna forget about it immediately.

Posted this from my phone!

This was a good use of my time, right? Now I just need some vinyl stickers to make this happen.