We're talking WordPress databases and WordPress hosting on this chat with Brad Touesnard. What's page caching vs object caching? How should you move a database from local to dev to production? And what should be in your repo for WordPress?
Time Jump Links
- 00:42 Ooops I'm full stack
- 01:50 Guest introduction
- 08:26 Moving data from local to dev to production
- 11:02 New product coming soon
- 16:06 Page cache vs object caching
- 18:03 Is WordPress' database difficult to work with?
- 21:25 Sponsor: WooCommerce
- 22:43 Serverless vs WordPress
- 24:26 Scratching your own itch
- 29:29 What do I get?
- 33:49 What should my repo be with WordPress?
- 38:14 Sponsor: Netlify
- 40:31 Pricing and ongoing costs
- 45:41 Who's the ideal customer?
- 55:15 Second album effect
MANTRA: Just Build Websites!
Dave Rupert: Hey there, Shop-o-maniacs. You're listening to another episode of the ShopTalk Show, a podcast all about front-end full stack Web design and development. I'm Dave Rupert and with me is Chris Full Stack Coyier. Hey, Chris.
Chris Coyier: Hey! Oh, thanks. Yeah, relating it to my recent talk I published. Thanks, Dave. Maybe we'll talk about that in another show, although we kind of already have, in a way.
Dave: Circling back, circling back to the, oops, I'm full stack.
Chris: Yeah. Oops, I'm full stack. It feels that way to me, although, it's funny, when I was writing it, thinking about all the future of the front-end stuff. I was like, you know what? It's not that different of a feeling when, years ago, a decade ago, with no [laughter] ability to make websites at all, I was able to struggle my way through, you know, buying a domain name, pointing it at some hosting, and installing WordPress there. Isn't that funny? Does that make me full stack too?
Chris: I did all that by myself and I installed the theme and I customized that theme and I made and deployed a website with nobody's dang permission. It's just kind of funny that now we're like, "Well, I connect to a headless CMS with APIs." It's like, yeah, I mean that's awesome. I love that stuff too, but it's not that different.
Dave: But have you ever synched your production in local database?
Chris: [Laughter] No, I would just give up on that and just not do it at all.
Chris: Anyway, Dave is trying to get us to the meat and potatoes. We have a special guest this week to talk about -- there's no avoiding it. We're going to talk about WordPress a whole bunch. We have Brad Touesnard. Hey, Brad.
Brad Touesnard: Hey, guys.
Chris: Hey. Thanks for being on. Brad, you've generously sponsored this show in the past and we've talked about your products on the show many times, one of them being one that I think people are super familiar with, should all be familiar with, WP Migrate DB Pro. [Laughter] Such an awesome product. I've sung its praises many times, although we should point out this show isn't particularly sponsored. We just wanted to talk to you, Brad.
Brad: Yeah. Yeah, I really appreciate you guys riffing on our product over the years. It's really helped us out, especially in the early days when we were first launching. Chris, you helped me out a bunch. You talked about it on CSS-Tricks as well, so a huge thanks to you for that. That's been really, really impactful.
Chris: Sure. Well, you don't need to thank me. It's funny. I'm sure you have a better beat on what the competitive market is like for this kind of product but, to me, it doesn't seem like there are any. It's not like there is no other way to move a database. If we're setting the stage here, it's like your production database is the one that's got all the stuff in it. On a site like CSS-Tricks, we have a huge database just full of posts, pages, custom post types, forums, and all this stuff. Sometimes, when you're working on it locally, you just want all of it, or at least a good subsection of it, so that it feels like you're working on the real site.
On CSS-Tricks, it's not even optional. There is so much going on there where it has to reference exact pages with IDs and stuff that I can't install a starter DB of stuff and have that be a useful way to work. It's just not happening. I assume most sites kind of become like that at some point in their existence that you're just like, I can't just have a "Hello, World" post and do what I need to do developing this product. I need a way to get my production database down.
If you're a nerd, there's a bunch of ways to do it. You can SSH in and MySQL dump, get an SQL file, and then download that to your computer somewhere, and run the MYSQL commands on your terminal to put that database into place. Then the second you do that, hey, it's great but now my password is my production password and my password manager forgot it. More importantly, all my images are broken because they're pointing somewhere weird. There are just issues with that that people come up with.
It's like, wouldn't it be nice if you could just press a button and have your production database get pulled down and have it munge all that data necessary to work just fine on a local database? You all made that product, so good job. [Laughter]
Brad: Yeah, thanks. It's funny that you mention that workflow because the genesis of that plugin is actually for staging sites. I was working at an agency and I needed to move a WordPress site from my local dev environment up to a staging server so that the account managers and all the other people on the team could add content to that site. I did this manually a few times and hit my head against the wall pretty hard-- [Laughter]
Brad: --each time because you have to do a find and replace on the database, but then that screws up all the serialized data because, if there is any find and replace that affect the serialized data, it corrupts it. Then you have to rebuild all your widgets and it's just -- oh, my God. It was just such a nightmare, so I built a plugin to do the find and replace, handle the serialized data, and just kind of spit out an export of the database. I just threw it up on WordPress.org not thinking anything of it at the time, and it was only years later when I was looking to get into products to build my first kind of serious commercial product--
Brad: --that I took a look at that thing and I was like, huh.
Chris: Really? You were like, well, let's resurrect that thing. Why not?
Brad: Yeah. Yeah, that's pretty much how it went.
Chris: So it's true that there are all kinds of data munging that has to happen for this plugin to do what it needs to do, right?
Brad: Yeah. Yeah, there is. There's some weird stuff. It will actually un-serialize the data and then do a recursive find and replace, like, looking through. So, if it's an associative array, for example, it'll go right down into the bottom of the nest and try to find a URL to replace. If it's an object, like a serialized object, it will actually go through the object, go through all the variables in the object, and actually try to replace some values there as well. Yeah, it does. You know there's some weird stuff that will account for some differences between versions of MySQL that we ran up against.
Brad: If you're migrating from an old version of MySQL to a newer version of MySQL, there could be differences in between the two, so it'll actually handle those for you.
Chris: I remember at some point in WordPress history, it decided to support emojis better or something.
Chris: There were some weird years there where it felt like half and half.
Brad: Mm-hmm. That was the utf8mb4 transition.
Brad: [Laughter] As you do, right?
Chris: Yeah, I guess. I mean I'm glad that it supports emojis now, but it made for some difficult years, but I guess that's just software, isn't it? It's funny because there's a cost to it in that not just a cost that, if you want this pro version, you have to literally pay money for it, but many well spent if you ask me. But on a huge site, I might have to leave it running overnight to get it all. I think, to a nerd, you'd be like, "Yeah, but even on a huge database, like a MySQL dump is a few seconds." That seems crazy that you'd be willing to put up with that kind of time thing. I mean MySQL; write a script. Dump it. Find replace. Import it. This can't be the way.
But I'm like, I don't even care. I don't even want that technical debt in my life. I'm more than happy to let it go overnight and that's an extreme case, anyway. When I use it on ShopTalk Show's WordPress site, it runs in like two minutes or whatever. So, it's like [scoff]. Most people are probably in that boat anyway.
I'm only talking about my suck production down to local thing, but you're even saying that's just one use case anyway. Isn't it? There's this, like, you can use it to talk in between staging and production in local too, so there is a workflow there that it does. And you have this other workflow that I don't think I've ever used, but I guess some people probably do, where you could treat your local database like the source of truth and push posts up if you wanted.
Brad: Yeah, I mean you can do that. You have to be careful with replacing production in a WordPress site because, your site, maybe it has certain plugins installed that add data from the front-end, so a user submits a form. Let's say you have Gravity Forms installed. If someone submits a form, that's going to go into your database. Well, you don't have that data locally while you're developing.
Brad: If you push up your changes, you're going to overwrite the database and you're going to lose that data that the person submitted via the form. That's just one example.
If you're running an e-commerce site with, like, WooCommerce or another e-commerce plugin, you're going to have orders coming in.
Brad: You have to be--
Chris: Why do you even have that functionality at all built into it, that push function?
Brad: Oh, for staging sites, mainly.
Brad: So that you can just quickly set up a staging site. You just set up a barebones WordPress site on a staging server somewhere and then you push and away you go. There's your staging site.
Chris: You push production to staging so that it's kind of a mirror.
Brad: Yeah, or maybe you just built the site for the first time, so it's a new site, right? You need to set it up on staging to get it populated by the content team. Then you could use Migrate DB Pro to push to staging to set up the site on that staging server as well. That's probably a pretty common workflow, actually. That was pretty much the primary workflow when I worked at an agency.
Chris: All right. Well, that's that product, which you still make and it seems likely that it's a staple over at Delicious Brains. That's the name of your company that umbrellas over these products, right?
Brad: Yep, definitely. It's kind of our flagship product, although we have a new product that's emerging. [Laughter]
Chris: Yeah. Well, right. I think that's how I heard from you most recently.
Dave: Ooh. We need some Price is Right soundbites. We need some, like--
Chris: You've kind of stepped your foot into the hosting market for WordPress, but in a somewhat unusual way. It's not like you've become a hosting reseller or started your own kind of hosting company or something. It's this really kind of unusual middle ground between -- I don't know. [Laughter] Let's see. What's the best way to sell this? Maybe I'll just pass it off to you to kind of do it and then I'll pepper you with my own questions about it. The product is called SpinupWP, right?
Brad: Yes, like spin up a server, but spin up a WordPress server, so SpinupWP. Yeah, so if anyone is familiar with Laravel Forge, it's Laravel Forge but for WordPress instead of Laravel, right?
Chris: I've heard a lot of good things about Forge.
Chris: I think, once in a while, I end up talking about, like, "Hey, what is your local WordPress development setup like?" because I think, for a long time, it used to be like, "I don't know. A lot of people use MAMP," you know.
Chris: But that was like, for whatever reasons, it's just endless problems with MAMP, it seemed like over the years. It would work fine for a couple months and then maybe a new version. Who knows? Something would go wrong and it was always like, when it did it sucked.
Then there was like--I don't know--along comes Vagrant, Docker, and all these things, but they seemed like a pretty large step up in nerdery that not everybody was comfortable with. I had a few minutes where I was trying to deal with Docker myself. Then I'm like, "You know what? I think I'll leave that to the Docker nerds."
Chris: Then along came Local by Flywheel, which is what I use now, which is just right up my alley. It's this nice little GUI. You just press a little button, it does some stuff, and it just makes your sites work locally just fine. I'm kind of a big fan of that product. Every time I mention it, people are like, "What about Laravel Forge?" I feel like just as many or more people like that as an alternative.
Brad: Mm-hmm. Actually, I think you're talking about Laravel Valet.
Chris: Valet. Sorry. Shoot. Yeah.
Brad: Yeah. Yeah.
Chris: Apparently, they have lots of products.
Brad: Yeah, Valet is pretty cool. Yeah, it's just like a command line based local dev environment that uses Homebrew to install NGINX and the whole stack just locally on your Mac.
Chris: It's meant for local whereas this Forge thing is meant for not local, right?
Brad: Yeah, that's right. Laravel Forge is a cloud app that allows you to quickly spin up servers and get you hosting your Laravel app really quickly on your own servers. That's basically….
Chris: By "on your own," you mean--I don't know--AWS or something.
Chris: Or somebody else's.
Brad: Yeah, exactly.
Brad: I mean you could. I think it works with anything with an IP address. I know our product does. Anything with an IP address that you connect to spin up WP will connect to it and set it all up.
Chris: Try to run bash commands or whatever?
Chris: Yeah. Yeah.
Chris: Okay. Well, that's cool.
Chris: So, that's -- and people use Forge for WordPress too because it's PHP specific, right?
Brad: Yes, they do. A lot of people use Forge. But the problem was, like, we were using Forge for WordPress, actually, as well.
Brad: We were on a company retreat and we were discussing that. Someone said something like, "Oh, it would be amazing if there was Laravel Forge for WordPress where it would go the extra mile and set everything up for WordPress for you." Then we all said, "Hmm. Yeah, that would be nice if that existed."
Chris: Because the checklist isn't particularly small, right?
Chris: There's all kinds of crap you have to do that's specific to WordPress.
Chris: I think it's kind of funny. There are so many hosts like Flywheel that are like, that's all we do is WordPress. You're like, aren't you just a host though? What do you care what we put on your thing?
Brad: Yeah. [Laughter]
Chris: Why would you say no to a Craft CMS site? It's the same crap, isn't it? I think it's like, well, no, I guess, not really. There's so much specific that you can do just for WordPress that it makes sense to specialize.
Brad: Yeah, there is a lot to do. For example, you'd want to set up Redis for object caching. If you don't do that then, basically, every time you hit a WordPress page, it will run queries against MySQL every time, right? If you don't have that object cache, you're really taxing your server.
Chris: There's NGINX reverse proxy caching, which I think would stop some of that. There's Cloudflare could be in the way that would stop there. There's the old school caching plugins that make .hmtl files for your thing.
Brad: [Laughter] Yep.
Chris: I don't know why nobody uses those anymore, but they seem out of fashion. What was it, WP Total Cache Pro or something like that that would do that?
Brad: Yeah. Yeah.
Chris: There are other ways, right? But is Redis the hip, the best way, perhaps?
Brad: Okay, so that's a different caching layer, actually. That's page caching.
Brad: Object caching is, so like if you're querying the database for a post, for example, you cache that object, so the post object. Then, subsequent requests--
Chris: If it's page cached, it's still not making queries, right?
Brad: That's right. That's right. The page cache is another level above the object cache.
Brad: There are some pages, like in my example before. If you're running an e-commerce site, your checkout can't be page cached, right?
Brad: That one there, that page is going to really benefit from an object cache for any objects it needs to bring back from the database.
Chris: Fascinating. Fascinating. There are multiple layers of caching in the way because what you're really trying to avoid is any database queries at all, if you can.
Brad: That's correct. [Laughter]
Brad: WordPress's database is not exactly optimized for performance. The bulk of it is basically an object value store but in MySQL, right? If you're querying any data, any metadata, it can be very, very expensive, especially if your database is large. It basically gets exponentially slower as you query your metadata and your database expands.
Dave: I have a question. Has that been hard to work with for you and your tools, or does it make it easier because it's all in a post table? Do you know what I mean?
Brad: Yep. No, 100%. It has made it challenging.
Dave: Does it make it easier or harder?
Brad: Yeah, very challenging, but that's kind of our competitive advantage is coming up with creative solutions around that to improve performance, basically.
Chris: I've had that been a criticism when I talk about -- lately, I've been more and more interested in JAMstack kind of stuff. Isn't it cool, all these other ways that we can build sites? Headless CMSs are cool. Flat file CMS-y kind of stuff is cool.
I wonder, more and more when I am going to build a new site, I want to look at a wider spectrum of what are the possibilities of building that site. Where do I keep the data? What do I trust to do the auth? What's the hosting like? What do I prerender and what do I not? All these kind of interesting things.
I like the idea; I can leverage a lot of that stuff but still leave my CMS on WordPress if I want to. It's just kind of cool to know that that's a possibility because, honestly, the CMS abilities of WordPress are pretty good sometimes. You slap advanced custom fields on there or whatever and you have a pretty capable CMS, at least that I'm super comfortable with, in a way.
Then the criticism part comes in. They're like, "But why would you do that when it's not really for that, in a way, and the database structure is antiquated, or whatever, or just not up for that task in the way that other DBs can be?"
I don't know where I'm going with that, necessarily. Am I holding onto WordPress in a way that I don't need to all the time or is it still cool for stuff like that?
Brad: I mean it's pretty stable, kind of reliable. It's that old staple, you know. [Laughter]
Brad: Old rock, you know?
Chris: Yeah. If you do the smart stuff that you're doing with it, with your page caching, Redis caching, and all that, you get the speed too, yeah?
Brad: Yeah. Well, yeah, and it's been around for so long that I think people are confident in setting it up for performance at this point whereas, if you adopt some new thing, you don't know where the bottlenecks are. You don't know, necessarily, how to host it in a performant way. Yeah.
Brad: I think that's the big advantage.
Chris: Knowing the edges of technology - clutch. That's so important, isn't it? This is what my cofounder Alex always says, "If you can't tell me three rough edges of any particular technology then you just haven't used it enough." [Laughter]
Brad: Right. Right.
Brad: I feel like serverless is kind of that right now. It's the new, shiny. The promise is so nice. It's like, I don't have to worry about servers anymore.
Dave: There are no drawbacks, absolutely no drawbacks.
Dave: It works for everything.
Brad: Exactly. I haven't heard any -- like, it's very rare to hear someone talk trash about serverless at this point, and I'm just like, I don't know where the drawbacks are. [Laughter] I don't know.
Chris Enns: This episode is brought to you in part by WooCommerce. Today, I want to tell you about three exciting updates to WooCommerce. The first one is that WooCommerce version 3.6 shipped recently and added product blocks, so you can build rich pages with blocks in sites running WordPress version 5 or higher that show products by category, best-selling products, handpicked products, newest products, and products with specific attributes, and more. Super handy if you're using the new Guttenberg WordPress editor.
The second cool WooCommerce update is the new WooCommerce mobile app for iOS and Android. You can track your store to see which products are performing best, check revenue, view and manage orders, and get real time alerts notifying you about store activity like new orders or reviews. Be sure to search for WooCommerce mobile app on the iOS app store or Google Play today.
Finally, if you're running a WooCommerce site right now, you should check out the new dashboard for WooCommerce admin that the team released as a feature plugin. It's what's going to eventually be the default dashboard in WooCommerce, but you can check it out right now. You get all sorts of data and analytics on your WooCommerce store with a completely customizable dashboard, new reports, and a new activity panel to alert you to what's happening right now in your WooCommerce store.
If you'd like to see a WooCommerce store in operation, go check out the CodePen shop at blog.codepen.io/shop and try ordering yourself a shirt. The CodePen shop is running on WooCommerce and the shirt is one of my favorite shirts that I wear each week.
Our thanks to WooCommerce for sponsoring this episode of ShopTalk Show.
Dave: I asked this question on Twitter and Brian LaRue. Gees. What was his jobs over the years? He's big into serverless and a very big advocate for it. He was like, "You know, if you're doing real time communication or something like that, it's not the right tool. You need an RPC server, RTC, or whatever. You need a server that does real time communication and serverless is not that." It was like, ah-ha. Okay. Now we're starting to get to what it's good at and what it's not. Stuff like that or long tasks, obviously.
Chris: The real edges, though, because I think, at a surface level, serverless is happy to tell you, "Well, it's not for long-running tasks. It's only for tasks that max out at 30 seconds or whatever, 5 minutes, or something, it is now these days." There is a scale situation that it's good at. Scale being like--
Dave: Right, like 500 micro apps doesn't necessarily make your life better.
Chris: No, and testing can be hard, and sharing code can be hard, but those are the obvious ones. Those are like, just look at the technology and see what the rough edges are at a high level. I think it's probably coming down the line. What are they at, at a real level? If the world starts using this, the real edges will come out.
Anyway, we've really diverged from this SpinupWP thing. Maybe we need to--
Dave: Circle back. Circle back…. What I like about your company, or it seems, is one of your missions seems to be, let's find pain points of WordPress and fix them. Is that kind of like your mission statement? Is that how you're operating?
Brad: Yeah, kind of. [Laughter] Actually, I think it's a lot simpler than that and probably not as noble. Really, I'm just spotting opportunities. I'm just scratching my own itch, basically. The genesis of Migrate DB Pro, we already talked about. I was scratching my own itch there.
With Offload Media, WP Offload Media is our second plugin that we built. That was to get around a clustered environment. If you have WordPress set up in a cluster and you upload an image, let's say, it'll go onto the machine that you're hitting for that request. But if you're in a cluster, you've got multiple machines running your WordPress site and that image is not going to federate to those other machines. [Laughter] I created this plugin that would basically hijack that upload an send it to Amazon S3 right away so that your whole media library would be in S3 and off of the servers in your cluster. Again, just scratching my own itch there.
Then SpinupWP, the same thing. We were like, "We would really love for this to exist. Let's build it." I mean there was a bit more to it than that. We talked to potentially customers and stuff and made sure that we weren't the only people in the world that wanted it to exist. [Laughter] Otherwise, yeah, that's what we're doing. We're just trucking along.
Chris: Some of it is cost. Some of it is like, okay, to save some cost, what I'm going to do is just use AWS directly because there are all the power in the world there if I want it and it's not particularly expensive, or I'm just going to get a raw DO box, DigitalOcean, and just use it for my WordPress site. I don't necessarily need help doing it. I would need help because I don't know what I'm doing, but somebody who has done this before, hey, it's a box. You install PHP on it. You install MySQL on it. You install WordPress and away you go.
Is that -- [Laughter] …that's a lot of responsibility and you're probably not nearly getting out of that box what you could be. SpinupWP, that's what it seems like to me. It's like, no, we'll spin up that DO box for you and just do a killer job doing it. We're going to make this thing scream for you.
Brad: Yeah, exactly. Kind of the other part of it is, we could have built an app that just did that, just spun up their server and kind of set them off to sea. [Laughter] You know?
Brad: Good luck, fella.
Chris: Yeah. [Laughter]
Brad: But the problem is, people have anxiety about maintaining their server, like, am I going to be able to keep up with security updates? What if? What if? What if?
Part of the mission for SpinupWP is to help educate people on how to maintain their server. One rule that we've kind of made for ourselves is to never push updates to people's servers without them knowing about it. I find that that, in the past, has largely been the cause of any downtime I've had is when I'm on a host and they do something to their infrastructure. Then something blows up, right?
Brad: Because with SpinupWP, you're bringing your own server and you're kind of owning it more, we feel like let's be very careful. If we need to upgrade something on a server, we're going to give them an upgrade routine that they can run at their preferred time when they want to do it, when they have time to do it, when they have time to babysit the server for a few hours afterwards. If there's a PHP security thing, any security updates automatically get installed on the server, so we definitely do that. But if you want to update your PHP version from 7.3 to 7.4 when it comes out, then you can just do that in your control panel just by choosing it from the dropdown.
Chris: That makes a lot of sense. SpinupWP, just for people's understanding, is not something you install. I don't NPM install SpinupWP or whatever. SpinupWP is a cloud thing that you have. It's a Delicious Brains product. You go have an account on it or whatever, but it's a website. It's a control panel thing. I pay for it and I use it to spin up something like a DigitalOcean box, but that DigitalOcean box is mine. I have to pay for that.
Brad: Yep. Exactly.
Chris: In a sense, here, it adds a little bit of complication because I'm paying two companies. I'm paying you and I'm paying DigitalOcean.
Brad: Yep. That's right, so a tradeoff.
Chris: That might be like, meh, I don't love that. But what you get out of it is a heck of a lot too. Whatever. It's not a marketing show, but I'm interested to hear. What's the whole checklist? What do I get?
Brad: [Laughter] What's in it for me? Yeah, first of all, the stack that we set up is going to be tuned for WordPress, so we're running NGINX. The object cache is set up. Your page caching is set up. All that is set up right out of the box.
Brad: You get Let's Encrypt SSL Certificates automatically configured for your sites that you deploy. One thing that's really cool for developers is a git-push-deploy, so you can set it up so that, when you push to GitHub--your GitHub repo or any Git repo, really; any Git repo that supports webhooks--the webhook will fire back to SpinupWP and will pull the latest from whatever branch you've configured as your deploy branch. That's pretty cool.
Chris: That seems massive to me.
Chris: All this super-tuned site and we're talking a $10 a month box on DigitalOcean, right? We're not talking about a massive--
Chris: Right. Not that $10 is trivial, but it's kind of trivial. It's nowhere near a managed host kind of thing.
Chris: HTTPS just comes along for the ride.
Chris: Backups come for the ride. You didn't mention that, but it looks like it says that here. That's pretty rad, right?
Chris: You just get that for "free."
Chris: Then this git-to-deploy thing, which is kind of a big deal in that I don't think you necessarily expect that in WordPress land. I don't know of any. I think maybe -- what's the one that just bought Flywheel? Weren't they in Austin for a minute, Dave? What's that one?
Dave: WP Engine.
Dave: WP Engine, yeah.
Chris: They're kind of Git focused, in a way, or at least they had some stuff going on, I think, right?
Chris: But it's not particularly common, right? It's not like Netlify. That's just a thing. It's just, you push and it deploys. That's a first class citizen. This gets stuff.
I'm seeing this more and more to the point where I hope and I think you're a part of this, so congratulations--golf clap--git-push-to-master should be the way to deploy. You know? I'm glad that it is in this case because there are plenty of things that have been middleman in between that and those apps have been great. Deploy-bot or deploy-hq and Buddy, who sponsored the show, I use Buddy. We're using Buddy right now to our deployment for ShopTalk because it's kind of this thing that can watch a GitHub repo and then you can teach it about SFTP and you can connect the two things that way. That's great. I'm glad that that exists. They do so much more than just that. It's like, it'll be a product that exists forever, but it's like the host should do that.
I feel like any host for any product, they need to look at this Git thing and be like, "This is what our customers need. We're the hosting destination. We should help our customers get their code to our servers." Git is clearly the way forward with that. It's weird almost that it's taken this long to make that a first class citizen in the world, I think.
Brad: Yeah. We love Buddy as well. We use Buddy. Our git-push-to-deploy feature in SpinupWP is pretty basic. Buddy allows you to have a pipeline and do builds.
Chris: Yeah, exactly.
Brad: If they fail, it won't deploy it.
Brad: There's really a lot to it, but ours is just kind of a simple git-push, which is probably good enough for plenty of use cases.
Chris: I do have a question about that. In the past, I've always been like, what should I make my Git repo anyway with WordPress? I need to have this discussion with somebody who knows what they're talking about. I've been like, should it just be the theme because that's mostly what I author? Then that's a tiny little Git repo and I can plunk it right where it needs to go. That seems really small and efficient, but that's not really enough because themes often depend on plugins. There's just more to it.
In the past, I've been like, you know what? Screw it. I'm going to make the whole root of this entire site on Git and then everything is here. If I need a static file at the root, I have it. It's in the repo. Everything I need to run this site is in the repo. It just felt good to me, but it also felt a little heavy handed, like, oh, my God; to update WordPress, I need to do it locally and then push WordPress core files to a repository. A little much.
Then I've most host recently where I can't. The host doesn't even let you touch core WordPress files, which is good. It's like a security thing. I was like, well, I have to change what my Git repo is now, I guess.
The general consideration is, pus the WP content folder; make that the root of your Git thing. Is that industry standard? What do people do?
Brad: [Laughter] Yeah, I mean there are plenty of different opinions on this. This is almost tabs versus spaces level.
Chris: Really?! Oh, wow. I didn't realize I was stepping in it, but it makes sense.
Brad: Yeah. Okay, it's not that big of a deal.
Brad: Nothing is that big. No, it's a pretty big deal. I think the way we recommend at this point is to use Composer, actually, which is the PHP package manager, to manage your WordPress plugins and then you can commit your Composer file, which would include the version of each plugin that you've installed via Composer.
Chris: Okay, so this keeps your plugins themselves, like the actual files, out of the repo.
Brad: Exactly. Then, as part of your deploy process, you would run Composer install and Composer install would update any plugins that have changed versions or install plugins that have been added to your Composer.json.
Chris: Does the SpinupWP work with that approach? I'm sure it does.
Brad: It absolutely does. It does.
Brad: You can run a script, a post deploy script. The only rub there that you have to be careful of is that, if your post deploy script fails, it's going to be broken. Your site is going to be broken, so you have to really be conscious of that.
Chris: Yeah, what's the emergency button look like then? I feel like, on my traditional host where I can login with FTP, I always have that as an emergency situation, like, "Oh, I'm just going to FTP in and I'm just going to literally delete that folder that's a busted ass plugin and WordPress will deactivate it for me. Then maybe I'll hand drag a fresh copy of that plugin back into place and cross my fingers that works." You know?
Chris: I have that available, I suppose, this way too, right? If it's a DigitalOcean box, I can get into it somehow.
Brad: Oh, yeah. Exactly. You have full SSH access. We've got two classes of SSH users, so each site that you create has its own site user and it's restricted to the site's files, so you can't access the NGINX configs on the server or anything like that. Then you've got what we're calling pseudo users, so those users have full root access, so they can run commands as pseudo. Yeah, they've got the keys to the castle, right?
Soon after launch, we actually implemented that because we realized; a bunch of people were telling us, "Hey, hey, yeah, this isn't going to work because our users need to be restricted to the site." [Laughter] "We're hosting a bunch of client sites, so we need to give them access to their sites but not other people's sites."
Chris: Yeah. It's not once site per box, right? One box can have a bunch of sites on it is the idea?
Brad: That's right.
Brad: Exactly. Yeah.
Chris Enns: We've talked a lot about Netlify Build before, the Git workflow for Web development where you can build, deploy, and manage modern Web projects super easily with Netlify. They just launched something new they're calling Netlify Dev where you can run their entire platform right on your laptop or your desktop, I suppose, too. You can preview it all: site generation functions and edge logic. Imagine the productivity boost of being able to locally test your site generator, API integrations, serverless functions, and edge rules all in a single development server. That's Netlify Dev, a powerful way to build and test modern Web apps on your local machine.
With one simple command, you can install Netlify Dev to use on any project. That'll spawn a fully local environment of Netlify Dev that automatically detects and runs your site generator, makes environment variables available, performs edge logic and routing rules, and compiles and runs cloud functions. The extra bonus is that you can even stream that live, so Netlify Dev takes hot reloading to a whole new level, allowing you to actually stream your dev server to a live URL, which is great for collaborative development. You can now share your work as you work and get instant feedback. I could be working on my site. I could have it open in a browser, open on my phone, see the changes live as they're happening, but you could also have it wherever you are in the world open on your browser or on your phone. Whenever I update something, the site will update, and you'd be able to see it.
Netlify Dev automatically detects common tools like Gatsby, Hugo, Jekyll, React Static, 11ty, and more, providing a zero config setup for your local dev server.
Local testing works too. It all comes with Netlify Live, a hosted service that continually runs your dev command just like you do locally, watching for changes. The result is an instant preview you can share with your entire team with live updates as code and content changes.
Like I said at the beginning, Netlify Dev installs with a Netlify CLI. Create new sites, set up continuous deployment, and push new deploys right from the command line. Netlify Dev is just the beginning. Take your local developments to Netlify Build, power collaboration through Netlify's Git-based workflow with deploy previews, branch testing, and more.
Check out Netlify Dev at Netlify.com. Our thanks to Netlify for sponsoring this episode of ShopTalk Show.
Chris: It's as cheap as $9 a month. You use this in perpetuity, in a way, right? SpinupWP provides all this stuff for you long-term, right? It'll help you upgrade your PHP version when that's ready, it does the backups for you, it helps you with the Git stuff, and all that. This is a thing that, this isn't a "help me spin up a box and leave" kind of product. You just leave it. You leave it managing your sites long-term, right?
Brad: Yeah, exactly. Exactly. If you cancel your subscription, you can totally do that and you can totally walk away with the box. It's totally yours. But there are a few things that will stop working that we're managing via the app. For example, Let's Encrypt Certificates will stop renewing.
Brad: You'd need to figure out a way to set that up. Another thing is backups. We're handling the backups via the app too, so the backups stop running.
Chris: That is kind of neat. Don't cancel or figure out your crap before you cancel. If you stop paying your host, the implications are a little worse than stopping paying you because your site will stop.
Chris: This, your site doesn't stop. Not that we want to encourage people to do that, but it's just interesting because you're not the host. You're just this….
Brad: Yeah, that's an advantage that we're totally proud of that we have kind of resisted the urge to have platform lock in, you know, lock in our customers, because it's so common today, right? You sign up for something and you're just locked into it and you can't get out no matter how terrible it is. [Laughter]
If someone is really not enjoying our service, it would be nice if they could just leave. Maybe they'll be back later. Who knows, right?
Chris: Yeah, perhaps. But it also means that, because you're this middleman, that let's say you want to move your DigitalOcean box to an AWS situation. You can also do that, right?
Brad: That's right. Yeah. Exactly.
Chris: If it's cheaper or something else comes along that offered some cool stuff or whatever. The offerings of those companies, they're a little bit more commodity, I guess, but they have their own interesting reasons why you might use them, don't they?
Brad: Who has their own interesting reasons?
Chris: Like a DigitalOcean, right? The fact that you've picked them to spin up your box on, you get some stuff from DO too.
Brad: Yeah. Yeah.
Chris: I don't know if they do WordPress backups. Ultimately, the speed of the site a little bit depends on them too. They have their own uptime and things like that. If they had a major outage, it would affect you, for sure, still.
Brad: Yeah. Yeah, absolutely. Absolutely, and I mean that's one thing that we get criticized is, you've got all your eggs in one basket because you've got the one server; your site or sites are running all on that server and, if it goes down or DigitalOcean goes down and you're on DigitalOcean, then you're down. There is no failover. Then we get requests from people, like, "Can you guys work on setting up a load balancer in kind of a clustered setup so that there's more redundancy?"
Chris: You don't off that just yet, yeah?
Brad: we don't offer that just yet. We're kind of on the fence about it, actually. The thing is, if DigitalOcean goes down and your cluster is on DigitalOcean, then you're going down. [Laughter] It doesn't matter if you have a cluster.
Dave: That's a good point.
Brad: Nine times out of ten, the downtime is because of a networking issue or something at the data center.
Dave: Like a DDOS on DigitalOcean, CloudFront, or something like that.
Brad: Or the network switch update caused a glitch and cascaded through the whole data center. Who knows, right?
Brad: There are so many things.
Dave: …that happen, yeah.
Chris: The thing that you put in place to prevent this is the cause of it.
Brad: Yeah, exactly. [Laughter] Exactly. There is so much abstraction now. These cloud servers are amazing. It's amazing that you can just spin up a little VM somewhere right now and it abstracts the whole infrastructure that's involved.
I actually started a Web hosting company way back in 2002. We literally drove to the data center with the physical servers in our car and put them in racks in the data center. That was 2002. Here we are in 2019, and I haven't touched a server in many years and I'm happy that I haven't.
Chris: Who is the ideal customer then, in a way? Are you kind of hoping this is a for everybody kind of thing here?
Brad: No, definitely not. We don't. Basically, you have to have a technical mindset, l like to say. Maybe you're a designer, but you right front-end code, you're used to debugging, and you're really enthusiastic about learning. Maybe you want to learn about how to manage your own server. That's totally, totally something for that type of person. You don't have to be a sys admin, a back-end developer, or anything like that. I think, if you have the right mindset, you can set out to learn how to run your own server.
We have a tutorial series that walks you through all of the steps to do it manually. When someone wants to learn, I think that's the best way. I actually don't recommend people use SpinupWP that are just starting out with managing their own server because it's like learning about Git by buying Git Tower. [Laughter] You're not going to learn how Git works by using a GUI app, right? You have to use the commands to understand it.
That's what I recommend; people go through the tutorial and learn that way. Then, after they've done that once, they kind of get it. Then they can switch to SpinupWP and they save a ton of time. It's just way more convenient to create a new site on their server and stuff, right?
Chris: Yeah. What's the DNS scene like then? You buy a domain. Where do you point it? You're still following whatever DigitalOcean's recommendation is then?
Brad: When you provision a server, you'll get an IP address. Then you just plug that into wherever your DNS is hosted, usually. We're not doing any DNS service.
DigitalOcean does have a DNS service. Personally, I prefer Cloudflare's DNS servers because it's, I think, probably the fastest out there and they have a great API as well. We have plans to actually--
Chris: Buy the domain name. Point your name servers at Cloudflare.
Chris: Point your A-records at the IP address that you got from DO.
Brad: Yeah, exactly.
Chris: Sweet. That seems nice, too, and you'll get stuff from Cloudflare too that, as good as your job you're doing with SpinupWP, there's stuff at the Cloudflare level that you can't do, which they can.
Brad: Oh, yeah. We love Cloudflare. It's great. We plan to integrate with them so that you can manage your DNS from SpinupWP without having to log into your Cloudflare control panel.
Chris: Sure. Bring that little dashboard over. Sure.
Brad: Yeah. Yeah, and then we'll be able to do cool stuff though, too, like if you're creating a new site and you've got your Cloudflare account hooked up. We could automatically create an A-record over there for that. It'll update your A-record.
Chris: Oh, I see, so you don't even have to touch it then. It's just automatic.
Brad: Yeah, and then let's say you're moving a site. We don't have a move tool at the moment, but imagine we did and you're moving your site from one server to another. We could automatically update that DNS record for you when we're at that very tail end of that move. That would flip everything over to point at the new server. There are definitely some really cool use cases for having access to the DNS, for sure.
Chris: Sweet. I guess one of my favorite things you said just recently was that technical mindset thing, which is kind of a fascinating way to think about it. It's not like you have to be a DevOps person or be a back-end person. Whatever. You're like, even if you're a front-end person, if you have a technical mindset in some way, you can perhaps do this.
Brad: Yeah. I think it's also the learning mindset. If you're keen to learn, I think that is the big ingredient because you can kind of work backwards from where you want to end up. If you want to end up managing your own server, okay, well, then you need to know how to SSH into a server, first of all. Okay, I don't know how to do that. How do I do that? [Laughter] Then just go backwards until you get to the place where there is something you know how to do. Then proceed from there.
Chris: Think of how many doors that opens up overall. That really has nothing to do with hosting your WordPress or anything. That's kind of like, if you're just up for that kind of stuff, that's a door opening mindset that you can have, for sure.
I don't know. I've never used React, but I can follow this tutorial. Oh, I kind of get it. Oh, here are some sports that totally have no idea what's going on but, eh, maybe I will the next time I play with it.
Brad: Yeah. I think it's also a mindset that applies to so many parts of life. For business, for example, you might be a great developer and you want to launch the next hotness of Sass app or something, right? Well, if you want to do that, you still have to have a learning mindset to learn marketing, to learn maybe business, how to forge business relationships, how to do customer development. There are so many things that go into starting a business other than actually building the app. You're going to have to learn all of those things if you want your app to be a success.
Chris: Yeah, I like that. It's not only technical; your learning mindset helps you in an entrepreneurial spirit as well. I'm sure it helps in everything.
Chris: I'm sure there are some doctors sitting around now being like, "You know what makes a good doctor? A curious doctor." Whatever.
Chris: An openminded mindset will serve you in lots of different ways. Speaking of learning, did you bring--? I'm sure this second product has lessons in it. It's not your second product. I realize you have the offloading product and don't you have an SES product too? I think I use that one.
Brad: We do.
Chris: We use that on CodePen's blog for some reason because the server that it's on, it's just easier for us to use AWS for some things.
Chris: I'm sure it's a smaller market for those things, Right? WP DB Migrate Pro.
Brad: Yeah. I don't know. I feel like the market size for SpinupWP might be bigger than all of them.
Chris: Yeah. Yeah. Yeah, but like the SES one is smaller, I think. Maybe not.
Brad: Oh, yeah. Yeah. Yeah.
Brad: I think it definitely is.
Chris: There are add-ons and stuff to the DB Pro, right? It's a big ball of stuff. In fact, I used one that I never used the other day because I have this situation where I want contributors. Let's say you have a WordPress site and you want people to work with you. It's not that you don't trust them, but it's irresponsible of them. It's irresponsible of you to give them a production database because a production database might have a bunch of users in it, their real email addresses, their real purchase history, and stuff like that. That's not cool, you know.
Brad: Yep. Exactly.
Chris: There's an add-on to anonymize stuff that works pretty well with DB Pro.
Brad: That's right. Yeah.
Chris: Which is cool.
Brad: We built that in the spirit of GDPR, all the GDPR snafu.
Chris: Oh, right on. My use case felt kind of perfect for it. I want collaborators, but they shouldn't have -- even if they worked for me, like employees probably shouldn't have the entire user database of a production site, in some cases, you know.
Brad: Yeah, I've thought about that too. I get apprehensive about that when I call customer support at my bank or anywhere else. I'm like, do they have access to everything? [Laughter]
Chris: They definitely do.
Brad: Yeah, exactly. Exactly.
Chris: But then it seems like you make this new product and it's not DeliciousBrains.com/SpinupWP. It's got its own domain. That type of stuff smells to me like a learning, like we're going to do this differently based on lessons from the past kind of thing.
Brad: Yeah, the anonymizer is interesting. I wonder if that's something that people are actually using. We're not even actually tracking anything about that add-on. It could be widely popular and we have no idea.
Brad: This is the first recorded instance right now.
Chris: No, that embarrasses me sometimes too where somebody is like, "What's your daily active users?" You're like, "What? I have no clue."
Brad: [Laughter] Yeah.
Chris: I don't even know what our revenue is. Don't ask me any more questions.
Chris: Anyway, what's on your mind over there, Dave?
Dave: It sounds like you all need some charts and graphs.
Dave: That's the Dave Rupert specialty.
Dave: No, I think it's neat. Have you ever heard of the second album effect or whatever? Musicians, it's really hard to make a second album. Are you guys familiar with this concept?
Chris: The sophomore slump or whatever.
Dave: Yeah, sophomore slump. Yeah.
Brad: Is that why Madonna and David Bowie, they reinvented themselves for every album or something?
Dave: Yeah. Maybe that's part of it is reinventing. You have a product, I feel like, a household name for WordPress developers, WP DB Migrate Pro. Did I do all those in the right order?
Brad: Close enough.
Dave: Close enough; 50% accuracy. I like the SpinupWP because -- I don't know. In the past, I've ran -- you set up. You can set up a WordPress. People do it all the time. But do you set it up great? Do you set it up poorly?
Chris: [Laughter] Isn't that what you've done? Now you've seeded doubt with all of us that I will never do my own AWS WordPress site because I'm probably going to do it wrong.
Chris: You've just made us all doubt ourselves.
Dave: Not like wrong, but it's the reason I use MAMP on my local host for everything. It's just like, okay, it's always the same every time or it has some predictability. I just think this is kind of cool that you're doing this for production environments. It seems like you're solving--
Back to the second album thing. You're solving a problem that people are having. You said you're solving your problem, but I think a lot of people are having this problem. I think, particularly in the WordPress shops that are doing 5, 10, 20 WordPress sites a year or something like that, this is kind of optimal for them because your updates, your everything, I guess it's all together. All your environments are the same and that's kind of a good thing if you're doing a lot of different sites, projects, or whatever. Your environments, at least, have some consistency and that eliminates just a major wildcard for WordPress development. [Laughter] I think this is very cool and I wish I had the opportunity to use it. I guess I actually could if WordPress comes across our plate, but this is very cool.
Brad: Here's something you could try that I just remembered when you were talking. Remember, I said that SpinupWP works with anything with an IP address? Well, technically, you could spin up a VM locally on your own machine with just Ubuntu.
Chris: Expose the IP.
Brad: Expose the IP and point SpinupWP at it and have it set up like a local VM on your machine with the whole stack. Yeah, if you kept the IP exposed, you could keep going. You could keep adding sites and managing your VM that way.
I had problems doing that, personally, because my stupid router has a block on port 22, so the SSH port, it won't allow incoming connections on that port. [Laughter] But I have the worst router in the world, so that's probably not a problem for everyone.
Chris: I mean it's fun as a thought experiment, but it's also -- isn't more and more the spirit of local development to be like, "We need to make this more like production," in that if it's fake production, that causes problems. I mean I'd say maybe once a month, we have a problem like that at CodePen in some small or big way. This isn't behaving on production like it is in local development. Lately, we've been trying to fight that in new and interesting kind of ways, one of them of which you'd probably be interested in because it's like we had to write our own WP Migrate DB Pro - kind of. Like, let's script it ourselves.
We can't use it because our database is entirely totally bespoke to CodePen, like a lot of apps in our case is. But in the past, we've been like, "Well, whatever. We'll just do a dump of the DB once in a while." I bet the DB I was using up until a few weeks ago was two years old, believe it or not. You know it doesn't matter. It's mostly fine. If I need a table here and there, I can bug DevOps to just export me one table and I'll import it. Fine. But it's been more and more of a problem because it's like--I don't know--we have some new areas of the site that have new data, yadda-yadda-yadda.
Finally, Alex was largely doing this over on our team. Scripted up his own fancy DB Script that has a YAML config file. It's like the UI you built for WP -- we almost need an acronym. I've got to say the whole thing.
Brad: We actually have an acronym internally. We just say MDB. It's very simple.
Chris: MDB. Gosh. Okay. MDB.
Chris: You know how you click which tables you want in there, like I want this one, this one, this one and, usually, you just grab them all, but sometimes you just want to limit the scope of what you're pulling. It's just a YAML config of that, but the YAML config is much deeper because it's like, do you want to wipe your DB first or not? Do you want to wipe this table first or not? Which users do you want? Should the users follow each other for various reasons? It's all these little, special things. The idea is that, when you run it, it takes a minute or two, not forever, and the final database is 100 megabytes, maybe, not even, rather than a terabyte or whatever the production CodePen database is. Sometimes you have to write your own little thing like that, you know?
Brad: Yep. Absolutely. Yeah, that makes a lot of sense. I think, yeah, that's interesting. Are you basically excluding all the pens, basically, the data, pretty much?
Chris: No, those come too. You say, "I want this user." It gets everything associated with that user: their pens, their projects, their comments, their activity. It's a little complicated how it has to snake through the DB and get all that stuff.
Brad: Right. Okay.
Chris: But it kind of just has to.
Brad: You do it by user, so this user and that user and all their associated data, but you limit it to….
Chris: Yeah. What would be too bad is if you grab them and, like, I'm following a couple hundred users. It'd be nice to go get those users too so that my experience locally is kind of like what they see because it's social in that way. But we're like, "You know what? That's a little too crazy," so the users that you pull, we just force them to follow each other instead. Then you get that social experience without having to snake too deeply.
Brad: Yeah. I'm looking forward to the first bug because of that.
Brad: It turns out, why am I following everyone? Eh… [Laughter]
Chris: Yeah, and we don't have to do that much data munging on the way back in like you have. It's pretty much we just pull it. The way that it works is, I think, SSH into the box runs a MySQL dump, grabs the file, pulls it locally, and then runs it. It's fast in that way.
Chris: But there's so much data that it's still a couple minutes to run. I don't know why I ended up telling that story. I just did for some reason. Oh, because we were talking about having local be more like production. That would be kind of cool if you did this with SpinupWP, point it at yourself so that what you're running locally is super very, very similar to the ultimate box. It wouldn't be. It's not a Docker container or whatever, so I guess it's not super exactly right.
Does DO give you local boxes to spin up? I wouldn't doubt that they do.
Brad: Well, so DO does have Docker, but we are just using straight up VMs. You just get a Linux. You get a Linux VM on wherever you want and then you point Spinup to add it and it'll do the rest.
Chris: It's still a VM. It would still be identical.
Brad: Yeah. No, it would literally be as close as you can imagine because DigitalOcean's VMs are virtualized environments as well, right?
Brad: It's not like when you get a $5 droplet at DigitalOcean. You're getting your own physical machine there. [Laughter] That would be amazing, wouldn't it?
Chris: I'm sure if you're willing to pay for it, they'd be glad to help you out.
Brad: Oh, yeah. Yeah.
Chris: Starting at $22,000 a month.
Chris: All right. Well, any final thoughts, Mr. Dave?
Dave: No. Again, thanks for all the work you do. I think you make WordPress a lot easier to work with. I'm just barely getting into this client's deployments, scale, all backing up DB, so thanks for all the work you've done. I think it's definitely made my life easier, so thank you and thanks for coming on the show. For people who aren't following you, Brad, how can they follow you and give you money?
Brad: Well, you can find me on Twitter. I'm just @BradT. I don't really blog anymore, so forget that.
Brad: DeliciousBrains.com and SpinupWP.com is kind of what we've been talking about. Yeah, that's it.
Dave: Awesome. Thank you and thank you, dear listener, for downloading this in your podcatcher of choice. Be sure to star, heart, favorite it up. That's how people find out about the show. Follow us on Twitter, @ShopTalkShow, for tons of tweets a month. If you hate your job, head over to ShopTalkShow.com/jobs and get a brand new one because people want to hire people like you. Chris, do you have anything else you'd like to say?
Chris: Oh, ShopTalkShow.com.