440: Serverless, Local Database, Edge Functions, and Using WordPress Serverless
Let's talk Serverless - including using WordPress Serverless, how to solve a problem like local databases, and edge functions with Netlify and beyond.
Guests
Chris Coyier and Dave Rupert
Time Jump Links
Transcript
[Banjo music]
MANTRA: Just Build Websites!
Dave Rupert: Hey there, Shop-o-maniacs. You're listening to Episode 440 of the ShopTalk Show. I'm Dave Rupert and with me is Chris Coyier. Chris, we are 440 years old today. How are you feeling?
Chris Coyier: I feel about 440. Actually, I've been feeling a little bit better, honestly, because I've had a rough year with the arm breaking and then the back trouble, and then I get hit by a car, and yadda-yadda. The last thing wrong with me is my left foot and I'd say it's 70% healed. I'm back doing some mask-wearing personal training and stuff and feeling good about that.
Dave: Okay.
Chris: I need to fix the diet again. That was good for a while, and now it's off again. Ah! It's always a struggle but, in the grand scheme of things, I'm feeling pretty darn good today.
Dave: You know, I don't understand diets. What's the deal with diets? [Laughter] Over the pandemic, I've basically given up diet Coke. It's basically gone, right?
Chris: You're a hero.
Dave: Well, I mean, I'm not going to get the virus just for a diet Coke. Listen. [Laughter] I like diet Coke, but I'm not going to get the virus for it.
I'm having smoothies for breakfast. I take the dog for walks in the middle of the day because--whatever--stress. Then I don't drink beer that much. I've just basically given up on it because the calories per fun are so out of whack. As a parent, you're just like, "Well, I'm not going to get fully blitzed or whatever because I also have to take care of children."
Chris: Yeah. Tricky.
Dave: So, the calories per fun are out, and so it's like two beers a week or something like that. Really just not much at all.
I've eliminated all these vices, if you will, and the things that people are like, "Oh, that makes you fat." But I'm not seeing any results. [Laughter]
Chris: [Laughter]
Dave: I don't know what's going on. Yeah, got a Vitamix. Eating smoothies for breakfast. I just don't even know what's going on, man. You think I would have some return on this investment but it's just not working out.
Chris: I'm sorry, man. That's all right.
Dave: Yeah, well.
Chris: Some of it is--
Dave: I don't understand.
Chris: Maybe you'll just delay the year that we all inevitably get cancer or whatever, you know.
Dave: Yeah.
Chris: There's some of that, too.
Dave: Yeah, I push it all back just a year or two.
Chris: All right, well, you know you mentioned serverless before the show. That's always been something I kind of keep my eye on. I don't think I'm a professional in it by any means, but it felt right when it started to be a little industry zeitgeist thing, like, "Oh, this is clever," and it really hasn't gone away. In fact, it's nothing but gotten bigger and better over the years. I know it's a little bit of a buzzword, but I don't know. Are you dabbling recently or what?
Dave: So, I'm building a little product, a little side project product thing. I'm kind of buying into JAMstack, right? If I would have built this in Rails, I would have been done and done a long time ago. [Laughter]
Chris: [Laughter]
Dave: So, whoops. But I think this idea is kind of getting into, how can I future proof myself? I think I've maybe said it before on the show. I've built something in Rails and it's cool, but then all the development kind of lands on my shoulders. Then spinning up the environment is hard, and stuff like that. I found with sites like 11ty sites and Nuxt is very close to 11ty or Jekyll, or something like that, in its structure.
Chris: Really?
Dave: Yeah. Just the file structure is very similar.
Chris: I guess you're right.
Dave: Even the sort of metaphors of pages and stuff like that that it's using or in layouts.
Chris: Yeah, it's like put stuff in folders and pages and we'll build the site structure based on that. I guess that's kind of how they both work.
Dave: Yeah.
Chris: I didn't really think of them as connected like that, but they are.
Dave: No, they're pretty similar, and so I've had success, I would say, with my coworkers, mostly in the design camp, success with getting them spinning this up locally and working on this little thing we're building. Then it comes time to -- we're deploying to Netlify just so we can make branch deploys and previews and all this stuff. You get a lot of benefits being in the JAMstack ecosystem.
Chris: I think so.
Dave: Then it comes to the data part, so we want to add stuff via forms. When you get into that, it starts getting more difficult because the architecture I'm kind of landing on, and I can tell you a 2,000-word story about my journey into GraphQL, but I'll pause for a minute just to say what you kind of realize is, in your Nuxt app, your Nuxt app needs to call some kind of server that has all the API keys, or something, that only lets the Nuxt app or whatever call it. Then it will spit out and call the actual database. There's sort of a chain that happens.
Chris: Yeah. You write your own little crud functions. Yeah.
Dave: You write your code functions. And so, in a Netlify world, those crud things happen on the serverless in your Netlify functions folder.
Chris: Yeah. In your functions folder. Yep.
Dave: You end up writing a bunch of functions, and this is kind of my first question. It's like page get all or get all pages. What would be a better function? Get all posts. There you go. That's the blog. We'll do the blog.
Chris: Right.
Dave: Then get all post comments. You pass it a post ID or something like that, or get all--
Chris: Yeah.
Dave: Get one page, or whatever. Whatever you want to call your functions, they're like that. I'm working in this and I'm repeating myself a lot, doing a lot of handler calls and then I'm just doing one thing, fetching one thing from the database and then spitting them back out. I kind of was like, you know, is the concept of doing crud like a Rails controller inside your serverless function? Is that a thing or is that a dumb thing?
Chris: Ah, you know. I mean if you're working with Rest APIs, I don't know that you have much choice. You've got to write some kind of handler for the things. But GraphQL kind of opens it up a little bit in that you don't write git all posts .js file. You just have one GraphQL handler and it just executes whatever GraphQL ask that you have of it.
Dave: Right. But don't you have to write resolvers and stuff like that? You're in the client. You're writing the GraphQL. Or is it in the API data layer that you're writing the GraphQL?
Chris: I think it could go either way, but it sounds like you're doing -- it's the client-side stuff, mostly. Yeah. Are you trying to get it to do it on build?
Dave: Doing it all client-side mostly, yeah.
Chris: Yeah.
Dave: Mostly just client-side.
Chris: I think it just depends a lot on where the data is going and what service it is and how much that service wants to help you with that data. There is Contentful and Contentful is very UI-driven. You can scaffold out your data on Contentful and be like, this is what my data is like, and it gives you these GraphQL endpoints. In that case, you're not hitting a Netlify function at all, really. You're just doing it right -- you're skipping right -- hitting their APIs. Probably.
I don't know how the API key thing works there. There must be something. Maybe you do need one little middleman to scoot through it. I don't know.
Then there is stuff like -- what's the whole list of them? Hasura, Fauna, and Amplify's data store thing.
Dave: Yeah.
Chris: They all have little different philosophies. Some of them, if you really want GraphQL, you write a GraphQL schema that's in your codebase, and that structures the data on their side. I think it really depends on which service you're using for that data, how it wants you to interface with that.
It doesn't feel right that you would have to structure all your data and then write a whole bunch of individual Netlify functions for every possible request you have. That does seem like a lot of repeating yourself, but I don't know.
Dave: Yeah because my thought is, I just either make a get, like a fetch get, or a fetch post with some content. Then it figures out what to do with it or how to write that data to a database. I guess I should say I ended up opting out of GraphQL because I didn't like it and that's maybe a controversial opinion. [Laughter] It's not my favorite and I can give more in-depth reasons. But I settled on this thing called Prisma, which is--
Chris: There you go.
Dave: --which is what RedwoodJS uses.
Chris: Okay.
Dave: Sort of by default, but it's sort of this ORM, almost like Rails active record style ORM, to basically scaffold out your database. You write almost like a schema, almost like a GraphQL schema exactly, but you write a schema and then you use that schema to generate your database for you. You can do migrations and stuff like that. I like that aspect of it. But then just even how it grabs stuff from the database is pretty cool. Maybe that's the tradeoff I'm making.
Chris: I think that sounds cool. I don't have as much experience, but it sounds like it's matching your kind of mental model nicely.
Dave: Yeah. Just even how I want to develop, and stuff like that. I took a GraphQL course, even, over at JAMstack Conf because I just was like, you know, I need to round out, fill in some holes of my understanding. It was educational, but it was like you realize the GraphQL landscape is ginormous. There's a map somewhere that has it all but, yeah, it's like this just enormous landscape of all this stuff.
Chris: Yeah.
Dave: That you kind of need to know. It sort of was like this, okay, I want data. It's like, "Cool. You want data? Well, you needed GraphQL." Then you have to write your GraphQL query. Then you want that? Well, you need Graphcool, or whatever, to play with that. But then you need GraphQL. Then you need to write a resolver, and that resolver tells you what to do with the data. Then once you have the data, then you have-- It felt like I was putting four or five different appliances in front of just getting data.
Chris: Yes.
Dave: That's what I'm trying to figure out is, like, how do I--
Chris: I wonder if it works better on larger teams where that responsibility is split out to the people that want that responsibility?
Dave: Yeah.
Chris: I'm not sure if that's -- that's kind of a baby thought, but I wrote a bunch of GraphQL this morning because we upgraded to Apollo Client 3 at CodePen and it's fast-moving and early enough of technology that they do stuff that seems crazy, like break a bunch of APIs and totally stop supporting big components that you'd think--
Dave: [Laughter]
Chris: The way they used to write it in React is you'd have a query component, like angle bracket query. You'd call a query and then you'd put the query information in there and variables that you're going to pass it and all that stuff. Then it's like a render function. I forget the right vernacular for all that, but you have some props that you can work with like data, so the result of the data comes back and you can use that in your templating - very handy.
But also, other stuff like the state that it's in, like a loading state. If this loading is true, that means you don't have the data yet, so render a skeleton or something, which felt like a modern way to do stuff. Then this new client comes out and the query components are deprecated.
Dave: Oh, wow!
Chris: No more query components.
Dave: That's a big change.
Chris: Yeah, it's a big change. What the heck?
Dave: [Laughter] So, is it now using hooks?
Chris: Yeah, it's hooks.
Dave: You use data or something like that?
Chris: You use query. You use query.
Dave: Use query. Okay. Okay.
Chris: Yep. Okay, so cool. We got use query. We can refactor. It's not the hardest thing in the world. In fact, I only have four or five left, I think I've got to write this morning. Then we'll be totally off the query components, thank God, and on the latest and greatest.
It feels good. We're pretty hook-based anyway, so having all of our hook stuff all hooks is fine.
Dave: I was going to say, what if you weren't even at that point? What if you're still React classes, you're still in React 14 land?
Chris: I don't know. Yeah, I think you don't upgrade to this.
Dave: Yeah.
Chris: Then you just stay on the old one.
Dave: Okay.
Chris: I don't know what that story is, but you know we like to stay on the latest and greatest, so whatever.
Dave: Yeah. Yeah.
Chris: We're there, and then another change I was doing was, one of the things you can do is put the query right in use query or have like GQL backtick thing and just have the stuff in there.
Dave: Mm-hmm.
Chris: But if you break it out into a .graphql file, your queries instead, and then import them like Webpack style or bundler style. You can then Lint those files, which is kind of cool. You can set up GraphQL Linting that says, like, "Any file that ends in GraphQL, here are some rules that I want to make sure, site-wide, we do." That was another refactor we did was break all of our queries into .graphql files and mutations so that we can apply Linting to them and make sure we're not making weird mistakes. Some of those are really important.
For example, in Apollo Client 3, they said every single thing that you pull needs an ID for caching reasons, which is totally fine. That makes sense. Cache should have an ID to match stuff to. I don't know how it worked before, but we found lots of places in our app that didn't have an ID in the query and it would cause weird caching things. That upgrade having a Lint rule for that was very useful for us.
I don't write the resolvers. I don't write any. I'm at one slice of this level and, Dave, you're rolling in and you need to be on all the levels.
Dave: Yeah.
Chris: You're writing all of it and I don't envy that. I think I would be much more lost than you are with all that and might come to the same conclusion if I had to.
Dave: You know it's interesting. As somebody who has decades of crud app experience, it was like this is a lot of action to set up, a lot more than maybe I thought it was going to be, which is naivety on my part. When I settled on this appliances metaphor, I think that really -- like you said, Apollo server and Apollo client, those are other appliances you have to kind of know--
Chris: Yep.
Dave: --to get the data flow to happen from your data store. Yeah, so it opened my eyes. I guess I'm not -- I think it's still cool, but I think it just felt like a little smidge repetitive, like you have to do it at this layer and then you have to do it at this layer and then you have to do -- so I was trying to--
Chris: Okay. Stuff changes. Even if you decided against it now, it doesn't mean you'll never use it. Maybe if the app gets absolutely humongous one day, you'll reevaluate things and see value there, or maybe you never will. I don't know.
Dave: Yeah.
Chris: It doesn't matter.
Dave: Yeah.
Chris: Things can change over time. I can tell you one of the things I like is the robust feeling of it. Even if it feels a little verbose, it's easy to trace what's happening.
Dave: Yeah.
Chris: It feels like there's a lot of control, also. Honestly, it's one of the things I like about React. I know you don't even like React, but I think this would be true in any JavaScript framework, for the most part, is feeling how -- just like, I'm describing what I want this interface to do and it just does it and it never breaks. Never do we have a bug that's like, if you click the button, it doesn't do the thing that it's supposed to do when you click the button. That's never a bug in our app because there's a button and it has an unclick handler and the unclick handler does the job that it's supposed to do.
Dave: Right.
Chris: It feels very sturdy. Maybe that's a better word.
Dave: Yeah, I guess, or I guess isolated, right? You don't -- I think I like that part of it, too. I just did a thing on form validation. I used to the six lines of JavaScript just to get a weird form validation with HTML attributes and stuff like that.
Chris: Yeah. Unblur. Do job.
Dave: Yeah. It's more like when the invalid event fires on the input, then put an error state. Then I can style the error state, because the way CSS works, it's invalid when the page loads, and so you don't actually want that.
Chris: Right.
Dave: You want it to show up later, after submit. Historically, I've reached for jQuery Validate or something like that. Then I was like, I'm just going to do it in six lines of JavaScript. Did that.
But now in Vue, it's like doing a bunch of query selector all feels really stupid, so I figured out a way just to do it without it and it's pretty simple but it's using directives instead that do all the query selectors and add event listener source stuff. Yeah, it's nice because it's all contained.
I do suffer from a little repetitiveness, like if I'm component one or page one in Nuxt and page two are very similar, and they both have forms, and I want validation on both forms, you end up with a little bit of repetitiveness. But there may be a way that I can optimize that out with some -- who knows. I don't want to do a wrapper component, like you said, [laughter] like you suggested, because -- I don't know. Wrapper components give me bad vibes for some reason.
Chris: Yeah. Fair enough.
Dave: Because you open the door to one. Its 16 cousins are going to show up with it.
Chris: [Laughter]
Dave: No, it's just sort of more like -- I don't know. You're building a dependency tree and this thing is now dependent on this.
Chris: How is Prisma going? Are you feeling pretty good?
Dave: Prisma, I really like. I wish it was "the thing." [Laughter] You know it's weird when you're not using "the thing." You know what I mean? You feel like, am I doing it wrong if I'm not -- am I doing it wrong if I'm not doing the GraphQL thing? It feels like you're doing a wrong, culturally, but it's actually pretty great. The docs are wonderful. The development is wonderful.
I think it is venture-backed by Tom Preston-Warner, so that's kind of cool. It just sort of tells me he is using it and it will be around for a bit, so that's kind of cool.
I think it's a different approach to getting data out of your database. One thing they do, and this is the other question I have about all this is, one thing they do is they say, "Just use SQLite," for local development or whatever. They offer that.
Chris: Okay.
Dave: Which is kind of cool because you can use Node NPM to do SQLite. Like it installs--
Chris: Yeah, not to mention that everybody else working on the project with you stays in sync data-wise, which is pretty cool.
Dave: Yeah. Yeah, and we've had a few syncing problems, like somebody will add data, and then it'll get stomped by somebody else because it's like a compiled file. It's almost like your Sass file, right?
Chris: Oh, it's like binary?
Dave: Yeah. Yeah, so it's sort of like a compiled file. I'm kind of just like, how do people -- what's the best way to work with local databases, local data? You can download and then get somebody set up on a full fricken' MySQL thing and Mamp or whatever. Then hopefully that never breaks or disconnects. Then they have to run the DBC to command or whatever.
Chris: Oh, yeah. It's tricky. It's tricky. At CodePen, Alex wrote a thing called "SSDB: Super Slim Database."
Dave: Okay.
Chris: It's rails, so we have that going on, which means active record and whatever. If there's a migration, you just run it. Actually, with our dev environment, those just run automatically. It's pretty nice. But it doesn't solve syncing, really. What if you need a new, clean set of data to work with in your database? Rails doesn't help you with that. You know?
Dave: Yeah.
Chris: I don't know. So, he's got this script and we just run it. He made a little command-line utility and it has a config file, which, largely, you don't need to change, but it has some cool stuff that you can change in it. For example, there's just like a YAML thing and you can put somebody's username in it. If I want to pull Dave's data, I just put your username in there. Then the next time I run it, I'll have a full copy of you, locally.
Dave: Oh, okay. Okay.
Chris: But it makes some smart defaults, like all of team CodePen is on there and there's certain popularity stuff and a random smattering of collections and all this stuff. The idea is that it'll be 0.001% size of our production database.
Dave: Yeah. Yeah.
Chris: It's just enough stuff that it's useful, locally, but it runs in three minutes or less. It's really fast to run. If there's a particular customer and they have a problem with their account, we can get an exact copy of that customer by putting their name in the file, running SSDB, and then we got them. But it's totally hand-rolled. No service helps us with this at all. It had to be handwritten.
Dave: Okay. Well, that's good to know. I'm just like the local development is a thing. We all do it, right? But then when it comes to local data, people are like, "Oh, uh, talk to Jeff on data."
Chris: [Laughter] Yeah.
Dave: It's such a manual thing. There's no NPM installed database that I have found yet. SQLite kind of does it, but you can't really NPM install Postgres or something major serious, like production-grade.
Chris: I've always wondered. What if you stopped caring about offline development entirely? You're like, "Well, if I'm on a plane, I just can't work on it." Why couldn't everybody locally connect to a staging DB or a development DB, but that just is cloud-hosted?
Dave: Right.
Chris: You can even do serverless. Why wouldn't that just be nice? You can just be like, then nobody has to sync anything. It just connects to that.
Dave: Well, I've thought of that, too. I've actually been in enterprise situations that do that. The problem is when somebody makes big changes to development in one database and then whoever was working on development in one database as well, they got just nuked entirely. You know?
But maybe the solution is you get your own cloud data instance, right? Everyone gets their own and then there's a set of migration tools to copy data or ferry data around or something just to get from one to another. I'm trying to imagine what the world would be like. What would be my ideal working with local data setup? Almost like our favorite WordPress Delicious Brains--
Chris: Yeah, WP DP Migrate Pro. I still use that crap all the time.
Dave: You're like, "Oh, the one on the cloud, I want you to put it on my machine and fix all the routes," or whatever.
Chris: Yeah.
Dave: It's like, done.
Chris: I would say, as long as we're pie in the sky, I do want a full copy.
Dave: Yeah.
Chris: That's just like not happening on CodePen because it's just....
Dave: Terabytes, yeah.
Chris: Yeah, very large. But, ideally, that's what it would be. Ideally, it would be an exact replica of production two minutes behind, or something.
Dave: Yeah. Yeah. Well, and could you even -- could you do that? I don't know. Just click a button and say, "Snapshot here. Give it to me."
Chris: I think there are other considerations, though, too. You almost want data that doesn't change too much, for testing purposes sometimes.
Dave: Well, shoot. That's another thing, right? You want different data to test with, right? I want some mal-formatted Markdown post in my data set so that I can test bad Markdown.
Chris: Yeah, and you don't want some other developer to just -- I don't know -- fix it or something, not knowing the implications of that.
Dave: Right. Yeah. Yeah.
Chris: It's all tricky.
Dave: I don't know. I'm curious what people are finding.
[Banjo music starts]
Chris: This episode of ShopTalk Show is brought to you by Automattic. We talk about them all the time: Jetpack, WooCommerce, WordPress.com, and a bunch of other products. They have started producing courses about WordPress itself at wpcourses.com. There'll be a link in the shown notes, of course.
Their very first course is called Blogging for Beginners. If you're out there, you're listening to this, and you're new to the world of making websites and interested in the world of blogging, certainly a world I'm very interested in--I've had a lot of luck, success, joy, pleasure, and everything from blogging over the years--if you just want a course to get you up to speed, what's it all about, how do you do it, this is the course to take. It's $49. There are office hour sessions, so you can ask questions, as well as, of course, the course itself. But then, in addition to that, you get these quarterly sessions led by WordPress product experts. It's really cool.
It looks like they're really committed to doing a good job with this, over time. You can check out the syllabus online and all that kind of thing. It's $49, again, for the course Blogging for Beginners, and it's just their first course.
They've listed an upcoming course called Podcasting for Beginners. If you're interested in the world of podcasting, certainly keep your eye on that one. It serves as a good reminder that you can run a podcast on WordPress.com, in case you didn't know. That's pretty compelling, too.
Interesting world for them to be in, so high five, Automattic team, for getting this out the door. Hot off the presses, wpcourses.com.
[Banjo music stops]
Chris: Well, now that you're writing some Netlify functions and stuff for your crud stuff, is this the first time you've done a lot of that?
Dave: You know this is my first big, yeah, "baby writes his first function" situation. I'm definitely -- I'm trying to be all in on this, but it's very easy to be like, "Oh, if we just had a little Node server here. You know, $5 a month. We could just kind of do a thing."
Chris: You can. Use Vercel or something.
Dave: No, you totally can, and Netlify has other products like background functions or something. You know stuff like that.
Chris: Right.
Dave: But I'm just trying to keep the overhead as simple as possible and almost lightweight or you could almost run this application, like your own instance or for another client or whatever, just with changing some config keys, basically, like API keys or whatever. You could basically just offload this or you could hot-swap some variables, some environment variables, and then it's running a whole new clean instance or whatever. That's sort of the target goal.
But then even with the idea, the constraint, that I want my coworkers to be able to work on it, which is maybe unfair to everybody else. But it's just the idea of, like, I want it to be easy to work on because, at a three-person company, if I make it really hard for two other people to work on it, again, it becomes my problem. Now it's all Dave Rupert's problems.
The Netlify functions, they make that really easy to work with. I just was trying to--
Chris: Yeah, I like how they have a deployment story, which, otherwise, you've got to all configure yourself. There are some tools like serverless.com that helps with that. There are a couple other contenders for that, but it's still nontrivial. It literally is trivial on Netlify. Put the fricken' Node function in a folder, deploy it, and you're done.
Dave: Tell it what folder you want, yeah.
Chris: That's stupid easy. The background thing is interesting. They just released that. All it does is just ups the thing from 10 seconds to 15 minutes, like how long it can run.
Dave: Yeah.
Chris: This is not like a total C-change.
Dave: Yeah. Yeah, it's just a limit of the amount of time, but you know that's something.
Chris: Well, yeah, it's a big deal.
Dave: There's something to be said for that.
Chris: It means they're eating some more costs for us. Thanks, Netlify.
Dave: Yeah, you get a little 15-minute Node server. That's kind of cool.
Chris: It is cool. It's very nice, but it's not always running. That's different.
Dave: Right.
Chris: If you need a thing that just responds to any request with Node, that's a different story, a little bit.
Dave: Right. Maybe that's, again, a dedicated endpoint or something like that.
Chris: You're starting to see, and they don't -- Netlify has got a beta of Edge functions too, right?
Dave: I believe so. Yeah.
Chris: Those are out now.
Dave: You can kind of like start manipulating stuff with the context, right?
Chris: Right, I just wonder if that's showing up in your brain, too. It's like, oh, I wrote this as a function, meaning that function -- if it's in your functions folder, now you have a URL to hit it.
Dave: Mm-hmm.
Chris: But the Edge functions, they don't need a URL because they just run on the request. They just automatically run and you can do stuff.
Dave: Oh.
Chris: I wonder if you'll start chewing on that pretty soon.
Dave: No, because you could pre-fetch data, basically, right? Because you're saving--
Chris: Yeah.
Dave: In the client, you say, "Okay, render the whole client." Okay, now that that's mounted or whatever, now do a fetch request. You could almost be like, "Peace out. I know this thing is going to do a fetch request, so I'm just going to call this recall function."
Chris: Yeah, at some particular URL. Like if it's the homepage, maybe not. But if it's /product/red/page3, you damn well know what query it's going to run. You just run it at that.
Dave: Yeah.
Chris: Run it in the handler.
Dave: Run it there. Yeah. No, that's a good point, but that's maybe kind of too far out, probably, for us. Again, it's something can everybody work on, you know?
Chris: Yep. Yep. Yep.
Dave: But, no, so far so good. I'm really on the lookout. If listeners have any links, like stories where people are purely serverless. You know what I mean? Maybe part of JAMstack is, you can just be whatever, man. But it's very pluralistic. If somebody is like, "We went on a fully serverless architecture and we don't have a hidden -- whatever -- API server anywhere," or a lot of the JAMstack case studies are like, "We pull down information from our big commerce, our Shopify API," you know, and so it's like, "Well, that's a Node server running somewhere." Just because you don't own that Node server doesn't make it not a Node server, or whatever.
Chris: Yeah.
Dave: But purely serverless architecture is sort of what I'm interested in.
Chris: Yeah. You went all in. You don't hear that much. Even when we had Ryan from Begin on, he said that's not normal.
Dave: Yeah.
Chris: It's more normal to piecemeal things.
Dave: I would be curious if there are any good, purely serverless stories. That's sort of what we're trying. We'll see how long it lasts.
Chris: I heard a little conversation. It was Swyx and Ryan Florence, maybe, talking about are the Edge functions things going to supersede some of serverless usage entirely? He made some analogy like some other VHS to streaming or something and skipped over Blu-ray, kind of. Something cheesy like that.
I wonder if it'll pan out like a hierarchy was my kind of response to that. If you can run it at an Edge handler, then do it - if you can. Usually, there is benefit there. But there are a lot of limitations to that. If you can't, then step down to another level, which is maybe a serverless function. But then are there limits to that too, like how long it runs or whatever? Then step to something else.
I was confused by the idea that I think when you write a serverless function--like you, Dave, when you write one of those functions, put it in a folder, and deploy it--it's deployed to one zone.
Dave: One zone.
Chris: It's deployed to, like, Iowa or something.
Dave: AWS East.
Chris: Yeah, but Lambda does have other services too. It's not a uniquely Amazon thing. Edge versions of their serverless functions, they're just more expensive.
Dave: Hmm. Okay.
Chris: You have to decide, can I ship it to everywhere or not? Is it worth it? Is it cost-effective to do that?
Dave: Yeah. Yeah.
Chris: Eventually, there'll be a pretty clear mental chart of what you pick and when.
Dave: Yeah, that's hard for me, too, because you're like, "I'm just building the thing," but then you're like, "Wait a minute. I have to decide where it runs, too, like this eight lines of Node. Where does it run?" That's a lot of mental overhead for right now.
Chris: It sure it. Yet another thing that's cool that it opened the door to front-end developers, but I don't know how to think about that stuff. I don't know how to price it all out and know how it affects performance. I just don't think about it at all. I'd like to not have to, and I think that's where it's kind of going. Watching AWS for a decade, they are amazing. Other companies are, too, but I really think this exemplifies AWS. They're constantly making it better.
Dave: Mm-hmm.
Chris: Significantly better and releasing the betterness and telling you. Oftentimes, It's better and cheaper. They're just like, "Here you go. We just shipped it. It's now better and cheaper." It's very easy to map out, look, and be like, "Holy crap. That's just way better than it used to be. Thanks. Thanks."
You know I didn't even have to write anything to make it better. It seems like if that's been the case for so long now, what else is going to get better? Well, probably the global CDN stuff. Maybe we'll get to a point where you don't have to ask Amazon to please run my Lambdas in multiple zones or all zones. It'll just do that.
Dave: Mm-hmm. Yeah. Yeah, that's--
Chris: Pretty cool.
Dave: Yeah. I'm curious where that goes. Brian LaRue, who you mentioned earlier from Begin, he follows all this stuff. It's interesting when he's at these AWS conferences, he's just like, "Oh, wow. They just made that faster." [Laughter] Every story is like, "Yeah, it's serverless, but it's like the cold start went down," or whatever.
Chris: Yeah, exactly. The CPUs it run on went ARM, so they're better for these things, or who knows what.
Dave: Yeah. Yeah, totally, and so I'm like, oh, speaking of, I think the Apple thing is happening. [Laughter] It's a lot of overhead. It's getting to be a lot of overhead, but it is very interesting what gains you can make.
What would you do if you did a fetch request on the Edge handler? Would you just pass it back in a header or something?
Chris: The data of it?
Dave: Yeah. Would you--?
Chris: I would try to manipulate the HTML right there.
Dave: Drop a script tag at the bottom or something?
Chris: Well, maybe.
Dave: Or the head?
Chris: I mean that's kind of clever, but I'm imagining that you get data that's like, uh, avatar URL and a user ID, and just stuff. You probably already know what you want to do with that stuff, so just do it right then. If you have to do a query selector all and find an image tag and replace the image source, do it then. Literally, manipulate the HTML before it responds. That way it doesn't -- the request doesn't come down and then client-side JavaScript has to pick it up and do something with it. That work has already been done. Now that's assuming a lot about what you intend to do with that information. I get what you're saying. It might be nice to just plop some JSON onto the page so that client-side JavaScript just has it and knows what to do with it and can use it from there.
Dave: Yeah. Yeah.
Chris: That's probably fine, but if you can just do the work at the cloud, do the work elsewhere.
Dave: Hmm. That's interesting. I wonder how easy that would be. In theory, you could just send your React app up to the cloud function and it has access. It could render it a string or whatever.
Chris: Right. In that Brian episode, that's what he was saying. Then it's cached too. You can run that fetch, but you might know enough based on headers that you don't even have to run the fetch. You could just be like, "Oh, I've already done this once. Get it from cache instead."
Dave: Yeah.
Chris: Which is kind of cool and clever. You might even know that based on the URL. It's really architecture-level decisions. This doesn't change that often, so just always fetch it from cache at this URL, or whatever.
Dave: Hmm.
Chris: It's like service worker in the sky.
Dave: [Singing voice] Service worker in the -- that's my favorite credence clearwater revival--
Chris: [Laughter]
Dave: [Laughter] [Singing voice] Service worker in the sky.
Chris: [Singing voice] Got a good job in the cloud.
[Banjo music starts]
Chris: This episode of ShopTalk Show is brought to you in part by Netlify. Ooh, Netlify is the best. So easy. They make deployment so easy. It's such a pleasure to have some files on your local computer or even just static stuff that you're just authoring by hand or you're playing around with the static site generator or anything like that. You have it and it's a nice, local dev environment, super comfortable.
You're like, "Hmm, maybe I'll put this live." It's already a Git repo, so I'll tell Netlify to look at this repo, run this build command, this folder is the one I want to be the live site," and I'm done. It's super-fast. Every time I push to the master branch on that repo or any other branch that I specify, it's immediately live. I see previews of the pull requests against it. What?! It's so easy. It is really gotten us spoiled, as developers, of how good a deployment, like a CICD kind of world, can be.
You can also run back-end stuff if you need to. Sure, it's static hosting. But if you want to run some Node code on the back-end, fine. Just make a folder called "functions." Throw your Node or Go code in there, and it'll deploy them as AWS Lambdas with nice, little, local URLs you can hit, so you're not worrying about cores or stuff like that. And run your back-end code, too.
Now, if you append dash background to those files, like myfunction-background.js, they'll run up to 15 minutes, which is the limit of AWS Lambda - period. Meaning, you could spin up a headless browser with Puppeteer, visit a bunch of URLs, make PDFs out of them, then store them in storage, and email them somewhere else. Do long-running stuff because, if you don't do the dash background thing, you only have ten seconds. You might get Puppeteer started up, but if you try to take a bunch of screenshots or something, you might bump into that limit. Background functions are where it's at. Thanks, Netlify.
[Banjo music stops]
Dave: Hey! We've got some questions, I think, here. We are a question and answer podcast. You might have missed that from the last pandemic. But, hey, we do have a question. Should we jump over to this, here, Chris?
Chris: Sure. Yeah. Do it. Do it. Yeah.
Dave: Here we go. Got a question from Eric Butler who writes in. It's an audio question, our favorite kind of question, so we'll try to patch that in.
Eric Butler: Hey, Dave and Chris. Love the show. Got a question for you. So, if I want to build a static site but use WordPress as a headless CMS for it, how do I make sure the WordPress site isn't publicly accessible online? As far as I know, there isn't a headless or API-only mode in WordPress that would prevent all the usual front-end pages and URLs from working. So, potentially, people could find that WordPress site and think it was my actual website. Or search engines could find it and then my static site is having to compete against the WordPress version for SEO. The question is, how do I set up the WordPress site to make sure none of that happens?
Chris: Yeah, you definitely need to deal with that. I don't think it would be that difficult to do. You could put it behind basic auth. There's stuff to do there. You definitely need to hide it, so hide it.
Dave: Yeah. I think this sort of gets into the whole serverless thing I was doing. You want some sort of keyed response to go fetch the data, right? You could always put no robots on the whole entire--
Chris: Yeah, robots.txt it, for sure. That's probably not quite good enough.
Dave: Oh, really? Because Google is still too aggressive about robots, huh?
Chris: Yeah, and they say--
Dave: Yeah.
Chris: Yeah, they say, like, "Don't count on this. We're still going to index the site."
Dave: Yeah.
Chris: I don't know. That seems maybe not good enough, but I looked at one recently called Get Shifter, getshifter.io.
Dave: Mm-hmm.
Chris: They will spin up a WordPress site for you super quickly and you can run it in two different ways with Get Shifter. One of them is, "We'll just host your WordPress site on purpose as an API because you're going to hit it, like you would Contentful or something. We'll just have your WordPress site always up and hit it like an API, how you want to, headless style." I imagine they hide it somehow.
Dave: Mm-hmm.
Chris: I'm sure that's just a default setup, if that's what you're asking them to do. I'm sure they have solved that problem.
They also have a second version of running WordPress that's really clever, which is, they'll statically build it. It'll run WordPress temporarily. It'll spin it up serverlessly. You work with the site. Get it looking how you want all perfectly, then you say, "Okay. Done," and it will statically render all of it for you and deploy it to like a JAMstack host. They'll do that hosting for you. Push it to Netlify or whatever, which is pretty cool.
Dave: Yeah.
Chris: There's that, too. That's the question. Eric, do you need a live version of WordPress all the time? If it's just you, for example, you could just run WordPress locally. That could be your install of WordPress. You might want to back it up somewhere, but maybe it just doesn't need to be online all the time.
Dave: Yeah. You could program WordPress. When you hit "publish," you have an after post function or after publish function or something, and then it just goes and hits your webhook to recompile your Netlify or something like that.
Chris: Yeah.
Dave: That would be kind of wild.
Chris: But it sounds like he's talking about using it heedlessly, like API only style.
Dave: Yeah.
Chris: In that case, you probably do need to host it somewhere. That's one of these things that's like not-not compelling to me. I kind of get it. You still get to use WordPress. But it's like when you hit the WordPress API, whether it's the Rest API or you put WP GraphQL in front of it or something, all you're getting is content. Then you need to totally rebuild the site.
The only value you're getting from WordPress is that you can hit a thing and it'll spit out some content at you. You still have got to build all the URLs. You've got to do blah-blah-blah-blah-blah. Not to mention that any plugin you use is probably useless in that way. It doesn't deeply integrate into the site and give you probably the full value of that plugin.
If there's a forms plugin or something, that's going to enqueue some CSS and enqueue some JavaScript on the regular WordPress site that makes it do its fancy thing. But then if you hit the API and get that form, all you're going to get is the HTML for that form and it won't work.
Dave: Mm-hmm.
Chris: You're getting a very slim value from WordPress when you just use it headlessly. You can't just decide one day, "Oh, I'm just going to make a Vue site then and hit the WordPress API for the data." You can, but you need to know that WordPress is giving you 8% of its value if you do it that way, if you ask me.
Dave: No, that's a good point. Sort of like you're ejecting from what WordPress already does, and so you're maybe losing some of the value. I don't know. I'm kind of like, if you just wanted to use it headless, couldn't you just make a theme that renders nothing? It's just a single index.php that does nothing? Wouldn't that work? Then all your HTML views would be blank.
Chris: That's very clever and I bet it would work.
Dave: Or you just say, like, leave a message. Put your no robots in there, meta no robots, and that's all it is, right? Now it's headless because it doesn't have a body, I guess. Maybe that works.
But again, I think I agree with Chris. You're using WordPress only for the editor, which gives me an idea to steal the editor and make a new product. But anyway --
Chris: Don't it? Yes.
Dave: --you're using WordPress to do this for the editor.
Chris: There needs to be, like, cloud-hosted Gutenberg because Gutenberg kicks butt.
Dave: Yeah. Anyway, yeah, you're using it just for the editor and I think that's -- you know, again, it's good, like we said, but yeah. Your mileage may vary.
Chris: I bet there's a plugin out there, too--just knowing that this is seemingly a common request--that would just turn off WordPress other than the API routes.
Dave: Yeah.
Chris: There's got to be something.
Dave: Yeah, there's got to be a plugin. What's your idea? what is the WordPress API for then if it's not for this stuff? What's it for? Like search?
Chris: It's for that. It's for this. It's just -- I think it's easier in greenfield projects. Just know that, okay, I'm going to build a site that I'm just going to use basically a post editor for and publish new content. I'm going to invest no time at all in the WordPress theme. I'm not going to make a WordPress theme and then try to replicate it elsewhere. You totally ignore the fact that WordPress even has themes.
Dave: Yeah.
Chris: You just very heavily look at what the API is giving you and be like, what can I build with this API data? Then it also installs no plugins, which is getting trickier because even Gutenberg itself, without adding any plugins, can do some kind of fancy stuff to the front-end of a site that won't work through the API, like columns.
Oh, Gutenberg can do columns. I find that incredibly useful. Even in day-to-day blogging, it's kind of often where I want to set something side-by-side with something, but not write my own bespoke HTML to do it because that probably won't age well. But if I use this tool that WordPress is saying this is -- WordPress is amazing at backward compatibility. So, if they make a columns thing for Gutenberg, they're going to support that HTML forever. So, I'm comfortable using that knowing that it's going to spit out some HTML that they're going to turn into columns.
Well, great. But you pull in that HTML through the API. Now it's totally on you to make sure that you have CSS in place that knows about that HTML that can make it column-like. It also makes some kind of assumptions, like the way it does columns is to Flexbox, for various reasons, I've looked into.
Dave: Hmm.
Chris: It's like, you know, be aware of that.
Dave: Yeah. What it spits out might be a little unique.
Chris: Yeah.
Dave: Yeah.
Chris: It's almost like, wherever you use that API, you'd want to be like, "WordPress, will you also please give me whatever the canonical admin CSS is that you expect to have on the front-end for these particular blocks?"
Dave: Yeah.
Chris: Blocks are starting to have their own CSS and HTML. They're like little plugins to themselves, so I think that kind of complicates API usage.
Dave: Mm-hmm. No, that's a good -- I mean, yeah. I'd be curious. I've heard people do it and then back out and just kind of go all Gatsby on building their own WordPress or headless whatever CMS kind of thing. Anyway, they started on doing the Gatsby plus WordPress thing, and then they're like, "You know I'm just doing Gatsby," so anyway. So, your mileage may vary.
Chris: Isn't that the big question? How much do I care about JAMstack? Do I just like the idea that they're static files because that's really fast? If that's the case, looking at something like Get Shifter is neat because you are getting an absolutely static site that you could even host on Netlify, which there's no -- it better be static because it's not going to work otherwise on Netlify.
If that's all you care about and are perfectly happy building the themes with PHP and all that stuff knowing that the end result is JAMstack, that's a great solution for that. Or are you compelled by a JAMstack approach because it opens the door for you to then write in Vue or write in React or whatever? I suspect that that's a bigger deal that there are people that are just like, "I just like React, so I'm going to work in Gatsby because I just like working that way. I feel more productive and powerful, et cetera." Then using WordPress as the source for your Gatsby site, that's why you're choosing it. It's not because you love JAMstack. It's more like because you love React.
Dave: Mm-hmm. Yeah. All right. I got more ideas. If you're just using it for the CMS part, Netlify CMS is pretty good. A little work to get it going, but if you just want the editor.
Chris: Yeah.
Dave: Man, my dream world, here's another product. Are you ready?
Chris: Yep.
Dave: Here it is. Free startups here on the ShopTalk Show.
Chris: [Laughter]
Dave: A VS Code theme that looks like a CMS, and it's like you could just--
Chris: Wow! You're blowing my mind here, Dave.
Dave: It's just a folder of posts or whatever, but it looks like a CMS. You click through, you edit Markdown, you save it, and then it auto sends crap up to the cloud or however you want to configure it. Maybe it's a little setting in your branch, but that's it.
Chris: That is great. That is great. There's some precedent for it because of how they turn their settings into JSON.
Dave: Yeah.
Chris: There's like a mapping between a UI and a JSON thing.
Dave: No, if you could download VS Code or maybe it's VS Press. This is where the big money Microsoft needs to give me a call. VS Press.
Chris: Oh, VS Press. Oh, yeah, you've got a killer name.
Dave: Its own product. Yeah.
Chris: Yeah.
Dave: There you go.
Chris: Somebody will do it. Stackbit, maybe you've seen.
Dave: Mm-hmm.
Chris: It kind of got popular because you could just click some buttons and it would scaffold out a JAMstack-y site for you.
Dave: Yeah.
Chris: But it was kind of neat because it'd be like, "Okay, I want to use this. I want to use Netlify CMS and I want to use this theme with this static site generator." It was kind of neat. They would support it all combined. That was kind of like their intro story.
They've gotten a lot more advanced lately. After you've done that spinning up, they drop you into this thing called Stackbit Studio, and it's not supposed to require -- it works really well with the stuff they scaffold, but it's supposed to eventually kind of support any JAMstack site. It will look at your JAMstack site and kind of figure out what's going on.
It turns into almost like a Webflow kind of thing. It's not nearly as fancy as Webflow, but it's for editing content on JAMstack sites in a very visual way. It figures out the mapping between your JAMstack site and where it needs to send that data to change it. If it's like, "Oh, I see. This data comes from this Markdown file over here, so I'll make that editable." When they change it in this very visual editor, it will change the Markdown file that it needs to do that.
It changes it at the smartest level. It'll make a branch called preview in your GitHub repository and literally change the Markdown file in that branch. That way, then if you click the publish button, it'll be like, "Okay. I'll merge the preview branch to the master branch then." How clever is that?
Dave: Yeah.
Chris: It will also know, like, "Oh, this JAMstack site doesn't use Markdown files. It uses Sanity," or Contentful or something. "So, I'll figure out the mappings on their Contentful account and map it out that way." Then when you change things in this visual builder, it'll port that data back to that account. It's like, how the hell does that work? It's incredible.
Dave: Yeah. No, I don't know how. They do diffs and stuff too. It's just weird. It's like how they even figure that out.
Chris: So, I mean, if you're compelled by -- I mention it because if you're compelled by Netlify CMS, which is cool and I use it on some projects, this is that but way, way more. Like way beefed-up version of that.
Dave: Yep!
Chris: Whoa! That was a lot. Let's see. Should we do the last one just so we can consider it complete?
Dave: Yeah. Less of a question and more of a tip from Russell Heimlich. On Episode 401 -- gees, that feels like a century ago -- Harry asks, "How can I add a header or footer without anything besides HTML?" You can do this with server-side includes in the syntax. I'm going to mouth blog it here is like:
angle bracket exclamation point dash dash
Almost like a comment. Then:
#include virtual="header.html"
Then:
dash dash angle bracket
Almost like an end of a comment.
Chris: Right.
Dave: Using an HTML comment, you can include contents of another HTML page. See The Geek Stuff server-side includes. This should work on Apache and NGINX, which are the two most common Web server platforms powering the Web.
Chris: I did a blog post a while back that had eight solutions like this.
Dave: Mm-hmm.
Chris: My opening premise was like, I think it's a little weird that HTML itself can't do this, after all this time, but I get why there are a couple of limitations there. But still, would like to do it. Anyway, it's such a common need. I've got a three-page site. I want the header on all three of those pages, so an include.
You know we've been solving this problem since the beginning of time on the Web. Every server-side language has an include statement for this reason, for writing things modularly like that. Now, all these static site generators have it. There must be dozens, hundreds of static site generators of some kind that have some kind of statement where you write something and it will go then get that file and include it when it's building the file. Pug, Nunjucks, and all of the templating languages that 11ty supports, blah-blah-blah. Everybody has an include statement.
Russell is pointing to the fact that your server can do it. Your Apache or NGINX thing can do it.
Dave: I didn't know NGINX could do it.
Chris: I didn't know that either.
Dave: But most people put NGINX in front of Apache, so I wonder if that's it.
Chris: So, maybe that's it. Yeah.
Dave: I hate to split hairs, but yeah.
Chris: I tried to prove this out because I had an old Apache server sitting around I could it test it on, and I could not get it to work. Then recently, somebody wrote to me and told me why it didn't work. It was because I needed some Apache module enabled or turned on or something that I didn't have and a lot of hosts turn it off these days because so few people do this, I think.
Dave: Yeah. Yeah.
Chris: It's still nontrivial. It's still not my favorite. It's kind of cool that you don't need -- there's technology that's already sitting there on your server you might be able to leverage for this. This is super old school, classic technique. High five, Russell. Thanks for sending this in. It is cool to know about.
Dave: Yeah. No, it's cool. I would consider this, and I know the Filament Group uses this quite a bit just to get some pre-caching and stuff like that because you can be like, "If there's a cookie, include this," or something.
Chris: Yeah.
Dave: You could kind of do server smarts here. You can, "If this is not the first-page view or if this is the first-page view, render out my critical CSS. Otherwise, don't." Now the page is even faster.
But that always sits weird with me. Maybe one point is because I've not used Apache for a long time. Rails is WEBrick, whatever WEBrick runs, or Puma. I was working on a Java project. That was all Tomcat and Tomcat is super weird even though that's Apache or whatever.
What does an HTTP server in Node do? I don't even know what brand of server that is. That's the only reason I don't think it's a panacea, but it is a cool technology for Apache, like a LAMP stack, very much.
Chris: Yeah.
Dave: You can do it but, yeah, I don't know. Usually, in that situation, I would just rename my file index.php and include PHP.
Chris: Yeah, use a PHP include. You know what I really like? This is weird. It's a lot of buy-in for this, but Pug does an awesome job of includes because it has a nice way of -- I like includes with local variables like Rails has got. You know?
Dave: Mm-hmm.
Chris: And lots of other languages have. I think that's neat to use an include and then pass it stuff, too. Even PHP can't really do that. It kind of does with globals, like you could set a global variable right before the PHP include.
Dave: Right and rewrite it every time.
Chris: Yeah. Sure, I mean, whatever. It's possible. But Pug does a nice job of this. I'll take the last little minute here to say that we just did an absolute crapload of work, Alex did, at CodePen on CodePen Projects, which has been neglected for quite a long time on CodePen. They 10x the speed of it. Just made it really hardcore reliable. It's all running on serverless stuff now, so it's faster. It's smoother. It's just a much more pleasant experience to use and we're going to keep evolving that, of course.
One of the things that CodePen Projects was always kind of good at was running Nunjucks and Pug. A use case clearly in mind from the beginning of that Project was, what if you need to make a 3-page site, a 10-page site, a 40-page site, or something, that has this situation? It has parts that it pieces together: blog posts, headers and footers, and little componentry and stuff. Pug is fricken' great at that and so is Nunjucks. I can't even decide which one I like better. I change from day-to-day, and CodePen Project supports it really well. If there's a site like that you want to play around with these include statements like that, including includes with local variables, try out CodePen Project. It's pretty great for it.
We rewrote all of our DNS stuff that was supporting it. One thing CodePen Project can do is deploy. It deploys to our own little hosting thing, but then we give you DNS information if you want to bring your own domain name and host it. That's all working really well, again. So, anyway--
Dave: Nice.
Chris: Yeah.
Dave: I have a few projects. I want to find them. I think I've deployed production stuff on projects. Just literally, I made a website and I was like, "Oh, that looks good. Let's make that the thing."
Chris: [Laughter]
Dave: It's perfect for when you're like, "I want to pen, but I want it to be multiple pages."
Chris: Yeah. Well, spoiler alert. We're going to really double, triple down on that in the coming months and years. We have lots of big plans for how cool CodePen could be, but we're a small team and baby-stepping it in that direction. Oh, I'm so excited, though.
CodePen is like, what? Eight and a half years old now. We're like, finally starting to run with gas here. Some decisions are starting to pay off in our long-term, like, we're here. You can trust us. We're growing. Yeah.
Dave: Oh, that's great. Awesome!
Chris: All right. Thanks, Dave.
Dave: Well, cool. Yeah, thank you. Thank you, dear listener, for downloading this in your podcatcher of choice. Be sure to star, heart, favorite it up. That's how people find out about the show. Follow us on Twitter, @ShopTalkShow, for tons of tweets a month.
If you hate your job, head over to ShopTalkShow.com/jobs and get a brand new one because people want to hire people like you.
Chris, do you have anything else you'd like to say?
Chris: ShopTalkShow.com.
Dave: 440!