Transcript

[Banjo music]

MANTRA: Just Build Websites!

Dave Rupert: Hey there, Shop-o-maniacs. You're listening to another episode of the ShopTalk Show. I'm Dave Rupert and with me is Chris Coyier. Hey, Chris. How are you?

Chris Coyier: I’m doing fantastically. Man, I've been living in the data world for a little while. You know? I'm a back-end developer now.

Dave: Oh, hey.

Chris: I'm sure I told you that. You know.

Dave: Welcome. Welcome, man.

Chris: I switched over in the last year.

Dave: Congratulations. You did it.

Chris: Yep. I know. I did it.

Dave: Back of the back now. Not front of the front.

Chris: They'll take anybody, these back-end people. It's amazing.

Dave: [Laughter]

Chris: But for real. You know I did. Last week--

Dave: [Laughter] Yeah. It's like Jurassic Park where it's like, "I know Linux," or whatever. Anyway, so...

[Laughter]

Chris: You've just got to say you are and then you are.

Dave: Yeah.

Chris: But you know I talked about a little side project I did the other day, and I ended up using PostgreSQL for it. But it was a decision. I had to make some choices about where am I going to keep a little bit of data. And I even mentioned that this show has a Discord. In that Discord, we ended up talking. It's just a forever conversation.

You know what the most common question on ShopTalk, the early years, was? How do I sync my WordPress data locally into the cloud. It was really weird.

Dave: A few hundred episodes on that one topic.

Chris: Yeah. It was unbelievable. But data is a forever issue, and we have some wonderful guests on today, two of the people behind the product Sanity, to talk about their data solution, which is called Sanity. Check it out. It's pretty cool. We're just going to dig into it.

I have Simen and Espen on. Simen, how are you?

00:01:52

Simen Svale: I'm good. Thank you. Thank you for having us.

Chris: Yeah. You're welcome. Thanks so much for being on. This is great. Sanity has been popular for a while, and it continues to grow in popularity, so it's cool to see. I don't know. It's a chance to get to talk to y'all about what this thing does and how you think about it.

Espen, how are you?

Espen Hovlandsdal: I'm great. Thanks for asking. It's been a good morning so far.

Chris: Nice! You two fellows, just because I guarantee there's crossover between our shows and plenty of it, you're with Wes and Scott. You were just on Syntax, so if you all listen to that.

I listen to it. Great episode. I'm going to try to not talk about the exact same stuff, just for the people that listen to both shows. We're going to crank it up. We're going to do a spicy edition.

Dave: Ooh...

Chris: No easy ones for you.

[Laughter]

Simen: We're ready for you.

Chris: I don't know. Okay. Cool. I think, Simen, I did meet you. I think, as you put it in, 1976 at the first Netlify Jamstack Conf back in, yeah, San Francisco, I think we were.

Simen: It was kind of like the Woodstock of the Jamstack movement, right?

Chris: Yeah, it was. You know it was.

Simen: [Laughter]

Chris: I was at the first Bonnaroo and the first Jamstack Conf. Okay, so yeah. That was cool. Your product existed then, too, though. It's not like... You know you were live, even back then, right? You've been at this for a while.

Simen: Yeah.

Chris: How old is Sanity? Have you had your ten year yet?

00:03:24

Simen: It's coming up. 2015, isn't that, Espen, when we opened kind of the public version? I think.

Espen: Is it that old?

Simen: I'm not good with time. Maybe 2016, something like that - 2016, 2015. It's been such a whirlwind. I feel like it's always like that.

I remember we celebrated the third year, and that was like, "Has it been three years? It feels like we just launched." It's going so fast. It's flying by.

But you're right, it was already open at that time. I think we were really trying to get to know the Netlify guys, Zite or Vercel, figuring out how to kind of partner with those smart people in the front-end deployment space. That was super interesting.

Chris: You know you picked well because, to this day, neither one of them have a data solution, which blows my mind a little bit. But those are some big players in the "I want to deploy my website" kind of thing these days. And I'm sure you are thinking -- correct me if I'm wrong -- people that make a Jamstack site, it doesn't mean they don't need data. They probably do need data. That was kind of the point of Jamstack wasn't necessarily that all these sites are static only.

The J in Jamstack was JavaScript, and the A was APIs, which means use JavaScript to use APIs to potentially go get some data. And a lot of people did that. A lot of people still do that. Then people wonder. They scratch their heads, and they says, "But data where? Where do I get it from?"

00:04:56

Simen: That was kind what prompted us in the beginning because we were like everyone else at the time, like early 2000, late 2000, early 2010, starting to just move away from every kind of monolithic solution. We just wanted to use JavaScript to build whatever we wanted and to use the latest whatever it is because things were moving so fast.

Chris: Mm-hmm.

Simen: I think I never used the same framework twice, and I just wanted a kind of solution that wasn't like WordPress with an API. I wanted a proper database. I wanted something with query language.

To my editors, it appeared as an intuitive editing interface. But to me as a developer, it was just a principle data store. And I think that's kind of what we've been working on all of this time, making sure these things are uncompromising a data store and uncompromisingly an intuitive authoring interface.

Chris: Well, that probably worked out well for you too is that the front-end moves fast. You almost want your back-end to move slow.

[Laughter]

Chris: You're like, "Stop. Don't change back here."

Dave: Yeah. Don't put the back-end in the front-end because that's going to get messed up real bad. Yeah.

Chris: Okay. Sanity is one of these solutions of "Data where?" Sanity, potentially. That's one of your options.

I want to make sure I have this right and that people get it because I think it's a bit of an unusual model in a way is that you have this thing called Sanity Studio, and you've just released V3. Apparently, you worked on that for many years, Espen. Congratulations on the release.

Espen: Thanks.

Chris: That you self-host, I think, right? The people manage that themselves - whatever. But the data is always in the Sanity cloud. Y'all hold the data. I host the front-end.

To me, that feels unusual. If you think of something like WordPress, which people probably have a strong mental model for, it's almost like WordPress provides the UI but you provide the hosting. It's almost reverse, which is kind of just funny to think about, I guess.

00:07:00

Espen: It's kind of an interesting take on it. I would say it's interesting because, when people talk about headless, they usually think about the front-end that talks to this back-end somewhere or API somewhere, and that's headless. But with Sanity, it's actually the full stack, so it is also the CMS that's using the exact same APIs. And that's very easy to see because of that model where you have this entire self-hosted React application that talks to the... You can open up the network tools, and you can see there's nothing but, like, JavaScript talking directly to our APIs that you can use yourself. So, it's the same APIs that you can use for making scripts. And back-end services can use those same APIs to patch and retrieve data, set up listeners, all of that kind of good stuff.

But it's also, at the end of the day, a single-page application. So, when you run Sanity build, like the command line tool, it uses a command line tool to do that. You end up with just JavaScript and HTML.

And so, it's very easy to host it. So, even though it is self-hosted in a way, you can also just run Sanity deploy, and we will host that sort of static bundler for you. But it's also easy now to take that to host it on Vercel, host it on Netlify. And with the new version of the Studio, because it's just React, you can even build this into your ... website, so you can have it live on, like, your /admin routes on your Next.js application or things like that.

Chris: Oh... Oh, that's new in V3, that kind of stick it on your same site?

00:08:33

Simen: You get wp-admin back.

Espen: Yeah.

Chris: Oh!

[Laughter]

Espen: Yes.

Chris: Uh-huh. DaveRupert.com/wp-admin is coming back, buddy.

Dave: Hey. Thousands of people visit that URL every day. They must really want it.

Espen: "People" in air quotes, right?

Dave: Yeah. Bots, basically. I thought about making that, turning that into some kind of honeypot where I just ... robots.

Simen: [Indiscernible]

Espen: Oh, yeah. Definitely do that.

Simen: But you know when we were working as a consultancy before we started Sanity, we looked at these kind of headless things coming up, and they were always--like you said, Espen--hosted. You can't really change the CMS, and we were like, "That's kind of for us as consultants, that's part of what we want to shape. Our customers are actually going to use that." Their customers are going to use the website, but there's a lot of value that we can give to our customers in the CMS, so we wanted to have a headless system where we could still do all those things. That meant we wanted that to be open source.

That was the good part of WordPress. The experience is open source. The bad part is you also have to kind of operate the database. I don't care about that. I don't want to do that.

Dave: Yeah. No, I was going to say when I was doing a lot of consulting and I built CMSs, custom CMSs for people, and there were some okay ones. [Laughter]

Simen: [Laughter]

Dave: But the big part of it was scoping down the data to, like, "No. I just want you to edit these three things. Put these three things in here. Then hit save, and you'll be done."

I've had experiences where I give somebody a Squarespace or a whole WordPress and they just don't use it - or Webflow. They don't use it because they're like, "I don't know what to do." You know?

It's just 5,000 bells and whistles. You know?

Simen: Exactly.

Dave: It was very beneficial for me to be like, "Here. Just put the headline in. Then we can be done." [Laughter] You know?

00:10:35

Simen: Yeah. The thing I really wanted was like with WordPress and that ilk. You start by stripping them down, usually, if you want to give them to a customer. That's the first kind of week, two weeks, three weeks. Then you can build in the things if you have time. If it's a cheap product, you have to give them all of the bells and whistles, so we wanted something that started out pretty clean, pretty basic, and then you can add things very easily. But most importantly, we wanted it to be so that you can structure and name things like what they do, like what the business that you're working for does.

Finally, the first job we did, the first big job we did for a huge agency, a ... agency, they asked us, "When do we have a seminar? Where's the documentation?" We were like, [laughter], "There's not going to be a seminar. We're going to come to the Christmas party in December, and you can ask us any questions then."

Dave: [Laughter]

Simen: Then we got the mail back. We were just, "Oh," because they had logged in and they'd seen that, "Oh, yeah. This just makes sense. There are no questions," and we never heard back with any questions.

That was the goal. I think a lot of people still have that experience.

Chris: Okay. What we're talking about is the UI and the UX of what it is like to manage data in a thing. You're saying that Sanity Studio is -- I don't know if "do whatever you want" is the right way to put it, but kind of.

Let's put a point on it, though. Let's say Dave is really serious about his library page on DaveRupert.com. He reads books, and he wants to build a beautiful website that displays the books that he read, so it needs an image. It needs a title. It needs an author. It needs--

Dave: Rating.

Chris: --ISBN.

Dave: Subtitle.

Chris: [indiscernible]

Dave: Yeah.

Chris: Sanity would be a good choice there, right, because I can somehow tell Sanity, "These are the pieces of data I care about, and this is the UX of what I want adding and editing books to be like." Would Sanity be perfect for that? Is that the case, Espen?

00:12:22

Espen: Yeah. I would definitely say so. This is sort of the power of Sanity is structuring those kind of data ships, especially when you go a little step beyond that. When you're saying, "I have an author," instead of just saying, "This is the name of the author," you'll want to create that as a reference to an author.

And so, all of a sudden, you have these hierarchies where you can find, like, "Find all the books that George R. Martin wrote," so you can see all of those in one place.

Chris: Oh... That's interesting.

Espen: Yeah.

Chris: I didn't even think of that. I thought, "Oh, author. That's a string." But it's not really because if we're thinking of how SQL would do it, there's probably an author's table, and it would be a join on the author's table or whatever because, of course, we would want to presumably have a filter or something. Not that you couldn't build a filter with a string, but I guess that's the relational DB way of thinking about it is to reference something else.

Is Sanity like PostgreSQL under the hood or something, or is it your own? What's the secret sauce there?

00:13:29

Simen: The secret sauce is called GROQ. We wanted it to be familiar in the terms of it is a database. The database part is just JSON documents. It's nothing. It's just a pile of JSON documents, and everything that is legal JSON is legal in our database.

SQL is not really good at curating JSON documents because they are more like trees whereas SQL likes tables. So, what we said, we're going to create something that looks like GraphQL but it is really about a free query language like SQL but for JSON. That's what we did, and it's turning out--

We expected to get a lot of flak for that because, like, who comes up with a new query language in 2017? But it turns out that people really are embracing it. They really are understanding the value of that because -- exactly like you said, Chris -- you need that power of SQL, but you want to use JSON. Like Espen said, you want that author thing to be a separate model because the next thing you want, Dave, is to have a separate author page with the picture and the bio. You want to have related authors, and so you're building this kind of structure.

Our goal was to be able to say, you build that structure in a way that makes sense to you as a developer and then it becomes a UI that makes sense to the content creator. These kinds become this harmonious union between the professions. This is our dream.

Dave: I think I love this idea of, like, I own the CMS. You know? It's not just a cloud instance because A) that can fit into my security model. Enterprise people are going to enterprise. They want to host it on some weird thing.

But can I also hack it? If I just put in the ISBN number for a book, could I hit some library API and get all that information to auto-populate my fields and then save that? Do I have that extensibility that I could hack a form?

Chris: Good question. Like a hook or something?

00:15:32

Espen: Yeah. We have a bunch of interesting APIs that can do things like that. You can either make your own custom input component, which you could use to, like, when you fill in the ISBN, you could go out, fetch that data, and then populate the document.

Chris: As long as you can do it client-side, right?

Espen: That's true. Yeah. You can either do that or now with the new version, like if you're building this into your existing Next.js app, it's easier, like whatever you use for that. It's easier to add API routes on top of that as well. So, you could always do /api, /books, /isbn.

Chris: Oh, see. A cloud function - or whatever - that would protect your place.

Espen: Yeah. Things like that. Yeah.

Simen: You can use our hooks to trigger a function, right? You can trigger a Lambda and just these things kind of behind the scenes.

Espen: Oh, yeah. Yeah. There are multiple ways of doing that. The webhook system is also quite powerful, so you could set up that even to React on drafts changing. So, whenever the ISBN changes on a draft, you could go out and fetch that and then, from the cloud function, you could annotate the document, which is kind of an interesting deal because it starts to lead into this landscape where you can have bots hooking into this thing and following along with the content and annotate again, enriching the content behind the scenes.

Because the Studio is all real-time updated, you'll just be on that page and, all of a sudden, things will start appearing. [Laughter] It's kind of an interesting way of doing things.

Chris: You're supposed to say ML. That's how you say that.

Dave: ML, yeah.

[Laughter]

Dave: Chris has his local copy of Sanity open, Sanity Studio, and I have my local copy of Sanity Studio. We're still seeing real-time updates?

Espen: Yes.

Dave: Whoo...

Chris: Wow. Web sockets, how does that work?

00:17:21

Simen: Oh, small patches and it's WebSocket, yes. But the beautiful thing, like Espen said, is we have customers who sync in data from all kinds of sources, like our biggest customers sync in a million different ... every day, and it's built in such a way that people can still work inside those documents and those changes just kind of magically merge together. That is, of course, a huge reason.

I think that's one of the beautiful things. This is very useful when you're a very small team because small teams are always working on the same document.

Chris: Hmm...

Simen: People think that this real-time stuff is just important when you're big, but it isn't. It's even more important for a like a ten-person team having a product launch. They are all hammering on the same three documents.

When you get big, it's even more important. I can go into the Sanity Studio. We are at about 110 people. I can go in. I can see where everyone is. I can just join, let's say, Espen, and I can see where he's working in the documentation, and I can join him.

This is not like Google Docs because Sanity models any kind of JSON document. These changes are much more complicated to model.

Chris: Hmm...

Simen: You can actually even go and review changes. You can see every change in history down to each letter, who wrote it, revert things, and do all of that. In a sense, you're talking about, Chris, why don't these other companies make your data back-end. I think, in the beginning, it looks like very simple stuff. But actually, like with everything, the devil is in the detail.

The same thing for us. We could think in the beginning, deploying Jamstack sites is very easy. But it's really, really hard to do that on a global scale really fast.

It's a similar thing. It kind of takes a huge company to do all of these tiny things that make it look super easy and super dumb from the surface. Sanity using it as a data store, I use it for my hobby projects. I'm working on ML stuff right now. I use Sanity data stores for the back-end for that. It's super easy. I don't need to think about all of these complicated things. But when I need to, it's all just there.

Chris: Speaking of the details there, you're kind of saying... Dave laid out a cool situation where you're both kind of working in real-time and such. What are the specifics there? I'm almost asking for my own, like, hmm....

Simen: [Laughter]

Chris: Do you lock something? If he's on the ISBN field, it's never locked. Does that mean you--?

Simen: You merge. We don't have a concept of locking. What we do is everything you do, every time you type, the system is trying to make the smallest possible patch, like you added a day around....

Chris: So, is it CRDTs or whatever?

Simen: Similar to that. Yeah. Yeah. The same concept, local flavor.

Chris: Same concept, local flavor. You invented your own?

[Laughter]

Dave: Norwegian grown. [Laughter]

Chris: Your own...? That stuff is freaking complicated. I mean just to give you some credit, I guess.

Simen: It was.

Chris: Yeah.

00:20:08

Simen: Yeah, you know we looked at... We didn't look at the CRDT article. We looked at actually someone who was designing a real-time network fighting game, those kind of streetfighter style fighting games. They were super....

Chris: Mm-hmm.

Simen: So, we looked at that. Then what they did was if you have... Let's say you pressed your hit button, like four frames ago, but I didn't know it, but I get the message now. My client will rewind four frames and re-simulate those four frames with the knowledge that you clicked that kind of hit button. Then I'll see the real data. There'll be a little bit of a twitch because suddenly, your hand is up here. It just skips....

Chris: Yeah.

Simen: It just looks cool.

Chris: Right.

Simen: We do the same thing. What we do is we just speculate on what's going on based on what we see, and then if something happens and it turns out that happened before you did something, we will basically rebase, like a GitHub rebase. We will rebase the state of the browser with that new information.

Chris: You're crazy.

Simen: It's crazy.

Chris: I would just not do that.

Simen: [Laughter]

Chris: I would just lock the field. Last write wins. Lock the field. You know? [Laughter]

Simen: Yeah.

Chris: Too late. You already did the hard thing.

Simen: Yeah, yeah, yeah.

Chris: [Laughter]

Simen: No, but you're right. But this is super important and it's one of those things where we took the extra year to do all of these things right because we know it's impossible to add it later. These things have to be kind of in the foundation of the system because it kind of bleeds into everything else you do.

Chris: Hmm... Don't I know it?

Espen: Yeah, it's also easier when you're talking about deep structures. It's even more important because, like if it's a string field, you could potentially just, like, "Yeah, let's lock that field," but when you have these arrays that have objects and, within that object, there's a string field or even a rich text field, there are so many levels here where you might want to lock it (if that was your approach). So, it actually turns out to be, like if you were doing actual real-time, collaborate things, you kind of can't have locking in that way. You need everything to be merging.

Simen: We used to work for this huge media company, loads of newspapers. We used to help them invent new things, and one of the things we wanted to was a number of things related to updating articles based on events, like bots coming in and updating sports scores and stuff like that.

That was amazing. Let's do that! Then we figured out, "Oh, but this system is locking," so everything becomes incredibly complicated because if a journalist is in there, we can't go in there and add our information. If he has left for, like, the weekend, we can never go in there.

We were like, "This just has to stop." Our system will be lock-free because also we think it's happening right now, like the GPT stuff and everything. It's going to be a world where I'm going to sit here writing. Something is going to fact-check me. Something else is going to kind of look for inconsistencies with the new XYZ plan names.

Chris: Mm-hmm.

Simen: This is stuff that's going to go on in the background. It needs to be able to interact with the things I'm doing in real-time, and that is happening right now. Right?

Dave: Yeah. Do you see that in your customers? Are they asking or is this pressure to be real-time pushing them to leverage stuff like Sanity?

00:23:21

Simen: To be honest, I think it's more like these are problems we don't see.

Dave: Mm-hmm.

Simen: Because people don't realize these things. Right? One example is, let's put it this way. When these people are working together at the same time, very often you just don't notice. You just don't see it because it's just so seamless.

Dave: Yeah.

Simen: To us, it's just like this is a pain that people don't feel. It is very, very valuable, but it's just invisible. But you're right. This kind of real-time, only a few very big companies are using this kind of real-time writing stuff right now. But we see out-years influx of services coming in, like real-time translations and stuff like, that will kind of come in.

Dave: No. So, I'm working on an app called Luro. #buyit. [Laughter] #ad.

But I think it was Chris who was like, "Oh, it'd be cool if this autosaved," like when you're editing a document. Just autosave it like Notion or anything. Everything kind of expects autosave.

We were like, "Yeah, cool," and that day, boom, we're autosaving. Then we were like, "Wait a minute. Uh... What happens when two people are playing autosave? It's going to go real bad."

Simen: It's even worse when you don't autosave.

Dave: Right! Right. Yeah, exactly, and so were just kind of like, "Maybe we are too naïve about this and we need maybe a locking situation." I think that's probably where we're going in this iteration, but it is interesting.

Simen: Or you just use Sanity as a backend.

Dave: Yeah, exactly. Well, now I'm like, "Maybe that's the call."

Simen: [Laughter]

Dave: But yeah. It's funny. You don't know it's a problem until you really, like that third tier of thinking about--

00:25:10

Simen: Yeah, exactly. It's like you said. Right now, you will just close windows. You just expect everything to be committed either to a draft or--

I was using Grafana the other day. I was redesigning a dashboard. Then I realized it isn't real-time, so now my coworker had done the same thing, and I had to just give up my changes, I guess. I had no idea. Now that is just not cutting it, I feel.

I think even locking is kind of a joke. You're not using any of these tools for fun. You're trying to achieve something and get on with your day. I'm in there and, yeah, I'm supposed to kind of wait for 7 minutes or 30 minutes (depending on what you're doing - I don't know). I don't feel like that's acceptable, actually.

Chris: Yeah, because it's possible that you're doing writing. Your brain is on fire doing something.

We use Notion a lot to share documents and stuff. Dave and I are in the same Notion document right now. We're not actively working on it, but can you imagine if Dave had this open so I just couldn't see it, or he's writing in a certain area of the document, so there's just a big red bar around it like, "Do not touch!"? Get out of here. You can't do that.

Simen: Exactly.

Chris: That's totally unacceptable. And that's not that different. It's quite literally the same thing. You wouldn't call Notion a CMS, probably. I know you kind of avoid that term, CMS, too. But whatever. It's a system that manages content.

Simen: [Laughter] That's because everyone hates their CMS. Right?

Chris: Yeah.

Simen: We are the CMS you don't hate, so what should you call that? [Laughter]

Dave: Yeah.

Simen: Perfect...

Dave: No. Well, I was going to ask. We have another problem, I'll call it, in Luro. Marketing. Marketers want to market, and so now--

And I say marketers. It's my coworkers. But just the idea of, like, "Let's put some how-to videos or something, or context, or new feature release messages and stuff like that."

It's like, "Okay, cool. Let's code this out. We'll just, I guess - I don't know - handwrite JSON to do these little feature blurbs." Some have videos. Some don't. Then you're just kind of--

I think we're just kind of doing it dumb right now. Then there's a whole deploy process. Boom. Staging, deploy, kind of thing. That's not great for the long-term. It's fine for right now, but it's not great for long-term.

Do you see Sanity as a place to hook in there for apps?

00:27:42

Simen: Oh, yeah ... both first kind of little sticky notes was like it should be easier than the JSON file locally.

Dave: Mm-hmm.

Simen: To me, I spin up... A Sanity free plan is super generous, so you could just spin up a Sanity for each of your JSON files. An API is super dumb. It's just a cURL, just a get. You don't have to get into any of the advanced if you don't need it.

And so, I would say it's easier than that JSON file and, immediately, you can now give that job to someone else who doesn't necessarily like to ... something or deploy pipelines. I feel that I'm actually on kind of a crusade right now in terms of I think people should be--

It's weird to me where you start your front-end project and step one maybe is Next's create app or Create React App or whatever you like. The second step is just all CMS. Install Sanity, I'd say, of course. But at least--

A lot of the CMSs now have generous free tiers. Use one of those. That should be your second step. Then you start writing your first kind of page or whatever you need because I think these things are now so cheap and easy and the cost--

I was doing the same thing. Our own website, we're always JSON with markdown, some kind of weird shenanigans with some YAML and some random APIs here and there. That worked because we did everything ourselves.

The thing is, suddenly, you grow. And you grow by two people, and that's 50% growth for a small company. Now you have to onboard people. Now half your company is onboarding people instead of just giving them a CMS.

I think that is a very obvious investment to make on my part. And our goal was for Sanity to be something that scales from that to global spanning corporations.

The fun thing now is we are used by someone's boyfriend's Korean food blog and Burger King and AT&T. [Laughter] It's the same system. There's no customization by us. This is the same piece of software everyone is getting. It's just a matter of scale. Yeah.

Chris: Oh, that's nice. That's nice. Burger King wants staging and dev, right? What's the story there? I'm putting words in their mouth, but the bigger the company, the more you want local development that's not touching my production data.

Dave: You don't want Dave Rupert hot fixing text. [Laughter]

[Laughter]

Chris: Especially a local dev. What is Sanity connecting to when I'm on airplane - or whatever - or in a no wi-fi situation? Is there a local dev story or just not?

00:30:34

Simen: No, it's not.

Chris: There's just none?

Espen: Yeah. In local dev, the local dev one isn't a story right now. But in terms of the multiple environments like dev and staging, what people usually do is they have multiple data sets. It's like a separate... Think of it as a separate database. You have the same structure. Well, potentially the same structure in those, and then you just have different data. You copy data between those if you want data that is as close to production as possible, or you have completely dummy data if you want to be completely just playing around, I guess.

Chris: That's cool.

Espen: That also allows you to change the content model and build new features without affecting production in any way. It's a pretty nice way of working, and it also allows you to model different. Sometimes, you just want different separation, so you want one part of the business to be in this data set and another part of the business in a different data set. It can be used for a lot of things, but environments is definitely one of the things that we see people using it for the most.

Dave: I think I have this right, but Sanity has a schema, correct? Duplicating my production in development or staging databases is really just like copy the schema. Is that kind of the idea there?

Espen: Yeah. The Studio is the part that defines the schema right now. So, theoretically, the database doesn't really care about the schema, at least at this point, which is kind of cool. It's because you can do things like just put some JSON data in there without having to declare a schema upfront, which allows you to use it for, like, just throw some content in there and start querying.

Dave: That's the number one reason I don't use GraphQL because you need a schema first. [Laughter]

Espen: Yeah, exactly.

Simen: Yeah. Yeah. You could just dump JSON in there and start curating it with joins and all the SQL goodness, but you don't have to--

Chris: So, it literally is just JSON.

Simen: Yeah.

Chris: It's just JSON files sitting on a disk.

Espen: Yeah.

Simen: It's just a pile of JSON. But the fun thing is we index it in such a way that you can use all kinds of the things you associate with relational databases. You can kind of reshape data and you can join it according to, let's say, equality in fields and all of those advanced things.

Chris: Yeah.

Simen: But you don't have to define anything upfront, which is kind of amazing.

00:33:02

Espen: Yeah, but then having this schema in the Studio is what automatically generates a UI for you. Right?

Chris: Mm-hmm.

Espen: It's like when you want to... It's very easy to play around with the data by just throwing some JSON at it. But once you want to have other people actually start editing that content, that's when you start defining that schema.

Chris: Is the schema literally JSON schema, like that exact spec?

Espen: Uh, no. It's not. It could, theoretically.

Chris: Yeah, unfortunately.

Espen: But it's not. It's a little more friendly than a JSON schema.

Chris: Okay, right on. Then let's say you're like, "Oh, the ISBN number should be above the title," or something. Do you just move that line higher in the schema?

Simen: Yeah. In VS Code, you just move it up.

Chris: And it just appears?

Simen: Yeah. Exactly.

Chris: Okay. That's awesome.

[Laughter]

Espen: In the future, we'll of course provide better APIs for figuring out how the forms should look. Currently, it is literally like moving the field up and down. But sometimes you want to be more prescriptive, so you want this field should only... Well, should be on the left side of this thing, and this should be on the right because then I can see all the context. There are a million things that you can do like that. But currently, it's the order of the fields and the schema that determines that.

Simen: Ah, but I think you're selling yourself a little bit short now because we saw... We haven't changed the APIs. We are really careful about changing it. We haven't broken any of our back-end APIs, as far as we know, ever. We even maintain all the bugs from 2015 for the people who use those versions of the API because I hate getting up at night because someone updated their APIs - because for no reason for me.

But then what we have saved up there is we saw people... Like one thing people really love about Sanity and people keep telling us about is this customization. It's cool that it's fast to get up and running with Sanity. You just need a short schema, one little file and it's up and running. But then our dream was (and what people like about it) is then you can basically do everything. You're not punished for getting a fast start because now still everything is possible.

We saw people do very intense kind of rebuilds of Sanity. So, what Espen and team has been doing is building this V3, which is basically, I would say, a framework for building custom CMSs. It decomposes the whole thing.

I think you're being very humble now, Espen, in terms of what you can do. But what you actually can do is incredible. I don't think there is anything like it out there.

Espen: Yeah. I mean that's the fun part right now is that, as you say, it's more of a framework, so it is individual React components that you can import and sort of build up smaller pieces of the framework or the Studio. Going forward, that will just be even easier, so each input component is its own thing that it can important and just expose.

If you just want to edit a single field or you just want one specific user to be able to edit a certain portion of the content model, you can do that both through configuration but also by actually programmatically doing these things. It's all just React, so it feels very familiar if you're used to building things with React.

Simen: It's it like, Espen, now Sanity is just like React. Targets are like Sanity, and that's the CMS?

00:36:28

Espen: Yeah, it literally is just like a React component that is called Sanity (at the lowest level). When you're in a route inside of Next.js or something, you can just import Sanity from Sanity, and then you render a JSX thing. Then you're off to the races. Just passes your project ID and configuration, and then that's it.

Simen: the whole of Sanity is just a React component.

Dave: If I'm understanding... I think I like your approach because there are sort of like three choices. Right? You use a CMS like WordPress, and you just have their data model. Right? Or you use something. You build out some UI, and you try to trick it to spit out your data model that you want, like custom fields.

It's almost like, "Hey, build your UI and we'll create the data for you." But it sounds like Sanity is very much like you actually start with your data model, your little blog of JSON, and then you create the UI off of that. I guess the question I'm kind of leading to is what's my first experience with Sanity? Is it five minutes? Is it one hour? Is it two days? If I'm like, "All right. Let me try this," what's that experience like?

00:37:54

Espen: The first thing people usually do is to start off with running the command line tool and starting or creating a new project, and so that will create a hosted database in the cloud. Then it doesn't have any schema yet. Then it asks you whether or not you want a template. We have a couple of ones to go from, like, if you want a personal website, a blog, a movie database, something like that just so you have something to get inspired by.

Then create some files. Mostly just configuration file and a few schema files. You can start editing those. Once you run Sanity start, then you've got a CMS up and running, and all of the content is stored in the cloud and it's ready to go. Then you can start editing those schema files to modify it to your needs or you can start from scratch and define the individual content models that you have.

The UI will just magically reflect this. It's just like once you change the schema files, it's a hot reload. All of a sudden, you have a new navigation section that's like, "Oh, yeah. Now we have books. Now we have authors." And you can navigate and start creating new documents.

The other way of going about is it of course, you create the project first, so you have this database in the cloud. Then you can start using our tooling to import data or just route it, write scripts to import data, and that's basically just JSON files, as we've been talking about before. It just has a document ID and a document type. Those are the only attributes that we really require you to provide. Even a document that will be auto generated for you if you don't provide one, but sometimes you want a little bit more control.

Once you enter that into the database, then you can start querying, as we've talked about before. We have the GROQ query language, which doesn't require you to define a schema. And so, you can just say, "Find all the JSON documents of type author," and then you have those. And then you can say, "I just want the name and the best friend fields," and you'll get those.

It's very similar to GraphQL, but it doesn't require that schema definition upfront, which is nice.

00:40:16

Simen: We also have the starters, right? They can also get started by, like, ready my project, front-end, back-end, same file or same project.

Chris: Mm-hmm.

Simen: Integrate it and up and running.

Espen: Yeah, that's really exciting now with the new version, V3. We can provide these starters, which gives you the entire stack. You have the actual front-end that talks to our API and also the CMS, and it's all in the same repository, the same project. It even deploys that straight onto Vercel, so you have everything up and running within a few minutes.

Chris: Hmm... So, if it's a Next.js site, you run your command line tool. You run theirs. Or maybe it's combined or something these days. I don't know how you're doing it.

But then it needs to get some data, so you probably encourage the "get server-side props" thing, right? That's how Next.js does it. It might as well ask for Sanity data so it can server-side render.

Simen: Exactly.

Chris: Then I don't know. I assume there are some API keys or something because it's got to do that.

Simen: Yeah.

Espen: Depending on your security model, you can choose to either have a completely open, public data set that anyone could query. In that case, you don't even need any of the API keys. Or you can have a private one in which case you will need to provide a token. Yeah.

Chris: Yeah, that's cool.

Simen: By default, it's all public. Just your drafts are secret.

Chris: Oh, I see. Yeah, okay. That probably is pretty satisfying to spin up a site with those, that modern of technology. Then what's already installed?

Would you consider GROQ your ORM or whatever? At some point, you have to write a query that says, "I want these library books," and I know y'all really like and advocate for your GROQ language. I would say let's spend a minute or so talking about GROQ because it is interesting. It's kind of your - I don't know. It's almost like, "GraphQL sucks. Use GROQ," thing. [Laughter]

Go ahead. You've got to do your thing.

00:42:27

Simen: Yeah. [Laughter] Our thinking about GraphQL is the same as what you mentioned, Dave. GraphQL is amazing for designing controlled APIs. If you want to create a specific API and have really great control over what's legal and what's illegal, and have tight control over that, then GraphQL is basically the err to Restful APIs, right? That's what it replaces. It's really much better than Rest.

Whereas what you need... Let's say when you're implementing a GraphQL API, you have to have some kind of queries underneath. How do you get the information to fulfill that query? Usually, in most systems, there will be some SQL or MongoDB queries or something like that.

GROQ is that, but it's exposed all the way to the client. So, if you really want to have, like, we created GROQ so that it basically is shaped like GraphQL. Espen wrote all our--

We have GraphQL as well. Our GraphQL API is just more limited because they have to be generic. They are just text transformation. They're just transforming text from GraphQL to GROQ and then sending that onto the backend.

It's low-level. It's the SQL to the Restful. Yeah. If GROQ is Rest, then this is SQL. Yeah?

Chris: Well, let's use the book analogy. Dave has got his bookshelf. There's a thing in the schema that says this is a book, and a book has all the stuff we talked about: the title, the ISBN, the image, and all that stuff.

Immediately, I can GROQ for that stuff, right? There's no setup or anything.

Espen: Yes.

Chris: GROQ just will work with that.

Espen: Exactly.

Chris: Is there any setup for the GraphQL, though, or do I get an automatic? Is there a query and a mutation for a book type just because it exists in the schema? Do I just get that for free?

Espen: Yeah. You'll have to define the schema with GraphQL because GraphQL requires a schema to even know what's available and what queries should be there.

Simen: We ... right? You already wrote one, right? We create one for you.

Espen: Yeah. Once you define the schema in the Studio, you can run Sanity GraphQL deploy, and it'll take a look at the schema and automatically generate a GraphQL API for you, which then allows you....

Chris: Oh, it does? Oh, wow. That's nice.

Simen: Yeah.

Espen: It's not that much more work. At least not if you already have the schema in the studio. But the thing that GROQ can do that GraphQL can't do is you can't just... You can just import data, like willy-nilly, and then you can start querying without defining any schema for it. That's really powerful.

00:45:14

Simen: Yeah, and the really nice thing that people like with GROQ is you know the code that you always have in your fetch. You will probably do a number of different fetches. Maybe you will reshape it a little bit before you give it to your view. You have this whole bit of code that runs before you render.

Chris: Mm-hmm.

Simen: We didn't anticipate this, but we made GROQ so powerful that basically most people just replace that whole bit of code. You basically express that whole reshaping. You can take your authors. Let's say you don't, for this particular page, need the data about them. So, you convert it to a string on the server-side. You concatenate the strings with a comma. That's what you send over.

You could be very, very specific about the shape of that you want and really rebuild it and join in data from other sources like other books. Go fetch that, like other books by the same author, and attached that. And then you give that to your view, and now the whole thing is done.

The cool thing about that is that whole thing is now CDN cached edge, so that whole processing is happening server-side. Also, we store it for you. So, the next customer coming in, that's one millisecond.

Chris: Hmm. Are we all crossing our fingers that this really takes off? Part of it, I hate to say it, but you're the only thing that has got GROQ. I heard you say on Syntax that other places could and have shown interest in GROQ.

Simen: It's open source.

Chris: But nobody has, right? But would it be nice if they would? I'm sure it would be nice for you to be like the originators of some awesome language. But at the moment, it is some form of, like, "If I choose to go all in on GROQ, I'm sure as hell not leaving Sanity." [Laughter] You know what I mean? That's fair to say.

00:46:51

Simen: No. Oh, yeah. That is far to say. It is, of course, one of the reasons we make sure this is open-source and well-documented. There's a test suite that is open source. The whole thing is... We even have a JavaScript implementation of GROQ that's open source. So, it's possible.

We just sure don't want to use it as kind of a mote to get people to lock in with us. But we do have to--

Let's put it this way. These GraphQL APIs that are kind of general, generic, these are terrible. You build this huge kind of tree of every possible thing you may ask for. These schemas are terrible. Using GraphQL as your query language is just nasty.

In a sense, we have to have a proper query language. We can't not have that. And you can say other databases -- let's say Mongo or Elastic -- they have a query language, which is proprietary. But it's just encoded as JSON. It's just not very friendly to write, but it's the same kind of thing.

You could see GROQ as that. It's just that with a nicer notation. So, we can not have that. It has to be a lower level query language.

But what is happening now is that we see people are... We don't advocate GROQ over GraphQL. We see them as two different things. It's like a plate and a fork. We prefer forks, but you need a plate, you need a plate.

That's how we view it, and I think we're going to work more on our GraphQL support in the future because I think it's awesome when you need a controlled API. It's nice to be able to just go into your data when that's the thing. It's nice that when you need--

Let's say you have seven different applications talking to the same data store, and you might not be able to deploy all of them. Maybe some of them are PlayStations, so might not control when they update. You might need to have a controlled API that might be able to be updated without your dataset updating. Those things, we support those things. We see those things, and we think that's important, too. We can have both. That's not a problem.

Chris: It is so funny. It's so far away from your brother. What did you say? Some boyfriend that's got a Korean food blog or something.

[Laughter]

Simen: Yeah.

Chris: That's real different.

Simen: We do both of those things.

00:49:05

Chris: Let me explain what we do at CodePen. We have a GraphQL API at CodePen, right? I am seeing where you're coming from here in an interesting way because I like the idea.

Well, let me describe it first. It's very handwritten. There are all kinds of queries and mutations, and they all have what you call a resolver. So, the first thing it hits is that resolver. And it's like, "What do you want me to do?"

The way we've structured it is, like, "Well, it might just return something if it's really basic to return," or it might hit a logic layer that says, "Oh, authorization," and permissions is important to this one, so it's got to go through that layer first.

Then eventually, it will get down to some lower-level place where an actual query has to be performed. That query is probably handwritten too. It probably isn't just raw SQL. Although, it could be. It probably uses what we call GORM. Not what we call... We use a thing called GORM, which is just a Go-based ORM for pulling data out of that PostgreSQL thing.

Now, we've been doing this long enough that some of that stuff is auto-generated. We have some fancy tooling that could look at a GraphQL type and make types and such out of it that it can perform queries on our behalf anyway. It's not like somebody is sitting there being like, "Every single query and mutation has a bespoke query that is handcrafted by a developer."

There's lots of automation to get us there. But because we have this whole setup and can look at all the code all the way down, we can do really bespoke stuff, like, "What does this mutation exactly do? Do I need to finesse it a little bit before it goes into the database?"

I query to get something out. I have this layer to do a little stuff to it before it gets back to the person (if I want to). That level of control is pretty great. but I like the idea that that query that's really deep in our codebase kind of gets sucked all the way up to the top with GROQ. You know?

Simen: Yeah. You know what I think is amazing? This concept came up early in our company's life. We were like, "I want a database API that respects the query, the security filters, so I can basically use SQL and the database takes care of hiding whatever you don't have access to." We did that very early on.

GROQ has that same thing. If you have a very complex enterprise, we have a very complex set of security filters they can use. Then you can still just give the query API to anyone. They would just only see what they're allowed to. Even if they specifically ask for something, they won't even know it's there. They won't even know they are not allowed to see it.

What I think is amazing is you can take... Let's say you have a company. Part of your company has been working for a long time on maintaining this database of product nutritional information or something very specific.

Now this other team needs that information. In the past, you would have to wait for seven months until that other team could write the API you need to there because they control that kind of API surface, and everyone is waiting for someone to implement some flag on their API. If you use something like a content cloud like the Sanity cloud, then that API is the same. So, I can just start leveraging that other team's data if I'm allowed to, and I can just start making my own API on top of that because....

Chris: The data is there. I can see it and use it.

Simen: I can shape it, I can get it, and I can do that EU kind of certification process that we were ... just based on the other team's data. Right?

00:52:33

Chris: Yeah, that's cool. I was at a local meetup the other day. This, I'm sure, will horrify some of you and maybe appeal to some of you.

[Laughter]

Chris: There was a Vue app, and this team had some theory that you only need a database and a front-end and that people put way too much code in the middle and their back-end, so they had these Vue components that had SQL queries just in the Vue component. It just went straight through the database. There was a little back-end because you need to do a little auth or something on the way there.

Simen: [Laughter] I hope so.

Chris: It had nothing. It was just like the back-end was just, "Accept user input. Perform query that was given." They would return data.

Simen: Yeah. It's the same concept, right? Yeah, it's the same concept.

Chris: I thought it was so weird. Yeah, I guess it is. I guess it kind of is the same concept. That's very strange to me.

Dave: I am scared.

[Laughter]

Dave: I am afraid.

Chris: I'd rather write it in GROQ, though. I'd rather have a little bit of abstraction between my raw database query.

Espen: Preferably not have mutations in there.

Chris: Yeah.

Simen: No, I just think the nice thing is to be able to have the knowledge about how things need to be shaped near where you use it, so that when you need to reshape that data, that's often in the same file. I find that super sweet.

Chris: Yeah. I do a lot of data reshaping, I'll say. [Laughter]

Your point, I admit, unfortunately, I've never written a GROQ query in my life. But you're saying that GROQ can do the reshaping as part of the query? It's reshaped right there?

Simen: We were shocked. You can do any... you can fetch any number of things, unrelated to related. You can ... the least number of queries. They might kind of build on each other or not. You can say, "I want to get this book," and I get the author. I want to get the other books by that author. But I just want the titles of those books, nothing else. But then also, I want to go get the articles that are related to any of these books, and I will build them into one single array. But I will only have title and the image of those articles.

Chris: Mm-hmm.

Simen: You can really kind of do... That is usually seven, eight API calls and some logic for each map - blah-blah-blah - stuff in JavaScript. All of that happens on our back-end once. Then it's one piece of compressed JSON over the wire. That is usually then also cached.

Chris: That's touted as something that GraphQL is good at. The thing is, somebody has to handwrite that in order to work.

Simen: Exactly. It has to be the thing you need, right? It probably isn't. [Laughter] And if you have a very small company or you have a single-person team, maybe you can do that. But in the real company setting, this leads to everyone waiting for everyone else to change their APIs or write lots of weird hacky middleware to just kind of cobble it together.

Chris: I think we've nailed on some limitations. It would be interesting to know if you have a couple that you're comfortable talking about. What are some things that Sanity isn't particularly good at? It would surprise me if it scaled to millions of records, but I assume it can. Right? You're talking about pretty large-scale. It blows my mind that GROQ can be fast at all.

00:55:53

Simen: I proposed a get to ten million documents project, and then we were kindly informed by sales that we have several customers with way more than ten million documents, and they are super happy. So, I guess you can make it work, but you're right. That is definitely a challenge.

We have a great team with incredibly smart people working on the data back-end and kind of the design philosophy is that we take all the complexity. To you, it will look a dumb list of JSON documents. We know that to us it's this complicated and dimensional challenge to get that to actually run anything fast, and it does. But you're right, that is a limitation. If your thing is very huge and also very complicated and you expect to run very complex queries, it will not work. It will be very slow. That is true.

Chris: Well, we don't have to dwell on that. But I would assume that.

Yeah, feel free to do other ones, but I'm wondering. Is Vercel going to buy this? What's going on here? Would it be a bad day for Sanity to wake up to hear Vercel announcing they now have a data solution and it's not Sanity? [Laughter] That would suck, right?

00:56:59

Simen: Oh, yeah. That would suck. But I don't think--

Chris: [Laughter]

Simen: I don't think our... I remember talking to... I won't mention names, but I talked to a proprietor of a very famous front-end framework saying, "You know, a CMS, that's just like a forms over database, isn't it?"

I'm like, "Yeah. Yeah, you just keep believing that. That's just what it is. It's a very, very simple, dumb stuff."

Chris: So easy. Yeah.

Simen: [Laughter] Exactly.

Chris: [Laughter]

Simen: I think I'm not scared about it in terms of I don't think... I know for a fact, let's say, Vercel, they have incredible ambitions on behalf of their own projects. Guillermo is an incredibly smart person.

Chris: Mm-hmm.

Simen: He understands how very, very complex problems, kind of state management, and that kind of challenge is. And they have so much to do on their own platform that I don't think it would be... I think it would be--

The worst part of that on my part would be, I think, a really bad day for Vercel if they were to spread themselves like that.

Chris: Too thin. Okay. Yeah, it's interesting to see other... And Netlify is the same thing. They don't have their own front-end framework. Although, I guess they pay Zach to work on 11ty and stuff but don't have quite as much investment into one particular framework like Vercel does. But also, kind of famously doesn't have anywhere to keep your data.

It's interesting that companies that really succeed at one thing tend to not do the other thing, you know, unless you're Amazon. But then they don't really have a framework that has really succeeded. They kind of tried, and they have so much that it ends up being kind of a confusing mess. I don't think anybody necessarily trusts them to build a framework. Maybe that's weird, but it feels like that ship has sailed for them.

Simen: No, but isn't it? I think that is so interesting. I think this is such an interesting thing.

You would guess someone like Amazon with definitely having infinite resources would be able to even make their own offering simple. But even companies can make a big diamond just making--

I mean Vercel. Let's be fair. Vercel and Netlify is just making stuff like AVS and GCP just easier to use.

Chris: Yeah.

Simen: It's basically just a facade over that, and you are willing. Even we. Our company is full of AAA developers. We'd gladly pay companies like Vercel and Netlify to make that easier for us. Yeah, gladly. That's easy.

For a particular project, if you can pick one, like Vercel or doing it raw on GCP, we will. Even if we still pay GCP hundreds of thousands of dollars a month, we still do that.

I mean that is very interesting, and I think that's a thing for a company that want to really crush in the market. Let's say you are doing front-end development or you are doing a CMS. We would never--

I got this question, actually, the other day, like, "Why aren't you making money deploying the studios? You have the simplest possible deployment solution." We were like, "There is no... Then we would be competing with Vercel or Netlify, and we don't want to do that." We don't think we can win that fight, and we don't want to.

It's about focus. And the people who win, focus. Right?

Chris: Yeah. Pick the fights you can win. Well, you picked a good fight, though, because I'm a little obsessed with the idea that if you can get customers to put their data on your thing and pay you to do it, that's a really super-duper solid business. [Laughter]

Simen: Yeah. Yeah. Yeah. You just have to be amazingly trustworthy, right? You can imagine we have storms in our server centers. That's an all-hands-on-deck situation because what we want is for everyone to just know that we just take care of everyone's stuff. That is kind of the biggest deal.

Chris: And it's not just data-data. It's like if all Vercel did was Next, I think I wouldn't bet on them anything. I'd be like, "That's a sucky business model." But the fact that you're deployment too, it's an amazing business model. They do have data. It's just files not structured data.

Anyway, we got to wrap it up. Do you have any final thoughts there after all that, Espen?

Espen: Uh... Nothing from me. No.

Chris: Okay. All right, Dave.

Dave: Well, I just thank you all. That was fun to talk about. It was very interesting. I mean it's always that... I feel like, every week on Twitter, it's like, "I'm building a site. What should I use for data?" Maybe people now have Sanity on their radar, so hopefully, that's cool.

But before we go, for people who aren't following you and giving you money, how can they do that? We'll start with Espen.

Espen: They can go to sanity.io to check out the product. I'm on Twitter, @rexxars.

Dave: Nice. Simen?

Simen: Yeah, so the same thing. I really think people should go check out Sanity V3. Even if they have tried Sanity before, it's really something. It's really amazing.

You can find me at Twitter, @svale.

Dave: Awesome. Well, thank you very much. We really appreciate it.

Thank you, dear listener, for downloading this in your podcatcher of choice. Be sure to star, heart, favorite it up. That's how people find out about the show.

Follow us on Twitter for as long as it exists [laughter] @shoptalkshow. Then head over to patreon.com/shoptalkshow to join the D-d-d-d-discord. It's always fun.

Chris, do you got anything else you'd like to say?

Chris: Oh, that's it. ShopTalkShow.com. Thank you, everybody.