Transcript

[Banjo music]

MANTRA: Just Build Websites!

Dave Rupert: Hey there, Shop-o-maniacs. You're listening to the take-two edition of the ShopTalk Show.

[Laughter]

Dave: I'm Dave--brand new wi-fi--Rupert. With me is Chris--in the office--Coyier.

Chris Coyier: Chris--where Riverside always works perfectly--Coyier! Yeah! Just kidding.

Dave: Whoa! Literally just telling our guest about how awesome Riverside is [laughter] and then it gave us a second opportunity to run the show back.

Chris: [Indiscernible] Normally, it's fine. Normally, it's fine. But it wasn't today. It wasn't today.

This is going to be great. We're going to do a wonderful show here. We have a guest. I'm just going to introduce him right away. It's Mr. Adam Coster. Hey, Adam. How are ya'?

Adam Coster: Hello, hello. How are you?

Chris: I'm absolutely fantastic. We know Adam from the Discord and also his work in the games industry and stuff. You are from kind of a small studio called Butterscotch Shenanigans, right?

Adam: Yep. It's a mouthful, but it is right.

Chris: Yeah, it's wonderful. It's like CodePen-sized, so I feel like we probably have similar spirit in that way. It's a small studio. You have nine at the moment, right?

Adam: That's right, yep.

Chris: Nine people.

Adam: We started with three, three brothers. We founded it back in 2012. Over the years, we've messed up hiring practices a few times and tried some stuff, and now here we are with our really solid, excellent core team of nine people.

Chris: Whatever. I cannot relate.

Adam: I know. [Laughter]

Chris: Just kidding.

Dave: If you work with your brothers, is that the best experience or is that you get wedgies experience? I'm trying to think of how it would be with my brother.

Adam: Yeah, we do get that question a lot. It is an interesting dynamic, definitely. We grew up together. We didn't not get along, but we weren't close friends, really, when we were younger. But after college and kind of once we got to the early part of our not being trapped in the same house, being siblings mostly kind of lives, we sort of started to reconnect.

The big advantage that we have is that we're just really aligned on long-term goals, why we're doing this, and that kind of stuff. Of course, then we have that history to pull on. And we have extra incentives to not mess with the bigger picture of our relationships, which the net effect has been really good for the business and also our just kind of family dynamic. But it easily could go other ways. [Laughter]

Dave: Yeah, yeah.

Adam: It hasn't, and it has been very good. I think it would be... Yeah, it's hard to imagine doing it with other people.

Chris: That's funny, though. Yeah, I think it'd be hard to give... It seems like it would be hard to deliver biting feedback to your family.

Adam: We've had to... Figuring out how to give feedback to each other has definitely been the biggest challenge of the whole deal, but we kind of worked out a lot of those kinks in the first few years of working together. It's still a struggle, just like it is with anybody. But it's also just made our sibling relationships better over the years.

Dave: Charlie horses, name calling, locking people out.

Adam: All that good stuff. Yeah, we did that as children, definitely. There was all that stuff.

As adults, though, we just kind of... We're more just like friends than we are like siblings, I guess, in terms of the dynamics.

Dave: Yeah.

Adam: Yeah, it's been good.

Dave: That's cool.

00:03:39

Chris: All right. It's called Butterscotch Shenanigans. Also, a podcaster. You're up in the many hundreds of episodes, as we are, so we're simpatico that way. Founded in 2012, again. And if you've heard of them or maybe you haven't heard of the studio, maybe you've heard of the game. Their biggest hit, I guess, so far is called Crashlands, right?

Adam: Yep, that's right, which it's an old game now. It was launched back in 2016. But we're working on the sequel right now. We're trying to wrap it up and get it published soon.

Dave: Ooh...

Chris: Yeah. Crashlands 2.

Adam: Crashlands 2.

Chris: Is that what it's called?

Adam: That's right, yep. Naming a sequel is always a big challenge because you're like, do I do "Crashlands:" and then come up with something?

Chris: Yeah. No, just say "2."

Adam: Yeah, just say "2." It keeps it easy.

Chris: That's what Moana is doing, so I think she's smart.

Adam: Yeah. I think that's the right--

Chris: Okay, so Crashlands. Let's hear about it a little bit. I do know that it wasn't your absolute first thing, though. That would be extra unusual (right) to just have a hit right out of the gate. It's something like your fifth game.

00:04:38

Adam: Yeah, it's our fifth game. We had four mobile-only titles before that. Our idea at the time was because we were all new to game dev and teaching ourselves by doing it and just trying to see if we could find a way in the industry. That was the lowest barrier to entry, especially at that time. You could have a small game. It didn't have to be high fidelity.

Chris: Compared to--? You're not going to just be like, "Let's make a Nintendo Wii game," or whatever was the thing right there.

Adam: Exactly. Yep.

Chris: I can't imagine people not even knowing where to possibly start. I wouldn't.

Adam: Yeah. Yeah. Yeah, and we didn't either. We just kind of started. Well, I guess we really started with game jams. For anyone not familiar with the concept, the old stuff used to be like film and stuff. But a game jam is basically you sit down for some short period of time (48 hours, usually), and you try to make a game. Just however you can get it done with whatever scope you can manage. And that's how we got our start, really.

My middle brother... I'm the oldest of the three of us. My middle brother is our game programmer, and he had tried to figure out how to program all the way up through college. It just never quite clicked. Then one day he picked up a piece of game dev software, which I called GameMaker, did a game jam, and he could do drag and drop, and at least a version of it. He was drag and drop and put stuff together, learning how to program as he goes. That was how we got our start.

It turned out that he realized he could do it, so having that slicker entry point into the process of at least the making games part, which is an enormous challenge because there's so much going on. There's audio stuff. There's video stuff. There's animation. There's lots of programming problems you have to solve. And the potential Web stuff, too, once you start to get into the online side of things.

It's just so much stuff that goes into it. It's just so multidisciplinary that having some kind of an entry point that smooths things out is essential. And so, for us that was the engine GameMaker, which we still use today.

Chris: Really?

Dave: Wow.

Chris: Wow.

Adam: It's evolved with us. I've never heard of any game studio ever who liked their engine and wasn't fighting it all the time and wasn't mad all the time. And so, we are certainly in that camp. We're mad all the time. But I do still have to give them props. They've made a great product, and we gave them feedback constantly. More than I'm sure they would like.

Chris: Is it a little bit like picking React native - or something?

Adam: It's the same idea.

Chris: You pick it, and then because you picked it, it poops out to a bunch of different platforms - or whatever?

Adam: Yeah. Yep.

Dave: Is it, or is it more like Jumla?

[Laughter]

Adam: I think frameworks are a reasonable comparison with the exception that, with frameworks, you're still working with JavaScript. You're still working with a Web stack, right? In game engines, you get the language changes. The whole model can change, which is more sort of kind of, you know, frameworks have different models for thinking about the problems that you're solving.

Dave: Or financial models.

Adam: Or financial models.

Dave: A lot of people got swept up in the ol' Unity switch-a-roo.

00:07:44

Adam: Yeah, Unity was a whole thing. Yeah. Yeah, and so it's a similar thing. For me at this stage in our progress of learning how to do this stuff, the most important thing by far that a game engine provides -- because these had to be compiled down for targets, right? You don't have the advantage of a browser where it's like, okay, you've got three browser targets, I guess. But it's the same code, more or less, that goes to each one. A single server target, right?

You may have compiled languages server-side, but you don't have to. There are just so many options there. But for video games, for performance reasons, you're compiling down to platform targets. The most important thing for us has been the fact that our engine (and many of the engines, but not all of them) can just kind of hit any target (to varying degrees of things going awry). But they can hit any of these targets. And so, that's what allowed us to be--

Chris: What's a target? Like Xbox or something?

Adam: Yeah, so Xbox, Switch, Windows PC, Mac as an OS or all the consoles.

Chris: I see.

Adam: Yeah.

Chris: To a varying degree. I just started Crashlands, so we'll get to that in a minute. But I noticed one of the things is you have to sign up your name when you start.

Adam: Yeah.

Chris: You don't get the iOS keyboard. You get your keyboard - or whatever. That does seem just unusual. But it felt like, "Oh, yeah, of course." Right? Because it gets spit out to all these different targets. How are you going to get the native keyboard on all six of these things? It's kind of like, "Screw it. We'll just make our own." [Laughter]

Adam: Yeah. The easiest thing was just to do it that way, which we're trying... For Crashlands 2, we're trying to see if we can use system keyboards and stuff instead. But also, there's a whole language problem there because we don't get to just use system fonts. It's all got to be compiled in, right?

Chris: Hmm...

Adam: In the browser at least, you can just be like, "I'm going to load a font from Google Fonts - or whatever. Straight from the server, right? Any font you want. You can reference system fonts. They have lots of fallback options.

Then also, the Web as a layout engine is so incredible. It'll make anything happen that you need. And games do not have that. All of your UI work is pixel-perfect kind of stuff.

Chris: Mm-hmm.

Adam: Extremely tedious, and so dealing with languages and fonts and all of that is a huge part of the nightmare of getting it all to work. We've had with our... So, original Crashlands supported English and Chinese. Our last game, Levelhead, launched in 2020. It supported 12 languages.

Chris: Whoa!

Adam: Crashlands 2 is supporting 13 languages.

Dave: Wow.

Adam: And so to manage fonts and rendering and all of this different stuff to make that all work, and then you pull up the user keyboard. It's like they could type whatever they want there and try to figure out how to make that map back onto the game stuff is just--

Chris: [groans]

Adam: It's a lot, so that's one of those problems we're trying to solve right now for Crashlands 2 before we get it done. And really, just so that people can name their pets. It's like the tiniest little interaction.

Chris: Yeah.

Adam: But it's one of those that so much of games are about, these little things that when you have them, people barely notice that they're there. And they don't realize they appreciate it because they don't have the alternative. But it's just tons and tons of these little, little details like that where the amount of work on our side, engineering effort to be like, "Yeah, you can name your pet," is just shocking every time there's something that seems straightforward.

I'm actually writing up a newsletter right now--because we have our newsletter audience for our players--about how buildings work in Crashlands 2, which is this series of problems that once you start going down these rabbit holes, it just never ends where it's like, "How do you know if a wall should go down when the player goes to the other--?"

The player goes through a door, so the wall should come down so you can see inside the room, right? It's like, how does the room--?

Chris: Oh, I see.

Adam: How does the room know it's a room? How does the door know it's a door?

Dave: Whoa!

Adam: What if you have two rooms stuck together? The players can make--

Dave: Existential. [Laughter]

Adam: Yeah, everything is existential in game dev. It's actually the most fun part to me of the whole thing. You sit down and do something that maps onto the real world, so it seems obvious when you're like, "Yeah, we just need doors. We just need a room."

00:11:49

Dave: It's funny whenever I hear people complain, like, "CSS is terrible." I'm like, "Have you tried any of the alternatives?" [Laughter]

Adam: CSS is incredible. Yeah.

Dave: Have you tried to build a UI in a video game, because there are some fun things, like anchoring they figured out, and we'll get anchoring soon. But beyond that, it's like build it yourself entirely, right?

Adam: It's just build it yourself. Yeah. Just top to bottom. Yeah.

Dave: And so, I'm just like, "Oh, CSS is so good." Then like you said, I should say Butterscotch Shenanigans is one of the best game studios to me.

Adam: I appreciate that. Thank you.

Dave: I watched all your GDC talks. You and your brothers talk at GDC quite a bit. I feel like you should be on the permanent roster. [Laughter]

Adam: That's also what we think, but every year, we just sit down and we're like, "All right. Time to submit five talks each and see if we get in the door."

Dave: See who gets through. So, that's how I know of y'all. Then you showed up in the Discord, and we're talking. I was like, "No, way! You work there!" I just think that's super cool.

But as part of your job for being the CEO of Butterscotch Shenanigans, you don't just make video games. There's a whole other world of tools and apps and VS Code plugins or GameMaker extensions and all this stuff that you have to build, right?

00:13:20

Adam: Again, it's that multidisciplinary thing, right? There are so many kinds of jobs that have to get done, especially in the indie side of games.

The money is tight just all the time, and all your money comes from games in the first few months. Then again, it takes years. We've been working on Crashlands 2 now for 4 years, right? It just takes so long to make these things.

Chris: Hmm...

Adam: And so, the money that came in from the last game in those first few months just has to sit in the bank and slowly running... It's kind of quickly running out, you know, but just running out as you're trying to get the next thing out. And so--

Chris: That's fascinating to me. I can see that, like a fathead on the game kind of thing. There's no recurring?

Adam: It depends on the financial model, right? There is some, but for almost any game model, because also, as an entertainment industry kind of a thing, it's all about, as you know, it's what people are into in the moment. And so, we still have a lot of people playing Crashlands original. We still have a lot of people playing our other games, kind of all added up.

Chris: I mean it's even on Apple Arcade, isn't it?

Adam: It's on Apple Arcade.

Chris: Yeah.

Adam: It's on Google Play Pass. It's on everything, right? We have this residual tail, but the size of the tail compared to that launch spike--

Chris: Is nothing? Yeah.

Adam: It's nothing, yeah, and so our strategy has been to stay as small as possible from a human resources scale because that's where all of our costs are.

Chris: Mm-hmm.

Adam: Which means, again, it's multidisciplinary. Everybody has got a role, a bunch of roles, and then my big focus when I joined the team was to say, "We've got to lean really hard into tooling and automation and anything we can do to make it so that each one of us can have the power of a million people so that we could just truly do the stuff just with ourselves," because it's just the risk with every single game launch are so high. And if it goes poorly and we're out of money, then we can't...

We don't have two to four years to build the next thing. And so, that's been the reality of our journey so far is that we've had three different points where we were looking at a few months left and got lucky, something good happened, we got a good business deal. Something allowed us to get that boost to get through the next thing.

Chris: Mm-hmm.

00:15:46

Adam: And so, there were a few times where we were like, if we just hadn't gotten a little lucky, we already wouldn't be here. And so, a lot of my job is also to stack the deck as much as we can with our marketing efforts, business efforts, and all of that kind of stuff. But still primarily it's, "How do we just make it so we can make games?"

My goal is not to make Crashlands 2. It's to create the kind of studio that can make a game like Crashlands 2.

Chris: Hmm...

Adam: And do it fast and effectively and with high enough quality that we're not chasing bugs for a million years plus launch because it's also different than lots of other software where you can kind of soft launch things, and you can launch with partial features and this kind of stuff.

In the game space, you more or less have to put it out on day one complete and ready to go. And it succeeds or it fails. And there have been occasional games where they came out of the grave a year or two down the line. But it's just so rare. It's usually it comes out, put all your eggs in that one basket, and then it works or it doesn't.

Chris: Yeah, that would be a real bummer if it actually was a pretty sweet game but there were some just crucial bugs that ruined the experience and it just ... the whole thing.

Adam: Yeah. And it doesn't even have to be bugs either. It could just be a bad market. Our last game, Levelhead, which was in Xbox Game Pass when we launched it, was in Google Play Pass, so it had a big subscriber or a big set of users that came out of subscription services because those players will just try stuff because there's no risk to them to do it.

And we still have a decent little sort of chunk of players who have been playing this game for a long... It's a hobby game, so they've just been playing it day in and day out for a long time, which is very fun to see on our side. But from a sheer numbers perspective, from a success standpoint, it would have killed our studio if we didn't have those deals. The thing too is the per-player sort of revenue that you get and stuff is just really low. You just have to move so many units to support even a small studio based mostly in the Midwest where our costs are low and all this kind of stuff.

Chris: So, is it a little bit like Spotify or something, like some band gets paid on plays, you get paid the same way from these pass?

Adam: Yeah, each one does it differently. Crashlands 2 is going to be on Netflix, on their game service.

Dave: Oh, neat.

Chris: Oh...

Adam: And we have Crashlands is in Apple Arcade. We've had a game in Google Play Pass and Xbox Game Pass that will have similar names. Each one has a different model, and so it varies from just like, "Here's an upfront fee and we'll get X number of years," or whatever.

Chris: Oh, it could be that simple? Be like, "Here's a check."

Adam: It could be that simple, yeah.

Chris: Yeah.

Adam: Which is honestly my preferred one because just knowing what our runway looks like is the most important thing that we can do on our side. But most of them do have sort of a Spotify-style model where there's a pool of games. They have some magical algorithm in the backend that they don't tell you anything about so you don't gamify it. They pay out based on that.

Chris: Yeah. You get a magical check at the end of the month, and it could be $4 or something.

Adam: Yeah, it could be nothing. And we've been on platforms because Crashlands is on just so much stuff.

Chris: Mm-hmm.

Adam: We have platforms where, in a given month, we'll make $20 on a platform.

[Laughter]

Adam: Which if we're doing this as a hobby and you just want proof that somebody cared, then that's really exciting to have even that, right?

Chris: Yeah.

Adam: At the very beginning, we were really jazzed when we just sold a few copies of games. But not when you're trying to run a studio.

Dave: trying to pay nine salaries.

Adam: Exactly.

Dave: Yeah.

Adam: That's nothing, yeah.

Dave: $20 really doesn't materially impact the business. Yeah.

00:19:25

Chris: No Nintendo or Xbox or anything for Crashlands 2. Steam and Netflix.

Adam: Our strategy has been to go as cross-platform as we can to have the biggest market that we can.

Chris: Yeah.

Adam: But like I was saying earlier about your game engine, choosing a good game engine that lets you hit all those different targets and does it really effectively doesn't take away all of the challenge because so much of what we have to do is have the different support for different kinds of controllers and controller configurations.

In our case, we support touch and controller and keyboard and mouse in the same game, and so trying to figure out how to make all of that work. But then you hit the Switch, and the Switch is like, "Oh, hey. Your controller can break in half, and now it's two controllers with fewer buttons each." That can happen in middle of gameplay, so how do you handle that?

Something like the Switch also has these really extreme just hardware requirements where how often you write to disk even is throttled and all this kind of stuff. And so, every time we add a new platform, we have this enormous new set of technical requirements, as well as design requirements, because each one of these platforms, too ... the Web where it's like you get to choose what you're doing.

In these, you'll have... We got rejected from one of our platform partners at one point because we had their sign-in button, and they were like, "Hmm... There are not enough pixels on the right side of this button."

Sometimes it's that level of stuff that we have to make work, and so the amount of... And build times are long and QA is expensive, and the complexity of games because all the systems interact, so you can't test the thing in isolation. So, every time we make a new build, it's a few days of QA time.

Dave: Whoa.

Adam: It's just so costly to do it per platform that we've kind of switched our strategy to say we want to be able to hit every platform or be close to it. but we'll only actually public to platforms where we think we can actually generate enough return to compensate for how much of a pain in the ass it is because it's just so hard to do that.

With Switch, it's extremely hard to be successful on as an indie because it's first-party focused. It's only Nintendo games. You get a few hits that kind of come up from the indie space, but they don't have good curation mechanisms or ways to float things to the top.

Steam on PC is the only platform that really does that, that makes it possible to launch a game, get it in front of people, let their algorithm test stuff out to see if people are into it.

Chris: Mm-hmm.

Adam: But most of the platforms, you just put your game on there with tens of thousands minimum of other games and just hope. You just hope that people play it.

Dave: Hope search works. Yeah. [Laughter]

Adam: Yep. Yeah. I think that's exactly it. On Switch, people only find Crashlands on Switch if they know it's there. That's the only way they find it.

Dave: Yeah.

Adam: And so, with that, it basically broke even for Crashlands in terms of our costs of getting it on there. And that's just too much effort.

Chris: Yeah.

Adam: It's just so hard to build these things already. You know?

Dave: Yeah.

Adam: And with such limited time.

Dave: Screw that.

Adam: Yeah, exactly.

00:22:30

Chris: Netflix is brand new. That seems pretty interesting. Do you have anything to say about that?

Adam: Yeah, I think everyone is kind of trying to enter the subscription space for video games. For a while, everybody was calling their services the Netflix of games, right? Then the joke was, why isn't Netflix getting in on the Netflix of games?

They started to do that. What has it been, like a year and a half ago or something like that?

Chris: It's early days for them. They might be shelling out bucks to get good stuff, maybe.

Adam: My bet is... I can't go into specifics of our deal, of course, but I would say that it would be unlikely that we would get the same deal today that we got when we first started working with them because, at that time... I think, at that time, they maybe had a couple dozen games on the platform. It was very early days when we first started working with them.

Chris: Yeah.

Adam: They've been our favorite ... [indiscernible].

Chris: Do they impart deadlines and stuff, too, then? You can't take another three years to finish, can you?

Adam: They've been so easy about it. So, they're not letting us cost them more money by our decisions how we do stuff.

Chris: Yeah.

Adam: But outside of that, they've been pretty flexible with timelines and all that kind of stuff. It is pretty rare to have a new entrant into the space that actually understands that the core aspect of making games is it takes forever. It's just so slow.

Almost anybody else coming into the space is looking for that quick turnaround, like it's an app or something. Right? And so, they throw somebody upfront, put really tight deadlines on it, and don't have flexibility. Because the other thing, too, with games is it's extremely hard to design it from the outset and then have that design experientially actually be good. You need that iterative process as you go, and you discover things as you go where something you thought was good just isn't, and you need to massage stuff all the time.

Predicting when a milestone is going to be complete or when the game can come out is just a fool's errand. You can't really do it. And you can mostly just try to keep limiting scope to try to hit some target over time.

Dave: Right.

Adam: But when people who are used to very milestone-driven development where it's like, "Okay, here Act 1 is done, here Act 2 is done, here Act 3 is done," that kind of stuff just doesn't work very well in the game space. But Netflix came to it basically like they clearly had some games people involved from the outset, so working with them has been really easy so far. Will it stay that way, who knows because these are all rapidly shifting.

00:25:06

Chris: I would think, because you mentioned the controller difficulties, what the hell is a controller for Netflix? It could be anything.

Adam: Yeah, so Netflix is starting off just mobile so that you're basically looking at touch controls.

Chris: Okay.

Adam: But I don't actually know exactly what the long-term plan is. I'm pretty sure they want to get on TVs somehow, but yeah, there's a big controller problem there.

Chris: Hmm...

Adam: With Apple Arcade, when we were working with them for Crashlands, that was also kind of at the same time. What they wanted to do with Apple Arcade was get games on TVs. But there's not a controller, right? And so, for a while they were trying to make it so that if you have a game on the platform, it would work with the Apple remote - or something.

Dave: Mm-hmm. Yeah.

Adam: Which is a very limited kind of a peripheral for that kind of--

Chris: We have that. I have Apple TV, and so some Apple Arcade games are on there. Some of the games just don't let you do it with the Apple remote. Some do and some don't, and so we have one. I just went to Best Buy - or whatever - and bought some little third-party thingy.

Adam: Yeah.

Chris: It detects it okay. I largely find it fine. But it wasn't--

Adam: Yeah, it just depends on the hardware details, right?

Dave: Yeah.

Adam: I think, for us, the main thing was most people who have an Apple TV aren't going to be playing games with the remote.

Chris: Yeah, yeah, yeah.

Adam: There's always just the market segment problem because that's the other thing with all the platforms, too. They're all trying to use our games to accomplish some bigger-picture business goal of theirs, and a lot of it is selling hardware. And so, they'll constantly be pushing and trying to require us to do stuff that does not make any sense on our side because the market for it is too small. But they're trying to create that market. And so, one of the challenges that we've had is, with some business partners, we can negotiate relative to that. Others are just like, "You can do it or not be involved. Those are your two options." [Laughter]

Chris: [Laughter]

Adam: And so, a lot of our biz dev effort is trying to figure out which platforms to go to and that kind of stuff. They're informed by a lot more than just, "Can we get it to work on the platform?" A lot of it is, "How easy is it to work with this business partner? What does that market look like?" and all that kind of stuff.

Chris: Eh!

Adam: Yeah.

Dave: Man, your--

00:27:14

Chris: I could see how you don't want those two people to be the same person at a game studio. The person who is talking to Johnny Business at Microsoft - or something - be like, "We need you down in the game making sure it's fun."

Adam: Yeah, yeah.

Chris: Not negotiating contracts about hardware.

Adam: Yeah, and it's just so expensive. We haven't had a deal yet that took any less than a few months to fully put together.

Chris: Ugh!

Adam: It's just so much back and forth and trying to finalize. It is part of it, too. Every game is a standalone software product. And so, trying to get... Them trying to have requirements about that adapt sufficiently to any given game without making us change things and us having design decisions about our games because some platforms require cloud saves or require achievements, require whatever, and there are some game designs that don't actually make sense for some of those features.

It's this constant balance of trying to figure out what is it we have to do, like what are some of our design constraints, that are just informed by, we're going on this platform, and they're going to make us do this. So, how do we incorporate that into what we're doing for the game in a way that still works and is good?

Yeah, when you're doing this stuff as a hobby, like when we first started, we were just making what we wanted because it was fun. To actually figure out how to make it successful as a career, it just shifts into biz dev so fast where it's just, "What do we have to do to get somebody to let us put the game on their platform or for somebody to give us some upfront money so that we don't run out?" - or whatever.

Dave: Yeah.

Adam: It becomes mostly about that, so it's like being a startup just all the time because every new game is basically a new--

Dave: A new company, kind of. Yeah.

Adam: It's a new company, yeah. It's just a whole new thing, so yeah, it's a lot.

Chris: Hopefully, a few open doors because of, you know, the 2 at the end of the title there.

Adam: Yeah, well, that's the hope (right) is that we can leverage the success of the one. That's why people do franchises because it's just so risky already to do all of this stuff.

Chris: Mm-hmm.

00:29:20

Adam: But I think one of the most fun things about this, though (because the original Crashlands came out in 2016, which is eight years ago, right?), the people who have been the fan base then is excited when they see your trailers come out and talk to us in various forms and places are talking about it as a game from their childhood. They're like, "I grew up on this," is the vibe, which is really cool because that's why we ended up in games is because we grew up playing Nintendo games and stuff.

We've had that experience of being, like, you play a new Mario game and you're like, "Oh!" you feel that nostalgia from the olden days. It was something we didn't think about that much, actually, was that nostalgia piece of truly literal kids.

Chris: Yeah, yeah.

Adam: Teens playing this game. It was actually an important part of their childhoods, and they couldn't tell us at the time because kids aren't taking to devs about this stuff, right?

Chris: Yeah.

Adam: But they could come tell us today that, like, "Hey, this was really important and I'm extremely excited to relive my childhood through this thing you're making."

Chris: Yeah, a game that will be better. Yeah. Dave and I, when we go to conferences, get that, "Oh, man! I used to listen to that show!"

Adam: Yeah, yeah. Right. Yep.

[Laughter]

Dave: Okay... Uh...

Adam: Is it still going? Yep.

Chris: Yeah. [Laughter]

Dave: Oh, you're still doing it? Wow.

Adam: Yeah.

Chris: Yeah.

Adam: Yeah.

[Laughter]

Chris: Good for you... I think.

Adam: Ah... Yep, yep.

Dave: Yep. Yeah, well--

00:30:45

Chris: Well, that's cool. So, there's some Web tech that jiggers into this. Partially, servers and syncing and stuff.

Adam: Yep.

Chris: That seemed like something that you did early on and then still leverage. I'm very curious about that. It's one of those things, these games can be long, right? At least multiday, right?

Adam: Yep.

Chris: Nobody is knocking through it in one session on the couch. So, how is your progress stored and that kind of thing?

Adam: Yeah.

Chris: It seems like a big deal. You're kind of on your own, right? A little bit?

Adam: Yep.

Chris: I'm sure different platforms have different ways of handling it, but if you're cross-platform, then what the heck do you do?

Adam: Yeah, well, it was one of the interesting problems upfront because even if you're not trying to take the data cross-platform but just want your game to be cross-platform with fully separate data.

Chris: Mm-hmm.

Adam: Yeah, each platform has its own cloud-saving system, and the APIs are different. The way they have you store data is different. Their limits are different in terms of how many megabytes can you have, how many individual blobs can you have. All that kind of stuff is completely different.

Adam: That was something that, very early on, we recognized was like, "Can we just not have to do that? Can we just have our own system that works one way? But then also, let's just go fully cross-platform with people's data." That was our original reason to do it an original strategy.

Over time, platforms have become less amenable to that because they're all trying to lock down data.

Chris: Does that mean, like, "Oh, would you like to save your progress? Why don't you log into our Web-based system"?

Adam: Yeah. Exactly.

Chris: That would seem... I'd be like, "Oh... really?"

Adam: Yeah, yeah.

Chris: As a gamer today, I'd be like, "That's annoying." But I'd maybe.

Adam: Yeah, there are two parts there. Exactly, so players don't like logging into stuff. Nobody does, right? Players really don't, but they've gotten used to the idea that if they're playing on Apple or playing on their device.

Chris: Will they even let you? It seems so not done.

Adam: It's become an increasing challenge, I would say, to have them let us do it.

Chris: Okay.

Adam: Yeah because it's becoming more of a requirement that we use their service to do it and more of a restriction to also prevent us from using ours. So, Apple Arcade is an example. This is public, so I don't think I have to worry about talking about it.

They don't allow any networking outside of the Apple ecosystem on Apple Arcade, right? All of the games just talk to Apple services and that's it.

Dave: Wow.

Chris: Mm-hmm.

Adam: That was part of us getting on the platform is we had to rip out all of the stuff that we were doing for our own system and only use theirs.

Dave: How does that work code-wise? Is it you copy folder--?

Adam: Right. Yeah, yeah.

Dave: --Crashlands, and you say, "Crashlands-applearcade," and then you delete all the network code? Or is it like you have - whatever--

Chris: A feature flag?

Dave: --load game is 20 billion miles long?

00:33:47

Adam: It's feature flag wild, just wild feature flag stuff because of all the targets and the variations within the targets. Yeah, it's all feature flag-based. Technically, the code is in there. It just isn't allowed to run because we have a separate build per target platform, and sometimes separate builds within the operating system. We have a Google Play build and a Netflix build, right?

Dave: Okay.

Adam: They are separate builds because they have to be deployed differently for different reasons and stuff. But usually, we just try to use feature flags and leave it at that. But certainly, for the builds themselves, the build will get made and then we'll have... Part of the automations that we do is we have some scripts we'll go yank out pieces automatically and try to do that kind of stuff for us. But for the most part, it's all the same code.

Dave: So, Netflix is Android or--

Adam: Netflix is both.

Dave: Okay, I guess...

Adam: Yeah.

Dave: Yeah. Huh. Yeah, I guess is Android a lie? Do you just go Play Store, or does Amazon have their store? Do you have to differentiate builds per--?

Adam: We do, yeah.

Dave: Wow.

00:34:55

Adam: We basically do, or again feature flags if we can use it sufficiently. But usually, you have to bake stuff into the actual executable with, like, the ID of your application, which is specific to platforms and some of this other stuff. As much as we can, we try to keep things within single builds for a given operating system target. But often, we just have to do whole-ass separate builds.

Right now, for Crashlands 2, I think we have like seven builds - or something - that get made for every... It's all through CI systems, right? But it's like we version it and then, an hour and a half later, we have seven distinct builds deployed to different QA systems on different platforms.

Chris: An hour and a half?

Adam: Right. Yeah. Yeah, we try and choose the time.

Chris: Blah.

Adam: And so, we do as much local testing as we can before that goes out. But a lot of it, too, is it's all... You have these huge SDKs and all this big stuff you have to pull in. And so, we try to get that stuff done fast and cost-effectively, hitting all the targets.

Dave: Wow.

Adam: It adds up to a lot.

Dave: It adds up. I know early, decades ago, I applied for a job at SquareSoft who makes Final Fantasy. Is that right?

Adam: Yep.

Dave: Yeah, in LA (Los Angeles), and I think my whole job was going to be QA, like walking into walls all day.

Adam: Yep! [Laughter]

Dave: Just like, I just walk into walls and report bugs.

Adam: Doing the automated QA in games is just impossible. It's just... on occasion--

Chris: I was going to ask about that. Is there Cypress for games or something? Can you write end-to-end tests?

Adam: There just isn't. Yeah, because--

Chris: There's just not?

Adam: No because the beauty of end-to-end tests on the Web is like you can just see into the... There's a common structure. It's HTML, right? There's a common kind of layout of how things work, and your ... are there. Everything is just visible, and you can find it and attach things to it.

Any two websites, yeah, they can look completely different, but you can just look in there. You can just dig into them and attach things to it. Also, your workflows tend to be a little simpler for Web apps.

Chris: Yeah, I could see that because you do have the controller. You could send commands in, right?

Adam: Yeah.

Chris: One after another. But then what? At the end of that, it would have to be a screen test, like, do the pixels look exactly like this?

Adam: Yeah.

Chris: At the end of these 100 key presses. That's the most brittle thing I can possibly think of.

Adam: It's brittle, and especially in games because so much of it is randomized so that you don't have exactly the same experience each time.

Chris: Oh... Blah!

Adam: Yeah, exactly. The creatures, they'll spin up their pathfinding AI. They randomly choose in any given frame, "Am I going to attack? Am I going to just sit here? Am I going to go eat something off the ground?" - whatever. No two experiences are even... They diverge instantly and then just keep diverging.

We rely so heavily, so we do have a lot of automated tests for just engine-level stuff, making sure that just the tech works, basically.

Chris: Yeah because there's got to be. Oh, no. Now I'm obsessed over this. Is there a test that's like, "Monster can walk in eight directions," or something? At least a component-level test that you didn't break the monster or something.

00:38:09

Adam: Yeah, we rely so heavily on human testing that the thing basically become an end-to-end human testing experience, if that makes sense. Say a monster doesn't do what it's supposed to do. Rather than trying to have individual tests that handle that, especially given that they're randomized and all of that stuff (whether they're even rendered or if their sounds trigger on time), there are so many components that go into it.

Rather than doing that, we just say, "Well, our QA have to just go experience the game. No matter what, they're going to have to go play test some stuff."

Chris: Yeah.

Adam: While they're doing it, they're going to offhand notice if something was weird and janky. Right? So, even when they're not... It is the nice thing about it is when you're not specifically testing a thing because games are so interwoven in all of their features and they all interact with each other, that's what makes it impossible to test automatically. But it also means that you just incidentally test stuff while you're testing the thing that you care about. And so, a lot of our time over the past six years has gone into figuring out how to do QA efficiently and really well.

Chris: Yeah. What happens when you notice something? Jira? What are you using?

Adam: We have our own system. We try to have our own system for everything so that way it's built for purpose.

Chris: Yeah.

Adam: But it's basically like an in-game report mechanism that scoops up some data from the game.

Chris: Ah! So, you get some metadata stuff along with it. Yeah, that's nice.

Adam: And a lot of that is version management because so much changes between versions and making sure we know, like, "Did this report that came in a week ago, does it still apply?" and that kind of stuff.

Chris: Oh, oh, oh! Yeah, I see! I see. That's funny. We've been having this company jam.dev be our sponsor, and this is one episode break where they're not on it.

Adam: Ah, yeah.

Chris: But it reminds me of it because they have this browser extension tool for reporting bugs that sucks up all this extra data of what is going on in the game.

Adam: Yeah.

Chris: It's hyper-relevant to this.

Adam: Yeah, and that kind of thing is just so useful to be able to say where they're at in the game. Sometimes some information about what's on screen, that kind of stuff. But mostly, we just want to know metadata level what state is the game in, and then we just rely largely on people taking videos of stuff of the stuff that they see.

Chris: Yeah.

Adam: Because it's also so hard to explain without, like, "Here's a video showing what I'm talking about."

Chris: Right.

Adam: Sometimes it's down to, like, yesterday somebody reported to us that -- the details don't matter that much -- the animations and how the attacks work and how that rolls into an automatic switch to a different weapon kind of a thing, right? Just kind of like action-packed kind of parts of game mechanics that they noticed that, depending on the animation time, there were some very slight, 100-millisecond sort of range differences between how long it would take to automatically switch.

As a consequence, they were finding that they were accidentally unequipping something. So, they put together this just absurd video where they carefully took little snippets, starting at the same rough time of showing each of the different things so they could watch them swap, right?

Chris: Mm-hmm.

Adam: All side-by-side. That whole video duration was 100 milliseconds, right? Then we were crawling through it frame-by-frame to figure out, like, "Oh, yeah. No, we can see this now that we've kind of boiled it down."

Chris: Oh, my gosh.

Dave: Do you just ever want to message back, like, "Shut up, nerd!" or something?

[Laughter]

Adam: It is--

Chris: No, you want to send them $100 for being amazing.

Adam: Yeah, exactly.

Dave: Yeah.

00:41:28

Adam: We love when... We love getting those reports, but there is this notorious challenge, and in any industry but definitely in games, of QA versus the devs because the QA's job is supposed to be to find all the stuff that you missed and didn't think about. And a lot of those little details are things that you're like, "Yeah, but how much does it matter?"

There's so much of trying to find those lines. And it gets really easy on the dev side to get overwhelmed by just feedback reports and stuff. A lot of what we try to do is just make sure we're maintaining an extremely good relationship between our QA and our devs.

Chris: Mm-hmm.

Adam: And a lot of that is focusing on triage where it's giving people feedback about their feedback and, as we're in different points of dev, at some points we want all those little details. At other points, it just isn't broken, that's all we care about, and trying to make sure we stay on top of that so that no one is getting overwhelmed or upset.

A lot of the human management side of that is just kind of a big part of the challenge. But it is also what makes it fun.

We have one of our QA members in particular who just loves doing it. He will find... He will take all day trying to find any edge to anything. He loves doing it. And he is so satisfied when he finds something that we have kind of the inverse problem where, when he can't find something, he starts feeling like he's not doing his job and that kind of stuff.

Dave: [Laughter]

Adam: Because QA is a weird role in any industry because the KPIs are whacky. If you're finding lots of bugs, that means lots of bugs are being produced and you're finding them. If you're not, either you're doing a bad job or there aren't a lot of bugs coming in.

Dave: Yeah, or you have a huge blind spot. Yeah.

Chris: Yeah, I find that gut-wrenching almost.

Adam: Yeah.

Chris: And I don't like the incentive system there. Yeah.

Adam: No. Yeah, so part of it, too, is trying to avoid because we don't have that as an incentive system in the studio. But we have to actively work against that as just a psychological internal KPI because a QA tester, they're thinking about how many bugs did I find, right? And if they keep not finding any, and they don't know if it's because they're just not finding them or because they're not there, then we just have this constant push and pull of trying to figure out are things going well, are they not, is everybody emotionally healthy, where are we at? And so much of all of this stuff, because of how integrated it is, requires that careful psychological work of keeping on top of everyone's mental well-being.

00:44:00

Chris: It also means they get to play the game, right? Isn't there some incentive there, like early access or whatever they call it?

Adam: We do that for betas and stuff. We'll have a beta coming up in the next couple of months or so. It'll also be closed, so it'll probably be 50 people or so that we'll invite to try to play the whole game.

Chris: Yeah.

Adam: Those are just people from the community who are just doing it--

Chris: Yeah.

Adam: --because they think it'll be fun to see it early. For our own QA, we have a couple of full-time QA and some part-timers that we have. And a lot of indie devs just don't have QA because it's hard and expensive.

Chris: Yeah.

Dave: Or it's them. Yeah. It's just--

Adam: Or it's just them. Yeah, exactly. Or they'll try to recruit people from out in the world to playtest stuff for them. But it's one of the things we've invested heavily in at this point.

It's like tools and QA. If we can do those things really well, then we can make sure because our first... When we launched the original Crashlands a million years ago, the post-launch stuff was the biggest nightmare. There were just so many bug reports coming in. Things were so broken. And what was supposed to be very exciting because, like, "We did it!" and people liked it and were buying it and playing it, and that was amazing, but also we were working 18-hour days for the next 2 weeks trying to fix it.

Chris: Oh, I got it. So, it's harder when it comes out.

Adam: Yeah.

Chris: It's not a big... Yeah.

Adam: It was so much harder, and so we learned that lesson. We were like, "Okay, well, if we actually do--" because success is its own... has its own huge costs that we just hadn't had to deal with previously because the impacts... because our prior games were free to play. They had lots of players, but any one player wasn't going to be too demanding.

Kind of going back to that question of, like, what does the data look like and how do we store it, it was easier to recover things because the data is like, "Oh, this number is bigger and this number is bigger," kind of stuff. It's easy to merge things and recover. When it comes to a long-form game like an RPG like Crashlands or Crashlands 2, we're saving tremendous amounts of state data covering huge amounts of player time. And so, when stuff goes wrong there, we feel absolutely horrible.

That's our number one nightmare is player data loss. We feel bad, of course, but not nearly as bad as the players feel about it when they've sunk 30 hours into their gameplay campaign and then suddenly it's gone. And so, avoiding that kind of stuff through really robust QA and being really careful about that has quickly shifted.

In our last game launch, we never really got the rewards from that as we had because we were in ... services, in the first week we had a few hundred thousand players, which is a lot of people to be managing.

Chris: Mm-hmm.

Adam: But it went off just fine. We got some feedback and some little bug reports and stuff, but actually, it was weirdly casual.

We had 12 languages that launched on 7 platforms. It was this huge ordeal to get the thing out. And we pushed all the buttons on release day and it goes live. People start playing it. And we were ready.

We don't crunch. We avoid it at all cost, but were like, "Weekend of launch, everybody just assumed we're crunching." We keep it as tight as we can but assume that.

We just all kind of hung out. We got to just watch videos of people playing the game, and that was that... We were like, "Okay, yeah. This was the right move."

Chris: Does stuff show up on YouTube and stuff because it's launched?

Adam: Yeah, yeah. Yeah, exactly.

Chris: Yeah, the YouTubers show up.

Adam: Yeah. Yep.

Chris: Cool.

Adam: And streamers are playing, and so it is actually getting it out and getting to see people respond to it is just so fun.

Chris: That's got to be pretty fun.

Adam: Yeah, it is.

Chris: Yeah.

Dave: That's fun.

00:47:40

Adam: Especially the fun surprises that we put in there, which is part of the challenge, too, of making games is like we want to keep people excited for the four years it takes to make it, so we want to show them stuff. But we also want to get that chance of surprise, and so we have an early thing that people already know about because of the demo we put out a little while ago. But we really wanted to be able to save somehow, so we could watch streamers react to it at launch day.

Chris: Mm-hmm.

Adam: But when your player takes enough damage that they "die" because they actually don't (in the game), they get teleported away and leave behind a whacky inflatable tube man that we just call a decoy (right) that's pretending to be the player with this EDM music sitting on top, just this absurd moment that we tried to have a lot of in the original Crashlands and got to see people react to it and just how much fun they had.

Chris: Mm-hmm.

Adam: We tried to do that again but then not leak that too early has been one of those interesting challenges, too.

Chris: Yeah. Yeah, what do you allow in the trailer and what don't you.

Adam: Yeah, that's always a big fight in the team because we want to put out new stuff so that it's worth watching--

Chris: Yeah.

Dave: Yeah.

Adam: --but also we don't want to put out any of the new stuff because then people will know it's in there. Yeah.

Chris: Yeah, yeah, yeah, yeah.

Adam: Hype-driven, you know. It makes it hard to do product launches.

Dave: Well, I unfortunately do have a hard stop, so I'm going to have to cut it off. I feel like we didn't even get into all the tooling y'all make for GameMaker. You have stuff on a VS Code extension, your Stitch. I guess Stitch is a suite of tools for GameMaker.

Adam: Yeah.

Dave: And you can use GameMaker in VS Code, which is kind of cool.

Adam: Yeah. Yeah, so we do all of our game programming in VS Code now through that extension instead of the official IDE because then we get to use all the cool stuff that VS Code has. You know?

Dave: Yeah.

Chris: That's cool.

Adam: We have a lot of other stuff. But yeah, if you guys want to get into that stuff on a future chat, we could do that, definitely.

Dave: Yeah. No, it's very cool. I think you've done a really good job of ruining my hopes and dreams.

[Laughter]

Dave: But I say that in the most honest sense. A lot of game dev videos, you watch YouTube, they're like, "Just make a game in a weekend. Unity. Done." You know? And I think you give a really good, honest, like, "Okay, cool. You did that. But here's what it is if you're doing it." That's been really educational.

Adam: Yeah. The hobby side and the business side are just so different.

Dave: Yeah.

Adam: But yeah, the hobby side, though, is super fun because you do... We used to do game jams all the way through most of Levelhead's development, so through 2019 we were still doing game jams because that was the time we got to do it again, where we got to just make a thing and not worry about, "Is this viable in the market?" and "Have we covered all of our platform target bases?"

Dave: Yeah.

Adam: Is it fully tested, you know? So, yeah. No, I think the fact is the hobby space in games is a super fun, cool place to be in. And it's just fun to make these things.

Dave: Yeah.

Adam: But yeah, it's a very challenging industry.

Dave: Cool. I mean thank you for your insight on it. It's very cool just to see what y'all have done. And good luck on Crashlands 2. For people who aren't following you and giving you money, how can they do that?

Adam: Yeah, I think the best thing you could do is just go to crashlands2.com. From there, you can get to everything. You can get to our podcast. You can get to our other games. You get to all that. But the biggest thing, really, is if you play games on PC on Steam, just wish list Crashlands 2. That'll help us get that thing as many eyeballs as possible once we go to launch it.

Dave: Okay, Shop-o-maniacs, give it the ol' ShopTalk Show bump. We know it works. [Laughter]

Adam: Yeah.

Dave: But yeah. And thank you so much for coming on. This has been very cool.

Chris: Yeah.

Dave: Thank you, dear listener, for downloading this in your podcatcher of choice. Be sure to star, heart, favorite it up. Follow us in the D-d-d-d-discord. Adam is there. Then, yeah, I'm out of words to say. But Chris, do you got anything else you'd want to say?

[Laughter]

Chris: My game is going to be like Wordle or Simpler or I'm out. You know?

Adam: Yeah.

Chris: That's... [Laughter]

Adam: Wise.

Chris: Yeah. All right, thanks. See you later. ShopTalkShow.com.