Transcript

[Banjo music]

MANTRA: Just Build Websites!

Dave Rupert: Hey there, Shop-o-maniacs. You're listening to another episode of the ShopTalk Show, a podcast all about frontend Web design and development. I'm Dave Rupert, and with me is Chris Coyier.

Chris Coyier: Hey! This is our second episode in kind of a loose miniseries we're doing kind of digging into the modern JavaScript side of frontend development, I guess.

On our first episode, if you listened last week, we talked to Henry Zhu of Babel and got his whole perspective on what Babel does for the universe. It was such a cool way to start this series because, if you think of a thing that has a huge impact on the JavaScript ecosystem, certainly it's Babel. What has an even bigger impact on the JavaScript? Hmm. I'd say you can't get much bigger than NPM, and so we have an incredible guest. Laurie Voss is joining us from NPM. Hey, Laurie.

Laurie Voss: Hi. Thanks for inviting me.

Chris: Yeah, this is wonderful. You are the CDO, the Chief Data Officer, at NPM. Can you tell us about that, what NPM is, and that type of stuff for us?

Laurie: Sure. Well, regarding the title, the thing that I tell people these days is, w, when you're a founder you get to make up a title that has the word "chief" in it. I don't know what Chief Data Officer means at other companies but, basically, what it means is I get to dig into the mountain of data that NPM generates every single day and try and figure out things that are useful to know about of that stuff.

In terms of NPM and who we are, we started out as the Node Package Manager, and then we became the JavaScript Package Manager. A couple of years ago, we sort of realized that we had become kind of the package manager for Web development, generally. A lot of the stuff that's happening inside of NPM isn't JavaScript anymore, which was a big surprise to us.

I work at NPM, Inc., which has been around five years now, but NPM, the open source project, started ten years ago in 2009. In fact, its birthday is coming up in September.

Chris: Hey, ten years!

Laurie: Ten years of people complaining about node modules being too big.

Chris: [Laughter] But there are people out there, perhaps for better or worse, that what they think of as NPM is just something that their teacher to type into their terminal and it just works somehow. They've installed it or it came preinstalled on their computer. It does some magic for them. It just pulls some files out of the sky and that's all they think of it as.

It's true. That's what it does. But, like you said, there's NPM, Inc. as well, which of course there is, right? There needs to be some company that makes that command work and shells out for the, I'm sure, extreme amount of bandwidth required to make that happen. That's why NPM, Inc. exists, yeah?

Laurie: Yes, absolutely. In 2013, NPM had something like 1.3 million users.

Chris: Whoo!

Laurie: It was still Isaac's sort of nights and weekends project when he was working at Joyent, and so he was looking at this thing. At the time, NPM was down for like eight hours a day. Right? Any time Isaac went to sleep, NPM would fall over and it wouldn't come back up until he woke up again. It was famous for its downtime.

He looked at the situation and was like, "Well, this isn't going to work. It's either going to collapse under its own weight or we have to make it sustainable."

We'd know each other a long time already by that point, and he was like, "All right, well, we need to make it sustainable and one of the ways you can do that is you can form a foundation like some other packaged managers have done."

We talked to the people who run the foundations, and their feedback was not good. Their feedback was like, "Well, we have a fixed amount of money that we have to beg for every year, but our costs go up all the time," and so it's just not a good model.

Chris: You're like NPM PR.

[Laughter]

Dave: Mm-hmm.

Laurie: Instead, we decided to go for a more sustainable model of a company whose revenue scales to how popular it is. That's been working so far. Now, NPM has 11 million users.

Chris: [Whistles]

Laurie: And they download a lot more stuff. The registry has grown 2500% since NPM, Inc. because can inc.

Chris: That's something else. It's a little closer to the GitHub model, right? Oh, you want privacy or enterprise, that's a paid thing.

Laurie: Exactly. Until quite recently, people were like, "Well, we know if GitHub's model is a good model." Now we're like, "Hey, remember them? They got acquired for $7.5 billion? It seems they were doing okay."

[Laughter]

Chris: Cool.

Laurie: We're like, "Yes. We've got a business model and we like the business model. The point of the business model is that it keeps the registry running forever."

The things that we give away for free are open source packages. Anybody can upload one. Anyone can download one.

The two things, the two primary things people pay for are being able to keep packages private, so if you want to keep it private to just your team or just your company, you want to have different permissions for people to be able to publish or install, that kind of stuff; and also, security, which is becoming a bigger and bigger part of what we do.

Chris: Mm-hmm.

Laurie: At the beginning of 2018, we acquired Lift Security in response to how big a deal security was becoming for us because people were very concerned about, you know, "I download 2,000 packages to make my app run. It's great that I don't have to write all of the code in those 2,000 packages, but I don't know who they're from and I don't know if it's good."

That's a valid question. We work on that and provide both security, free security services in the form of the NPM audit command and, for enterprise customers, we provide deeper paid services.

Dave: That runs every time you NPM install, right? I see a little message, like, "You have 53 vulnerabilities in Gulp.

[Laughter]

Laurie: You should probably take a look at those. Yes, yes.

Dave: Yeah, yeah.

Laurie: What runs every time you install is what we call a quick audit, which just tells you, you've got something wrong or you haven't. Then the NPM audit command will tell you everything that's wrong in detail.

One of the fun things is that it will tell you how to fix it if it knows how to fix it. The fix is usually that you find the vulnerable module and you upgrade it to the fixed version of the module because, usually if somebody is finding a vulnerability, it immediately results in the author patching that vulnerability.

This is so funny because we released the NPM audit command and, a week after releasing the NPM audit command, we were like, the audit command basically just tells you to run NPM. Why do we have a command? Why is there an NPM command that tells you to run NPM? Why doesn't it just run itself? Then we added the NPM audit fix command, which is what that does.

[Laughter]

Laurie: The NPM audit fix command is just fixes everything. It's just like, okay, you've got 53 vulnerable packages. Here are the 53 new packages that you need to install instead to fix everything. Now you have zero vulnerable packages. You're welcome.

It doesn't work for everything. Sometimes, to fix the vulnerability, you need a breaking change, and that means that you have to say, "Okay. It's fine. My code will break. I will need to manually fix it." But it really takes a lot of the effort out of keeping your stuff secure.

Chris: That's a big deal. It's nice to know. It would be very unfair to think that NPM doesn't care about security. You do a ton of work for this, including integrate it directly into the tools that people use every single day, which is pretty rad.

How do you spread the word about something like that? It's nice that we are able to do it here in a very small way, but with 11 million users.

Laurie: Well, building it into the command helps a lot. You'd heard about it because you saw the security command. Lots of people just run the command, run NPM install, and go, "Oh, there's a security command?" Then they find out about it that way.

You're right. Letting people know what's going on with NPM is a really big challenge for us. Something like half of the 11 million people who use NPM, they think about it the way that you said that people think about, which is, it's just this thing that my teacher told me to run. Half of them don't know that there's an NPM, Inc. They don't know that there's a company. It's not that we make products that they don't want. It's that they don't know that there's a company.

Chris: You're a data officer, so when you say half, you mean it, right?

Laurie: I mean half.

Chris: Yeah.

Laurie: I've got the data. They don't know.

[Laughter]

Dave: That's cool and hell of a frightening, isn't it, that people don't understand the infrastructure but then, also, cool that the infrastructure is invisible? Is that exactly where you want…?

Laurie: Yeah. It's a testament to our reliability and uptime now that people don't think about where NPM comes from and what it's doing because it just always works or it nearly always works. The downside there is, they don't think about it. Therefore, that half of the people, we can't talk to them to tell them that there's new stuff that they need.

We've been doing our best. We've been broadcasting on every channel we can find, going through every article, going to every conference we can manage to go to. We are making some progress.

In particular, one of the things that we've been trying to get across to people is, the average Web app these days, a rich Web app, it has 2,000 NPM modules in it. You can't just hope they're okay. You have to be using security services.

The sign that that's working would be, has the number of people who are concerned about security gone up? Everybody should be concerned about security. The numbers should be 100%.

Last year, it was 77%. This year, it's 83%. So, we made some progress. A lot more people have realized that this is a thing that they need to take care of, and a lot more people are taking care of it.

Last year, only 39% of people were running code scans to check for security. This year, 46% of people are. That's still less than half, right? It should be 100% of people. But it's a big improvement.

Chris: It is not a lot of people. I live in a somewhat small town, and we have a tiny little JavaScript meetup. We choose one week to just sit around and talk about JavaScript security.

It starts with NPM because, of course, occasionally there are major news articles that highlight some particular problem that happened throughout the NPM thing. But then I think people realize that it's not everybody. I'm sure maybe it feels differently to you being as intimately involved as you are, but not everybody jumps right on NPM and says, "Oh, can you believe they let this happen again?"

Some people are like, "Well, it's really the maintainer's responsibility or it's the people that use packages at all. Why would you let some obscure thing happen here?" Where you kind of place--I don't know--responsibility or blame can be in lots of different places. But, of course, I guess you, as ",Inc." want to do as much as you can.

Laurie: I feel both ways simultaneously. On the one hand, when people say, "JavaScript security is because NPM sucks," I'm like, "That's not fair."

Chris: Uh--

Laurie: That's not true. But also, is NPM the company that is best placed to be doing something about JavaScript security? That's absolutely true. Is it always the case that NPM could have done--?

Every time there's a security problem, is it the case that NPM could have done more and that would have made this less likely? That's also true. Right?

It's simultaneously not our fault, but also kind of our responsibility. It is a thing that we should be taking care of, which is why we're taking it so seriously.

Chris: Mm-hmm.

Laurie: But I think the biggest misunderstanding people have about JavaScript security is that the thing that gets all the press, the thing that gets people to notice JavaScript security is malicious code. They hear about event stream. They hear about less than.

Chris: Right.

Laurie: Those are the two big ones people can remember. But those, they happen once a year, maybe twice a year. They're really rare. Malicious code on the registry is a very rare event.

What happens all the time, what happens 99% of the time, is accidentally vulnerable code. Somebody made a hole in a server. They accidentally left a vulnerability to flood attacks. They are mis-parsing something. Their regex is bad.

Chris: Mm-hmm.

Laurie: Ninety-nine percent of the things that are vulnerable, the author didn't know that they were doing something wrong, and there's always a fix, but people haven't picked them up.

Dave: That's usually kind of like--I don't know--deep nerd security stuff. Sort of like, oh, if you throw this character at this regex, you can actually--I don't know--trick the DNS to do something. Those are the kind of problems that are kind of hidden.

Laurie: Absolutely. The problem is that that's the thing that gets you owned. Right? The one that everybody hears about, nobody gets owned by those things because everybody heard about it and everybody is like, "Holy crap! We need to fix that."

Chris: [Laughter]

Laurie: It's the ones that stay under the radar that sit there for months unpatched and people get owned by them because, by the time a hacker gets around to figuring out how to really exploit that bug, they still haven't fixed it.

Dave: Yeah, so how many packages in NPM? Do you have that number?

Laurie: It goes up by 700 a day, so I always have to check the homepage to find out what the number is going to be today. Today, it's 892,000.

Dave: Wow. We can only really pluck out two famous--

[Laughter]

Dave: Two famous explosions, I guess.

Chris: It's high, but it's not astronomical. It's a huge number, but it's not in the billions or something.

Laurie: We've also discovered that the number of packages, number of popular packages, is relatively small within that group. If you want to talk about the number of packages, if you want to say, to provide 100% security for everybody who is using the registry, you would have to secure all 892,000 packages, right? Everybody is using one of them for something.

Lots of them have only a handful of users. So, if you wanted to take care of 50% of downloads, if you wanted to make sure that 50% of all downloads were secure, you'd only have to secure --

Chris: Let me guess.

Laurie: --about 500 packages.

Chris: Five hundred! Oh, my gosh! Wow, so it's way less than one percent. The fat head of NPM is super fat.

Laurie: Right but, the bigger the percentage you try to cover, that number gets real big real fast. Right? To go from 50% to 75%, it goes out to 10,000. [Laughter]

Chris: Okay.

Dave: Wow. The bulk of NPM, I guess most projects only use or the majority of them use 500 projects? Is that how I'm understanding it?

Laurie: No, it's not quite like that. It's 50% of downloads. Nearly everybody is using something in the long tail, but nearly everybody is also using packages in that 500.

Dave: Okay.

Chris: Wow.

Laurie: If we completely secured those 500, we would probably not completely secure any particular NPM user.

Dave: Mm-hmm.

Laurie: Right? Because everybody is using something weird.

Chris: React. Express. What's the top ten?

Laurie: It's weird. Nothing that you'd think of as being a famous package is in the top ten because all of the really top ten are sub-sub-sub-dependencies like really, really popular shunts of other stuff.

Chris: Fascinating! Really?

Laurie: It's like WS and Q. You don't think about either of those as being a popular package, but those are the two. Those are the top two.

The only one that's in everything, as you probably have heard of is, is lodash. Lodash is just in everything. Everybody uses lodash for something.

Dave: Oh, you can't escape lodash. [Laughter] People use lodash for things they probably shouldn't.

Laurie: Yeah. [Laughter]

Chris Enns: This episode of ShopTalk Show is brought to you by something really cool. It's an alternative to .com. It's the .design domain name. If you're a designer and you've thought of a sweet name for your website and it isn't available under .com, check out .design. Chances are, the domain name you want is waiting for you. You can head to porkbun.com and use the coupon code SHOPTALK on the checkout page to get your free .design domain name for your website.

And .design is super widely used, too. It's not like it's a super weird one anymore. There's airbnb.design, facebook.design, uber.design, adobe.design. There are so many of them. It's exactly the same. It's just a domain name and Google doesn't really care these days. It functions the same way as .com or .org. It's just more interesting.

It looks great on resumes or business cards, on email addresses, and it's free. Did I mention that? You can go to porkbun.com and use coupon code SHOPTALK at checkout. You get a free year of email hosting, whois privacy, SSL certs, and all that stuff, which is pretty fantastic. You should go get one right now. Thanks to porkbun.com for sponsoring ShopTalk Show.

[Banjo music]

Chris: You're a data officer, but you produce a report, yearly. Is it survey-based or a combination of survey and usage-based and stuff? There are all kinds of interesting data to uncover here. Are you in the early days of processing that or is it about to come out?

Laurie: Yeah. Last year, we ran a survey. It was kind of a large one. We were just like, "Let's run a survey and we'll see what happens."

We got 16,000 responses, which really shouldn't have been a surprise. We had ten million users in 2017, we should have expected that a lot of people would reply.

There was so much data in there, we basically just dined out on it for the rest of the year. We were like, "All right, well, this informs what we should be doing about security. This informs our product strategy. This informs how marketing should be talking to people."

That was when we found out how big a deal TypeScript was. That was when we found out how many people are looking at GraphQL, which was a big surprise for us. A bunch of stuff that we didn't know was as popular as we thought it was going to be turned up in that data.

We ran it again at the end of last year and we got 33,000 responses this time, so more than twice as many. I am just, just beginning to dig into that. In fact, last night was when I finished the first pass of digging through that data and found a bunch of fun stuff in there.

Chris: You have access to data all the time of what's getting yanked right now. To some degree, that will inform you that TypeScript is popular. Presumably because there's--who knows--some webpack loader for TypeScript or something that gets pulled.

Laurie: Mm-hmm.

Chris: Or maybe you have to poll TypeScript itself, but that's different than asking somebody, "Hi. What do you like?"

[Laughter]

Laurie: Yeah.

Chris: And they pick it.

Laurie: We also do a lot of mining of the registry data itself.

Chris: Yeah.

Laurie: How much is this being downloaded? Is this trending up or down? I have this metric that I like a lot and I use it all the time called "share of registry" because it's this funny thing. Every package author looks at their package and goes, "Wow! I have 10% more downloads this month than I did last month. My package is getting super popular." The thing is, the whole registry grows by 10% every month.

Dave: Oh!

Laurie: Every package gets 10% more downloads every single month because the whole registry is getting bigger. Tens of thousands of people show up as brand new JavaScript users every single day. To really figure out if your package is getting more popular relative to everything else in the registry, you have to so math.

Dave: I'm out. I'm out.

[Laughter]

Laurie: Luckily, it's not super complicated math, just a little bit of math, but it produces some really interesting, interesting data.

Chris: That is interesting. Last year, you find out TypeScript and GraphQL are cool. What else is on that list and is that list the same this year or did we see any new players?

Laurie: I think the biggest surprises last year were quite how popular TypeScript was. Forty-two percent of people said they were using TypeScript last year. We asked again this year, and that number jumped to 63%.

Chris: No kidding, of 33,000 people. Wow.

Laurie: We were like, all right, last year the question was, "Do you use TypeScript?" This year, we were like, 46% is a big number. When you say "used," what exactly do you mean? How are you using this? How often are you using this? Are you using somebody else's TypeScript? Are you writing TypeScript yourself? Did you try TypeScript one time and that counts as using it?

Chris: Right. You already know that it's cool, so you marked "yes."

[Laughter]

Laurie: Right, so we went into a lot more detail this year, and we got numbers that were lower, but a lot more believable. So, 63% of people said they use TypeScript, but 15% of people just say that they're using TypeScript by which they mean they're using a framework or a library that uses TypeScript.

Dave: Yeah.

Chris: Yeah. I know Angular is written in it, and I use Angular. Thus, I use TypeScript.

Laurie: Exactly.

Dave: Yeah.

Laurie: A bunch of people who use Angular report themselves as using TypeScript because they do. Of course, they do. They're using Angular. But they don't write TypeScript.

People who say that they write TypeScript is only 42%, which is still a lot. We dug deeper into those people, and we were like, "Do you write it all the time or are you just trying it out once in a while?"

Within the 42% who say they write TypeScript, 52% say they primarily write TypeScript. They write it all the time. It's the thing they do most of the time. Another 34% say they do it some of the time.

If you do the math on that, it means that, overall, 36% of NPM users are writing TypeScript some or most of the time. That is amazing. Right? A third of NPM isn't JavaScript anymore. A third of NPM is this other thing Microsoft invented.

That's going to be a huge shift in 2019. It's going to change how we talk about Web development because NPM has been talking about ourselves -- you know we stopped calling ourselves the Node Package Manager five years ago and changed to calling ourselves the JavaScript Package Manager, but that's not true either. Right? There's just tons and tons of TypeScript in there now and tons of people writing TypeScript. They're all using NPM every day.

Now we're like, we're the script package manager, I suppose. I don't know. [Laughter] We're some kind of package manager.

Dave: Does this put NPM in a place where it's a bit of a trendsetter? People, I'm sure, are going to look at this jump in TypeScript usage and be like, "Oh, I've got FOMO. I've got to get on this TypeScript train." Do you feel like NPM has a place in that? Is that good or bad?

Laurie: That's a really good question. I think the fact that we are talking about TypeScript now and saying, "Yes, this is a trend. It's real. This is definitely a thing that people are using and not just saying that they're using," will get some more people to use it than were using it before.

But we didn't make it up, right? Thirty-six percent of people, that's more than four million people. We didn't plant those 4 million people and force them to start writing TypeScript. They found it all on their own.

Dave: You're not some George Soros super company, [laughter] Microsoft funded dark money.

Laurie: Right. We're not creating this trend. At this point, we are just reporting this trend. Generally, that's what we try to do. NPM doesn't try to tell people the right way to do things. We try to observe what they're doing and make that easy.

We're going to do that with TypeScript. If TypeScript is what everybody is doing, then TypeScript is what we're going to let you do.

There is an experimental new version of NPM called Pink that I'm going to be writing about some more in the coming months. It's like node except it runs TypeScript natively. You don't need to transpile anything. You can just write TypeScript and run Pink and your file and it works. You don't have to think about where the module is coming from. You don't have to think about how the transpilation is happening. You just write TypeScript as if it were a first class language.

It works today. It's really magical to play with. We're going to be doing more and more of that. All of the stuff that people do all the time that currently is a third party tool, we're going to start going, "Well, if everybody does it, we should just do it by default."

You use webpack all the time? Webpack is going to be built in.

You use Babel all the time? Babel is just going to happen by default. We are just going to make it as easy as it possibly can to do the thing that you were already doing.

Chris: Pink makes a lot of sense to me. It's like Wayne Gretzky. Let's skate where the puck is going to be, kind of thing. We can see these trends happening. Let's build tools to support what people want to do.

Then, at the same time, I feel exactly the opposite. Why? Nothing at NPM seems like it needs to fundamentally change to support trends. It's just some files, so who cares? Why don't you just let trends come and go? It's kind of awesome to have this technology that transcends trends.

Laurie: [Laughter] I think I've been doing Web development a while now. I've been doing Web development for 20, 30 years. I think one of the constants of my career has been that old-timers like myself look at something and go, "Ah, that's just a flash in the pan trend." Then it turns out to be how things work now. Right?

When I was just starting, I was using PHP, and people were like, "Ah, it's just a trend. You're just using a framework. You should learn Pearl like a real programmer."

I was like, "All right, but I'm not going to. I'm just going to keep using PHP because it's much easier."

Whatever is easiest tends to win. That seems to be the guiding principle. If people are finding TypeScript solves problems for them and a lot of people are adopting TypeScript because they find it easier to get stuff done, then it's not going to be a trend, right? It's just going to be the new normal.

Chris: Yeah, yeah.

Dave: Wow.

Laurie: React, I think -- I don't know that I know that TypeScript is going to be that yet. TypeScript might still be a trend. It's not big enough. Thirty-six percent isn't big enough.

What is big enough is React. React is at something like 63% adoption. That is more than twice as big as any other Web framework has ever been. It's gigantic. It is unprecedented.

It means that there's a whole generation of developers who have come up in the last two or three years for whom the React model of building a component where we break all of these rules that we thought were true in 2005, they were like, "Yeah, your CSS, your JavaScript, and your HTML, they're all mixed together in this one big file. It's fine." That's how things work now. That's how they grew up.

Chris: Right.

Laurie: That's how they expect things to work.

Chris: I would think that that trend is such a big deal that it will affect what choices browsers make because they could be like, "Hey, hold on. I don't think this is a good way to ship code to us." The authors are going to be like, "Well, get on it. I don't care what you think. [Laughter] This is what we're doing."

Laurie: I had a fascinating conversation, with the people who work on Chrome, late last year where I'm not even sure if they have publicly announced it yet, so I won't go too far. Basically, they said, "We are going to go where the river is going. We're going to stop trying to push back against it. Everybody is doing this React thing. We are going to make it so that browsers are good at React."

What does that mean exactly? I don't think they've fully decided. Obviously, Web components exist. The people who use Web components say that they are fast because they are built into the browser, but they also say that they don't like writing them. They're kind of a pain. The developer ergonomics are not good.

Everybody likes the developer ergonomics in React. So, if you built React into browsers, then you would have the developer experience everybody likes and you'd suddenly get the performance benefit of it being built into the browser. Plus, all of these websites that already exist and are built in React would just get faster for free. That would be amazing. I think that's a really exciting direction.

I think the thing that points the way here is jQuery. Everybody talks about jQuery as, like, "It was this library that we used to use. Nobody uses jQuery anymore." That's not cool. That's not true. Everybody uses jQuery, right?

The thing that jQuery invented that was really great was being able to use CSS selectors to get DOM elements. They built that into the browser. That's just part of the browser now. You didn't stop using jQuery. You just forgot that you were using jQuery because it's part of the browser now. I think that's going to happen to React.

Chris: It'll be fascinating because it seems like a more clear A to B with jQuery of, like, oh, that's a nice little one-liner syntax.

Laurie: [Laughter]

Chris: When you say, "We're going to make React work in the browser," you're like, "What do you mean? Like state management? I don't quite get it."

I'm sure they are. If they're not dumb, they also are thinking about [the future]. Their business is huge and important too, and everybody is doing the same thing, trying to go where the people are going.

Laurie: There are tons of smaller examples. A lot of the drag and drop APIs that exist now, those used to be browser shims. There's a ton of stuff that's built into browsers now that sort of transcended from being a library. I think, given that we've done it successfully in the past, we will find a way to do it successfully again for React.

Chris: Just because you mentioned PHP, and this is just my ignorance--I don't quite understand it--why doesn't NPM serve that type of thing too? Why can't you be the de facto PHP package manager too? Do you not want to? Is there a technological fundamental difference between JavaScript code and PHP code?

Laurie: The most obvious reason is that PHP has a package manager and it's not broken, so we don't need to fix it. It's something we talk about once in a while, especially when we look at the bad time that the people who run the other registries are having.

Those people, they were not having fun in 2014, and they are having even less fun in 2019. They do not have the money. They do not have the resources. They do not have the time to keep up with the security and scaling challenges that they face. That is a problem for them, and it's a problem for the languages that they serve.

I think part of the growth that JavaScript has seen has been because NPM has been so robust.

Chris: Yeah.

Laurie: Occasionally, we do look at those other languages and go, "Well, we could become the package manager for PHP. We could absorb Ruby Gem.

Chris: Yeah. Is that a growth opportunity for you or is it just too dangerous? Surely, they sit around and have meetings and say, "Hey, why can't we do what they're doing?"

Laurie: They way that we've been successful is focusing on what we are very good at. We're very good at JavaScript. We're very good at Web development. We know what those kinds of people want.

We've seen other companies build products that try to be everything for every language simultaneously. They are always 80% solutions. They're like, "We do everything, but we're kind of crappy at all of them."

If we tried to build a Ruby solution, if we tried to build a PHP solution, we are not primarily PHP developers; we're not primarily Ruby developers. We wouldn't understand at the sort of visceral level how people in those communities want those tools to work, so we'd get it wrong, and they probably wouldn't like our tools.

Also, we don't want to be the 800-pound gorilla, right? We don't want to stomp in and be like, "All right, we're your package manager now. Get onboard."

[Laughter]

Laurie: No. If those communities want to start using NPM, they can invite us in and we'll help them, but no one is currently begging us to come around, so we're not going to just stomp in and mess up somebody else's garden.

Chris: That's sounds very, very fair and smart. This is kind of a classic NPM question, perhaps. Dave, I think you were wondering about what makes for a good node package or a bad node package. Is that unfair to have a spectrum of good versus bad for a package?

Dave: Yeah. Is there a quality rating system, like--I don't know--NPM inspector that goes around?

[Laughter]

Chris: I'm sure there's fart apps of NPM packages that are just dumb jokes.

Laurie: There are so many dumb NPM packages. Let me tell you.

Dave: Or imagine an NPM package that just wants to count node modules and, once installed, it recompiles Lib Sass at some point. [Laughter] Perhaps that module exists on your system somewhere. How would you go about vetting the quality of that?

Laurie: That's not a hypothetical example at all.

Dave: No, nope, not at all. Not at all, nope.

Laurie: [Laughter]

Dave: The package that counts node modules also installs Lip Sass.

Laurie: It's very hard to define a simple metric that counts the package good. My favorite example for years has been mkdirp. What mkdirp does is it runs mkdir-p. Right?

It creates a directory. Even if you create a directory eight directories deep, it creates all of the subdirectories it needs to get there.

Chris: That sounds useful.

Laurie: That's a really simple function. It is useful. It's a really simple function. It's different on every single operating system, so you have to write it differently for every operating system you're on. Nobody wants to get it right, so they just install mkdirp, and they use that.

The package looks like it's broken because it's in version zero point something, and it hasn't been updated in, like, three years. The reason it hasn't been updated in three years is because it's done. It's perfect. It doesn't need to change.

If you were using metrics of, like, maintenance and quality, updates, and stuff like that, none of those metrics work for mkdirp, and that's true of a lot of the really low level, heavily dependent packages that get infrequent updates because they're small, they do one thing, and they are doing it really well at this point. Downloads is a proxy to popularity and popularity is an imperfect proxy to quality. Downloads is the metric that we have always exposed, but we know that it's not perfect. We know that there are lots of very popular packages that actually have superior alternatives--that we think they're superior--that don't get as many downloads.

You're right that it is part of NPM's job to help figure that out. One of the things that NPM does is it provides discovery for all of these packages, not just the downloading of them.

Has somebody focused chiefly on NPM's data? That is part of my job. Getting a better sense of what goes into a good package and how to show that to everybody, that's one of the things that we're going to try and improve in the next year.

Chris: I'd see how you might want to just stay away from it entirely because maybe it just doesn't matter that much. But if it could be done well, that would be cool to see, too.

I'm also curious about this. Okay, so there's mkdirp, or let's say there's some equally old package that is written in JavaScript. It's meant to be digested in the client, like in the browser context, but it literally was written six years ago, say, and it's still perfect. It doesn't need to change. It still does something wonderfully good.

But something changed around it, like it's not ready to be imported as a module because that technology wasn't around then or whatever. You can't import it. It's still good. The code doesn't need to change. It's functionally perfect, but it's just not being exported correctly. Is there a way to surface that, like, this package is ready to be used in webpack with your regular way, or this one isn't?

Laurie: Web-ready packages is definitely one of the things that we should be able to surface. One of the numbers I don't have that I'm definitely going to get this year is, how much of the registry is already ES6?

Dave: Ooh, that'd be interesting.

Laurie: A lot of it is CommonJS, but a lot of it is ES6. React is entirely written in ES6, although it's sort of an imaginary version of ES6. It's like Babel's version of ES6. It isn't quite ES6.

Chris: Really?! Oh, that's what we talked to Henry about last week in some way, like if somebody at some point wrote some stage zero crap because they just didn't care or whatever and the language moved a little bit. Then you're kind of locked to that weird version forever until somebody writes it, rewrites it.

Laurie: Yeah. Actually, I'm really strongly expecting that React is going to switch to TypeScript this year. Nobody at React has told me that that's going to happen. I just think that's going to happen.

Chris: You heard it here first. I'm going to put it on my calendar.

Dave: You heard it here first. Yeah, put it on the old calendar.

Laurie: [Laughter]

Chris: Ooh, there's another one coming up here, Dave--

Dave: Clam chowder.

Chris: --that's really embarrassing for me that was a clam chowder that I epically failed on, but let's see. Is January 1st next year cool?

Laurie: [Laughter] Sure. No problem. I saw this morning that Yarn is rewriting itself in TypeScript.

Dave: What? See, there. That would send my sniffer going.

Laurie: Right. The people who write Yarn and the people who write Babel, they're not a million miles apart, right? A lot of those are the same people. I think if one of those is happening, React is going to be the next thing to go.

[Banjo music]

Chris Enns: This episode is also brought to you by Jetpack, Jetpack.com, the folks behind WordPress Automatic and make this awesome plugin for WordPress that enables a whole bunch of awesome features, stuff you've heard about before in the podcast, but this time I want to talk about Activity, a new thing they've just added to Jetpack.

What it allows you to do is have a complete record of everything that happens on your WordPress site. When you go to a website with multiple users, whether you're maybe the developer and you've got clients that are using the website, or maybe it's a team of folks who are writing and updating a website, it's nice to have an audit trail so you can see what happened when, especially if something breaks. Activity is a comprehensive list of all the activity. It takes the guesswork out of site management, debugging, and repair on your website.

Even the free Jetpack plan gets you access to the 20 most recent events on your WordPress website. Customers on any paid plan will see events for the last 30 days. If you have the professional plan, you can see events from the last year. Any site with a paid plan has the added ability to filter activities by type and time range, so you can quickly find the information you're looking for.

Activity is just the latest feature that's been added to Jetpack. Of course, there are tons of other things that they give you as part of Jetpack. You can get WordPress themes, optimizes your site's photos and videos without slowing down your site using WordPress as a CDN. With monitor, protect, and restore, you can have automatic defense against hacks, malware, spam, data loss, and downtime. While WordPress.com is still the best place to manage your site, you can also do a lot of it from the mobile app they have for Android or iOS.

If Jetpack isn't working quite right for you, they have free support for customers and priority support for paid plans, so be sure and check out Jetpack.com for the full list of features and options that you have. If you use WordPress, it's a no-brainer. Just click on and install Jetpack. Thanks to them for sponsoring this episode of ShopTalk Show.

[Banjo music]

Dave: You mentioned Yarn. I hope it's not a sore spot.

Laurie: Mm-hmm.

Dave: [Laughter] How does Yard and NPM, that relationship work? Does Yarn still just use NPM under the hood? Then even some of the advantages of Yarn, like it shipped with a lock file first. That was the big thing, but now that's not the case. What's the relationship between Yarn and NPM?

Chris: We use … network and I don't even know why.

[Laughter]

Laurie: A lot of people who use Yarn fall into the category that you just put yourself in. When Yarn was released, it did two things. Well, it did one thing that we didn't do that had a really great side effect. It did the lock file and, because it did the lock file, it had to do a lot less checking on the server because the lock file just says, "Download these exact versions. Don't bother to check if there are newer or better versions. Just download these."

That made Yarn twice as fast as NPM because it was doing half as many requests because NPM needs to go, "Okay. What versions are available? Okay, I'll download that version. Two requests." Yarn just goes, "I'll download that version."

Dave: You'd have to do the negotiation of 2,000 modules per project to figure out which ones you needed to download first, right?"

Laurie: Exactly, so Yarn was a lot faster than NPM because of the lock file, and the lock file also made builds a lot more predictable. The reason that Yarn did that is because Facebook was one of the first Web applications to be big enough to run into the problem of semver addressed as a constant problem, right? They were constantly getting weird, new patch versions and, because they had 2,000 packages in there, it just became untenable.

They were like, "No, we need to switch to lock files. Let's do that," and they were right. They showed us … , so we adopted a lock file. NPM has lock files by default ever since version 5 and, as a result of us adopting a lock file, we also got twice as fast.

Now the two package managers are pretty much the same speed and have pretty much the same feature set. One of them is written by NPM and one of them is written by a bunch of other people, but they both talk to the same registry. So, in terms of just installing packages, they're fine. They're both equally good.

Where we think we have the edge is all of the security features that we've added to NPM in version 6. Those are either not yet adopted or only just being adopted by Yarn. Of course, we're going to be releasing more this year, so we think security should be a big concern for everybody.

We think that our client is more secure than Yarn's client, so we think that people should continue to be using NPM rather than Yarn. But, you know, if you don't want to, that's fine. It all works the same. It all comes from the same registry. You can still use our safe products, even if you're a Yarn user.

Chris: I wonder if, say--

Laurie: The client doesn't care.

Chris: I don't know. This feels gross to talk about hypotheticals, but right now you're like, Yarn is fine. We don't really care that much. It's not like we're going to somehow try to break Yarn or something. It seems like that would have bad voodoo for the community at large.

Let's say somebody made a version of Yarn that really was malicious in some way and harmed NPM in some way. I imagine you, as a company, would probably take steps to stop that from happening. I imagine you have the tools and capabilities somehow to eliminate alternative package managers from working, but you probably have no plans to do that.

Dave: The McConnell Option.

[Laughter]

Laurie: Yarn has its own registry. Rather, it has its own registry domain. It used to be a proxy to our registry. But now, we're both on Cloudflare, so it's literally just an alias to our registry. It's literally just an alternative name for our registry, so we can tell the packages that are being downloaded by Yarn's registry.

If something were to go horribly wrong with Yarn, that would be their big red button. We could make that; we could make it stop there. We could be like, "We're not serving packages to this alias."

There are a lot of Yarn users. That would be; it would have to be terrible. It would have to be a really bad problem.

Yarn has already had a couple of problems with NPM. It famously can't publish to NPM. I think they're trying to fix that, still, because it doesn't follow the right tarball format to publish the packages.

It's done a couple of other things that either mess up our data or mess up the packages when they come down, and we've had to fix those, but none of them have been anything close to as bad. Also, the people who run Yarn, they're not our enemies. They are our coconspirators in the world of JavaScript. When we tell them that something that Yarn is doing is not good for the registry, they are very good about fixing that stuff.

Chris: That's great.

Dave: Yeah. Yeah, it seems like, especially at the time, a necessary fork just that they could achieve their lock file goals, you know.

Laurie: It was honestly an experiment that we weren't brave enough to try. We thought that the people who would be pissed off by a lock file would be greater than the people who would benefit, and we were incorrect.

Dave: Well, you know, to be fair, a lock file--and I believe Yehuda Katz worked on it, who also worked on the bundler lock file in Ruby--it's awful. It's always wrong, and you don't know if you should commit it or not commit it. Then everyone is mad because their thing doesn't work.

Chris: You commit it. I think you're supposed to commit it, right? That's the point, to commit it.

Dave: Who knows? [Laughter]

Laurie: The funny thing is, Yehuda switched back to NPM.

Dave: Yeah.

Laurie: Yehuda doesn't use Yarn anymore.

Dave: Right. Right. I mean I understand why somebody wants it. For sure. For sure. For sure. But also, it's not always the funest thing to be in a locked state from your gem file or packaged file.

Chris: It's pretty important.

Dave: Or you're just trying to hop into a project and work, and somebody committed something weird.

Laurie: Yeah. There are some old-timers who, when I go to conferences, they're like, "Why is there a lock file? I hate it."

I'm like, "You could just disable it. It's still optional. It's just on by default."

Chris: I know why we use it at CodePen in our preprocessors, our little Lambda functions, because we can't update something and then have the way that code is processed for our users be all of a sudden different one day. That's not acceptable.

Laurie: In general, it's just a function of NPM being used for Web development more than it's being used for service side applications just because Web apps, they use so many more packages and the Web environment is just a lot more fragile in the server environment. It's a lot more sensitive to a sudden change in something, even quite a minor patch version. Web apps, they break more often when you change the underlying code and service side code doesn't do it as often.

Ninety-seven percent of people who use NPM are writing Web apps, so we have to reflect that. That was a surprise, [laughter] and we had to adapt to that new reality.

Dave: Yeah, I mean I guess with the untimely death of Bower, we all inherited a lot of that front-end kind of code.

Laurie: Yeah, I always expected the Bower people to be sort of mad that people stopped using Bower and switched to a combination of NPM and Webpack, but I don't know if you've ever heard of NPMS.io.

Chris: No.

Dave: Uh-uh.

Laurie: It's a search engine for NPM that is or was better than the search engine that we built for NPM. The two guys who built it, they open sourced it and literally just handed it to us. They're like, "We don't want to run the servers. That's a pain in the butt. But here is a much, much better search engine."

We were like, "You are correct. Our search sucked and this search is great because it's written by two people who are very good at search." Now, NPM search is powered by this open source project, NPMS.io.

Chris: Oh, really? Wasn't it Algolia for a minute, or no?

Laurie: No. Algolia helped us out for a little bit there, but no, it's primarily this open source NPMS.io.

Chris: Cool.

Laurie: One of the people who wrote it is the guy who wrote Bower.

Dave: Oh, there you go.

Laurie: [Laughter] I didn't find out until years later. He didn't mention it when he was giving us all of this amazing code to make search better. He wasn't like, "Yeah, no, I'm the same Andre who wrote Bower."

[Laughter]

Laurie: I was like, "Oh…."

Chris: Some heroes wear capes.

Laurie: I guess you're not mad.

Chris: Some heroes just contribute code. That's great.

What about more; can we talk more landscape stuff? I think that's of interest to people watching this show. Where do you see things going? Just because you have such a wide view of it, it'd be fascinating to hear about this. Especially because, what, 99% of people who use NPM use it for literally building websites, yeah?

Laurie: Yeah, so that was why I was excited to come on this show at this time because you were saying you were trying to set the stage for the conversations for the rest of the year. I'm like, "Let me tell you about the stage because I have this great view."

Chris: I want the stage.

Dave: I want this. Tell us the stage. We should know the stage, but please tell us the stage. [Laughter]

Laurie: Definitely, React being the bedrock of everything, I think, is going to be -- it was already true in 2018 and it's going to stay true in 2019. I think the strengths and continued adoption of TypeScript is going to be another of the biggest stories in 2019.

I think, as you've already mentioned, security is going to become a bigger deal for more and more people. It should have been already, and people are just beginning to catch up to it, especially in big companies. Big companies have a problem with security, which is that JavaScript kind of snuck up on them. Right?

Five years ago, there were three people in the corner who were writing some jQuery and it was fine. You didn't need to think too hard about it. Now, I walked into a building in a major bank in New York the other day, and they had a thousand people just in that building who did nothing but write JavaScript, and it wasn't the only building they had. [Laughter]

Chris: Wow.

Laurie: I was like, "All right, and what is your JavaScript security story?" They were like, "We were hoping you could tell us."

Chris: Ohhh!

Dave: Oh! Oh, fantastic."

Laurie: Everybody is using tons and tons of JavaScript and so much more than they were using five years ago. The security systems and processes and all of that stuff just have not caught up to this reality yet. It's why we are pushing so hard on that thing.

The other thing closely related to that, which is another, like, JavaScript has grown up concern, is compliance. We heard from a lot of big companies using JavaScript that licenses were a problem. They were like, "A lot of the code in the registry is either ambiguously licensed, it doesn't have a license, or it runs two licenses at the same time, and we're not sure which one wins, because you hire a lawyer and tell us."

We were like, "How many people actually cared about the license?" We discovered that 58% of people think that the license is important. It affects their decision on whether or not they use a package.

We were like, okay, fine; they care, but are they actually prevented from using a package if it has a license they're not allowed to use? That's true. Fifty-five percent of the people who care are actively prevented from using certain licenses.

Those two numbers together means something like a third of all NPM users can't use packages with certain licenses. That's big news, right? That's well into the category of, oh, well, if it's that important to that many people then NPM should be doing something about that. Right?

There are some third-party things that you can use right now that will do license checking and stuff, but there were third-party things that you could do about security before we invented Audit, and they weren't good enough either, right? Building it into the tool makes it the default and making it the default means that everybody else does it, even when they don't know yet that they're supposed to. Making sure that you are not downloading packages that are illegal to use, which is the primary problem with a bad license, is the thing that we're going to have to look at.

Dave: I guess, would you restrict that in your package.json? GPL comes to mind because if I use any GPL, my code has to be GPL and, therefore, has to be public domain. Even my server code would have to be public domain. Yeah. In my package.json, would I have a no GPL preset or something? [Laughter]

Laurie: We're still working on the exact shape of the solution.

Dave: Yeah.

Chris: Okay.

Laurie: Like with security, with security, we can be a little bit more prescriptive. We can say it's not secure or it is.

With licenses, it's a little bit more flexible, right? Some people are okay with GPL and some people are not, so we can't just be like, "No GPL ever." That would be bad. We can certainly make it configurable, and we can certainly pick some defaults, so that's probably what we're going to do.

Dave: But that could jam up the tree, the dependency tree too, couldn't? Like if I say, "I don't want GPL," a fifth level dependency might be GPL.

Laurie: I mean it could, but the fact of that matter is it's already doing that, right?

Dave: It already is. Yeah.

Laurie: If GPL is five levels deep in your code then, hey, surprise; you've been writing GPL code this whole time. [Laughter] People do not like hearing that.

Dave: Right.

Laurie: It does jam up the code. The GPL is a problem, especially for people writing commercial software.

I don't want to hate on the GPL as a concept. It's great, but MIT is a much more business-friendly license and people prefer it for that reason.

There's also just a ton of code on there without a license. I always get the jurisdiction wrong, but there is a difference between the U.K. and Europe about how code without a license works. In one of them, if your code doesn't have a license, that automatically means it's public domain. In one of them, if your code doesn't have a license, it automatically means it's copyrighted and nobody can use it for anything. [Laughter]

Dave: Ah, a twist! [Laughter]

Laurie: Right, so if it doesn't have a license, it's unusable in half of the two biggest markets for JavaScript.

Chris: Even in the U.S. it's like that, isn't it? At least that's how I thought it was.

Laurie: I think the U.S. is the one where it's automatically copyrighted and the E.U. is the one where it's automatically in the public domain.

Chris: Right. I've been yelled at that before because it's so tempting to just not think about it and just push some crap to GitHub, NPM, or whatever. Then you'll hear from people that it's like, "I get that you don't care about licenses, but I can't use it until you do."

Laurie: Exactly. There are a lot of people who are using comedy licenses like the unlicensed or the WTFPL. There are lesser and greater degrees of comedy.

There are lesser and greater degrees of taking things seriously, but lawyers don't know what they mean. If a lawyer doesn't know what it means, then you can't use the code. It doesn't matter if the intention was clear because lawyers have different standards for this stuff.

Chris: Yeah.

Laurie: JavaScript is growing up, and we're going to help people navigate that.

Chris: The stage so far: React, TypeScript, security compliance.

Laurie: Yeah. The last three are the ones that are not going to be big this year. You should start paying attention to them this year.

Chris: Okay.

Laurie: The first one is serverless. We asked people where they deploy their code, and this wasn't an exclusive question. It was like, you could answer multiple, but 33% of people said that they are using serverless code.

What does that mean? The serverless environment is very different from node running on a server and it's very different from node running in a browser. What do we have to do to make life easy for people who are running Lambda all the time?

I already mentioned GraphQL. Only about 23% of people are using GraphQL some of the time or most of the time, but 50% of people are considering it. Fifty percent of people are looking at GraphQL and going, "Maybe this is the year." I think that means that GraphQL is going to get really big this year.

Also in that category is WebAssembly. I think WebAssembly is the newest of the new. Only 3% of people say that they're using WebAssembly because you can do almost nothing in WebAssembly right now. But 54% of people are interested in using it.

Chris: It's good marketing because all I know about it, I mean I know a little, but it feels like the marketing is, "It's super-fast! You're going to love it! It's compiled!"

Laurie: It's so funny that you say that because we had the same question. We were like, "What do you think WebAssembly will do?" because 23% of people haven't even heard of it yet.

But within the people who have heard of it, 65% of them think it's going to be faster than JavaScript. That's funny because the people who write WebAssembly, the people who write V8 and wrote WebAssembly, the concept, not WebAssembly the code, they're like, "It's not actually that much faster." If you write good JavaScript, it'll be just as fast. It's just easier to get fast with WebAssembly because it more often stays on the optimized path.

Chris: Maybe it's that I don't write most of the JavaScript I use. I just pull it down. And so, if the whole ball of wax gets WebAssembly, you've got to imagine there's some--

Laurie: Yeah, I think that's one of the places it's going to see some uses, like in deep dependencies where people are like, well, we call this thing a million times a second. Can we make it faster? Yes. Then the whole app gets faster, so it'll start turning up in sub-sub-dependencies.

Chris: Hmm.

Laurie: But I think the more interesting thing, and 47% of people agreed, the more interesting use case for WebAssembly is that you can take a library that was written in some of the language, and you can just bring it to the Web. You can just cross-compile it, bring it in, and publish it as an NPM package, and it will interact with your JavaScript no problem.

Chris: Wow!

Laurie: Which is amazing. So, you know, OpenGL libraries, really complicated machine learning stuff, anything where somebody wrote a lot of really fiddly, complicated C that does something very hard, very efficiently, you don't have to rewrite it in JavaScript to use it on the Web anymore. You can just bring it in as WebAssembly.

That is going to be amazing because we talked about 900,000 packages in NPM. How many more packages exist in other libraries? Suddenly, they can all be in NPM. How much bigger will NPM get? How much more useful will JavaScript be if we can pull in literally any library in any language and use it on the Web? That's going to be super, super fun. That's why I'm excited about WebAssembly.

Chris: Wow. That does seem like a big deal. Good.

Dave: That feels like a really great place to stop. Thank you very much, Laurie, for coming on the show. For people who are not following you and giving you money, how can they do that?

Laurie: Our website is NPMJS.com, and we provide services both for small and medium businesses and big enterprises, anything you want starting at $7 per user a month. If you want to follow me personally, I am @seldo on Twitter S-E-L-D-O.

Dave: Great. Thank you very much. This has been super informative, so I really appreciate it. Thank you, dear listener, for downloading this in your podcatcher of choice. Be sure to start, heart, favorite it up. That's how people find out about the show. Follow us on Twitter at @ShopTalkShow for tons of tweets a month.

If you hate your job, head over to ShopTalkShow.com/jobs and get a brand new one because people want to hire people like you.

Chris, would you like to NPM install anything else at the end of this show?

Chris: [Laughter] ShopTalkShow.com.