Why 炒炒冷饭？Because this speech was made 10 months ago.
感觉 DHH 挺幽默的，但是很多东西又很有哲理。虽然是在 Rails 年会上发表的演讲，但是对于互联网行业的技术方面都有关。
以下是 Transcript，比较枯燥，不如看视频（快）。
[Music]
All right, I am incredibly happy to be here on the 20th anniversary of Ruby on Rails. It is an amazing achievement to make it five years in technology with a framework or a library. To make it 20 years is almost unbelievable. And not just to make it 20 years, but to be here 20 years in and to be this fired up, this excited, and this motivated to continue to improve the framework, to improve the ecosystem, and to work with everyone around it is just an absolute honor.
So, it is incredibly flattering timing that we had Rails World, the first Rails World, the first event of a brand new foundation, happen exactly this year. I started back in 2003, and we’ll talk more about that in a second, working on Ruby on Rails. I’ve not stopped; I’ve worked on it ever since, every year. And this year, in the 20th anniversary, it is wonderful to announce that today, maybe it’s happened already or it’ll happen shortly, we will be pushing Rails 7.1.
[Applause]
As another major release of the framework, packed full of all sorts of good stuff, Raphael is our release manager, and he’s done an absolutely excellent job getting everything ready. The last release, the last major release of Ruby on Rails, was Rails 7.0, which was almost two years ago, I think. Since then, this entire room of people, around 700 people, have made code contributions to the release. About 4,000 commits have gone into the framework since that last release. It’s just absolutely a pleasure to see.
And that’s one of those gifts that I think this is really all about. I like to look at the Rails community and the ecosystem as a gift-swapping environment. We are here to make gifts for ourselves and to share them with everyone else. You bring a gift, I bring a gift, and together we have a whole stack of gifts. And I think that realization that the environment we’re in is one set up in the ethos of the MIT license, and we’re not here in a transactional way—I’m building something for you, and then you have to give me an equal and proportionate part back. No, we’re here to give each other gifts with no strings attached because we simply enjoy making gifts. I enjoy making these gifts, I enjoy making them for myself, and I really enjoy sharing them and seeing all the gifts that come back.
Some of those gifts, a lot of those gifts, are code. As I said, we have 700 people who have contributed to the latest version of Rails, and we have almost 7,000 people who’ve contributed to Rails over the 20 years that we’ve been tracking contributions, which is just truly astounding. Not just that we have had so many people contribute to it, but we have so many people continue to contribute to it and new people coming in all the time. I know there’s a lot of people here for whom Rails World is their first Rails event. This is really a key part of why we’re doing this, why we’re putting on these events: to welcome new people into the ecosystem and get them involved in this gift-sharing ceremony.
And while a lot of those gifts are, of course, code—and code is important, it’s foundational—it’s certainly not everything. You can have the best code in the world, but if you do not wrap it in understandable documentation and flattering marketing, no one’s going to care. And I think the magic of the Ruby on Rails community is that understanding that the code is really important, but it’s not everything. There’s so much more to it.
And I think one of the most important gifts we’ve been able to, in some way, give ourselves is the foundation of new institutions. When you’ve been around for 20 years, there’s just a lot of stuff that accumulates, and sometimes you have to take a brand new sheet of paper and write down, like, you know what, with everything I know now, what would I do? And I think one of the most rewarding things that I’ve been involved in over the past year or so has been the Rails Foundation, setting that up—something we knew that we probably should have had for a long time, but now we do.
And I think I just want to thank the people and the companies who, when I reached out with just an idea of forming this foundation, were eager and willing to sign up and help us. I first reached out to Tobi from Shopify about just under a year ago with this idea of creating a foundation and setting a new institution in place for creating events like this. And he was on board right away, and these companies signed up shortly thereafter. It’s Cookpad, Fleek, GitHub, Intercom, Procore, Shopify, and my own company, 37signals. If not for them, we wouldn’t be here today. These were the eight companies that kicked in a million dollars to get us started, to be able to have a treasure chest to book an event like this before we even had anyone showing up to pay us for a ticket.
And then since then, we’ve added a whole slew of contributing members: AppSignal, who’s also the lead sponsor of this conference and also an application we now use at 37signals to track our performance; BigBinary; Cedarcode; Planet Argon; Anduin. It’s incredible to have this kind of support this long into it, to have people who are willing to invest not just their time but their money into facilitating something like this.
But even with all of that, none of it would have come to fruition, and I wouldn’t be on this stage if it hadn’t been for the executive director, Amanda Perino. I just want to give her a huge round of applause.
[Applause]
She has been not just instrumental but everything about making this happen, the foundation, setting it up, and creating this wonderful event.
And so here we are, 20 years into this. It’s such a weird concept to try to wrap your head around. I am 44 years old, which means I’ve spent about half my life working on this framework, half my life being involved with creating Ruby on Rails from a technology perspective, from an ecosystem perspective. And that’s sometimes hard to contextualize.
So one of the ways I try to make sense of time is to go back in time. We are as far away from the inception of Ruby on Rails in 2003 as we were from 1983. This is me in 1983, the year that the modern internet really got started with ARPANET converted to TCP/IP, the year that the first home Nintendo system was released, and one year before the first computer I really had an experience with, the Commodore 64, was released. That’s a long time; that feels like a completely different era, which reminds me of just what a young industry this is—that we are basically the same distance from the creation of Ruby on Rails as we are from the distance of a lot of people’s first computer when these things came out with one megahertz.
And then if we jump to that year of 2003—I think this might actually be 2004—this was my home office in Copenhagen where I worked on Ruby on Rails before moving to the United States. And these setups remind me that we’ve come an awful long way, but also that a lot of things are the same. If you look at the applications that are up on my screen, you can squint and it would look exactly like the applications that are up on my screen today at home. We’ve got TextMate, we’ve got my terminal, and okay, we’re not using Trac anymore, we’re using GitHub, but there are a lot of things that change and there’s also a lot of things that stay the same.
But we can get drunk on the grapes of nostalgia all day long. We can look back, and I could have filled an entire keynote with just funny anecdotes from 20 years of being in this community, developing this. But you know what? I think we’re going to have plenty of time to look back very shortly after AI puts us all out of a job or destroys the world, whichever one comes first. So what I’m interested in is looking forward, and that’s what this morning’s keynote is mostly going to be about—all the things that we’re doing, all the things that we’re building. And there is so much.
I think perhaps this is the most packed time that I’ve been involved with, the most number of new frameworks and libraries that I’ve seen grow and that we’ve planted recently. It’s going to be exciting. Now, I was going to ask AI to suggest, like, what should we even do about this phenomenon that this is coming in, this is something new? I’m sure all of you have had fun playing with AI, maybe already integrated it into your workflow, and perhaps some also are a little ambivalent about what this new world is going to be.
So I was going to ask AI to sort of represent this in a sign, and I thought the cliché it might offer me back is this Chinese expression for crisis—these two signs which the lore goes like, “Oh, this is danger but also opportunity.” So we have AI coming, and it’s threatening but it’s also exciting, and it would probably state it with complete confidence, and it would be kind of wrong because that is what AI does. It dreams of things. The second character actually does not mean opportunity; it means change point. But it’s been adopted by speakers and business people for a long time, and this is what we get back. We get back from AI what we put in, so we need to put in the best that we have, the best we have of humanity, the best we have of development, our very best ideas to get something great back.
And I think this is what’s exciting about this moment. Now, that crisis idea is also, I think, even if it’s not literally true that these characters reflect the opportunity of it, I like to believe that it is one of the opportunities that is right now. In 20 years of developing Ruby on Rails, we’ve gone through at least two and a half major crises in the industry. We’re sort of in one right now. This 5.5% is the American Federal Reserve fund rate, which is the main interest rate governing things in the US. It is heading up still. We are in this weird space with the economy where things are going, but we don’t quite know. But I think this is actually good—some compression, some force, even if it’s painful, just like AI can cause us to rise to the moment and realize what it actually is that’s important, what we’re actually good at, and force us to improve.
Because if I look back at the last, let’s say, decade plus of the tech industry, you can sort of map it to this graph of the long-term rates from the Fed fund rate. Right around 2009 was when the rate went to zero, and it stayed at zero for the next six years. And in my optics, this is exactly what’s given the sort of mold for a sloth, an indifference to productivity, an indifference to how capable individual programmers needed to be. And I think we’re rediscovering now, as rates go back up and the price of innovation becomes more important, that you know what, this was not a good place to be. We need some pressures. We need pressures from the economy, we need pressures from startups, we need pressures from each other to stay productive.
This was the original pitch of Ruby on Rails, that we had to be. And I think if you just sort of chart some of the things that were released during that era, it’s not exactly a mystery if you view it through this lens that it seemed like we got more and more tools that enabled us to create slightly better versions of the same things but required so much more effort to do. This wasn’t a good direction. This wasn’t a good setup. The low productivity tolerance was a zero-interest phenomenon, and I’m really glad that we’re going to blow it up because we have to.
Because as I see it, that phase from 2009 forward was a kind of productivity Dark Ages. Yes, we were able to produce slightly more grain, but we were using such primitive tools that required so much labor and division to do so. And part of this is I look back at my formative experience with Rails, with Ruby, which was building Basecamp. We built Basecamp in about six months with one developer and two designers part-time, and that turned into a wonderful business, a business that’s yielded and created hundreds of millions of dollars over the years. That was the inception. If we could do that in 2003-2004, we should be able to do so much more in 2023. Yet when I talk to startups and companies and I hear about team sizes and specialization, it’s the opposite. It takes longer to do marginally different, mostly less. That’s not a good place to be.
And I think part of this syndrome, part of these Dark Ages, were ushered in, in fact, by the companies that had that early success and ended up realizing, “Oh, we need something different, we need something more.” And then the second round of ideas just, not in all cases, were that good or didn’t apply to as many people in as many circumstances. There was a discussion on Twitter a while back about, well, Twitter, and the fact that Twitter moved off Rails. I was always really curious about that story. Here’s a testimony from someone who actually worked at Twitter when they moved off. Yeah, they moved off because they had a badly designed Rails app. That happens. Okay. And then they moved into what they thought was better, a constellation of Java microservices, and then nothing happened for 10 years. They poured concrete over the application and hired thousands of people to make no progress at all. Twitter, in many ways, is the example of what can happen from these dark ages where we think we’re making progress, but the incremental yield is just so small.
The same thing, by the way, happened at Airbnb. This was a trend. I could have put that on the slide next to the JavaScript frameworks of things that have slowed this industry down. Now, I think what’s going to happen now is that the pendulum is swinging back because it has to swing back because the forces of the economy are going to force us to swing it back. And we’re going to go away from this idea that part-stack developers are actually what we all should be aiming for, that we should be aiming for ever smaller niches, ever smaller specializations of things that we’re working on. No, we should be working on designing and installing and maintaining and knowing how to do all of these things ourselves and being capable of doing all these things ourselves.
There’s a funny anecdote in the Elon Musk biography where they talk about these solar panels that they’re installing, and the solar business is going terribly because it’s taken too long to install this and it’s not profitable. And the company looks into why is this happening. A lot of it was happening because the designers weren’t actually present when things were being installed. They were separating who’s coming up with the ideas from who’s implementing these ideas from who’s maintaining these ideas, and every single split created just inefficiencies and a cumbersome product to deal with. And I think this is exactly what codifies the split between part-stack developers, the specialization drive that we’ve had over the past 10-15 years, and the full-stack idea that we so cherish in the Rails community. Unless you are both designing and implementing and feeling the pressures of all the little bolts, you’re going to get it wrong, and you create something cumbersome.
One of my favorite articles on the internet ever in the development world was one by Joel Spolsky from 2001 called “Don’t Let Architecture Astronauts Scare You.” And I think it really encapsulates this. Remember that architecture people are solving problems that they think they can solve, not problems that are useful to solve. Tell me something new that I couldn’t do before, astronauts, and stay up there in space and don’t waste any more of my time. That really sums up the last 10 years for me. What new stuff have we been able to do and achieve and accomplish in the last 10-15 years? There is some, and we should embrace that, and we should extend that, but there’s also just so much roundness that didn’t bring us forward but just spun us in circles.
So the ideal that I look to is the idea of Leonardo da Vinci in the 1500s. This guy not only painted the Mona Lisa, but he was exploring the specific gravity of solids, coming up with ball bearings. He was an engineer, a scientist, a musician. The interface and merger of art and engineering, I think, is at the root of what we should try to be as full-stack developers. We should appreciate and embrace what Ruby gives us in terms of the beauty of the language, the productivity of it—all of it. That we are not just engineers, we are writers, we are artists, we are, and I’m pushing it, code poets. And we can allow ourselves to have the joy of those things at the same time without diminishing anything about what we build—in fact, enlarging it.
So I’d like to think that we are not just, or should not just be, full-stack developers. That term is kind of perhaps aged out a bit. We should be Renaissance developers, Renaissance people, interested in all aspects of it—the design, the implementation, the maintenance, the deployment, all of it. And I think this is what’s so wonderful about the Rails community is that we work tirelessly to lower the barrier for new people to be able to come in, have experiences of success, to see “Hello World” with the absolute minimum of fuss. But we also give a decade-long journey for someone to learn more and to explore more and to find out more and not have the division between who makes the framework and who uses the framework.
Ruby is this magic environment where, at any time you’re puzzled about something, you can simply use bundle open active_support to look up exactly how a method you’re using works. It’s just the same Ruby code that you’re already writing, and you can actually understand what’s going on and level up.
This leads me to an introduction to where we’re going with the new stuff. Progress is good, and there has been progress. I am not up here telling you that the height of all web development was 2005—absolutely not. We have developed a bunch of really great technologies and made progress on fundamental technologies in that time. But we’ve done so on a bridge of complexity, trapped in complexity, wrapped in complexity, and it doesn’t need to be that way. Progress is not synonymous with complexity. Complexity is a bridge we use to get to simplicity, and I think this is what’s so exciting about this moment—so exciting about some of these fundamental core developments—is that we can see the destination now. We can see that this was arduous, this was hard, this was difficult, but it’s going to get easier.
Alright, so I am going to talk about seven frameworks and libraries that have either already recently been announced, recently been released, or haven’t been released or even announced yet, in the next 37 minutes. So this is going to be a tasting menu; this is not going to be a deep dive into every single framework and every single story. A lot of this conference is packed with sessions that follow up on most of these things for the particular frameworks, and I encourage you to dig deeper into those. But there are these three bridges that I think have been packed with varying degrees of complexity over the past 10-15 years, where we’re now seeing a destination of simplicity that looks really appealing.
So let’s start with the front end. The front end has had the greatest amount of churn, the greatest amount of, to some degrees, running in circles, and to some degrees also making real material progress and changing what the expectations are for the kind of applications that we make for the web today. But it’s also been draining to try to keep up with, and sometimes not without reason. Some of that blame falls on Rails because we have tried to keep up with that. We have not just taken the August position that, you know what, we can just sit down, we can wait it out. No, you can’t. If you want to be relevant, you got to be in the mix, and if you want to be in the mix, you got to be in the mix with the ways people are actually building applications now.
In 2019, we added Webpacker by default to the Rails stack, which was perhaps one of the only times we’ve added a major new framework to a major component of Rails where I knew going in, “I hope this doesn’t last. I hope this is a bridge we need for now, but that there is a destination on the other side that we will get to and we can yank it out.” Now thankfully, that has happened, and it didn’t even take nearly as long as I thought it would because a lot of really exciting things have been happening on the front end, including in the JavaScript community, which I like to poke fun at but I also like to embrace. I actually really enjoy writing JavaScript these days. JavaScript is a wonderful second language, and that’s not nothing. There’s tons of languages in the world—I’ve tried a bunch of them—and JavaScript legitimately is my favorite second language. It just so happens I like Ruby more, and a lot more, but that doesn’t mean I can’t appreciate the times where I have to write JavaScript and that things are getting better.
I think one of the big changes that really got me excited that the pendulum is swinging back, even in the JavaScript world, was the introduction of ESBuild, which seemingly just came out of nowhere, integrated a bunch of things, and just got orders of magnitude faster for some of it. Here’s the comparison to that Webpack system we were using in Rails 6—it’s about 40 times, actually twice that, 80 times faster. That was a real change; that was a real moment of progress and something we should applaud and something I was really happy to work on and to integrate with Rails with JS bundling Rails. And what’s under this is once these wheels get moving and once we realize, you know what, we’re not happy with the complexity of the weight or the speed of it, it keeps rolling. So just recently, Bun has been introduced, which has added a marginal improvement on the progress and speed of ESBuild. But this is the kind of momentum there is.
What I really like about Bun is this sign that even in the JavaScript world, there is a weariness, a tiredness of splitting everything into five billion different packages. Bun does everything from being a runtime to being a bundler to a package manager—all of it. These are ideas that we hold dear in the Ruby on Rails world: that we should not have a billion trillion different packages. And that’s great. So you look at that and you go like, “Yes, I can see we’re moving over the bridge; we’re getting to somewhere simpler.”
But at the same time, other things are happening. Import Maps was the big gamble in Rails 7. I went back and forth: Should we make this the default? Are we skating too far? Where’s this going? Are there any other browsers even adopting it? In 2020, when I started working on this, this was an experimental feature in Chrome. Now it’s supported across the line in everything, and it is even better, even more exciting than ESBuild and Bun and all the other tooling that’s happening. Together with HTTP/2, this is a waterfall of loading a bunch of individual scripts from, I think it’s, yeah, Hey. As you can see, it’s all loading at the same time. There’s no longer a penalty for splitting up your big bundles of JavaScript. All the stuff we were so excited about making easier and faster with ESBuild and Bun is no longer even necessary. We don’t even have to do it because what is even better than ESBuild? What is even better than Bun? No build. No build is way better than fast build. You can make the build as fast as possible, but it’s never going to be as fast as no build.
Here is a chart demonstrating just how much faster not building is than building, even with the fastest builder. It is literally infinitely faster than Bun and certainly infinitely faster than ESBuild as well. And it has all of these wonderful features like source mapping without the mapping because it’s just the source. That is a major step forward, a major moment of back to the future where you can open view source on any website—this is actually Hey—and you don’t even need to use any source mapping, you don’t need to use any bundling. These are the files that we are writing when we develop Hey. These are the plain text JavaScript files that are not compiled at all, that are shipped, no build, straight to the browser. And they’re really nice to write these days, and they’re really nice to work with because of all these complexity bridges that Webpack and others paved for us that we got to this place with ES6 and the other fundamental steps of progress in the JavaScript world just so we could arrive here.
And I think this is what a lot of technologists often forget when we get excited about new progress and we realize, yeah, it’s a little bit complicated, but look at all the things we can do. I’m there with you. I’m like, this is great, but we’re only at step two. We got to get over the bridge. We got to just get to the progress without all the hardship. And I think this is what we’ve essentially been able to do now: that we can build modern web apps without any form of JavaScript bundling or transpilation. And this is not like some vision of a thing that could happen in the future. This is how we’re writing our apps right now.
I remember when I had debates with people over single-page applications versus multi-page applications, and the argument was always, “Yeah, yeah, yeah, you can use your multi-page applications for, I don’t know, some CRUD app, but you couldn’t do Gmail. You couldn’t do something as highly sophisticated and interactive like an email client.” Well, that is actually literally what we did with Hey, and we wrote all the JavaScript, like I just showed you, in plain text with zero build. Now that has me really excited. This is the workflow that we’re operating in, and it’s freaking great.
What I’m also excited about is the fact that no build as a mindset is expanding because now it also comes for CSS. It took a long time, and I credit the JavaScript folks and TC39 with dragging JavaScript forward and saying, “No, you know what? We can actually make the browsers better. We can make the browsers do all the things that we’re using this complicated tooling to get.” And then maybe it took a decade, but the CSS folks are catching up, and they’re catching up fast. CSS nesting, which is one of the primary features a lot of people were excited to use Sass for or other forms of CSS compiling, is now supported in all browsers. Custom properties, that is, variables, are supported in all browsers. We are using both of these things—no build for JavaScript, no build for CSS—to write the new applications of 37signals. The latest stuff that we have started has no build step whatsoever in any of it.
This is a screenshot of some CSS I snapped out of one of the new applications we’re working on for this once idea that we have. It’s using nesting, it’s using variables, it’s freaking awesome. If you are working directly with CSS, no build is not just possible, it’s great. So this is sort of the underlying plumbing, the underlying progress that took us about a decade’s worth of complexity to arrive at, but it is here now, and I very much encourage you to check it out.
Now I dance a happy dance because I no longer have to use either Webpack or even ESBuild or even Bun. But I’d actually give it one caveat, and it’s not just because Adam is in the house, but if you are going to build your CSS, Tailwind is a great way to do it. And we have a wonderful plugin for making this super easy. Adam and team wrapped up their entire framework into a single executable so you don’t even have to have Node or Bun or whatever installed on your system to use it. It’s pretty awesome. So if you are going to use a tool for it, I highly recommend Tailwind CSS. But at 37signals, we’re not using Tailwind CSS. We have designers who like to work directly in CSS, and we can now do it entirely with no build. It’s awesome.
Now what I really like about getting to that place of simplicity, or at least getting over the hump of the bridge and rolling down, is that you’ll get some first-order wins, and then you’ll start collecting the second-order wins. Sprockets was and is the original asset pipeline for Rails. It was introduced in 2009, I think we started working on it. This was trying to do bundling before the JavaScript world was interested in bundling, trying to solve a lot of these problems at an earlier stage. Yet we’re still using it. I don’t know how many of you have looked at the source of Sprockets or tried to fix a bug in that codebase recently, but it’s pretty rough. It’s full of wonderful integrations like an integration with a JavaScript bundler called Bower that I think was popular in 2011, or this compressor for YUI Compressor I think was also from 2011-2012. It is a graveyard of experiments and ideas and clutches that we needed well before no build, well before the modern HTTP/2 approach to shipping things, and we don’t need it anymore.
So one of the things that I’ve been working on for the past year and a half or so has been a clean sheet replacement for that that just does far less. It’s Prop Shaft. It is a new library for doing the asset pipeline in a world where you no longer need the Rails side of the asset pipeline to do any compiling at all. So it just does far less. It doesn’t have any compression, it doesn’t have any CoffeeScript compilers or Bower integrations or anything else like that. It has basically two features: it provides a load path for all your assets such that your gems and wherever you place your assets are accessible in any view, and then it provides digest stamps so you can do far future expiration and make sure you have good cache dynamics. That’s it. That’s what’s left when you drain the asset pipeline of all the complexity, when you drain it of all the clutches that were needed in an earlier time.
And I think this is what’s so exciting about this movement in Ruby on Rails. There’s a bunch of these opportunities where the fundamentals that we’re working with have gotten better, and we can drastically improve and simplify the entire framework one step at a time. This is not a big framework. I’ve been working on this together with Breno, who is presenting on Prop Shaft I think later today. And if you add it up, there’s less than 3,000 lines of code. I think Sprockets probably has 35,000-40,000 lines of code, something like that. It is a huge library. This is an order of magnitude simplification. Anyone who’s traversed the Sprockets GitHub issues page will know that the more code you have, the more issues you produce. So when you have an order of magnitude less code to deal with, you’ll have an order of magnitude, or maybe even two orders of magnitude, fewer issues to contend with, and you will have people who can actually understand what they’re dealing with.
I’ve had to bundle open Sprockets a couple of times, and I almost want to sort of recoil right away and just close the editor and not even look at it because it’s just such a horror show of implementation. I open this up now—you can open it. I encourage you after this talk, later today maybe, install Prop Shaft, open it up, see what it is. It is a reference implementation of how little you need once you boil it all down, and I guarantee you can absolutely understand what’s going on.
Now the price of having been around for 20 years is that if you try to make changes like this, try to swap out something like Sprockets that has a lot of dependencies, you’re going to run into this fight: stability versus progress. Now the Rails Doctrine that I wrote in 2013-2015, something like that, tackled this question directly. What should we choose when we are faced with a choice of some momentary disruption for moving forward? We should choose progress. This is how you stay excited about what you’re working on. This is how you guarantee that a 20-year-old framework can become a 40-year-old framework.
It’s actually funny, it calls out here at the bottom: “Is Rails going to be better off adopting another problem domain like job queuing or WebSockets in the years to come? If yes, then let’s suck it up and do the work.” Exactly. And by the way, since this was written, we did adopt WebSockets. That is what Action Cable became. And in just a moment, I’m going to be talking about job queuing, another area building on top of this.
This is really, I think, of the asset pipeline as just the pipes, the plumbing. What are we going to flow through those pipes? Hopefully some wonderful, clean, cool water. And this is one of the things—the pendulum has been swinging for about 10 years. Everyone was super excited about sending JSON, JSON, JSON over everything, for everything, all of the time. And now there’s a realization, even in the hardcore JavaScript world, that, “Oh, do you know what? HTML is actually kind of good. You can send it directly to a browser. There’s not any translation step. Search engines can reach us. We should also do this.” I’m not even—that’s good. That’s a good realization to come to, even if it is 2020. HTML is good. HTML, in fact, is better most of the time for web applications.
There are other frameworks here who’ve come to that conclusion too. HTMX has gotten some traction with that same idea that, “Oh, actually HTML is really good. HTML has gotten a lot better. We can do a lot more things with it. We should double down on it.” That’s what Turbo did, and we are working now on a major new update to Turbo called Turbo 8 that will hopefully be out maybe this year. We see three major new things that we’ve been working on I wanted to present to you and talk about.
The first one is morphing, which is one of those ideas that is foundational for React, for example, that you have a shadow DOM and you can do fast merges, and it’s happening through this morphing process. What I love about the interplay between people going in different directions is that there’s a convergence going on right now. Even in the React world, even in the heavy JavaScript world, “Oh, actually HTML is good.” And at the same time, people who’ve been like, “Yeah, HTML is great. I thought so too for a long time,” can learn something from the other side too. And morphing is exactly that.
So Turbo has these three different modes that if you’ve worked with Turbo you might be familiar with. The default one is the full body replacement—that’s Turbo Drive. That’s where we get the main speed increase just from the fact that we’re not reloading the JavaScript, we’re not reloading the CSS, and it’s just dropping. You don’t have to do anything. That’s sort of the peak of developer happiness: when we can just improve the base of how you’re working with a system without requiring you to learn something new or do something different. That’s great. Turbo Frames allows you to cut up the page into smaller segments and simplify your backend to dealing with. Great too. Slightly more complicated, requires some awareness about how you’re using, when it’s good to use. And then finally we have Turbo Stream and their actions where you update individual pieces of the page, and that’s great too. You get more responsiveness the further you go up the chain. If you’re just updating a single div on the page, Turbo Streams can do that in the most efficient way. Doing the full body replacement is a little less efficient.
Here’s where Turbo Morph fits in, allowing us to do more responsiveness, more of the actions we would otherwise have reached for Turbo Frames or Turbo Streams for, getting the same developer feel as though we were replacing the whole thing. Here is a quick video demonstrating Turbo Morph. So this is the card table in Basecamp. We built this just using Turbo Drive for most of it. When you add a new card without Turbo Morph, we’re just going to add it and we’re just going to reload the page. And you know what? It’s sort of fine, but as you can see, it resets the whole thing. The scroll position is lost. You lose a little bit of sense of where you actually are when you’re doing it. It’s fine, it’s okay, but you know what? It could also be a lot better.
Here’s where Turbo Morph fits in, allowing us to do more responsiveness, more of the actions we would otherwise have reached for Turbo Frames or Turbo Streams for, getting the same developer feel as though we were replacing the whole thing. Here is a quick video demonstrating Turbo Morph. So this is the card table in Basecamp. We built this just using Turbo Drive for most of it. When you add a new card without Turbo Morph, we’re just going to add it and we’re just going to reload the page. And you know what? It’s sort of fine, but as you can see, it resets the whole thing. The scroll position is lost. You lose a little bit of sense of where you actually are when you’re doing it. It’s fine, it’s okay, but you know what? It could also be a lot better.
The main thing we identify that morphing can help us with is to maintain scroll position—that’s huge—and reduce the amount of flickering that’s happening on the page. So here we’ll scroll over this; it even has both vertical and horizontal scroll, which makes it even worse when you lose your scroll position. And you can add a new column and you don’t lose where you are. If you add a new card to that, it’s going to be the same thing. Here we throw it in the trash, but this idea that with morphing we can get Turbo Drive ergonomics—you don’t have to do anything. It looks at the two DOMs that you have, does the comparison, inserts or updates the divs or other elements that have changed, maintaining that scroll position, maintaining everything without jitter. It’s a wonderful upgrade.
Jorge is going to talk, I think, later today actually in this room about Turbo Morph and how you can use it. I think that’s the main thing or the lead thing I’m excited about with Turbo. But part of this with Turbo is also keeping up with the changes of the web. We went for the puck with Import Maps when it was only an experimental feature in Chrome in 2020. Now there’s a new thing also coming out of the wonderful Chrome team called View Transitions. I don’t know how many of you have looked into that. It is a really cool way to make web applications feel much more like native applications, and we’re doing the work to integrate that with Turbo in anticipation that this is going to be shared everywhere.
Here’s a quick demo from the Chrome team showing just how View Transitions look. They look like native apps. You can apply animations; there’s no jitter, there’s no reload on it. Right now it is only supported in Chrome, but I think this is part of this—we’re going for the progress here. We’re going to skate where the puck is going to be. We’re going to invest in the things that are changing the web in a good way.
And then finally, we’re looking into local storage, making it such that Turbo Drive can upgrade your application to work with the cache across multiple tabs at the same time. It gives us a gateway to deal with offline, which has long been a difficult subject. Anyone who’s wrestled with service workers would probably wish someone would just abstract all that away and make it easy to use. That’s one of the things that we’re looking into.
Oh, and then we had this minor ergonomics upgrade to anyone who wants to work on it. I mean, it wasn’t like a big thing at all. It was hardly controversial on the internet. But this goes exactly to what we’re doing. Do you know what? There’s nothing that’s written in stone here. If we don’t like the tools we’re working with, if we don’t like the way we’re working with it, we will change it, and we will change it for the better as we see fit.
Strada, which is probably the longest-running piece of software that I’ve been involved with—we announced Strada in, I think, late 2020 and said like, “Oh, it’s coming in December.” Well, it took a few years, but Strada is finally here. It is the last piece of the Hotwire bundle. It is used to drive native controls from your web app, which means that you can, if you’re working with native applications, do more of your work on the web side of things while still making you feel entirely like a native application. It allows you to sidestep the App Store approval regimes for a ton more updates. As anyone who’s ever had to battle with Google or Apple over an update or even had to just wait on it knows, it is not a pleasant experience. The less of that you can do, the happier you generally are.
Here’s an example. I think this was Miles who actually made this example, and it’s really great. It shows exactly what Strada helps you do. On the left is a standard set of web dialogs, and with Strada showing how to use the markup in the HTML that you’re sending from your server to produce a native interaction and be able to change that native interaction without changing the native app itself is pretty awesome.
Alright, that’s the front end. On the back end, we have some really exciting new upgrades as well. Some of them we’ve barely even talked about, and they come from sort of the same source: what fundamental improvements are happening in our industry with either software or hardware or browsers. This is a chart showing you the transfer speeds of storage since about 2009. In 2009, a hard drive would transfer about 100 megabytes a second. As you can see, in 2023 with Gen 5 SSD drives, we are now two orders of magnitude faster. Two orders of magnitude is the kind of change that should reset all your fundamental assumptions about how you build anything that interfaces with storage. Things change.
Now, funny thing about 2009 is 2009 was when Redis first came out. I’m a huge fan of Redis. I think it is an absolutely beautiful piece of software. I love using it. I’ve even taken to the arcane command structure and logic, and I think two years ago finally stumbled upon the right abstraction for Redis and its integration with Rails in something called Kredis that allows you to add Redis aspects and keys to your models. It’s wonderful. But Redis was born in a world of 100 megabyte transfer rates. In a world of 10 gigabyte transfer rates, I don’t think we need Redis for as many things or the same things anymore.
One of the main things that we’ve used Redis for over the years is caching. It’s really fast, it’s in memory, it’s got good ergonomics, it’s got nice trimming, but RAM still is expensive. If you contrast it to storage for us, it’s about five to ten times as expensive to use RAM as it is to use storage drives. SSDs—Solid Cache was our inquiry into what would actually happen if you started using drives again for caching. Would it be fast enough? Could we use that? And if so, what would happen? Well, what would happen was you can extend the cache life to months instead of days and hours. If you have ten times as much storage for your cache or a hundred times as much storage for a cache, you can keep your cache for a lot longer. This makes your application way faster. If you don’t have to redo something that you’ve already done before, it is going to be virtually instant.
Now what we found was that storage is still slower than RAM, but it’s at the point where it doesn’t matter. For us on, I think it was Hey, we measured this. We will do a cache request in 0.8 milliseconds with Redis and 1.2 milliseconds with Solid Cache right now. And there’s room for improvement. Yeah, that’s 50% slower on a scale that doesn’t matter. So Solid Cache is a new caching library that uses the database as its backing, built on the idea that solid-state drives, from the name, Solid Cache, are now fast enough, good enough, better than RAM for caching. It has cache trimming, so we’ve set it up not by how much storage we can use for this, but what our data retention policies are. So we have 60 days of caching for Basecamp, for our fragment caching, for our rush dolls, for all of that stuff—not because that’s the maximum we can do, but because that’s what we promise customers of when their data is deleted if they cancel their account. It has encryption, which is a wonderful upgrade, especially for us. We did a bunch of work on Hey. This was how encryption got into Active Record by default because we didn’t want to be exposed to customer data with their emails. Now we can say the same thing is true for caching. That’s a massive upgrade. And it ships with sharding built-in by default.
Here’s the results. This is Basecamp before and after we introduced Solid Cache. Before we had a Redis-backed cache; after we had Solid Cache, we had an improvement on the 95th percentile from about 350 milliseconds to 250. You don’t even—you can see the difference. Long cache lives radically improve the performance of your application if you have anything that doesn’t just constantly churn. It’s a huge upgrade. And as I said, the performance difference, yes, it’s there. It just doesn’t matter. Solid Cache is available today. Actually, it was available yesterday. Donal is talking about it right after this talk, next over how to use Solid Cache, how we’ve been using it, how it’s been built. It’s available right now. That is a major upgrade. I think it’s one of those things—free performance to me is always the best. This is the closest thing you can possibly get. Actually, it’s not free; it’s cheaper than performance. It is—you get more performance for less cost. That’s wonderful.
Following this same idea, we’ve started looking into creating a whole new job backend for Rails built on the same idea that if we don’t need RAM for caching, why do we need it for queuing? So at Basecamp, at 37signals, we’ve been using Rescue for 10 years, 12 years, a long time. There’s a bunch of other queue options in the Rails world that you can use. We looked at all of them when we finally grew tired of this. This is a number of different gems, and I don’t think it even includes all of them, that we use to make Rescue run at 37signals. This is from, I believe, Basecamp. Here are six different gems to be able to do the interaction with jobs that we want to do. That’s a mess. You should not need six different pieces to solve one problem.
So we went back and looked at all these options that are out there. Rescue we were quite familiar with. Sidekiq is a popular option that a lot of people use. It also happens to be commercial software, which is something I have tremendous respect for but don’t want in a framework like Rails that’s built on open source. Then there’s Delayed Job and an add-on and an upgrade to that called GoodJob. So if we go through these, as I said, Sidekiq is a nice piece of job infrastructure a lot of people use, and it’s also commercial software which should exist. I’m a huge fan of that existing next to it. It should absolutely in no way ever be required. Ruby on Rails is built on the foundations of MIT, that it’s free and it’s open source. It should remain so forever. Delayed Job was built by Toby originally 15 years ago, ironically on the same principles that we’re looking at now. Delayed Job was built for the database. It’s built in a way that makes it a little bit difficult to scale, and this is why a lot of people move away from it. There’s a fork of that that has been upgraded, which is also an option, but also hasn’t been maintained a lot. GoodJob is one of the key inspirations we had for Solid Queue. It is built on top of some new query mechanisms in Postgres that make this so much more viable to use the database as a back for jobs. It’s pretty awesome, but it’s locked to Postgres. We don’t use Postgres. It’s also built on Delayed Job.
So what could happen if we started with a clean sheet of paper that was trying to fulfill this? It needed to be free and open source. It needed to support MySQL and particularly MySQL 8, which gets some of these features that Postgres has had exclusively for a while that make this fast. Be a new reference implementation that has the power to be a new default. You should be able to install Ruby on Rails, add a database, and then have excellent caching and excellent job running without having to change anything at all. That’s the goal.
So that’s Solid Queue. We’re not quite ready to release it. We’re hoping to release it within a few months. We’ve been running it in production for a number of workloads on and off at 37signals. I’m really excited to slot this in as a new default potentially in Rails 8. Together with that goes an upgrade to something called Mission Control that we actually built for originally Rescue but we targeted Active Job and we built on top of that to give insight and access to all your queues, be able to pause them. This was actually born out of an outage we had for Basecamp, an outage that was exacerbated by the fact that we couldn’t stop some queues from running a lot of expensive jobs when the database needed to have less load. Now there’s a web interface to be able to manage this. We’re building Mission Control to work against Active Job so any backend will work with it, which of course also means Solid Queue.
Here’s an example of diving into a specific queue, seeing what’s queued, being able to interact with these specific jobs, either rerunning them or pausing a single line. Both of these things we’re hoping to put out in 2023. Let’s see, let’s hope it’s before 2025, I suppose. 2023, we’re putting it in production, getting it out there.
And the final bit, the final piece of complexity that I’ve been excited to work on and tackle and realize some of the fundamental improvements we’ve had is distribution. So Heroku was a godsend when it came to the Rails world in, I think, 2009 was when it first premiered. It took something that was really actually quite complicated and cumbersome and hard to understand and made it really easy, which is to deploy a Rails application with no fuss at all, virtually no knowledge at all. Well before containerization and other technologies made it trivial for everyone else. I have the utmost respect for the work that Heroku did. It is evident in the fact that for 10 years, even after they essentially stopped evolving the service, they were still ahead of everyone else. It is only quite recently that competition like Render or Fly or others have been able to sort of catch up with the developer ergonomics. That is incredible, and I’m so grateful that Heroku made this possible in the early days for Rails when it was difficult.
But 2009 was also like four years before Docker. I think Docker came out in 2013. Some of the fundamentals have improved. Some of the fundamentals have been made better in a way that we don’t need to rely on a single or multiple commercial offerings to make it easy to deploy Ruby on Rails. They’re still great. There’s still a lot of good things to be said about them. I think there’s still places where it makes sense. But just as with Sidekiq, just as with everything else, Ruby on Rails from end to end should be free, open-source software that is not owned by any commercial entity and that we can all work on together to improve. And we should never accept that something is too hard that it has to be done by commercial vendors. Commercial vendors can add alternatives, and it’s wonderful, and we’re all better off if they do, but it should not be a requirement.
So I’ve been working on something called Kamal, which is actually a tool we came up with to get out of the cloud. It wasn’t so much about the Herokus of the world, but about getting off AWS, about cutting down on our exorbitant cloud bill and moving on to our own hardware. Now it turns out that that is exactly the same kind of developer ergonomics, deployment ergonomics as set by the best, and in my opinion, Heroku is and was the best in that, certainly the pioneer on it. So I built Kamal in that spirit that it should be as easy to deploy your Rails application to any vanilla VM in the cloud anywhere as it is to work with Heroku. I’d say we got pretty close. Not entirely there, but there’s a strong growing community already dealing with that.
And then finally, we have been working at 37signals on a new line of applications under the banner of Once that impacts this idea of distribution. So for the past 20 years, I’ve been all SaaS all the time. You run things as a service. It’s really great in a lot of ways, and then also in a lot of ways, it’s annoying. It means that you don’t have full power and control over your data. You have to trust whoever you give it to, and you have to pay recurring revenues for every single piece of software that you use online. And frankly, I’m personally sick of it, and I’d like to make a plea and a kick against it, getting us back to distributing software as something people can own and run on their own. But it needs to be easy. It needs to be Heroku easy to install your own software. It can’t have 15 pages in a technical manual for anyone normal to be able to adopt this.
So this is something that we’re working on right now. There’s a bunch of things spilling out of it. One of the things that’s spilling out of it is a newfound love for SQLite. SQLite is actually amazing, and it’s a database that a lot of people have thought of as a development tool. It is absolutely not. If you can run your entire application in a single process, it is, in my opinion, the best database you can use. It radically simplifies the setup, the maintenance, the backup of your application. So if that’s a choice, and it’s a choice for us with Once, you can use it, and we will be improving it greatly. There’s a bunch of other SQLite fans who’ve been invested and interested in this recently.
So that is the seven major tools that I’ve been working on in the last year and a half. We have Prop Shaft, Turbo 8, and Strada on the front end. We have Solid Cache, Solid Queue, and Mission Control on the back end. And we have Kamal for distribution. That’s a lot. That’s awesome. That’s exactly why I’m still here. There’s so many frontiers for us to continue to push forward and push them forward in service of the idea that Rails should be a one-person framework, that one individual should be able to make everything happen from start to end, to be able to go from zero to one, to be so fortunate that one day you too can end up with a mess that you rewrite as a swarm of Java microservices, just so you can do nothing for 10 years and just cash in. A framework that would make Leonardo da Vinci proud. A framework that can grow and nurture a generation of Renaissance developers.
Thank you so much.