Janet for Mortals

I wrote a book.

It’s called Janet for Mortals, and it’s free, and it’s on the internet, and you can read it right now.

And you should read it right now, instead of reading this blog post, because this blog post is not very interesting if you haven’t read the book. Heck, this blog post is not very interesting even if you have read the book. This blog post is a thinly-veiled promotion for my book to slip into my newsletter and RSS feed, with just enough additional content to pad it out to the length of a real post.

The book is about Janet, a programming language that I have written about before. I’ve been using Janet a lot lately, and I’ve been having a lot of fun with it, and I think that more people should know about it so that they can have fun with it too. People like you.

I’m not really going to talk much about why you should read the book, or even why you should care about Janet in the first place – that will come in a later post. Instead, this is going to be a short retrospective of what it was like to write my first technical book.

We’ll start with some numbers.

It took me twenty weeks to write the book, working in my spare time. I had originally estimated twelve weeks, which turned out to be a really good guess for how long I spent writing the book, but I didn’t account for how much time I would spend working on book-adjacent coding side quests.

The final book is pretty short: 44k words of English prose, if you don’t count any of the code snippets. I tried to find an example of a famous book with a similar word count to put that number in perspective, and the best I can do is The Great Gatsby, which clocks in at 47k words. It’s right on the border between novella- and novel-length, but it’s less than half as long as my series of posts about Nix, which is sort of terrifying.

But writing English words was only a fraction of the work. Over the course of those five months, I also spent a lot of time on:

• the website itself
• jimmy, bindings to a C++ library of persistent data structures
• Toodle.Studio, an interactive turtle graphics playground
• cmd, a command-line argument parsing library
• judge, an inline snapshot testing framework
• to do, a command-line todo list manager

These things are not very interesting by themselves, but this blog post just exists to promote the book, so I’m going to reflect on them now. You are welcome to stop reading at any point and go read the actual book instead.

The Website (2 weeks)

The most interesting thing about Janet for Mortals is that it has a built-in repl. At any point you can press escape and pull it up, and it’s docked to the bottom of the page, just out of the way of the text. I’m sure that it’s not the first programming book to include a repl like this, but I’ve never actually seen it done before.

The editor portion of the repl is CodeMirror, which I had used previously in Bauble. CodeMirror doesn’t know anything about Janet out of the box, but I had already implemented some basic language support when I wrote Bauble.

But I skipped a lot of the Janet language when I was writing the grammar for Bauble, like ``multi`backtick`quoted`` strings, because they didn’t really matter in Bauble’s constrained DSL. But they mattered for the book, so I had to spend time figuring out how to implement them.

But fleshing out the CodeMirror grammar (or more precisely, the Lezer grammar that CodeMirror uses) had an unexpected side effect: it let me re-use the grammar to do syntax highlighting for the code snippets in the book itself.

Janet for Mortals is just a static site, but no static site generators that I know of know how to highlight Janet code. For a long time the book was entirely black-and-white, which I don’t mind, but I knew I needed to add some color before I released it upon an unsuspecting public – I think Janet’s syntax is pretty unfamiliar to most people reading the book, and anything to make it look friendlier helps.

So I knew that I’d have to roll my own syntax highlighter of some kind, and I ended up just writing a simple static site generator in redo, which was not a very good fit, but… well, least bad choice that I know of. The meat of the generator is written in JavaScript, so that I could plug in the Lezer grammar for free, but it’s all tied together by a fragile web of shell.

I used Remark to implement the parsing of the book’s source, and it was nice how much control I had over the generated output. I even added a simple extension to label code blocks, which was pretty easy.

I also re-used Remark in the client itself, as part of the repl. The docstrings in the Janet standard library are written in Markdown, and I’m actually parsing and rendering them to HTML on the fly as part of the repl autocomplete.

I spent a long time getting the repl autocomplete working well – because this book was written for newcomers to the Janet language, I thought that the help-as-you-type would be really useful for people trying to follow along in the repl.

Autocomplete works by dynamically querying the Janet environment via WebAssembly at repl startup. This means if you define a new symbol with a docstring, it won’t actually appear in the autocomplete output, but I think that’s… fine. I could re-generate the autocompletions after every command is run, but… I don’t think there’s much value in that.

The most interesting part of the repl is probably the (report) function, which takes a string and POSTs it to a simple web server that sticks it into a SQLite database for me to peruse later. It’s not really any different than a comment box, but I feel like there’s something fun about doing it from the repl. I’m really glad that I added it – it’s been fun reading people’s feedback, and I’ve fixed quite a few errors because of it. I’m sad that I didn’t implement any way to respond, though!

The backend for reports is not written in Janet; it’s a tiny Haskell application that just listens for POST requests and sticks them into a SQLite database.

There are people using Janet to make websites, but I am not one of them: the primary thing I want out of a web server is security, and I just don’t think Janet or its HTTP libraries are “battle-tested” enough for me to connect them to the internet.

I also just think the idea of using a dynamically-typed interpreted language to build a web service is crazy, when there are optimizing compilers right there, but that’s a whole other conversation.

jimmy (1 week)

I spent a little bit of time writing bindings to immer, a library of persistent data structures. I never finished them, and probably won’t, at least not until I have a use for them. But as a demonstration of how to interop with C++ code from Janet, I think it was successful.

Last year I wrote a little art playground called Bauble. It was my first time embedding Janet in the browser, and I had a pretty tough time figuring out how to do that.

There weren’t a lot of resources back then about embedding Janet period, and doing it in the browser added an extra layer of difficulty. I’d never used WebAssembly or Emscripten before, or even TypeScript, and it turns out there are no tutorials on how to write TypeScript Emscripten WebAssembly Janet bindings, so I spent a while figuring out how all the pieces worked together.

And I’m glad I did, because I think the final product is really neat: it’s a website that is not written in JavaScript. I mean, a lot of it is. The UI is, still. But the actual application logic is all Janet.

I thought that that was a really useful superpower of Janet, and I wanted to make the technique more accessible. In fact this was a big motivation for writing this book about Janet – I wanted people to know that this was possible in the first place, and I wanted to make it easier to get started with it.

But Janet for Mortals doesn’t talk about Bauble at all. Bauble is actually not very interesting from an interop perspective: Bauble is completely stateless, and basically uses Janet to implement a pure function from strings to strings (they’re… pretty complicated strings; Bauble is a Janet-to-GLSL compiler, but they are strings nonetheless). I didn’t think it was a very good showcase for everything you can do with Janet, so I briefly considered talking about how I implemented the repl in the book, but I decided that that was far too boring. So I wrote Toodle.Studio – an obvious fork of Bauble – instead.

Toodle.Studio seems a lot simpler than Bauble, but the interop with JavaScript is much more involved. Toodle.Studio has to execute long-running Janet programs asynchronously over time. It has to think about memory management, as the JavaScript code retains multiple references to the same Janet values. It has to pass complex nested data structures to and from Janet, going through C++ as an intermediary. It does a very simple version of all of these things, but it’s a pretty good showcase for the techniques.

But the most interesting part of Toodle.Studio isn’t the interop or the memory management. The most interesting part of Toodle.Studio is the logo.

I wasn’t really planning on making a logo – this is a demo project for a book, after all – but sadly I had no choice. When I was getting ready to release the website, I showed it to my partner, because it’s rare that I work on something comprehensible to normal human beings. I thought she’d like it, but she was aghast.

“You said you were working on turtle graphics,” she said. “Where are the turtles?”

I tried to explain that the turtles aren’t really turtles, that it’s like a flea circus, and the turtles are metaphors – but she was having none of it. The lack of turtles was a base betrayal, so I had to spend a day or so modeling a cute animated turtle in Bauble to act as the logo. And making its eyes follow the mouse, of course.

Relationship repaired. The logo wound up being my favorite part of the site, and it was fun to get a chance to use Bauble to make something “real.”

cmd (2 weeks)

One of the things that I spent the most time on, oddly enough, was a command-line argument parsing library. The library itself only gets, like, three paragraphs of screen time in the book, but it was very important to me that it exist before the book came out, so that I could unambiguously claim that “Janet is an excellent scripting language.” Before cmd, that was still true, but the phrasing was more “Janet is a great choice for scripting and writing CLI tools, except that the argument parsing is kind of janky, sorry, but hey at least it’s better than Bash.”

cmd was heavily inspired by Core.Command, which is the best command-line argument parsing library that I have ever used. I’m extremely spoiled by how easy it makes writing CLIs, and I wanted to replicate that experience in Janet. cmd is definitely not as good as Core.Command – types, my goodness, types make everything so much easier – but it has 95% of the features I care about, and the concise notation makes it more pleasant to use in ad-hoc scripts.

One thing that I miss, though, is that Core.Command autogenerates Bash completion functions. I want to add that to cmd one day – the API is designed so that that will be possible to do. But… so many projects, so little time.

judge (1 week)

judge was one of the first things that I wrote in Janet, all the way back in 2021. I think that it worked pretty well considering that I didn’t know anything about Janet when I wrote it, but now I do, so I rewrote it from scratch. Not only is the API much nicer to use now, but the implementation is way simpler – and easier to make changes to.

The main differences between Judge v1 and Judge v2 are that tests can now appear inside regular source files, not just the test/ directory, and I added the test-macro and test-stdout helpers, which are extremely useful. The OCaml equivalent of test-stdout is pretty much the only way that I write tests professionally, because OCaml doesn’t really have a way to embed arbitrary data in source code, so we turn everything into a string.

After publishing the book – which has a whole chapter on testing with Judge – I had a chance to spend a little more time improving Judge, and I finally added an --interactive mode, which I’ve been wanting for a long time. And since I’m not spending all my time working on this book anymore, I’ve actually had a few opportunities to use the new Judge, and I gotta say: it’s nice. It’s really nice. I know I can’t impress upon you just how nice it is in this post – it really needs a demo, and I’m too lazy to record one right now – but I’m very happy with how it feels to use it to write Janet.

to do (2 hours)

I picked this project to highlight for the scripting chapter, because it’s a non-trivial thing that I had done in Bash before, and actually found it pretty painful. Parsing multi-line text with Sed is not fun, and trying to do date manipulation with date in a way that works the same on macOS and Linux is… basically impossible, as far as I can tell. I quickly ran into the limits of my patience, and gave up on the idea some years ago.

It was really fun to return to this with the full power of PEGs and sh and cmd at my disposal. I immediately surpassed all of features of my original Bash todo list, and was able to add quite a few more (like fzf multi-select – good luck constructing null-terminated strings in Bash).

The book covers a very simplified version of the app – it can’t schedule tasks for the future, and there’s no concept of “skipping” tasks. Those features are important for my todo list workflow, but they are probably not important for your todo list workflow, so the book only discusses the core functionality of adding things to a list and crossing them off. I think that it makes a good starting point to run with and make your own – paired with zsh-autoquoter, it’s actually a surprisingly useful app!

If you put all of these projects together, I was writing code for almost half the time that I spent working on the book. Eight out of twenty weeks, plus some periods where I was doing both at once.

I didn’t really budget for that going into this. I thought that I’d improve Judge, and I thought that I’d write an argument parser. But I thought that writing an argument parser would be way easier than it actually was. And I thought that I’d just talk about Bauble – it never occurred to me that I’d write another art playground just because Bauble was too easy.

So that’s the story of writing the book. Or really, everything but writing the book. All the other things. The writing itself isn’t that interesting. I wrote it in Markdown, in Sublime Text, which is my favorite editor for writing long-form prose. I have nothing interesting to say about that part.

Two new versions of Janet came out while I was writing the book, and I did have to go back and update the chapters on debugging and native modules to keep up with changes to the language. I plan on keeping the book up to date with the latest Janet release – we’ll see how long I can keep that up.

I haven’t done much to promote the book yet. I submitted it to Hacker News, and I submitted it to Lobsters, and I wrote a very half-hearted tweet about it. The reception was pretty much as good as I could have hoped for: it was on the Hacker News front page all day, and even held the number one spot for a while.

What does that actually mean? Well, according my Nginx access logs, I got:

• 30,025 unique visitors on Friday
• 9,568 unique visitors on Saturday
• 3,777 unique visitors on Sunday

Those numbers don’t mean very much, though. Those are just people who clicked on a link – the number of people who actually read the book is much, much smaller.

I don’t actually know how much smaller exactly, because I don’t have any client-side behavior-tracking analytics on the site. But I can sort of try to guess, by looking at my access logs. It seems like retention is not great:

• Chapter One had 22% as many visitors as the home page.
• Chapter Two had 20% as many visitors as Chapter One.
• Chapter Three had 69% as many visitors as Chapter Two.
• Chapters Four and Nine, “Pegular Expressions” and “Xenofunctions,” had more visitors than Chapter Three.

I’m guessing that last bit is because people clicked on those chapters to see what they were about, which just goes to show that unique visitor count is not a very dependable metric.

My best attempt at answering the question “how many people are actually reading the book” is 387, as of the end of the launch weekend. So far 387 unique IP addresses have loaded five or more distinct chapters, which is probably a decent proxy for the metric I care about.

I really had no expectations for what these numbers would be before I launched the book. It’s a big time commitment to read a weird book about a programming language you’ve barely heard of, and 387 seems simultaneously low (compared to, say, any blog post) and high (I don’t think I’ve ever read a book off a HN link). But it’s more than zero!

Alright, I think that’s enough. I’ll close with some fun facts:

• The Janet language is named after an immortal being in The Good Place who helps mortals navigate the afterlife, hence the title.

• The chapter with the fewest visits is currently “Testing and Debugging,” despite being the third-to-last chapter. This does not surprise me at all, but I think it’s a shame: the last three chapters are by far the most interesting in the book, and the style of testing described in that chapter is one of the biggest productivity upgrades that I have personally experienced in my engineering career.

• So far I’ve received 494 reports from the built-in repl reporting feature. Most of these were of the “hey nice book” or “testing” variety, but I’ve gotten several dozen typo reports, clarification requests, or otherwise useful comments through it as well.

• The most interesting report was just “you should listen to this song: https://www.youtube.com/watch?v=46i3LbIbbhI.” No context, no explanation, and I have no way to reply for clarification. But… thanks! It’s a good song. I’m into it.

• A few people asked me questions without including any kind of contact info, so I have no way to answer them. I hope that they found peace, wherever they are. I’m not ignoring you. I just… I only implemented an extremely primitive one-way feedback function.

• I’m going to plug the book one last time.

With feeling: Janet for Mortals! Out now! The first infinity visitors get their copy for free!