Re-writing Firn in rust (pt. 2)

Well, I did end up starting a re-write of firn. I'm having a pretty good time! I'd say I'm about 30-40% done and have probably spent about … 35 hours on it? It seems like a lot for a project (and I suppose it is), but the learning curve for rust has been steep. I am moving faster and faster with each commit, however, and I suspect it will certainly not take as long to complete the re-write as the original took. After all, I'm mostly porting old features from clojure to rust - I don't have to do any experimentation with the actual infrastructure of how the program works. All in all, this has been a good way to learn a new language. At the end of the experience I'll have acquired some new skills and knowledge and will also, arguably, have a better/faster product too.

I'm happy I wrote the original version of Firn in Clojure - at the time, I had been taking a break from programming and it was the right way to get back into the world of it - I still loved Clojure, and I knew I could get back into the swing of building things quickly with it. Getting re-acquainted with Clojure would later also gave me the momentum to launch into building Trunk.

Now, Clojure wouldn't be my first pick anymore for building a CLI tool such as Firn. I want/ed Firn to be an easily distributable binary; and from that I found that I was challenged by the use of GraalVM. I was originally excited at the idea of being able to build native-binaries in clojure (it still is very cool), but the workflow for creating and testing artifacts just wasn't optimal. Compile times were long and the system felt complex. The technology is new and there was limited documentation that I could find. Further, I was using Clojure to talk to Rust and embedding all of that in a binary - again, a formidable technological feat but I felt like I was stacking tools on top of tools.

Speaking of stacking tools on tools and layers of technology, I am reminded of the talk Preventing the Collapse of Civilization by Jonathan Blow - which was particularly eye-opening for me (not just regarding programming but regarding history and the "losing of history" as a whole). But programming wise, I have been thinking more and more about layers of abstraction and complexity in the tools we use and going forward I'd like to be a bit more careful when I'm walking into a project that involves more layers than I'd like (or are necessary). A part of this discussion deserves its own post, but largely I think that I see this shift as one that simply comes with time and experience.

Firn should have been written in Rust from the beginning as the core library that parses org content (Orgize) was written in Rust. All Firn really does is wrap that functionality in a way that 1) reads org files into memory, 2) call to orgize and pull out metadata / content 3) render content/metadata with a templating system (with extra niceties for "prebuilt renderers") and 4) write the output to a file.

All of the above can simply be done in Rust. At the time I didn't have the energy to learn Rust when I built Firn. But, switching over now there is plenty of simplification as a result (I think I listed these in a previous post when considering doing the re-write.):

• using only a single language:
  • not gluing two languages together with some kind of interop.
  • only one set of packages to manage.
  • easier for contributors to pick up.
• Faster compile times (2 seconds vs 5 minutes).
• Easier (read: more documentation) for cross compilation.
• Smaller binary size.

Another benefit is that I'm learning enough rust that I could fork the Orgize parser and vendor it if I need to have better customization over the tool, or make contributions upstream. I'm sure there are some smaller other benefits that are coming from simplifying Firn that I will discover over time.

As for progress, here's where I'm at in development:

• [X] CLI prompt
• [X] firn new command built
• [X] parsing of content, collecting of tags, links and logbooks.
• [X] implementation of templating system (handlebars)
• [X] writing to file.
• [X] Build support for partials
• [X] Development server
• [ ] Handle movement of attachments (images, etc)
• [ ] Build renderers:
  • [ ] sitemap
  • [ ] table of contents
  • [ ] backlinks
  • [ ] Logbook stuff
• [ ] Figure out cross compilation
• [ ] Update docs

Possible new features I'd consider building:

• [ ] Theme support, as in Hugo
• [ ] A blog/posts feature that better suits chonolgoical content / has it's own tagging system.

Looking at that list - there is still a fair bit of work to do. However, I've become more confident in using Rust. The first 20 hours might have moved me forward 5%, but the next 20 hours will probably move me forward 20%.

We'll see in a future update I suppose.