Musings on Innovation, Tools For Thought, Open Source, Venture Capital, Communities, Software Sustainability, and the Future of Foam. July 26, 2020
It’s been a month since I released a very early prototype of Foam on Twitter:
I’ve built something I’ve thought of for a long time, and it’s finally ready for preview! 🚀
Foam is a personal knowledge management and sharing system built on VS Code and GitHub 🧠
Try it out and let me know what you think! 👀
Foam is my humble contribution to the burgeoning ecosystem of Tools for Thought: A developer-friendly approach to note-taking, personal knowledge management and digital gardening, built on top of Visual Studio Code and GitHub.
I shared it on the internet hoping to get some feedback on my ideas, but instead I accidentally ended up creating a community of people excited to discover better ways to work, together.
In one month, Foam has been installed 8000 times, despite its awkward, multi-step Getting Started experience. It has collected 5800 stars on GitHub, the same amount as my previously most starred repository did in 3 years. Our Discord community has been joined by 550 people, we’ve merged 68 Pull Requests from 30 individual contributors, and our first weekend we hit #1 on Hacker News, Product Hunt and GitHub Trending.
In short, it’s been a whirlwind of a month. In this post, I want to pause for a moment, recount the brief history of Foam, and give you a peek at what the future of the project might look like.
A week before I launched Foam, a friend and I decided to start a book club. The first book we selected was Where Good Ideas Come From by Steven Johnson, where the author explores patterns of innovation in nature and in human creativity.
In retrospect, this book explains how Foam came to be, but in a weird twist of fate, caused Foam to exist even before I read the first page.
I’ve recently become a sucker for personal productivity books, and none has gripped me more than How to Take Smart Notes by Sönke Ahrens. In it, Ahrens describes a note-taking methodology that can significantly improve your learning and information retention over time. So before starting Where Good Ideas Come From, I knew I wanted to take notes on it, but I had an inconvenient problem:
There was a hole where my note-taking tool used to be.
Earlier this year, I was introduced to Roam Research, the most hyped and, arguably, the genre-defining Tool for Thought. Part note-taking app, part graph database, Roam aims to organise information in a way that matches how our brains work: associatively instead of hierarchically. Our brains aren’t indexed by categories: You can’t simply access all of your thoughts related to “soup”; instead, you can think of gazpacho, and then follow the string of thoughts, facts and memories that unravels as you tug on the ball of yarn. Roam works similarly, by allowing you to create bi-directional links between ideas: not only documents, but individual thoughts within a document, and visualise and traverse them as a graph.
The power of this idea is hard to explain, and it took me a month of use to realise the benefit of this mode of note-taking, but once I figured out that Roam happens to be the perfect companion to the methodology described in How to Take Smart Notes, I knew there was no going back to traditional note-taking.
Now, Roam is a good idea, but like most early-stage startup software, also an infuriatingly imperfect implementation: slow to start, clunky to write in, occasionally losing information when editing offline, and not available on mobile. I was fine with this, until they moved from their private beta to a paid subscription model, priced very… confidently at $15/month.
As a software engineer, I like to pay for software, and for a tool that can improve my learning and life-long knowledge retention, the price of a meal per month is very reasonable. But at the same time, I had enough frustrations with Roam that made me question whether it was the companion I wanted on my life-long learning journey.
As a software engineer, I also have the ability to think up software and will it to existence.
So, I had the classic developer thought…
Building a “Roam, but better” on my own seemed like a daunting task. To conjure up a pun that would surely elicit groans and eye-rolls at the Roam Research offices, Roam wasn’t built in a day. As a salient data point, the folks over at Athens Research have been working on an open source Roam-like app for 4 months, and despite being way smarter and dedicated than I am, haven’t shipped yet.
What I ended up building took me six days from idea to production.
To understand how, let’s go back to Johnson’s Where Good Ideas Come From.
The book is laid out in seven chapters, each describing a pattern of innovation, and a few chapters in particular are very relevant.
Chapter 2, The Adjacent Possible, explains that good ideas rarely occur out of the blue. In fact, when they do, they rarely tend to be successful in the market, and are considered either to be wrong, or “ahead of their time”. Instead, good ideas are are often just one step away from what was previously possible — or a combination of established ideas that together form something new: an innovation.
Foam is very much an innovation borne out of The Adjacent Possible: It’s the ideas behind How to Take Smart Notes as interpreted by Roam Research, combined with my own personal experiences using Visual Studio Code and GitHub. So in the end, I didn’t need to invent much.
Chapter 3, The Slow Hunch makes the case that good ideas are rarely a lightbulb moment, but a slowly evolving hunch, or a series of ideas that build on top of each other until reaching their final, innovative form.
In 2018, I wrote and published a serialised novel in a fashion that, in retrospect, is much like Foam. It was only after falling in and out of love with Roam that I realised the true power of these tools, when transposed outside of the software development context. So, Foam didn’t just come to me: it’s a natural conclusion (or at least a milestone) of a path I’ve been on for a while.
(In a mind-bending coincidence, this chapter also discusses the need for good note-taking in order to cultivate the slow hunch, and describes the tradition of Foam-like journals from the Commonplaces of English enlightenment and thinkers like John Locke, all the way to Tim Berners-Lee’s World Wide Web!)
Finally, Chapter 7, Platforms, makes the case that platforms and ecosystems have the ability to foster and supercharge innovation:
“The most generative platforms come in stacks, most conspicuously in the layered platform of the Web. […] Tim Berners-Lee was able to single-handedly design a new medium because he could freely build on top of the open protocols of the Internet platform. He didn’t have to engineer an entire system for communicating between computers spread across the planet; that problem had been solved decades before. All he had to do was build a standard framework for describing hypertext pages (HTML) and sharing them via existing Internet channels (HTTP)” — Steven Johnson, Where Good Ideas Come From p.189.
Just like the Web was built on top of the infrastructure of the Internet, Foam is built on a tall, open source stack of turtles.
Precisely because of the platforming abilities of these tools, I was able to build a Roam-like user experience (well, a very barebones experience…) in a matter of days, rather than weeks, or months.
The idea for Foam occurred to me on a Friday afternoon. By the following Friday, we had our first 1000 users. What happened in between is a true testament to the power of open source and software interoperability. The most credit I can take is the credit a curator takes of a well-put-together exhibition.
Earlier this week, Geoffrey Litt wrote a piece, Foam: Software as Curation, where he asks a question:
We tend to think of software in terms of monolithic “apps.” But what if we thought about software design as a process of curating existing parts into a holistic user experience?
He also explains, in layman’s terms, Foam’s “mostly air” philosophy:
While Foam does install its own extension, most of the value comes from installing other existing extensions and choosing good default settings. For example, it installs a few extensions that provide a nicer Markdown writing experience, and that tweak lots of things about how Markdown docs link to each other. And it picks a color theme well suited to writing Markdown. I would have never come across this whole configuration on my own, but I’m glad that someone else figured it out for me.
The piece is very good (and not too rosy-eyed about Foam’s approach, either, which I appreciated), but it ascribes to me, the curator, too much prescience: I didn’t build Foam out of existing extensions because I have faith in Software as a Curation. I did so because it was the easiest way to get it done.
Hard things are hard. I, personally, have always been a big fan of the easy road, especially when the payoff of the work is uncertain, as it is with ideas. Building Foam by combining existing extensions was an act of prototyping, not a philosophy.
It was only after I had built the initial version that I realised that what I had in my hands wasn’t just a prototype, but a seedling of a novel(ish) approach to product development.
Because of the ecosystem effects, Foam already has real-time collaboration (via Visual Studio Code Live Share) and a cross-platform mobile app (via GitJournal), something that other personal knowledge management apps like Roam sorely lack.
Most other tools have a Learning Curve, but Foam has what I like to call a Discovery Curve. Using a novel combination of tools or extensions, you may discover a powerful feature that didn’t exist before the moment you put them together.
Foam is a bet that by leveraging standard platforms, extensibility frameworks and commonly accepted interoperability surfaces, we can scale open source software to create a best-in-class product for our users.
Once I interrogated this idea of Software as a Curation, I found it yielded many interesting qualities that formed Foam’s Principles (read later, not important right now).
However, this approach has also many drawbacks, some of which are in the category of subtle-but-gnarly, though most of them are, in retrospect, pretty bloody obvious: We’re attempting to build a best-in-class product as a distributed team, but most of our team members haven’t ever agreed to be a part of the team, and few still don’t even know that a team exists.
As a self-identified fan of the easy road, had I been able to predict the kind of interest Foam has received, and the community it has created, I would have likely not elected to take this approach, and would have instead gone for a more traditional monolithic architecture.
And that would have been a mistake. Mistake, partly, because I would have never shipped it. But also a mistake, because nothing good comes easy.
It sounds like a cliché, but what we’re building with Foam isn’t software, it’s a communi—
That made even me roll my own eyes.
Let me try again.
What we’re building with Foam is a distributed, loosely-affiliated group of like-minded people interested in solving their own problems, and contributing those solutions to a commons.
In the Tools for Thought space, people keep using this term Collective Intelligence. I’m not sure what it means exactly, never bothered to Google it, but this sounds like that could be it?
It also sounds a lot like another thing, which I do know a thing or two about: Open Source.
Yadda yadda “the worst form of Government except for all other forms that have been tried”, or whatever Churchill supposedly said about democracy, applies very much to Open Source Software.
I’ll leave out the usual critiques (misaligned incentives! corporate profiteering of free labour! toxic culture of overwork!), and hone in on one critique, recently wielded by one of the Great Leaders of the personal productivity movement, and the manufactured-drama Antichrist of the #roamcult religion, Tiago Forte:
I can’t recommend open source software in general. It’s designed for the philosophical preferences of software developers, not the needs of everyone else — @fortelabs, 12 July 2020
The most annoying part of this critique is that it’s true. I don’t agree with its conclusion (I recommend open source software in general), but (with few notable exceptions), open source software, specifically end-user applications, tend to be unapproachable to non-developers.
Partially, this is because the movement generally fails to attract, and respect the value of the labour of, talented folks from disciplines other than software engineering. Great products are designed, not engineered. In addition to being well-designed, great products are also well-tested, well-documented, well-managed and well-marketed.
This is especially true of free, non-corporate-sponsored open source software (FOSS). Some open source software, like Visual Studio Code, stands out because they have the resources of world’s most valuable company at their disposal. Turns out it’s not closed-source license terms that make it easier to build good software, it’s money.
Tiago Forte also offered thoughts on the role of capital in the longevity of software, specifically in the space of knowledge management apps like Foam:
Knowledge mgmt apps want to be open and standards-compatible so they’re available to the largest number of people
So they have no moat and fail to build sustainable businesses
Which means they go out of business and are available to no one
We need cutthroat capitalist PKM — @fortelabs, 12 July 2020
To respond to this, suffice to say: Pffffft. Just watch me.
Reading back the previous section, I realise that this sounds like I’m gearing to asking you for money, either in form of a paid product or sponsorship. Doubly so, if you have read the Athens Sponsorware announcement shared earlier today.
Rest easy, feel free to let go of your purse strings, I am not going to ask you money. Though, if you were excited at the prospect of sponsoring an open source personal knowledge management tool, I recommend sponsoring Athens on their OpenCollective! ❤️
This is not to say money is bad.
It’s true that money makes some things more complicated, but it sure as hell makes a lot of things a lot easier!
Money is the best known mechanism of incentivising people to do things they otherwise wouldn’t. If Foam had (a lot of) money, we could hire UX researchers, UI designers, documentarians, community managers, and all the other valuable but underappreciated-in-open-source roles that are necessary for building great products. (And yes, developers too, because it’s still easier to pay someone to write code than to hope they’ll do it out of their own free will, and in a way that you hoped they would.)
“Stop daydreaming!”, you may say. This is open source software. We can’t have nice things.
But, what if we could?
What if I told you I was already offered between 1-3 million dollars in exchange of a small stake of Foam? Not by one, but multiple venture capital funds.
And they were (at least on paper) on board with keeping the software open source.
For a piece of software that’s more curation than code. More glue than… whatever it is you want to metaphorically glue.
For a non-entity entity that’s not even a registered company, and technically speaking, owns nothing.
Not going to lie, I did consider it.
It’d be nice to have money.
It’d make it a lot easier to build a world class product.
It’d be nice to finally exchange all these pointless GitHub stars for some real-world value tokens.
It’d also kill whatever this fucking thing dead in a heartbeat, both spiritually, but also, probably, as a product.
Fuck Tiago Forte’s “opinions”. I believe Open Source can create sustainable, world-class software, and that’s what we’re going to do.
I’m not going to take the easy road.
(Side note: I’m not disavowing money entirely. One day, I might ask you to loosen your purse strings, either in form of (voluntary, no-strings-attached) sponsorships, or in the form of your credit card details in exchange of a paid product offering. But that offering will have to earn your money by providing some outsized value that a loosely coupled Open Source organisation cannot give you for free)
So, what’s the plan? What’s in store for Foam? How do I plan to overcome the odds stacked against Open Source, and build a great product despite systemic challenges?
Why can’t open source end-user applications (as opposed to frameworks, libraries, developer tools), typically compete against their commercial alternatives?
My plan, at a very high level, is to attempt to address each of these issues:
So, how do we do this? Well, with all its warts, Open Source does have one thing that corporate competitors (usually) don’t:
We’re not in a rush. Foam is intended to be a lifelong companion on your quest for knowledge. Things Take Time, and to do them well takes twice as much. We’re going to move intentionally, and with focus.
We’re going to focus on building a sustainable organisation that can rise above the usual pitfalls of Open Source (poverty! misery! drama! people fucking off to take paid jobs because they need to eat!).
How exactly we’ll do this? That’s not solely up to me, that’s up for us as a
community distributed team to decide.
I do however, have some thoughts I’d like to share, but it’s getting late and I don’t feel like explaining them right now (another benefit of not having taken anyone’s money! No accountability!), so I’ll just drop this here without further comment:
(Ok, maybe just one comment, which is to say that this is a draft, and I’ve thought about this about as much as I thought about the first version of Foam before I launched it, which is to say, not enough. This is a conversation I’ll aim to open as a RFC
by August 1st, and hopefully ratify an organisational model by September 1st in early September, when I plan to transition to working on Foam full-time, paid by my savings until a further strategy emerges.)
If you’re interested in joining us on this journey. check out Foam. Foam 0.3.0 is out now with a few new features, and much more to come. Whether you’re a software engineer, designer, writer, community organiser, or just a personal knowledge management enthusiast interested in contributing to a commons, read our Contribution Guide or get in touch over at Foam Discord.
On a personal note, I feel optimistic about Foam, and our ability to create a best-in-class software product that truly helps people. Just as Foam, the tool, is a product of my slow hunch, I believe the work I’ve done in our industry as an engineer, leader, community organiser, hype man, and a general hard-to-eradicate nuisance has prepared me for the role of Foam’s curator, and I’m excited to see what’s in store for us.
It’s been a hell of a month.
Looking forward to the next one.