Ruminations on the idea of a spark file

Thanks to a post over on David Pottinger’s blog, Steps & Leaps,* I was reminded of the concept of the spark file, the invention of writer Steven Johnson. As Johnson describes it, the spark file is “a single document where I keep all my hunches: ideas for articles, speeches, software features, startups, ways of framing a chapter I know I’m going to write, even whole books…. There’s no organizing principle to it, no taxonomy–just a chronological list of semi-random ideas that I’ve managed to capture before I forgot them.”

The key to the effectiveness of this exercise, according to Johnson, is periodically reading the spark file from start to finish.

“I end up seeing new connections that hadn’t occurred to me the first (or fifth) time around,” he writes. “Sure, I end up reading over many hunches that never went anywhere, but there are almost always little sparks that I’d forgotten that suddenly seem more promising. And it’s always encouraging to see the hunches that turned into fully-realized projects or even entire books.”

This is a tantalizing idea, not unlike the bullet journal in some ways. It’s a single place to record data. In the bullet journal you rapid log events, tasks and random information. In the spark file you keep ideas that you don’t want to forget, at least not now. To get the most out of either system, you need to review the contents on a regular basis, the bullet journal probably more frequently than the spark file.

But there are also crucial differences in the two systems. In a bullet journal it isn’t so much the expression of the information, but the information itself. You bought book Z at your local bookstore on February 3. You completed Y project on March 12. That kind of thing. With a spark file, it seems to me, the expression of the idea is almost as important as the idea itself. Or, to put this another way, recording the idea will instant start to change the idea, and you will want room to be able to explore those changes. It might take two sentences, it might take a whole page. And when you review your ideas months later you will want to be able to append thoughts about that idea.

While a bullet journal could serve as the staging point for a spark file, where you make a quick note of something you want to write down in more detail later, you likely would never move information from the spark file to your bullet journal, unless it was a note to follow-up on an idea.

Johnson says his current spark file is over 50 pages (as of two years ago). So the spark file needs to be able to grow organically. I don’t flatter myself that I have that many ideas floating around in my head or ever will, but do wonder if I would have more if I had a more systematic way of non-critically capturing my “hunches.” Is a spark file something that would work for me? The only way to find out, I think, is give it a go.

And that leads to the next question: What’s the best way for ME to implement this concept?

From Johnson’s description, I can see three essential attributes of an effective spark file:

  • I must be able to access it for both reading and writing from anywhere, at any time.
  • Capturing my thoughts and ideas must be quick and easy, because any road bumps might make me decide to check my e-mail instead of writing down that “hunch.”
  • I must be able to read the entire set of hunches from start to finish in one window. But does this mean it needs to be a single document like Johnson uses? Or can it be composed in an application that provides concatenation of smaller documents into a longer one for reading?

One more essential attribute can be inferred from Johnson’s description, though he does not make it explicit. Regardless, it certainly is important to me. And that is, writing and editing needs to be easy, because I find that once I start to write down an idea from my head, that idea instantly begins to morph into something different. To allow myself the ability to explore this evolution, I have to be able to write in a nimble editor, one that makes it easy for me to re-write as I go. This may sound like a no-brainer, but it is nevertheless important for me to make this a requirement, as it will impact the tools I choose to use to keep my spark file. For example, it is one reason why a paper notebook would not work for me, as I am a terrible writer and editor with pen and paper.

As I first started thinking about creating my own spark file, I also began considering options for how to manage it. I rejected hierarchical free-form databases like DevonThink or UltraRecall because the information stored in those applications is too atomized. While those apps may work for others who want a place to store their ideas, they are not in keeping with the major principle as suggested by Steven Johnson of providing a single, readable view of all your hunches.

But does that mean only a single document will work? Not necessarily. It occurred to me that the remarkable writing application Scrivener, could be an ideal environment for keeping a record of hunches. As I envision it, I would create a spark file project and then add individual documents for each “hunch.” There are a lot of reasons Scrivener would be an excellent application to use for this purpose:

  • It’s a great writing environment, in which it is easy to compose and edit text.
  • There are many ways to add meta data to your hunches, if you want to do so. The date you create an entry is automatically recorded for instance. You can also add a synopsis, or annotations, and lots more. That seems useful.
  • You can easily export your entire spark file as a single document in many standard formats, so you are not locked in to Scrivener, and you can make your spark file portable.
  • You can use the scrivenings view, which concatenates your entries into a single window that is virtually indistinguishable from a single document.
  • You can save your spark file project in Dropbox so it is accessible on all your computers. Scrivener is available for both the Mac and Windows platforms.
  • If you want to add information to any of the hunches in your spark file, you can either append them to the original document or add supporting documents, which can appear as sub-documents to the original.
  • If an idea looks like it will blossom, you can easily move it to its own Scrivener file, where you can focus on nurturing it.

There are, of course, a few reasons why Scrivener might not be a good choice for maintaining a spark file. First, there is no current version for mobile apps. While this may be easily overcome using a quick note capture app with easy import into a Scrivener file, it does add a layer of complexity that works against the whole concept of ease and speed. The good news is that an iPad version of Scrivener is in the works and should be available soon.

A bigger issue for me is that in order to share a file among computers via Dropbox, you need to get into the habit of closing the file once you are done with it, as having the same file open on more than one computer currently can create problems. This, in fact, is the issue that has made me decide that Scrivener won’t work for me at this time. I know myself, and if I have to re-open a file in order to record a half-baked idea, I’m probably going to check my Twitter feed first and then… what was that idea again?

So I’m back to square one, which is doing just what Johnson does and having a single document. I can keep it in Dropbox for access from any of my devices. It should be in a standard format. Plain text or a Word document. I’m inclined to choose plain text, as there are more good options on the iPad for opening and editing that format. Okay. Now that I have limited my options to this, it is time to start my spark file and let the genius begin. Or something like that.

And maybe one day Scrivener will solve the problem of synchronizing among various computers using Dropbox.

*Please forgive the incestuousness of my referring to a blog post that refers to one of my blog posts; that’s how I was made aware of it in the first place.

16 thoughts on “Ruminations on the idea of a spark file

  1. I’m reminded of the idea that used to float around the Internets of keeping *everything* in one gigantic text file.

    You might like nvALT. It has tags. Notes are easy to search. It’ll sync across various devices. I use it on two Macs, syncing with Simplenote on a phone.

  2. How about using an nvAlt document called “Spark File” which consists of links to various files? The links provide the topic for the idea pages, but the elaboration of the idea takes place on the various pages. nvAlt syncs nicely with Ulysses on the Mac (just set the nvAlt folder as the external folder) and Byword on IOS.

  3. For those sparks I maintain a single file in my Dropbox. On the Mac I’m using nvAlt for capturing and reading, on iOS it’s Drafts with an URL action for capturing and Notesy for reading.

  4. Michael, James and Acky, it’s interesting that two of you suggested nvAlt. I almost included it as an alternative option in this post, but it didn’t meet the criteria I set for providing a single-document view of all the ideas in the spark file. In fact, I believe someone — maybe the original developer of Notational Velocity — even suggests that the more atomized your information, the more effective it is. That doesn’t mean nvAlt would not work as a great repository for ideas. The real question is how crucial is it to have all the ideas in a single view for review? Johnson certainly endorses that concept, and it is what intrigues me about the whole notion of a spark file. Thanks for reading this post and sending along your suggestions!

  5. A few weeks ago, after reading Steven’s post, I started a spark file using OmniOutliner. I also wanted to use an app that I can use anywhere. So far, I’ve entered about half of my ideas on my MacAir, and the other half on my iPad mini. I sync the two via OmniPresence. It’s working great so far.

    I really enjoy your blog!

    1. I was considering OminOutliner. It seems a good choice, except for lack of a Windows version — and I spend 8 hours a day, five days a week on Windows at work. I would welcome any other details about how you use OO as a spark file.

      Thanks for the nice comments about my blog!

      1. I can’t say it’s become a solid habit yet, but I often leave the file open on the right side of my screen (about 1/4 of the screen). If something hits me, I do I quick ⌘+tab or ⌘+` to pull it up and then add the idea. When I read back through it, I like to go full screen so that the “sparks” are limited to one or two lines on screen.

        As for organization, right now I’m simply using parent rows for [Month Year]. Initially I felt the urge to date each idea, like I’ve done in my idea files in Notational Velocity that have bold headings for the basic idea/topic and date, with idea points beneath (and there, I often include a simple note about what I think may have triggered the idea). But I think it might be helpful to keep my spark file as simple as possible. First, I don’t think it’s really necessary to know the exact date. But secondly — and this is just a psychological trick that I’m testing — I think it might be more encouraging to go back and read 30 or so ideas for the month than to read 30 or so ideas that (as the dates remind me) came from just a handful of days.

        And a few other things I’m testing…

        Topics: I usually begin the entry with a one to two word topic tag followed by a colon. That is helpful for quick scanning to find an idea. But I’m also beginning to think that it might defeat part of the purpose of a spark file if (a) the small pause and extra effort to classify it somehow dampens the spark and/or turns the process into one requires more than the minimal amount of effort needed to simply record the idea and get back to what I was doing, and (b) assigning ideas to categories / topics at this stage somehow becomes an obstacle to cross-pollination of ideas (if that makes sense).

        Check boxes: I’m checking the check boxes of ideas that I implement. But I’m not yet sure if that’s particularly helpful.

        Outlining: When I have two or more specific ideas that relate to the same basic idea, I enter them as parent (basic idea) and children (specific ideas). It’s working for me so far.

        Formatting: Looking back over my spark file, I find one line all in bold. I obviously thought it was an extra important idea. But right now I’m thinking that the spark file is not the place for marking ideas in terms of importance, practicality, and so on. Instead, I think it would be better to create a different kind of file (possibly using Scrivener, Tinderbox, or something else) for the “better” ideas that I want to turn into projects, etc.

  6. Randall, thank you so much for the added details of your spark file system. I enjoy reading how others use software and adapt them to specific uses. It helps illuminate my own efforts. In my spark file system, I am going to try first to NOT classify the contents in any way, other than to keep the entries in chronological order. I have a sense that trying to add those details will prejudice how I compose these ideas as I set them down. I’m going to try to write about this in an upcoming blog post.

    Anyway, thanks again for the engaging reply!

    1. You’re very welcome. I appreciate the interaction.

      I think you’re right about the topic classification at that stage. There’s the problem of prejudicing how it’s composed (good point). And then there’s the prejudicing of my later review because I will have essentially pigeonholed the ideas, and that will probably make it more difficult for me to make creative associations/connections among different ideas (that’s what I meant when I wrote “becomes an obstacle to cross-pollination of ideas”).

      I look forward to your next post.

  7. I use nvALT for Sparkfile. Indeed I have 2 files in my nvALT that I call “Sparks” and “Scratches” (I used to call “Flashes” before Johnson advocated the idea; and I used to writ them in Onenote before I get my mac). Sparks are good ideas. They have the potential to grow to a bigger idea, or will be included to another project. the Scratch is for writing scratchpads on timely basis: what I am doing every hour. Some people call this one “log”. I have an Alfred workflow that directly appends Sparks and Scratches into their respective files. I don’t even need to open nVALT or any app for dropping my flashes of ideas; I CATCH them fast in Alfred.

    I can share the workflow if u want to.
    (there is a way around for syncing Scrivener’s files to dropbox bzw—have a look at here:

    1. Thank you for the comments, Dellu, and the link to your article about syncing your nvALT notes with Scrivener. Very good stuff!

    1. I actually have licenses for both FoldingText and TaskPaper — both of which are inspired concepts. I get the feeling that FoldingText is intended to eventually replace TaskPaper, and FoldingText still needs a lot of work (just my opinion). But I’m skeptical about Jesse’s commitment to getting these apps right before he moves on to another project, which has been his history. So for now I don’t use either of them, I’m afraid.

      The previous post, where I was speculating about using Scrivener was sort of just thinking out loud about the idea of a spark file and what would or would not work. I had come to the conclusion, even in that post that Scrivener would not work for me as a spark file manager because it didn’t have a good iPad option and keeping the file in sync across computers requires closing the file when done so there are no complications. The second article on the topic was reporting on my further conclusions and the solution I have settled on.

      Thank you for the comment and for reading my blog.

  8. Thanks for your reply Steve,
    I understand your point on FoldingText and TaskPaper, it’s hard to use something we think do not get support in the future (at least this is my impression), your speech only had me reminded the characteristics of these two software.

    I tried it today for the first time Ulysses III, and I understood the differences compared to Scrivener (which I use since many years), so this clarify the reasons why you decided to use it for this project (of course, also the question of ‘use on the iPad is essential).

    – Marco.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s