Building a Better Research Ingestion Engine

The right mindset and tool can help authors increase efficiency, organize their content better, save time, and yield superior results.
Feb | 12 | 2024
  Feb | 12 | 2024
BY Phil Simon
  Phil Simon

Building a Better Research Ingestion Engine

The right mindset and tool can help authors increase efficiency, organize their content better, save time, and yield superior results.
Phil Simon
Feb | 12 | 2024

Building a Better Research Ingestion Engine

TL;DR: The right mindset and tool can help authors increase efficiency, organize their content better, save time, and yield superior results.
Phil Simon
Feb | 12 | 2024

Rookie non-fiction authors typically make plenty of mistakes when writing their first books. I sure did back in 2008.

I can summarize a biggie in four simple words: They just start writing. Put differently, they fail to do the necessary pre-writing work. Yep, I’m talking about proper research. It’s unfortunate because, of all the hundreds of things authors need to do, research-related tasks represent some of the most important ones—especially at the start.

Of course, experienced scribes have already learned this lesson. With rare exceptions, such as breaking stories, they dedicate considerable time to the following activities before sitting down to peck away at their manuscripts:

  • Conducting interviews.
  • Reading academic articles, research reports, case studies, and other books on the topic.
  • Performing primary research à la Jay Baer or Josh Bernoff.
  • Listening to podcasts.
  • Developing frameworks.
  • Sussing out the competition, especially if you’re about to jump into a red ocean.

(If you lack the time to do these things and still want to write a quality non-fiction text, you’ll need to find a ghostwriter or a generous, well-compensated writing partner.)

Today, I’ll show you how to use a single tool as the fulcrum of your research and writing process: Notion.

A Note on Tools

Before we start, let’s say that I don’t convert you to Notion. Fine, but note two things. First, with slight modifications, you could probably build a similar system in Microsoft Loop, Coda, or some of the other tools I describe in Low-Code/No-Code: Citizen Developers and the Surprising Future of Business Applications. After all, theft is rampant among software vendors.

Second, regardless of your weapon(s) of choice, I hope that this post plants a seed in your head. Say that your research approach is a tad disjointed like mine was. Maybe even scattershot. The following discussion may cause you to seriously rethink it.

Let’s light this candle.

Meet RacketHub

Prior Research Process and Applications

By way of background, my research, writing, and publication processes have evolved over the years. (Stasis and I typically don’t get along too well.)

In 2013, I began using a separate app (Pocket) to clip articles and blog posts for my books, often from the RSS reader Google Reader and, after Google killed it, Feedly. I also saved intriguing PDFs, datasets, and images into separate Dropbox folders with organized subfolders. Ultimately, much of that content would find its way into spreadsheets and Microsoft Word or Google documents.

To be sure, that’s a lot of tech, and those were just the research-oriented tools. Other parts of what some call the tech stack included:

  • Microsoft Word to write the manuscript like a pro.
  • Todoist to manage the project.
  • Canva to create near-final figures, although the designers took them to the next level.
  • Slack, Zoom, and Calendly for efficient communication.
  • Google Forms to collect book blurbs in an organized Sheet.
  • Other websites and programs that served discrete purposes, such as MyBib for endnotes and PDFpen for efficient text extraction.

This begs the question: Did my admittedly tech-heavy process work?

Sure. It yielded 11 books that don’t suck. To borrow a line from Jodi Foster in True Detective, though, that was the wrong question. The better one: Was there significant room for improvement?

The clear answer was yes.

The Ultimate Ingestion Engine: How Notion Databases Changed Everything

For starters, some of my tools overlapped, and I spent a good deal of time toggling among different ones. Sometimes, I’d forget if I linked to a PDF in a Word doc or stored a downloaded copy on Dropbox. Did an interviewee email me, DM me in Slack, or share a Google document with me? You get the idea. In my defense, I was unaware of a single app or program that would let me consolidate and simplify my research efforts.

In early 2022, I stumbled upon Notion, the aspiring magnet tool. After some experimentation, a lightbulb went off. I then went all-in on it. Notion transformed how I work in general, but I’ll focus below on my new research and book-writing processes,

Saying Adios to Pocket was a no-brainer for two reasons. First, and as you’ll see below, Notion makes it obsolete. The former is a far more robust, multi-purpose, and collaborative tool offering myriad benefits. Second, Pocket sucks now. Damn shame. Many Pocket customers have flocked to Readwise, but I digress.

Arrivederci to Google Sheets as well. When find an interesting article now, I just clip it from my browser into the following Notion database. (Yes, you can do this on any device.) Here’s a screenshot of some links that I’ve saved for a recent AI writing project of mine:

Simple Notion database view for book project.

What is the fulcrum of your research?

Why use a database to track this information? Lots of reasons. Most relevant here. it allows for easy sorting and filtering. (As I’ve said many times before, structured data is a beautiful thing.) And here’s where the magic happens. Want to know the sources added in the past week? No worries. Need to see the articles that you haven’t reviewed yet? Easy peasy. You’ll want to create multiple database views.

Benefits Galore

The database view above is intentionally simple. In reality, I track key elements of each entry through additional fields—technically called properties in Notionland. For full-length book projects, I’ll track:

  • Content source: Is it an article, video, interview, newsletter, chart, dataset, tweet, or a book? Pro tip: If it’s the entry relates to a proper book, your bibliography just got much easier to create.
  • Usability: Should we use the content in the book? A simple dropdown containing the choices yes, no, and unsure makes this decision clear. (Note that you should never delete an entry. You never know when you’ll change your mind.)
  • Subject: It might be current events, business, tech, or something else.
  • Who added the article: I share the database with my clients so they can add articles as they see fit. As Kafka so wisely said, “Better to have, and not need, than to need, and not have.”

Just like with Excel filters, you’ll be able to quickly manipulate the data to your liking. If staring at database records all day isn’t your jam, view the entries more visually on a Kanban board.

Finally, there’s an ancillary benefit. Politely reminding my clients that all research-related content goes here reinforces the importance of following a defined process. In 15 years as a professional writer, I can count on one hand the number of my projects that succeeded when clients routinely just did their own thing.

Looking Forward: Leveling Up via AI

On forthcoming projects, I plan on using the recently released Notion Q&A with Racket clients. Once you’ve aggregated oodles of content in a Notion workspace, the AI-powered feature makes it easier to collate it and—as its name implies—ask and answer all sorts of interesting questions. Returning to the database above, in theory, Q&A will be able to:

  • Quickly summarize vast quantities of information.
  • Save time trying to manually locate pieces of information.
  • Find patterns and discover hidden insights.
  • Suggest supplemental articles, much like what Feedly’s doing with AI these days.

I could keep going, but you get my point.

What You Need to Know

Remember two things. First, while you are probably excited to start the writing process in earnest, pooh-pooh the import of dedicated research time at your own peril. Constantly expecting to MacGyver your way out of problems on the fly is a recipe for rework, incoherent manuscripts, and other disasters. (Cue Eisenhower quote.)

Second, there are myriad ways to gather the ingredients for your non-fiction book. Some tools are better than others, but none is perfect. Despite its warts, I prefer Notion, but maybe you dig a similar no-code app mentioned earlier. Regardless of which one you use, it’s imperative to consistently use it. A slapdash approach will only hinder your progess and, in the end, the final product.

Get All Racket Posts in Your Inbox

Valuable tips, news, and insights about publishing, marketing, writing, and more.

Comments close 120 days after the post’s publication.


Related Posts

The AI Razor

Thanks to today’s über-powerful tools, it’s not at all hard to shave off a few hours of manual work.