We Deserve Better Screenshots
What if your screenshots had context?
Featured: replay, activity log, recent notes, + prompts.
Over the summer, Donna and I spent time thinking about annoying instances in our lives that deserve better experiences. One such experience was the screenshot. Why can’t I write any notes on it? How come I never see them again? Where do they go?
This problem is iPhone agnostic, although Apple is the primary culprit discussed here. Donna has an Android phone, and while the Google Photos app affords her more annotation opportunities, there are limited ways to reconnect to any past screenshots.
So, I tweeted this, and within a day, Donna and I began a design agency, Delta Beta Design, to tackle this as our first conceptual design project. A month later, we’re ready to share a glimpse of what Screenshots in Context could look like. We hope someone will take us up on this offer to design the experience. We’ll build it with you.
In short, we want to pose the question:
What could screenshots look like if we prioritized context, rediscovery, and reflection?
I’ll summarize our perspective as this: add context and reflection to the screenshot process to resurface good ideas and build on them. Notes, visuals and videos on how and what that looks like, below.
*Necessary caveat - while this followed a valid research approach, this is intentionally loose. If you know me, you understand I spend my time dreaming in a light manner as to paint a picture of what may be. Treat this as such. It's fun. It’s not designed to be a technical model.
Quick - how many screenshots do you have on your phone?
Go find out. I’ll be here.
(Apologies if you have an Android phone. Some don’t even show how many screenshots you saved! )
My guess is that you struggled to find exactly where the screenshots were even located. About half of the people (n=40) I talked to did. Once you did find it (if you did), you probably realized you have hundreds, if not thousands, of screenshots with little context of what they are. There’s no groupings. No words. No reference at all unless you click into one to see the date in which you took the shot.
My point should be illustrated. Screenshots are not considered important to our phone software makers.
Why do we take screenshots?
The best part of this whole experience was asking friends and strangers about their screenshots. After 40 mini interviews, we found a few prominent trends.
We capture screenshots to snap a moment in our mind's time, but we rarely revisit these captures. Less than 20% of people claimed to review their screenshots more than “once in a blue moon.” This is a missed opportunity to resurface and contextualize moments that mattered to us in the moment. Why did we save this?
Most screenshots are captured after a feeling. Most people we talked to claimed they most commonly took screenshots because "it reminded me of someone." This suggests screenshots tie closely to embodied cognition. We experience something — in scrolling, listening, or reading — and we feel a sensation that tells us to capture this moment in our mind's time. Even better when it leads us to reach out and text those we love, which most people did following this type of screenshot.
The third insight was expressed through excitement that a question seemed to expose a problem in their lives. People take a lot of screenshots (data below), and they almost unanimously realized they had saved far more screenshots that they had once found important.
What’s wrong with screenshots?
Screenshots contain insights that excited us in a moment, but are often lost due to current phone designs. This problem is felt routinely; this Twitter thread shows screenshots to be a comically common problem. Seriously, go look at that and laugh. I could fill this entire piece up with notes in that thread.
A small amount sample data (n=40) acquired by us backs up the belief. Most of us have hundreds into thousands of screenshots (median - 260, mean - 1060, min - 0 (never used, lol), max - 20,047) buried in our photo libraries that we don't fully recall and almost never return to. And yes, you read that correctly. One person interviewed had over 20,000 screenshots on their device. For what it’s worth, they offered a lovely explanation for their variety of use cases.
This data requires a deeper dive. Without our enormous outlier, the high is 4038. Still large, but comparably a drop. This provides us with a more realistic look, as the mean total of screenshots is 540. You might also ask - but what of the people with new phones, Bren? Good question!
On average, our sample group took ~50 screenshots a month. This varied greatly depending on overall volume, but it's important to note the frequency, even with phone upgrades and storage backups. There are other factors, as well. Google Photos automatically uploads and clears your active screenshot count on your Pixels. Apple does not. This all makes for interesting data collection of educated, mostly American mid-30 year olds, and you can apply any relevant caveats you wish.
Still, our focus is on total screenshots. We’re most interested in volume as it signals accumulation without review. “Interesting!” bells should be going off in your head, if you’re a builder.
On the iPhone, where I do my bidding — 904 times, to be exact — screenshots are seemingly treated as the lowest rung of data management. While there are many things you can do with your screenshots, they are little promoted — nor intuitively afforded — to us. They live within a folder within the Photos Library, in the ninth line of the fourth section. By my amateur count, there are 17 calls to action above my option to click ‘Screenshots’.
When I finally get to my Screenshots folder, there are nothing but standard image tiles — identical to the photos and videos on other pages. There's no immediate information of when I took the screenshot. There's nothing in sight about why I might have saved I off. There's no tags, titles, or text of any sort to trigger my memory. That is all more than a bit hilarious to me.
We believe this can be improved.
When do we take screenshots?
I take screenshots liberally, but nearly always with the intention of using what I took a shot of. Sometimes I immediately text a friend — what I snapped reminded me of them, whether it be a song I'm listening to, a picture I saw, or something I read. "This reminded me of you!" reflects the most common use case for screenshots from those we discussed with, followed closely by people using screenshots as a swipe file — a familiar concept in the design an ad worlds — to save off beautiful illustrations, cool diagrams, important passages of text, and examples of good ad copy. The final primary category fit the mold of reminders to self, like the temperature to grill meat or a quote to look at later.
What could screenshots look like?
We propose four small changes to screenshots. The patterns here are generalizable beyond the iPhone — Android phones, iPads, tablets, and even computer screengrabs are ripe for improvement. We all deserve better screenshots.
1) Text as a first-class citizen
When I take a screenshot on the iPhone today, there’s no sign I can add a note. I can scroll down on the screen and a comment box appears, but the default screen doesn’t afford any opportunity to think, “Why is this important to me?”
This…should change.
Simply add a note box below the screenshot. After I jot a note, prompt me to add any related tags.
This helps with future composability and note remixing.
I found this important. Let me note it. Then later, let me see it…
2) Activity log as a contextual trail
Context is critical to remembering why the screenshot was important. Screenshots capture a moment in time, yet currently lack details that help us remember why we took the shot to begin with.
Enter phone history. What was I listening to as I snapped the shot? What apps or sites was I visiting? Was I texting someone?
Added to the notes and tags I’ve added, I can see a more complete picture. I have context of why I thought it was smart to save the screenshot to begin with.
3) Rewind as a time machine
Let’s take that idea a step further. The moments leading up to taking a screenshot are probably meaningful.
Why not allow me to rewind and see what I was doing before I saved it?
By the way, the iPhone already has this feature. It’s called Screen Recording, and it’s rather fantastic!
This turns the screenshot into a living, contextual artifact of my thoughts.
Rewind + Activity Log for each screenshot
Press play for a :30 clip or check the history to see what you were doing before saving the screenshot.
4) Reminders as memory vitamins
Reviewing old screenshots rarely, if ever, happens. In fact, only one of our interviewed parties (not me!) mentioned reviewing old screenshots “from time to time,” while everyone else claimed to never review them. Hence, we're missing out on things we found important in the past.
Why not tap into a spaced repetition system? We know these are useful tools to improve memory and synthesis. We can base the prompt on screenshots we took notes on, those we sent to others, or shots we revisited or tagged in a certain way. Orbit is a great example of an engine that could back this model.
Also, the idea of memory vitamins really works for me as a metaphor. They’re good for you! IDK, roll with it.
Prior Art
We’re not the first to propose changes to everyday software. In fact, it’s a favorite pastime of many admirable software magicians (seriously, go check them all out). The base principle is that our experiences can, and should, be better…so let’s design them.
So what exists already today? Where is the prior art? Oddly, a number of tools that handle parts of the problem, but none devoted solely to resurfacing and contextualizing insights from screenshots.
The most popular apps seem to be screenshot threading tools — an extension to create longer images. The closest thing to noting comes from the image annotation apps, which admittedly look promising, but lack a primary focus on synthesis and knowledge building. Jaadoo gets the search and tagging functions down, but stops there for now. Omar Rizwan’s Screenotate is a move in the right direction, but it’s made for PC/Mac only (aside: I actually love this tool, and the freemium model is great if you want to be convinced).
In short, people have scratched the surface, but have not built exactly what we envision as a true advancement of the phone screenshot experience.
Also, it should be noted that Apple has these features already, separately! This suggestion is merely a combination of: Screen Recording + Screenshots + Notes, with a touch of love to the interface and interactions. We want you to be able to experience your screenshots in context, so you can turn static, fleeting snapshots into dynamic, contextual memories.
What else may come?
This design effort was limited in scope. We had some time this summer to think through things together, and my background in product design focused on essentials for sharing. But plenty more opportunities lie within this untapped mine.
Synthesis is a primary objective in the knowledge building community. How can you enable synthesis in the form of your screenshotted notes?
We modeled a synthesis review process pairing notes by:
Matching and/or related tags
Location
At random
The goal here would be to resurface, recompose, and compress your thoughts with the aid of context. This multiplicity helps embed complex ideas into a synthesized, shareable form. It’s a simple addition, and the literature in this realm is excellent to build atop (and admittedly, is part of my own research right now). Ultimately though, it felt beyond the scope of this design project, but should a form of this be built, you will see a form of this.
Beyond that, we subscribe to the belief that it should be more accessible for us to review our thoughts from a variety of tools — emails, texts, note apps, photos, and on. Linus Lee’s Monocole is a huge inspiration here.
The point is, our words and ideas are everywhere. Screenshots are just another strand in the thread of our thoughts. Let’s tie them together.
A full walkthrough
After the screenshot is taken (1), a text box prompts the user ‘What does this tie to?’ (2), which they can write in + highlight the relevant piece (3).
Later, a planned spaced repetition prompt asks you to build on your past notes (4), which takes you to a dedicated page for the screenshot, complete with an activity log, past notes, and the ability to replay the moments leading up to the shot (5). The video plays in screen as to not remove context for you to write a follow-up note, “What excited you about this?” (6).
This first clip (it’s only 22 seconds), walks through me taking a screenshot and adding notes.
This second clip (it’s 1 second longer), shows the value of returning to the same screenshot days later.
This was fun. I encourage you to reflect on it through that lens.
And that’s v1 of Screenshots in Context.
Love,
Bren + Fig
Delta Beta Design