Second Look

The facepalm problem

As a parent (and son), I spend a significant amount of time reviewing what family members find and share on the Internet. Facepalm is nearly the default reaction when one of my teenagers or parents sends me what they've "found" on Facebook, Buzzfeed, or X. While I'm more skeptical than most, I'm still amazed by what people take as fact just because they see words (or pictures) on a screen. With AI in the mix, it's more important than ever to scrutinize just where information comes from and how to process it.

We're starting to have conversations with our twins about how to evaluate what they read and see: assessing sources, applying reason, validating facts, etc. While these things are hard in the beginning, the hope is that they'll develop a lifelong muscle they can use to handle any information encounter down the road.

Yet, as we know, technology sometimes moves faster than we mere mortals can keep up with. Across all the generations, there are people who are getting fed information that looks valid to them, but they haven't had to build the habits the rest of us picked up or lucked into. I wanted to see what I could build that might help give back an edge, something to quickly call in where a second set of eyes might help.

The space between the apps

I've been tinkering with Shortcuts lately because I think it's a fun place to experiment with all the powerful features of the iPhone that sit between the apps. Shortcuts let you build miniature workflows and experiences that can work across and independently of apps. It's also the place where you can get raw access to some pretty powerful capabilities like Apple Intelligence or automated data extraction. Given that iOS does not (yet) provide a native way to provide a gut check on what you're seeing on screen, Shortcuts seemed like the perfect place for some experimentation.

One press, three answers

My Second Look Shortcut lets iOS users quickly evaluate a web page, screenshot, or open screen for its potential validity. The Shortcut requires no external software and only uses Apple Intelligence's Private Cloud Compute to provide the evaluation (however, it does require an Apple Intelligence-capable device). The Share button makes it easy to pass along the context in just about any app, but users can also configure a Control Center button or even program the Action button to trigger it.

The Shortcut itself collects as much information about what the user is looking at (a social post, a news article, a text message) and packages it up to Apple Intelligence for a look. The prompt is focused on making broad judgments as to whether the intention is to look like a news post or something humorous, then it simply states whether the item might require some further evaluation and suggests how to start.

• A text from an unknown number

  Worth a second look. A text from an unknown number asking for your email. Who sent it? Acting could risk money or personal data.

• A CNN article, shared from Safari

  Looks like a real source. This link goes to cnn.com, CNN's own site.

• An Onion story, shared from Safari

  Probably meant as a joke. A satire piece from The Onion. It's clearly labeled as their site.

I intentionally stayed away from having the model judge what is actually true. This isn't about judging posts about aliens landing in Washington D.C. from Facebook, it's about challenging users to think twice about what they read. That being said, the Shortcut includes instructions on how to adjust the prompt to increase the level and scale of judgment.

With the shortcut built, I tested across a wide variety of situations on my phone: news articles from CNN, social re-posts of sketchy takes on Facebook, text messages promising Instant Piles of Money… the results were strong but not perfect. Every result probably deserves its own second look, but the whole point of this is to get people thinking.

What I learned

1. How it handles what it can't see. Shortcuts doesn't divulge the active app for screenshots, and we don't try to guess. So, we ask, "Are you reading this in the CNN app?" just to plant the seed.
2. Fixing what it overbuilt. At one point the Shortcut ballooned to over 130 actions. Deliberate requests to streamline got it back down to a sleek 31 while retaining all the desired behavior.
3. Intelligence isn't universal. This shortcut requires Apple Intelligence features that are only present in the most recent iPhones. We would need to incorporate a different AI provider for older devices.

If there's a version 2

I felt like this was a good stopping point for a tool intended to be shared and used broadly, but here are a few items that might make it into v2:

• Better native language personalization for the user.
• Ability to swap out with an external LLM if the user is comfortable sharing their data.
• On-demand research of additional sources.