How we tested with real product teams

Most apps get tested in controlled environments. Controlled environments lie.

Before BoardSnap went public, I put it through 12 real product team sessions at three different companies — one early-stage startup, one mid-size SaaS, one consulting firm. These weren't usability tests. They were real meetings that happened to have a BoardSnap beta installed.

Here's what I saw.

### Setup

I recruited through my personal network. I asked for teams that:

1. Used physical whiteboards in meetings (not just Miro)
2. Had at least one meeting per week where the board mattered
3. Were willing to snap their boards for 3–4 weeks and send me honest feedback

The three companies were diverse enough to stress-test different use cases: the startup did sprint planning and architecture reviews, the SaaS did retros and cross-functional syncs, the consulting firm did client workshops and deliverable planning.

### What broke in week one

The project-per-client assumption. I'd designed Projects assuming one project = one work context. The consulting firm immediately showed me the flaw: they have clients with multiple workstreams that belong in separate projects but share a client relationship. My data model didn't support nested projects or shared brand context across projects. I added multi-brand context support (you can link multiple URLs to a project) as a temporary fix.

Action item ownership for larger teams. BoardSnap extracts ownership markers from boards ("→ Sarah", "[Engineering]"). On small teams (3–4 people), these markers were self-explanatory. On the 8-person cross-functional sync, "[Engineering]" was too vague — there were three engineers. I added a prompt improvement to extract more specific ownership language when it's present on the board, and flag ambiguity when it isn't.

The glare problem was more common than I expected. I knew about overhead lighting glare from my own testing. I didn't expect how many different configurations of fluorescent lighting could produce it. Three different teams in three different rooms hit the same failure mode. The fix — move your angle slightly — works, but I hadn't made it prominent enough in the UI. I added a contextual tip when the VisionKit confidence score is below threshold.

### What surprised me

Teams adopted the tri-state model faster than I expected. I thought in-progress would be a feature users would ignore for a while before discovering. Within three sessions, every team was using it naturally. The visual design (the dot-in-ring icon) apparently communicates its own meaning without instruction.

The summary was often more accurate than the note-taker. Two teams had dedicated note-takers in their meetings. I compared the note-taker output to the BoardSnap summary from the same session. The BoardSnap version picked up items the human note-taker missed, typically items written in the corner of the board or written quickly during a tangential discussion.

Nobody read the full summary. This was the uncomfortable one. Users skimmed the summary and went straight to the action items. The summary is there and technically correct — but it's not what people are using. This changed how I think about the information hierarchy: action items first, summary as supporting context, not the other way around.

"I would have just taken a photo" was the most common alternative. When I asked what teams would do without BoardSnap, the answer was almost always "take a photo and post it to Slack." The photo-to-Slack workflow is so embedded that people didn't even consider it inadequate. This told me the competition isn't Miro or Notion — it's the built-in camera app and Slack. That framing changed how I talk about BoardSnap.

### What changed in the product

Based on these 12 sessions:

1. Action items moved above the summary in the UI hierarchy
2. The glare tip was added as a contextual in-finder prompt
3. Multi-URL brand context was added for consulting firm use cases
4. Ownership ambiguity flagging was improved in the action item extractor
5. A "quick copy" button was added to the action item list for one-tap paste into Slack/Linear

None of these changes were on my roadmap before the real-world testing. All of them came directly from watching people use the product in situations I hadn't designed for.

This is the argument for testing with real teams before launch, not after. The issues that matter aren't the ones that appear in controlled usability tests — they're the ones that appear when a room full of people who have 45 minutes to run a sprint retro encounter your product for the first time.