User Feedback Triage

A major developer of consumer apps experienced sudden growth and went from receiving a few dozen items of user feedback per day to a few hundred. Their manual review process was becoming a big source of wasted time: Reading hundreds of items of user feedback to find a handful of valuable insights was cumbersome, and as the company grew, getting the right feedback to the right people was becoming difficult.

We saw two key opportunities:

1. Use AI to surface the valuable feedback.
2. Provide a way for staff to declare the types of feedback they were interested in.

First, we took a sample of 1,000 items of user feedback, and manually classified each item as either helpful or not helpful. Then, we created an AI automation which used GPT to perform the same task. We could prove to the client that GPT could do this work just as well as a human, but faster, and with minimal cost. We set up a daily automation to triage the previous day’s feedback.

Then, we created a simple internal tool which integrated with Asana. With this tool, anyone in the company could list keywords they were interested in, and they’d then get a daily report of all the feedback which mentioned those terms.

We cut the amount of time wasted reading unhelpful feedback to zero. We also helped more people in the company see the feedback they needed, promoting a more user-centric culture across the board.

In this case AI automation didn’t reduce the impact of humans, it amplified it.