Building an AI Writing Tool Nobody Used

In early 2023, like many companies, our company was asking how to become part of the AI revolution. Leadership moved quickly with a clear top-down directive: build an AI content tool for B2B subscription customers, including marketing managers at engineering hardware and manufacturing companies publishing technical articles on the platform.

On paper, the ambition made sense. Subscription customers were struggling to produce enough content. AI could, in theory, accelerate publishing. Our startup had strong SEO authority that these companies wanted to leverage. The pieces seemed to fit, except for three that never made it onto the table before design began:

• A clearly defined business outcome
• A validated user problem
• A measurable hypothesis for success

As the designer on the project, I pushed to define what success should look like before committing engineering capacity. The conversation happened, but momentum and market pressure were already pushing the initiative forward. In a founder-led environment, speed won over rigor, and we shipped.

What happened next taught me more about product thinking than any successful feature I've worked on.

We built an AI Copilot: a content generation tool embedded directly inside our company's CMS for our subscription customers (which were few to begin with). The target users were marketing managers at engineering hardware and manufacturing companies publishing technical content on the platform.

The flow was intentionally straightforward:

• Paste existing content or provide a few keywords
• Generate a structured outline
• Expand sections into full article drafts
• Publish directly to our company's published content

The UX was clean and intuitive. Early usability feedback confirmed that the flow was easy to understand. One early tester even called the outline generation "pretty impressive."

We rolled out to a select group of B2B subscription customers. Roughly half tried it at least once. But only about 1 in 10 used it more than a single time.

That gap, between easy to use and worth using again, is where the real story begins.

Key UI States

Within weeks of rollout, the quantitative signals were clear. Trial rate looked promising: curiosity was high enough that half of eligible customers tried the tool. But repeat usage,our strongest signal of value, was almost zero.

We didn't stop at the numbers. We ran interviews, monitored behavior in the CMS, and tried re-engagement campaigns. Across these inputs, the same themes surfaced:

• Outputs were often too long and cumbersome to edit.
• Technical claims were sometimes incorrect, increasing the editing burden instead of reducing it.
• Regeneration felt random, customers wanted more control and less of a black box.
• Many teams had already adopted general-purpose AI tools for their writing workflows.

That last point was the sharpest signal. Our users weren't avoiding AI, they were already using it, just not with us. The problem wasn't the category. It was our specific implementation.

One discovery during customer conversations reframed everything: subscription customers weren't primarily trying to save time on writing. They wanted their content to rank on Google and reach a wider engineering audience. Many were copying existing published articles into our tool and getting nearly identical content back, content that couldn't rank because it wasn't original.

We had built an efficiency tool for a problem our users didn't actually have. Their real problem was reach.

Quant + qual synthesis

Instead of explaining away the low adoption, we worked backwards from the data to understand where the initiative had broken down. Five core issues emerged.

1. The solution was defined before the problem.
   The project started with a technology decision, "we need an AI tool", rather than a validated user pain point. There was no written hypothesis, business objective, or success metric before engineering capacity was committed. Without a clear target, we couldn't know if we were building the right thing until after it shipped.
2. AI intervention happened at the wrong stage of the workflow.
   The tool optimized rewriting and expanding existing content. But research revealed that the real friction was earlier: understanding what to write so content would actually rank and reach an engineering audience. We solved the visible problem, not the most painful one.
3. In technical domains, accuracy is non‑negotiable.
   Engineering content lives or dies on credibility. When the tool generated technically incorrect claims, it didn't save time, it created more work. Editors now had to fact-check every output. The editing burden erased any time savings and eroded trust.
4. Trial rate and repeat usage are fundamentally different signals.
   Early adoption looked healthy. Curiosity got people in the door. But curiosity is not product-market fit. Repeat usage is. We learned to treat first-time trial as noise and return behavior as the metric that matters.
5. We were solving for a problem customers didn't have.
   Customers weren't asking for better content generation. Their marketing content was already performing adequately. What they wanted from our company was distribution, more engineers reading their work. Our AI tool added effort without directly impacting the outcome they cared most about: reach.