Notion's AI leak shows 'zero data retention' was a clause, not a fact

Your invoice says Notion. No invoice says Fireworks AI. Yet Fireworks was holding Notion AI chat transcripts that, under contract, it should never have kept. Do you know every company your AI tools quietly hand your prompts to? Neither do I.

What happened

Fireworks AI is a platform developers use to run open AI models; TechCrunch says it has tens of thousands of customers, including Harvey and Cursor. Per TechCrunch, attackers got in through its cloud provider and reached a development system; Fireworks says they took billing information and some datasets. It is Fireworks' second incident in two months. On November 12, Notion confirmed to TechCrunch that Notion AI chat transcripts were among the stolen data.

Notion's own security notice puts the impact at fewer than 1% of workspaces. The data is chat transcripts sent to Fireworks, meaning anything typed into or generated in the chat, plus data from LEAP, Notion's program where customers volunteer data to test new AI features. Notion cut the connection on November 3 and deleted the data there. The Insights feature is switched off.

The leak wasn't the AI. It was the dashboard about the AI.

Here's the part I find telling. Insights is an admin feature that shows how workspace members use AI, and Fireworks ran the classification behind it. So the exposure didn't sit in the core product. It sat in the analytics layer next to it and in a test program, exactly where teams stop looking because it's "just" reporting.

Then there's the line that sticks. Per Notion, Fireworks had contractually committed to zero data retention. Notion's investigation found data was retained anyway. Zero data retention turned out to be a clause, not a state of the system. Notion also spells out that its Enterprise ZDR covers API requests to model vendors, not stored features like chat history and Insights.

The catch for marketing teams

In my view, few teams pour more into AI chats than marketing: customer lists, campaign numbers, briefs, draft press releases. My rule of thumb predates this: customer data and final calls stay in-house, standardizable work can go outside. This case shows why. "Outside" isn't one vendor, it's a chain whose links you never see. AI doesn't fix broken habits, it widens the cracks. If you never settled what goes into a chat, you won't find out at purchase. You'll find out from the vendor's incident notice.

What I'd do now

• Notion admins: in a desktop browser, open "?" in the sidebar and go to "Security incident review," as Notion describes. If you haven't heard from Notion, it says your workspace wasn't identified as affected.
• Rotate any passwords, API keys or credentials someone pasted into Notion AI. That's Notion's own advice, and it applies to every AI tool.
• Write one line of team policy this week: what never goes into an AI chat (credentials, customer lists, unreleased numbers). Then ask your three most important AI vendors which features send content to which other providers, and which features sit outside their retention promise.

Bottom lineA contract clause is a promise, not a deletion log, and your AI tool has AI tools of its own.