Three months to remove a CSV export

The conversation began in a product review on March 4. Robin, our PM for the workspace surface area, had the floor. She was walking through the quarterly roadmap. About fifteen minutes in, she put up a slide listing three "deprecations under consideration". The first one was the CSV export from the legacy dashboard. The slide had a single line of justification: assume low usage, deprecated UX, low engineering cost to remove.

Robin paused on the slide. She said it was uncontroversial and she expected it to be a quick yes. There was a moment of silence in the room. Lila, the staff engineer who owned the dashboard's repo, said "before we remove it, can we check who's using it." Robin said "sure, that's reasonable, how long would that take." Lila said "let me get back to you." The meeting moved on.

The "get back to you" took twelve weeks.

This is the story of those twelve weeks. It is also the story of a different conversation, three months later, in which a different PM proposed a different deprecation and decided, in real time, not to ask whether anyone was using the thing. That second conversation is the one that mattered. The first one cost us twelve weeks of engineering time. The second one is the kind of cost that does not show up anywhere, because it is the cost of a question that no longer gets asked.

Week one: looking at what we already had

Lila spent the first week looking at what telemetry we already had on the CSV export endpoint. The endpoint was /v1/dashboards/legacy/export/csv. It had been added in 2022 by an engineer who left in early 2024. The endpoint had basic access logs. The logs recorded the timestamp, the IP address, the response status, and the user agent. They did not record the authenticated user, the customer account, or any other identifier that we could trace back to who, specifically, was hitting the endpoint.

Lila wrote a short query against our log warehouse:

select
  date_trunc('day', timestamp) as day,
  count(*) as hits,
  count(distinct client_ip) as ips,
  count(distinct user_agent) as uas
from access_logs
where path = '/v1/dashboards/legacy/export/csv'
  and timestamp > now() - interval '90 days'
group by 1
order by 1 desc

The result was a hundred and seventy three rows of data. The endpoint had been hit between twenty and eighty times per day for the last ninety days. The number of distinct IPs ranged between four and eleven. The user agents were a mix of two values: one was our own dashboard's user agent string, and the other was curl/7.81.0. The curl requests were all coming from a single IP range.

Lila showed me the data. We talked through what it could mean. The dashboard user agent suggested at least some humans were clicking the export button in the dashboard. The curl requests suggested at least one customer had automated the export. We could not tell from the data whether the dashboard hits were from one customer with eleven employees or from eleven different customers. We could not tell whether the curl IP range was one customer's CI pipeline or three different customers behind the same corporate egress.

The access logs answered the question "is this endpoint receiving traffic". They could not answer the question "do customers we care about depend on it". The two are different questions, and the difference is the entire conversation.

Lila wrote back to Robin on a Thursday. The message said something like: "We have data that says the endpoint is hit between twenty and eighty times a day, but we can't attribute it to specific customers. We'd need to add proper telemetry to know who's using it. Want me to spec that out?" Robin said yes. The work to spec out the telemetry took the rest of the week.

Week three: the JIRA ticket

The ticket was filed as DASH-2841: Add per-customer telemetry to CSV export endpoint. The description said:

The legacy CSV export endpoint (/v1/dashboards/legacy/export/csv) needs per-customer usage tracking so we can answer "who uses this" before deprecation. Current access logs do not attribute requests to authenticated customer accounts.

Scope: add a small instrumentation hook in the endpoint handler that emits an event to the analytics pipeline with the customer ID, plan tier, last-active timestamp, and request count for the session. Pipe the events into the existing usage dashboard.

Estimate: 1-2 days of engineering work. No data backfill needed. Decision-readable data available 4 weeks after deployment.

The ticket sat in Lila's team's backlog for a sprint and a half. The sprint and a half was not negligent. The sprint and a half was that we were in the middle of launching a different product line, and Lila's team had three engineers, and two of them were assigned to the launch. The ticket was not high priority because the deprecation was not on a deadline. The deprecation could wait two weeks. Two weeks turned into three.

When Lila finally picked up the ticket in week three of waiting, the work took her two days. She added the instrumentation. The code was small:

// inside the existing handler
const customerId = req.auth.customerId;
const accountTier = await accounts.getTier(customerId);
const lastActive = await accounts.lastActiveAt(customerId);

await usageEvents.emit({
  feature: 'legacy_csv_export',
  customer_id: customerId,
  plan_tier: accountTier,
  last_active: lastActive,
  timestamp: new Date().toISOString(),
});

// ... rest of the handler unchanged

The PR went through review in a day. The deployment went out on the following Tuesday. By the end of the week, the dashboard had its first events flowing.

Week four: the wait

We agreed in the original spec that we needed at least four weeks of data to be confident about the answer. Four weeks because some customers run quarterly reports, and we did not want to mistake "no usage this week" for "no usage at all". Four weeks turned out to be too short, but we did not know that at the time.

The waiting was uncomfortable. Robin asked twice during the wait whether we had preliminary data. Each time Lila said the data was looking sparse but it was too early to call. The first time, two weeks in, the dashboard showed three customers. The second time, three weeks in, it showed seven. By the end of week four, it showed twelve. Robin came back to the product review the following Monday with the answer.

The slide said: "Legacy CSV export, 12 customers in 4 weeks." She looked at the room. The room looked at the slide. Eduardo, our VP of product, asked the question that mattered. He said "twelve out of how many." Robin said twelve out of about twelve hundred customers who had logged into the legacy dashboard in the same window. Eduardo said "so one percent." Robin said yes. Eduardo asked who the twelve were.

This is where the next two weeks went.

Week eight: the twelve

Robin pulled the customer list. It was twelve customer accounts. Five of them were on our enterprise tier. Two of them were on our top revenue tier specifically, which meant their accounts were each worth more than the rest of the dashboard's user base combined. The other seven were on smaller plans. Two of the smaller-plan accounts were curl users, which made them probably one company with multiple pipelines, although we could not tell for sure without going further. The remaining five were a mix of dashboard clicks and automated calls.

Eduardo looked at the list. He said two of these are accounts we need to be careful with. He said let me think. He said let's bring this back next week.

The "next week" became three weeks. In between, Eduardo had separate conversations with the customer success leads for the two top-tier accounts. The customer success leads checked in with their primary contacts at the customers. The contacts at the customers said they would need to confirm with their data team whether the CSV export was part of any automated workflow. The data teams took ten days to respond.

The responses came back the second week of May. One of the top-tier customers said yes, they had a quarterly report that depended on the CSV export. They said they could migrate to the new dashboard's export within two months if we gave them a migration guide. The other top-tier customer said they were not actively using it but had it scripted as a backup path for the new export, in case the new one failed. They asked us not to remove it for at least six months.

Eduardo brought this back to the product review on May 18, ten weeks after Robin's original slide. He said we are removing the CSV export. The deprecation timeline is sixty days. We will email all twelve customers. We will provide a migration guide for the one with the quarterly report. We will note the request from the backup-path customer and proceed anyway. Robin said good. Lila said good. The meeting ended.

Week ten: the email

The email took two weeks to send. Two weeks not because the email was hard to write. Two weeks because the email needed legal review and customer success review and product review and a final sign-off from Eduardo. The email said:

Dear customer,

We are writing to inform you that the CSV export functionality on the legacy workspace dashboard will be deprecated on August 18, 2026. The new dashboard's export feature provides the same data in a richer format. A migration guide is available [here].

If you have any questions, please reach out to your account manager.

Thank you, The blastmap team

That was the final draft. The first draft had been thirty percent longer and had used the word "sunset" instead of "deprecated". Legal had asked us to use "deprecated" because "sunset" sometimes triggers customer churn provisions in enterprise contracts. Customer success had asked us to add the line about reaching out to account managers. Product had asked us to remove a paragraph about why we were removing the export. Eduardo had asked for the migration guide link to be more prominent.

The email went out on June 2. We got three replies. Two were "thanks for letting us know". One was the top-tier customer with the quarterly report, asking us to push the deprecation date out by thirty days. We pushed it out by thirty days. The final deprecation is now scheduled for September 17.

What happened in September

Three months after the original product review. Six months after the original question.

In late September, after the deprecation has gone through, the CSV export will be removed from the codebase. Lila estimates the removal itself will take her one day. The day will include deleting the route handler, removing the dashboard button that points at it, and cleaning up the half dozen helper utilities that were specific to the legacy format. There will be a small amount of cleanup of the access log queries that referenced the endpoint. The removal will close out the original ticket from week three.

That is the entire saga. From "uncontroversial deprecation" in week zero to "code removed" in month six. Net engineering time: about a week, distributed across multiple people. Net elapsed time: six months. Net customer migration impact: one account had to update a quarterly script. Net executive time: maybe four hours, distributed across the product reviews and the side conversations.

You might read that and say, this is fine. The system worked. We checked before we removed. We gave customers warning. We accommodated the one customer who needed it. The process was thorough. Nothing went wrong.

I would say the same thing if it were not for the meeting on September 14.

The meeting on September 14

The meeting was a different product review. A different PM, named Mateus, was walking through his roadmap for the next quarter. He had a list of features in the analytics product that he suspected nobody was using. He had not yet asked the question. He was thinking about asking the question.

He said something like, "we have this reporting widget that I don't think anyone is using. Should we remove it." There was a moment of silence in the room. I knew what was coming because I had been in the same room six months earlier. Mateus knew the story of the CSV export. Eduardo knew it. Half the engineers in the room had been on the affected sprints.

Mateus said, "or, actually, let's just leave it for now. We can revisit next quarter." Eduardo said "are you sure". Mateus said yes, it's not blocking anything, and the time to verify whether anyone uses it is probably more than the time to maintain it for another quarter. The meeting moved on. The widget will be on the next quarter's review with the same status: not blocking anything, not worth the cost of asking.

I do not know if the widget has any users. I do know that the question of who uses it will not be asked, this quarter or the next. I know that Mateus's decision was rational, given the cost of asking the question. I know that the rational decision will be made in another product review next quarter, about a different feature. I know that the cumulative effect of these rational decisions is that we are accumulating features at exactly the rate that we are adding them, with almost no removal. The product is growing in a way that we did not intend and that nobody is explicitly choosing.

This is what the twelve weeks really cost. The twelve weeks cost us the next time a PM thought about asking who used a feature, and decided not to. The cost is invisible because it is the absence of a question. It will be invisible next quarter too. By the end of the year, we will have ten or fifteen features we have not asked about. None of them will be removed. The cost of each one will be small. The compound cost of all of them, multiplied by the next two years, will be significant. It will be measured eventually as "the product feels bloated" or "engineering velocity is dropping" or "new hires can't find the surface area they're supposed to own". The diagnosis will not say "we stopped asking who uses features in March 2026". The diagnosis will say something abstract about scale and complexity. The abstract diagnosis will not be wrong. It will also not name the cause.

Why telemetry is not the answer

You might be reading this and thinking, OK, the lesson is to add telemetry to everything. Instrument every feature when you ship it. That way the question is cheap to answer next time.

I have worked at companies that tried this. The instrumentation goes in for the features that the team thinks will matter. The team is wrong about which features will matter. Three years later, somebody wants to know about a feature the team did not think would matter. The instrumentation is not there. The question is again three months away. The work to instrument retroactively is the same work that Lila did, multiplied by however many features you want to ask about.

The way to fix this is not better instrumentation discipline. Better discipline does not survive personnel changes, team reorgs, or the relentless pressure of the next launch. The way to fix this is to have the question be answerable from the system itself, not from the instrumentation that was prescient enough to anticipate the question.

The system already knows who is hitting which endpoints. It knows from the request logs, the auth context, the access patterns, the runtime traces. The information exists. The information is not assembled in a form that answers "which customers used this feature last quarter, broken down by plan tier, sorted by frequency". The work of assembling it is the work that nobody has built. It is the work that almost nobody builds. It is the work that, when it is built, turns every deprecation conversation from twelve weeks into a query.

What we are building

We are building the substrate that lets the question be a query. Not because we want to deprecate features faster, although that is a side effect. Because we want product managers to feel free to ask the question. We want the cost of asking to be measured in minutes, not months. We want Mateus's September widget to be answered in his next 1:1 with Lila, with a direct number of customers and which plan tiers they are on, and for the decision about whether to keep or remove the widget to be a five minute conversation instead of a non-decision.

The thing we are trying to bring back is the experimental posture. The posture where you can ship a feature, see who actually uses it, and decide based on the data whether to keep it. That posture is what you have when your product is small. You lose it as the product grows, because the cost of measurement grows faster than your willingness to measure. Eventually you stop measuring. You stop measuring because measuring is too expensive, not because it is unimportant. Once you stop measuring, you stop pruning. Once you stop pruning, you accumulate. The accumulation is what makes the product feel slow.

The substrate brings the posture back. It makes the measurement free. It makes the answer to "who uses this" the same cost as the answer to "what is the latency of this endpoint". Both of them become queries. Both of them become things you ask casually, in a 1:1, in a product review, in a Slack message, without budgeting a quarter of engineering time to find out.

Robin will get her next deprecation through in a week instead of three months. Mateus will ask his question instead of not asking it. The product will get smaller in the places where it should, faster than it grows in the places where it should not. Six months from now, we will look at the CSV export removal and think of it as the last one that took twelve weeks. That is what we are working toward. Not the removal. The asking.