Thirteen steps, three tools, one click
Context
Turning features on for a paying school district took thirteen steps across three internal tools, with the knowledge of the sequence siloed in a few heads. The company was simultaneously moving to tiered packages (essential, plus, premium) which the process had no concept of.
Problem
The feature flag service refreshed its cache every ten minutes. Onboarding
teams would enable a feature, see nothing change, and conclude the tools
couldn’t be trusted. Sometimes they got lucky near the end of the interval
and saw it apply instantly, which made the system feel random rather than
slow. Features hung off an unholy mix of paid_at dates, boolean columns,
and remote flags, and the only audit trail was a Slack bot.
The decision
One tool, one click, and a real data model: migrate feature state out of the flag environment into Postgres tables, give tiers cascading enablement (a district enables its schools, a higher tier includes the lower) while deliberately not cascading disablement, so accounts could move down a tier without collateral damage. Every change writes an audit event to Redshift, surfaced in Looker from the tool’s front page.
REJECTED: patching the existing process across three tools with better documentation. The tools weren’t misunderstood; they were genuinely inconsistent. No document fixes a ten minute lie.
Execution
The cache was the trust killer, so it went first: working from Datadog load data and the service’s memory and CPU headroom, I showed the refresh could drop from ten minutes to three seconds with no capacity risk. Changes now appeared essentially live. Along the way the new audit queries exposed that districts synced via API were namespaced differently from CSV ones, which had quietly granted unpaid rostering features for years; fixing the audit recaptured that revenue. Expiration dates flowed in from Salesforce and disabled paid features automatically, with renewal emails at 30, 15, and 1 day. I delivered roughly two-thirds of the project solo after the second engineer rolled off, shipped Loom training and docs for three audiences, and the tool ran free of regressions from rollout to my final day.
Outcome
13 → 1
Steps to enable a feature package
10 min → 3 s
Flag cache refresh: the trust killer
Zero
Regressions, rollout to handoff
Takeaway
Distrust of internal tools is usually a feedback loop problem. Make the change visible in seconds and the trust follows.