You can have the best infrastructure in the world. But if nobody watches the spend, you're just burning money efficiently.
62 Azure subscriptions. Different teams. Different projects. Different budgets. And not a single automated cost anomaly alert across any of them.
Let that sink in. Sixty-two subscriptions worth of cloud resources, running 24/7, billing by the minute — and the only way someone noticed a cost spike was when the monthly invoice arrived. By which point, the damage was already done.
I've seen it before. A forgotten VM running for three months. An undeleted test environment burning through premium storage. A misconfigured autoscaler that decided 3 AM was the perfect time to spin up 50 instances. All discoverable. All preventable. All expensive.
Why Native Cost Alerts Fall Short
Azure has built-in budget alerts. You set a threshold, get an email when it's exceeded. Simple.
Too simple.
Budget alerts are reactive. They tell you "you've already spent $10,000" — not "something changed and you're about to spend $10,000." By the time a budget alert fires, the money is gone.
Cost anomaly detection is different. It uses machine learning to establish a spending baseline, then alerts when actual spend deviates from the pattern. Not "you hit a number" but "this is unusual for your environment."
The problem? Azure's cost anomaly alerting doesn't work at the Management Group level. It's subscription-scoped only. Which means for 62 subscriptions, you need 62 individual alert configurations.
Nobody is clicking through the Azure Portal 62 times.
The Automation Runbook Approach
I automated it. An Azure Automation Runbook that creates cost anomaly alerts across all 62 subscriptions using the Cost Management REST API.
The runbook:
- Authenticates via System-Assigned Managed Identity — no stored credentials, no service principals, no secrets in variables (sound familiar?)
- Iterates through each subscription
- Checks if a cost anomaly alert already exists (idempotent — safe to rerun)
- Creates the alert if missing, updates if configuration drifted
- Sets expiration to 2030 — because "temporary" alerts that expire and nobody renews are worse than no alerts
The alert configuration:
- Type: InsightAlert (anomaly-based, not threshold-based)
- Scope: Individual subscription
- Notification: Email to the central IT engineering team
- Auto-expiration: December 2030 (set it and forget it — but actually monitor it)
One runbook execution. 62 subscriptions configured. Consistent. Repeatable. Auditable.
Why Not Azure Policy?
I tried Azure Policy first. Seemed logical — policy-based enforcement of cost anomaly alerts across all subscriptions.
Doesn't work.
Cost anomaly alerts aren't ARM resources in the traditional sense. You can't deploy them via Policy's deployIfNotExists effect. You can write an audit policy to check for their presence (I did — it's useful for compliance dashboards), but you can't auto-remediate.
So: Policy for auditing, Runbook for creation. Two tools, complementary purposes.
The Daily Scheduling Pattern
The runbook runs daily. Not because alerts need daily creation — they're persistent. But because:
- New subscriptions get alerts automatically
- Configuration drift gets corrected (someone deleted an alert? It's back tomorrow)
- Expiration dates get extended if needed
- The audit trail shows continuous governance, not one-time setup
Daily scheduling with Managed Identity means zero maintenance. No token rotations. No expired credentials at 3 AM. No "the automation broke and nobody noticed for a month."
The FinOps Conversation
Here's what most engineers miss about cost management: it's not about reducing spend. It's about knowing what you're spending and why.
If your team spins up 10 extra VMs for a load test — that's expected spend. The cost goes up, but it's intentional. No alarm needed.
If a rogue process starts writing 500GB of logs to premium storage — that's anomalous spend. Unexpected. Unplanned. Exactly the kind of thing that costs $3,000 before anyone opens the Azure Portal.
Cost anomaly detection catches the second scenario without crying wolf on the first. That's the difference between useful alerting and noise.
Multi-Team Notification Routing
62 subscriptions don't belong to one team. Different subscriptions, different owners, different budgets.
The runbook supports per-subscription email routing. Production subscriptions notify the infrastructure team. Development subscriptions notify the project leads. Shared services notify the platform team.
Same anomaly detection logic. Different notification targets. Because the person who can fix a cost anomaly in the data team's subscription is not the same person who manages the networking team's budget.
What Changed
Before: Monthly invoice review. "Why is this number higher?" Finger-pointing. Post-mortem. Promise to "keep an eye on it." Nobody does.
After: Anomaly detected in real-time. Right team notified. Investigation happens while the anomaly is still small. Cost impact minimized.
The first anomaly we caught was a test environment that someone forgot to tear down after a sprint demo. It had been running for 11 days. The cost anomaly alert fired on day 2. We caught it early. Previous approach? We'd have found it on the invoice 30 days later.
The Numbers
- 62 subscriptions covered
- 1 runbook manages all of them
- 0 stored credentials (Managed Identity)
- Daily automatic compliance checks
- 2030 expiration date (because "temporary" is never temporary)
FinOps isn't a team. It's a practice. And like any practice, it starts with automation.
Because nobody is going to manually check 62 subscription invoices every day. But a runbook will. Every single day. Without complaining.
62 subscriptions. One runbook. Zero blind spots.
That's FinOps at scale.