A paused rollout does not disappear; it queues, and when the pause expires the queue is the deployment. That is the part of Microsoft's post incident review for tracking ID 7Q30-010 that transfers to any estate with a change freeze, a patch deferral or a hold. Between 20:30 UTC on September 30 and 02:15 UTC on October 1, 2026, Azure networking customers saw intermittent failures in management operations across dozens of regions, and some ExpressRoute gateways lost connectivity to on-premises networks. This note keeps to what the review states. Anything about your own estate is a question to ask, not a failure mode Azure was shown to have elsewhere. ## What the Review Establishes About the Paused Rollout The Azure status history lists ExpressRoute Gateway, Application Gateway, Azure Firewall, Azure Bastion, Azure Virtual WAN and VPN Gateway among the affected services, and says Azure VMware Solution customers may also have lost connectivity to their software-defined datacenters. The review introduces its list of 33 regions as including those names, so treat the list as a floor. Impact times differed by region because the trigger was a rolling update moving through them. For cloud architecture strategy, the useful part is not the outage but the release mechanics behind it. The mechanism, as the review gives it: Azure applies operating system and security updates to networking gateways gradually, across deployment stages and redundant instances, so health can be evaluated before updates proceed. Before the incident, a temporary deployment restriction had paused those updates for a group of gateways. When the restriction expired, a large number of updates resumed within a short period. That produced a surge in gateway restarts and configuration-retrieval requests. At the same time, an earlier deployed change had lengthened the processing time of configuration operations on a shared regional networking control plane service. Microsoft says the combined volume and processing time overwhelmed that service, and that existing request throttling and load protection did not sufficiently protect critical processing. The response ran long. Monitoring flagged ExpressRoute degradation at 20:43 UTC and broader control plane degradation at 20:58. Further gateway updates were paused at 21:45. Traffic shifting gave temporary relief that did not hold, and attempts to add control plane resources ran into constraints with dependent platform services. The earlier change was tied to the processing-time increase at 23:55, rollback began at 01:15, and impact ended at 02:15. _Two contributing conditions; their combined load overwhelmed the shared service._ ## The Expiry of a Paused Rollout Is the Deploy A gradual rollout is a rate control. A paused rollout can reduce the release rate to zero without cancelling the work waiting behind it. Expiry then releases that backlog according to whatever resume behavior is configured, which is only safe if the resume path carries the same staging and redundancy discipline as the original rollout. Microsoft's own repair item says as much: paused updates will resume under staged rollout practices instead of across large numbers of gateways at once, with an estimated completion of October 2026. The load on a control plane is roughly requests arriving times the time each one takes. The review identifies two contributing conditions: more requests arriving as paused updates resumed, and longer processing times for configuration operations. It describes their combined effect overwhelming the shared service. It does not establish that either condition would have been harmless in isolation, and the product framing is a reading of the review, not a sentence in it. Neither a rollout gate nor a performance review owns the combined load. Two neighbors on this site are worth separating from it. Blast radius argues that a passing guardrail does not bound how far a failure reaches; this note is about a different control, and the question is narrower: at what rate does a lifted hold release? Retry storms cover clients amplifying a recovery; the review describes this surge as resumed updates and does not attribute it to client retries. The opposite failure also exists, an exception that outlives the freeze it was written for, and vSphere lifecycle governance covers that one. _Marked healthy, state not restored: the affected pair waited roughly 30 minutes before falling back._ ## The Restart That Needed the Control Plane ExpressRoute had a second problem, and it is a separate test. The review scopes the connectivity loss to a subset of upgraded ExpressRoute gateways. In the failure sequence Microsoft describes, instances restarting during the update were marked healthy before their configuration and routing state had been fully restored. The subsequent update reached the redundant instance before the first had completed restoration, leaving the affected gateways unable to retrieve the required routing configuration. After about 30 minutes of unsuccessful retrieval, those gateways applied locally stored last known good configurations and restored connectivity. Microsoft lists stronger ExpressRoute health checks, so redundant instances update only after the first has restored its state, with an estimated completion of March 2027. The restart is where the paused rollout turned from a management-plane problem into lost connectivity. Two things are visible here that steady state hides. A data-plane device whose restart depends on the control plane has a dependency that no traffic graph shows until the control plane is slow, and in the sequence the review describes, redundancy gave the affected gateways nothing once both members were inside the same fetch window. Healthy did not mean the instance's state was restored; Kubernetes health checks draws that distinction for readiness probes, and this note does not extend it. Which of your devices fetch configuration at start is a question for dependency architecture, because the answer is a dependency inventory item before it is an incident detail. ## Two Rows to Run on Your Own Estate These are questions for your environment. The review does not say your holds or devices behave this way. The first row tests the hold; the second tests the restart that follows it. Run them as separate tests, because passing one says nothing about the other. Test | Question to answer | Evidence that closes it ---|---|--- Hold expiry | For each freeze, deferral or paused ring you run: how many changes are queued behind it, and what is the release profile at expiry (everything at once, or waves with a cap and an abort test against the shared dependency)? | Queue size at expiry, wave size, rate against the shared management dependency, and the named person who can stop the release Restart dependency | For each gateway, firewall or edge device that fetches configuration at start: what happens if the source is slow or unreachable, is there a last known good, and how long before it applies? Does the second of a pair wait for restored state or only for liveness? | A restart test with the configuration source blocked or throttled, the measured wait before fallback, and the measured connectivity during it ## What the Review Does Not Establish The review does not say why the deployment restriction existed or how long the paused rollout ran, and it does not name the earlier change that lengthened processing time. It does not say how many gateways resumed at once. Microsoft states there was "nothing that customers could have done to avoid or minimize impact" from this incident, so nothing here is advice Azure customers could have used on September 30. The remaining repair dates, October 2026, March 2027 and June 2027, are estimates; only the rollback is marked completed. The Azure Incident Retrospective livestream on October 22 and 23 is not incorporated, and anything said there that changes these facts will arrive as an update. ## Architect's Verdict A paused rollout is a queue with a release date. The pause protects the system while it lasts, and the expiry decides whether that protection was a deferral or a loan. Azure's own repair item states the point: resume paths need the staging the original rollout had. The real problem is that nobody owns the interval between a hold ending and the first change landing. Every team can name who paused the rollout; few can name who decides how it resumes. Freezes have owners, rollouts have owners, and the transition between them runs on whatever default the scheduler has. Pair that with a device whose restart needs the control plane, and a hold expiry can reach the data plane without anyone having decided to deploy. Write down the release profile before the hold lifts, not after the first wave fails. ## Additional Resources * Cloud Architecture Strategy — the pillar covering the reliability, governance and dependency decisions this note sits inside. * Dependency Architecture — CS1 stage: classifying what devices and services depend on before an incident forces the list. * The Third Incident Is the One That Should Worry You — the neighboring argument on guardrails versus bounded impact; this note asks a narrower release-rate question. * The Retry Storm Is a Self-Inflicted DDoS — client-side amplification of a recovery, distinct from platform work resuming on a schedule. * What Kubernetes Health Checks Actually Guarantee — why a passing health signal is a narrower fact than restored state. * vSphere Lifecycle Management Is a Governance Problem, Not a Patching Problem — the opposite failure: a freeze exception that outlives its freeze. * Azure status history, PIR 7Q30-010 — Microsoft's post incident review, the primary source for every incident fact in this note. * Azure Incident Retrospective, tracking ID 7Q30-010 — Microsoft's livestream registration page for this incident; the session content is not incorporated in this note. _Originally published at rack2cloud.com_

You might have read my previous blogs about subnet peering, like this introduction to subnet peering when it was launched and the effect of subnet peering on ExpressRoute routing a bit later. Subne…

Rethinking connectivity for distributed AI infrastructure AI is changing more than the applications organizations build. It is reshaping where the...

Microsoft Azure suffered a widespread network outage across 18 global regions.

Direct link offering provides private georedundant access across some 24 cities

Curate Feeds | Make Collections | Customize Email Briefs

Curate Feeds | Make Collections | Customize Email Briefs

Your datacenter needs to talk to Azure. You can send that traffic through an encrypted tunnel over the public internet, or over a private circuit that never touches it. That single choice — shared road or private rail — decides cost, speed, and reliability. Almost every organization moving to Azure keeps _something_ on-premises, and those two worlds have to connect privately. Azure gives you two hybrid-connectivity options, and they take opposite routes to the same destination: **VPN Gateway** and **ExpressRoute**. Understanding them is really understanding one question — does your traffic ride the public internet, protected by encryption, or a dedicated line that bypasses it entirely? ## VPN Gateway: an encrypted tunnel over the internet Microsoft's description is exact: Azure VPN Gateway _"can be used to send encrypted traffic between an Azure virtual network and on-premises locations over the public Internet."_ Your traffic still travels the ordinary internet, but inside an IPsec/IKE tunnel, so it is private even though the road is shared. It comes in a few shapes: **site-to-site** (your datacenter's VPN device to Azure), **point-to-site** (an individual remote worker to the VNet), and **VNet-to-VNet**. It is quick to stand up, needs no third party, and is inexpensive — the pragmatic default for dev/test and small-to-medium production links. ## ExpressRoute: a private, dedicated circuit ExpressRoute takes the other road entirely. It _"lets you extend your on-premises networks into the Microsoft cloud over a private connection with the help of a connectivity provider."_ The defining fact: because ExpressRoute connections _do not go over the public internet_ , they offer _"more reliability, faster speeds, consistent latencies, and higher security than typical connections over the internet."_ You are not tunnelling through shared roads; you have a private rail line into Microsoft's network, arranged through a connectivity provider. That extra reliability and consistency costs more and takes longer to provision — but for a bank, a hospital, or a latency-sensitive enterprise workload, it is the answer. | VPN Gateway | ExpressRoute ---|---|--- Path | Public internet (encrypted tunnel) | Private dedicated circuit Reliability & latency | Best-effort internet | More reliable, consistent latency Setup | Fast, self-service | Via a connectivity provider Cost | Lower | Higher Best for | Dev/test, small-medium prod, remote users | Enterprise, regulated, latency-sensitive > A VPN tunnels privately through the public road. ExpressRoute gives you a private road of your own. Same destination, very different guarantees. **They are not mutually exclusive.** The grown-up pattern often uses _both_ : ExpressRoute as the primary private connection, with a **site-to-site VPN as a secure failover path** if the circuit goes down. Microsoft supports this coexistence directly. So the question is not always "which one" — sometimes it is "ExpressRoute with a VPN safety net." ## How to choose * **Getting started, dev/test, or a small-to-medium workload where good-enough internet connectivity is fine?** VPN Gateway. Fast, cheap, no provider to involve. * **Remote individual users needing into the VNet?** A point-to-site VPN. * **Enterprise scale, strict reliability or latency needs, regulated industry, or heavy sustained data transfer?** ExpressRoute — the private, consistent circuit is worth the cost and the lead time. * **Mission-critical link that cannot go dark?** ExpressRoute primary, VPN failover. The mistake is treating them as "cheap one vs expensive one." They are "shared road with a lock" versus "your own private line," and the right pick follows from how much reliability, consistency, and privacy the workload genuinely demands. Say "a site-to-site VPN for now, and ExpressRoute when we need consistent latency and an SLA on the connection — with the VPN kept as failover" and you are talking hybrid networking like an architect. ## Questions people also ask ### Is ExpressRoute more secure than VPN Gateway? ExpressRoute keeps traffic off the public internet, which removes a whole class of exposure a VPN carries by design. But an ExpressRoute circuit is not encrypted by default. For data that must be encrypted in transit, add MACsec on the circuit or run an IPsec VPN over ExpressRoute. VPN Gateway is encrypted from the start, just over a shared path. ### Can you use ExpressRoute and VPN Gateway together? Yes, and Azure supports the coexistence directly. The common pattern runs ExpressRoute as the primary private circuit and a site-to-site VPN as the failover path, so if the circuit drops, traffic falls back to an encrypted tunnel over the internet instead of going dark. ### How much does ExpressRoute cost compared to a VPN Gateway? ExpressRoute costs more. Its tiers run from a metered Local option up to Standard around $300 a month for 1 Gbps, plus a Premium add-on, before you add the connectivity provider's own charges. A VPN Gateway has a much lower flat cost since it rides the internet you already pay for and needs no third-party provider. ### What is the difference between ExpressRoute and a site-to-site VPN? A site-to-site VPN connects your on-premises network to Azure through an encrypted tunnel over the public internet. ExpressRoute connects the same two networks through a private circuit arranged with a connectivity provider, bypassing the internet entirely for more consistent latency and reliability. ### Which is faster, ExpressRoute or VPN Gateway? ExpressRoute is faster and more consistent. Because it does not share the public internet's congestion and routing variability, it delivers predictable latency and higher throughput. A VPN Gateway's speed depends on internet conditions between your site and Azure, which vary and can degrade under load. ## Further reading — the Microsoft docs * Azure ExpressRoute — overview * About Azure VPN Gateway * Hybrid connectivity options This is the comparison. Class 15 — Hybrid & DNS takes you hands-on: reason about connecting on-premises to Azure, wire consistent name resolution across the link, and see where a private circuit beats a tunnel. > This article was originally published on CAMPUX, a free Azure cloud-engineering bootcamp.

Subscribe to Microsoft Azure today for service updates, all in one place. Check out the new Cloud Platform roadmap to see our latest product plans.

Subscribe to Microsoft Azure today for service updates, all in one place. Check out the new Cloud Platform roadmap to see our latest product plans.

Azure ExpressRoute resiliency guard is now available in public preview for ExpressRoute virtual network gateways. The new resiliency model property lets you specify whether a gateway is intended for a single-homed or multi-homed configuration, helping align your deployment with the resiliency requirements of your workloads.

Microsoft erweitert Azure Networking um zonenredundantes NAT Gateway V2 und kündigt 400-Gigabit-ExpressRoute für 2026 an. Neue Sicherheitsfeatures inklusive.