Interactive parts stop responding in production

Problem

An interactive page works on your machine, but once it's deployed, parts of it stop answering. Controls still move, since IzzyBlazor's script runs in the browser: a slider's thumb and value follow the drag, a menu opens, a dialog closes. But nothing that needs C# runs: a value bound with Live doesn't reach what follows it, and OnClick does nothing. It can work on one load and not the next, or stop after a while. Blazor's reconnection message may appear, unless the app hides it. The browser's console shows an error like this one:

No Connection with that ID: Status code '404'

Because the controls still move, it looks like a bug in one component. It isn't: every interactive part of the page has stopped.

Cause

An interactive Server page runs over a circuit: the page's components and their state, kept in memory on the server that rendered it, and a SignalR connection to the browser. Every later request for that connection has to reach the same server. Behind a load balancer that spreads requests across servers, one of them reaches a server that has never heard of the connection. That server answers 404, and the circuit is lost.

Over WebSockets the browser holds one long connection, so this shows up less, but the request that sets the connection up and any reconnection can still land elsewhere. When WebSockets aren't available, SignalR falls back to long polling: a chain of separate requests. The page breaks the first time one reaches another server. Which server each request reaches is chance, so it fails sometimes and not others. The console says when the fallback happens: "Failed to connect via WebSockets, using the Long Polling fallback transport."

Static pages don't have this problem: nothing is kept on the server between requests, so any server can answer any request. The servers still have to share their data protection keys, which protect forms, sign-in cookies and the state of prerendered interactive components.

Fix

Keep each reader on one server, or run only one:

• Session affinity. Turn on the load balancer's sticky sessions, so it sends each browser back to the server it reached first: ARR affinity (Session affinity) on Azure App Service, stickiness on an AWS target group, cookie affinity on a Kubernetes ingress or Gateway controller. Fly.io has no affinity setting; it can route a request to one machine with the fly-replay header, but one machine is simpler.
• Azure SignalR Service. It holds the browsers' connections and routes each one to the server with its circuit. Set its ServerStickyMode to Required for Blazor.
• One server. Fine for a small app, and the simplest fix. Some hosts start two by default: Fly.io's fly deploy creates two machines for a new app unless it's given --ha=false.

# See how many machines the app runs
fly machine list

# Go back to one
fly scale count 1

# Deploy without adding a second
fly deploy --ha=false

Check that WebSockets get through too: a proxy in front of the app has to pass the Upgrade and Connection headers, or every page falls back to long polling, which is slower and makes this problem show up sooner.

A page that doesn't need C# on every click can be static, and then any server can answer it. IzzyBlazor's components post, link and run their script on static pages too.

Related

• Render modes: what needs a circuit, and choosing a page's mode.
• OnClick does nothing on a static page: C# events need an interactive page.
• Host and deploy Blazor Server: Microsoft's guide to scaling out, WebSockets and Azure SignalR Service.