VDI Storage Has Two Scale Problems: Login Storms and User Density
How we characterized real FSLogix® workloads, built LoginSim and CapacitySim, and evolved Roadrunner SMB™ 1.3 around what happens when thousands of users hit storage at once.
When I first wrote about why I built Roadrunner SMB, the basic idea was straightforward: Amazon EFS® is an excellent cloud-native filesystem, but Windows® environments still need native SMB3, Active Directory® integration, Windows ACL semantics, and the performance characteristics Windows workloads expect.
Since then, Roadrunner SMB has come a long way.
And one workload in particular has increasingly shaped the product:
VDI.
More specifically, AWS®-hosted virtual desktops using FSLogix profiles plus the departmental and corporate Windows file shares those same users depend on every day.
That turns out to be a very different storage problem from simply asking, "How many megabytes per second can this file server deliver?"
VDI storage has at least two independent scaling problems:
- How quickly can thousands of users log in without overwhelming the storage system?
- Once they are logged in, how many concurrent users can each storage node support predictably?
Roadrunner SMB 1.3 is the release where we went deeply after both questions.
The Monday-morning problem
A 5,000-user VDI environment does not behave like 5,000 people steadily accessing a departmental file share.
At 8:00 Monday morning, hundreds or thousands of desktops may come alive in a very short period.
FSLogix logins generate a concentrated pattern of SMB activity: file opens, directory operations, metadata requests, ACL evaluation, profile-container operations and small-file I/O.
That creates a workload whose bottleneck often isn't raw storage bandwidth.
It is concurrency.
A storage system can have plenty of aggregate throughput available and still struggle when a large number of clients simultaneously drive latency-sensitive metadata and file-open operations.
We wanted to understand that workload quantitatively rather than guess at it.
Start with real Windows clients
Our benchmarking did not begin with a synthetic workload generator.
We started with real Windows clients running FSLogix and real login workloads.
We captured the SMB conversation those logins actually drive: which protocol operations hit storage, in what mix, at what timing, and at what concurrency as a user authenticated, attached a profile, reached the desktop, and then kept working.
That is a different artifact from an IOPS target. It is a measured FSLogix SMB workload model - CREATE and file-open patterns, directory and metadata traffic, ACL evaluation, profile-container I/O, and the small-file mix that follows.
From that model we built two internal tools. LoginSim exists to replay that same SMB pattern faithfully, not to invent a prettier benchmark.
LoginSim: reproduce the login storm
LoginSim reproduces real FSLogix login-time SMB workload patterns at scale.
Instead of needing thousands of Windows desktops for every test, it speaks SMB and drives the same storage-facing operation mix we recorded from those clients.
A single session is not a one-shot connect. The profiled Windows login itself is about 5 seconds (connection established to desktop ready). Then the session keeps generating about 30 seconds of post-login SMB activity - the settle tail we measured after the desktop came up. LoginSim replays both halves: the login storm and the seated-user I/O that starts immediately afterward.
That overlap is the onslaught.
Our standard 1.3 storm shape launched 100 new login users every 5 seconds, for 15 minutes, without a pause. That is 20 arrivals per second cluster-wide. Because each session is still in login or post-login I/O when the next flight lands, the product is not serving 100 users at a time. It is serving new logins stacked on hundreds of already-active sessions - metadata and file-opens arriving on top of settle-tail reads and writes.
We can vary the same model:
- number of simulated users;
- login arrival rate;
- concurrency;
- login-window duration;
- cluster size;
- EC2 node size;
- success and failure behavior.
Most importantly, we can keep increasing load until something breaks.
That is exactly what we did during the development of 1.3.
And when testing exposes a bottleneck, we don't simply lower the benchmark.
We go back into the product and remove the bottleneck.
CapacitySim: what happens after everyone sits down?
Login arrival is only half the problem.
A system might survive a furious 15-minute login storm and still have poor economics if every storage node can support only a modest number of ongoing users.
So we extended the same workload-modeling work into CapacitySim.
CapacitySim extends that model to characterize per-node concurrent-session capacity and CPU/RAM planning limits.
The distinction matters:
LoginSim asks: How quickly can users arrive?
CapacitySim asks: How many users can remain active on the infrastructure once they're there?
Together, they give us a much more useful VDI sizing model than a traditional storage benchmark based primarily on sequential throughput or synthetic IOPS.
They are also no longer just R&D tools. LoginSim and CapacitySim are now part of our QA gates. We run them to validate a build before we release a software update, so a change that survives ordinary file-serving tests still has to survive the same arrival-storm and seated-user shapes that broke 1.2. A passing gate is release evidence. It is not a published performance SLO.
What the characterization showed
Roadrunner SMB 1.3 currently uses these FSLogix workload planning rates:
c6i.2xlarge - 5 logins/second/node
c6i.4xlarge - 10 logins/second/node
Because Roadrunner scales horizontally, cluster login capacity scales with node count.
A four-node cluster therefore provides a planning arrival rate of 20 or 40 logins per second, depending on instance size.
For concurrent-user capacity, our current 4-node Dual-AZ planning reference is:
4 x c6i.2xlarge - up to 20,000 users
4 x c6i.4xlarge - up to 40,000 users
Production clusters are supported from 2 through 8 nodes, allowing both horizontal and vertical scaling as the workload grows.
These are capacity-planning figures derived from our characterized FSLogix workload model, not published performance SLOs.
And an important qualification: Amazon WorkSpaces® itself was not used as the test harness.
We characterized real Windows clients running FSLogix, reproduced those SMB workloads through LoginSim, and extended the model with CapacitySim. The resulting numbers are intended to help size WorkSpaces and other FSLogix-based environments.
What broke when we pushed 1.2 hard enough
Roadrunner SMB 1.2 was already stable under normal file-serving workloads.
LoginSim changed the definition of "normal."
Once we began hammering clusters with concentrated login flights, we exposed conditions that simply don't appear during ordinary steady-state SMB traffic.
That work directly drove several of the most important architectural changes in Roadrunner SMB 1.3.
Eight NFS channels per share
FSLogix generates a lot of small, concurrent storage operations.
We found that the path from Samba into EFS could itself become a concurrency constraint long before the underlying EFS service had exhausted its capabilities.
Roadrunner SMB 1.3 therefore uses an 8-channel NFS mux for every customer share.
Instead of driving EFS through a single backend I/O path, Roadrunner can use eight independent efs-proxy/NFS sessions.
The design target was small-file and file-open concurrency during login storms.
A useful byproduct is significantly greater parallelism for high-throughput large-file writes as well.
Admission instead of collapse
Every system eventually has a finite capacity.
The interesting engineering question is what happens when demand exceeds it.
Roadrunner SMB 1.3 adds accept-path admission.
When a node reaches capacity, a new SMB client can receive STATUS_SERVER_UNAVAILABLE after a short hold rather than encountering an uncontrolled TCP reset.
A TCP reset leaves the client recovering from a broken connection. STATUS_SERVER_UNAVAILABLE, by contrast, is an SMB-level response that Windows can retry cleanly.
Roadrunner refuses excess new work before allowing it to destabilize an already-saturated serving node. Existing sessions continue to be served while the client retries through the normal cluster endpoint.
The objective isn't to pretend capacity is infinite. It is to keep behavior predictable near saturation and give Windows clients a clean retry path rather than allowing overload to manifest as connection resets.
Moving AWS calls out of the login hot path
Roadrunner's Windows ACL architecture already takes an unusual approach.
We largely separate ACL metadata from the file-data path.
Samba's ACL requests are intercepted by our vfs_acl_DDB layer.
A hot ACL lookup is normally served from Roadrunner's L1 RAM metadata cache, where lookup latency is measured in microseconds.
When authoritative state is required, the backing ACL store is Amazon DynamoDB®, which scales independently of EFS file I/O.
That architecture means Windows authorization metadata does not have to live in the same storage path as bulk file data.
During extreme login testing, however, we found some ACL-related operations that could still generate AWS calls in a critical login path.
At ordinary concurrency, a few milliseconds is insignificant.
Multiply a synchronous dependency by a large login storm and it becomes a very different problem.
Roadrunner SMB 1.3 removes those AWS calls from that critical path.
Metadata and file data are different workloads
This gets to a broader architectural principle behind Roadrunner.
ACL metadata and file contents have very different performance characteristics.
ACL information is small, repeatedly accessed, highly cacheable and extremely latency-sensitive.
File data can be huge and is often throughput-sensitive.
So Roadrunner treats them differently.
Windows metadata path
Samba -> Roadrunner L1 RAM metadata cache -> DynamoDB® authoritative ACL state
File-data path
Samba -> 8-channel NFS mux -> Amazon EFS
And sitting in front of the serving tier is admission logic designed to keep concentrated connection demand predictable as capacity is approached.
This is one reason Roadrunner is increasingly becoming specialized around VDI rather than simply being "another SMB gateway."
User density matters just as much as the storm
One of the lessons from CapacitySim is that VDI storage can't be sized solely around the morning peak.
After the storm ends, those users remain.
That means a useful VDI architecture needs both:
high login-arrival capacity and high ongoing user density per serving node.
Those two dimensions have different resource profiles.
LoginSim lets us characterize the former.
CapacitySim lets us walk the latter while watching CPU, RAM and other resource constraints.
That gives us a practical basis for deciding whether to scale Roadrunner up to a larger EC2 instance, out to more nodes, or both.
Why this matters operationally
Large VDI estates often eventually become storage-geometry problems.
As user populations grow, conventional file-server architectures may require administrators to divide users among additional storage clusters, maintain capacity and HA headroom within each one, and continuously decide where populations should live.
Roadrunner is designed around a different model.
The SMB serving tier scales horizontally over a common EFS storage foundation.
That means the operator can increasingly think about:
How much SMB serving capacity do I need?
rather than:
Which storage silo does this next group of users belong to?
For an MSP managing many customer estates, that distinction becomes particularly important.
The economics follow the workload
VDI is also unusually bursty economically.
The infrastructure needed at 8:05 AM may be very different from what is required late at night.
And the file estate itself often contains a very large amount of data that hasn't been accessed recently.
Roadrunner combines:
- horizontally scalable EC2 serving nodes;
- Amazon EFS Elastic throughput;
- Amazon EFS lifecycle management and Infrequent Access;
- usage-based Roadrunner SMB metering.
The idea is simple: build the storage system around actual demand rather than permanently provisioning every layer for the peak.
That becomes increasingly important as an environment moves from hundreds of desktops to thousands.
1.3 isn't the end of the benchmarking
In one sense, Roadrunner SMB 1.3 is the result of our benchmark work.
In another, it's the beginning.
We now have something we didn't have before: a repeatable FSLogix-derived workload model, tooling that can drive it against storage infrastructure at controlled scale, and a release process that requires LoginSim and CapacitySim to pass before an update ships.
That gives us a much better foundation for continued optimization - and a much harder time shipping a regression we cannot see in a smoke test.
VDI is now a primary design center for Roadrunner SMB - not simply another workload we happen to support.
It also gives us the ability to start asking another important question:
How do other AWS file-storage architectures behave under exactly the same workload?
That's a benchmark for another day.
For now, Roadrunner SMB 1.3 is GA, and we have a substantially better understanding of both sides of VDI storage scale: getting thousands of users logged in and keeping them there efficiently once they arrive.
See the benchmark and capacity-sizing details
The Product & Architecture page includes the current login-rate and concurrent-user planning charts, the 4-node Dual-AZ reference architecture, and additional details on Roadrunner SMB 1.3.
Roadrunner SMB is available through AWS Marketplace for evaluation and production deployment.