The server model that never saw a traffic spike

Question

An engineer fit a straight line predicting a website's page-load time from the number of simultaneous users, using observations from $100$ to $1000$ users. In that range load time rose gently and linearly. She now uses the line to predict load time at $5000$ users.

Explain why this prediction is unreliable, and what could make the true behavior differ sharply from the line.

Accepted Answer

At 5000 users the line just adds up a gentle per-user increase, but a real server does not behave gently once it is overwhelmed, so the prediction is likely far too optimistic. Systems have breaking points. Between 100 and 1000 users, the server had spare capacity, so each extra user slowed things only slightly, and a line described that well. But as load approaches the machine's limits (memory, connections, database locks), response time does not rise linearly, it can spike suddenly as queues back up. Near saturation the true curve shoots upward far faster than any straight line, and the sit…

The server model that never saw a traffic spike

Your answer

Discussion

Learn the concepts

Related questions