Scaling Up vs Scaling Out

There are two ways to handle more load. Make the machine bigger, or add more
machines. Industry writing tends to treat the second as the sophisticated answer
and the first as a stopgap, which is misleading for most systems.

Scaling up

Move to an instance with more CPU and memory. On every major provider this is
a resize operation taking minutes, usually with a reboot.

The advantages are substantial and underrated. Nothing about your application
changes. There is no new failure mode, no session handling problem, no shared
state to arrange, no load balancer to configure. If the bottleneck is memory,
which it often is, this resolves it directly.

The limits are real but further away than people assume. Cloud instances
reach very large sizes, and a single machine today handles traffic that would
have required a cluster not long ago. The genuine ceiling is usually reached
long after most applications stop growing.

The honest downside is that one machine is one failure domain. Scaling up
does nothing for availability, which is a separate problem from capacity and
worth keeping separate in your thinking.

Scaling out

Run several machines behind a load balancer. This adds capacity and, done
properly, removes the single point of failure.

It also requires your application to tolerate it, which is where the work
is:

• Sessions cannot live in the memory of one server. Move them
  to a shared store, or make requests stateless.
• Uploaded files cannot live on one server’s disk. Move them
  to object storage.
• Scheduled jobs must not run on every instance
  simultaneously.
• Caches become per-instance unless shared, which can produce
  inconsistent behaviour between requests.
• Deployments have to handle multiple machines and a period
  where versions differ.

None of this is exotic, and an application designed with it in mind handles
it without difficulty. Retrofitting it to an application that assumed one server
is where the effort lands.

The database is the real constraint

Application servers scale out easily. Databases do not, because they hold
state that has to stay consistent.

Read replicas help read-heavy workloads and introduce replication lag, which
your application must tolerate. Writes remain on one primary, and scaling writes
means sharding, which is a significant undertaking.

In practice this means the database is usually scaled up long after the
application tier has been scaled out, and the practical ceiling of the whole
system is often the largest database instance available.

A sensible order

1. Find the actual bottleneck. Not the one you assume. Check
   whether you are constrained by CPU, memory, disk throughput or the database,
   because adding capacity to the wrong one changes nothing.
2. Look for the cheap fix first. A missing database index, an
   uncached query, or an unoptimised asset frequently delivers more than a resize
   and costs nothing per month. This step is skipped surprisingly often.
3. Scale up. It is fast, reversible and requires no
   architectural change.
4. Scale out when you need availability, or when you have
   genuinely reached a sensible ceiling on one machine.

Autoscaling is a different question

Autoscaling adds and removes instances automatically in response to load. It
is valuable when load genuinely varies by a lot, and pointless when it does not.

It also requires everything scaling out requires, plus instances that start
quickly enough to be useful, plus sensible rules to avoid adding capacity in
response to a problem that more capacity will not fix. For a workload with a
predictable daily shape, a scheduled resize is simpler and achieves most of the
benefit.

What this means for provider choice

If your path is scaling up and occasionally out, the flat-rate providers
handle it perfectly well: resizing is a control panel operation and load
balancers are available. If your path genuinely requires autoscaling against
unpredictable demand, that is one of the clearer arguments for a hyperscaler, as
discussed in
when a cheap VPS beats a
hyperscaler.