How Tomcat Works: Architecture and Request Lifecycle

Tomcat in one sentence

Tomcat is a servlet container: it accepts TCP connections, parses HTTP off them, turns each request into an HttpServletRequest, finds the one servlet responsible for that URL, calls it, and writes the resulting HttpServletResponse back onto the socket. Everything else — the XML, the directory layout, the classloaders, the valves — exists to make that path configurable, isolated between applications, and manageable at runtime.

Knowing the path in detail is what turns Tomcat tuning from cargo-culting into engineering. Almost every production question — why does maxThreads not fix this, why does the app see a different class than I deployed, why do sessions vanish on redeploy — is answered somewhere along that path.

The component hierarchy

Tomcat's runtime is a tree of nested components, and server.xml is a literal description of it. Each level has one job.

Server                      // the JVM instance; owns the shutdown port
 └── Service                // binds connectors to one engine
      ├── Connector :8080   // HTTP/1.1 — protocol handling + thread pool
      ├── Connector :8009   // AJP — for mod_jk / mod_proxy_ajp
      └── Engine            // the request-processing entry point
           └── Host         // a virtual host, e.g. localhost
                └── Context // one web application (one WAR)
                     └── Wrapper  // one servlet

<Server port="8005" shutdown="SHUTDOWN">
  <Service name="Catalina">

    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               maxThreads="200" minSpareThreads="25"
               acceptCount="100" maxConnections="8192"
               redirectPort="8443" />

    <Engine name="Catalina" defaultHost="localhost">
      <Host name="localhost" appBase="webapps"
            unpackWARs="true" autoDeploy="true">
        <!-- Contexts are usually discovered from appBase -->
      </Host>
    </Engine>

  </Service>
</Server>

A few properties of this tree matter operationally:

• Connectors belong to a Service, not to an application. The 200 threads on port 8080 are shared by every web application deployed under that Engine. One slow application can starve every other one — which is the argument for separate Tomcat instances per critical application rather than one instance hosting ten.
• Everything implements Lifecycle. Every component moves through init → start → stop → destroy and fires events at each transition. That uniformity is what makes hot redeploy of a single Context possible while the rest of the server keeps serving.
• A Context is the unit of isolation. Its own classloader, its own session manager, its own ServletContext. Two applications on one Tomcat share a JVM and a thread pool but almost nothing else.

What happens at startup

catalina.sh start launches org.apache.catalina.startup.Bootstrap, and the sequence from there is worth knowing because most startup failures are a specific step in it.

1. Bootstrap builds the server classloaders — common, catalina and shared — from the paths in conf/catalina.properties, then loads Catalina reflectively through them. This is why Tomcat's own classes are invisible to your application.
2. Digester parses server.xml and instantiates the component tree described above, applying attributes as bean properties. A typo in an attribute name is silently ignored here — a genuine source of "my setting had no effect".
3. init() propagates down the tree. Connectors bind their sockets at this point, which is where a port conflict surfaces.
4. start() propagates down. The Host deploys applications from appBase: it expands WARs, creates a Context and a classloader for each, and merges the global conf/web.xml with the application's own descriptor and its annotations.
5. Each Context starts its filters and listeners, then loads servlets — eagerly if <load-on-startup> is set, lazily on first request otherwise.
6. Connectors begin accepting. Only now does the port actually serve traffic.

The connector: sockets to requests

The connector is where nearly all Tomcat performance lives. Since Tomcat 8.5 the default is the NIO protocol handler, and its internal division of labour is the thing to understand.

            ┌──────────────┐
 TCP  ──────▶│   Acceptor   │  1 thread. Calls accept(), sets the socket
             └──────┬───────┘  non-blocking, hands it to the poller.
                    ▼
             ┌──────────────┐
             │    Poller    │  1–2 threads on a Selector. Watches many
             └──────┬───────┘  sockets; wakes only when data is readable.
                    ▼
             ┌──────────────┐
             │ Worker pool  │  maxThreads threads. Parses HTTP, runs the
             └──────┬───────┘  filter chain and the servlet, writes out.
                    ▼
              Response ──▶ socket

The separation is what makes NIO scale. A connection that is open but idle — keep-alive between requests — is registered with the poller's selector and costs no thread at all. A worker thread is taken only while a request is actually being processed. That is the fundamental difference from the old BIO connector, where one thread was pinned per connection for its entire life.

Following one request through

1. The acceptor accepts the socket and registers it with a poller.
2. The poller's selector reports the socket readable and dispatches it to the worker pool.
3. A worker thread reads bytes and Coyote parses the request line and headers into an internal org.apache.coyote.Request — a lightweight, recyclable object.
4. CoyoteAdapter wraps it as the HttpServletRequest the Servlet API defines, and asks the Mapper which Host, Context and Wrapper should handle the URI.
5. The request enters the Engine pipeline and descends through Host, Context and Wrapper pipelines.
6. The Wrapper builds the filter chain and finally invokes service() on your servlet.
7. The response is written back through the same socket; if keep-alive applies, the socket returns to the poller and the worker thread is released.

Thread pools and their limits

Four connector attributes govern concurrency, and confusing them is the most common Tomcat misconfiguration there is.

They form a funnel, and the behaviour at each stage is different:

incoming connections
   │
   ├─▶ up to maxConnections   ── accepted and tracked by the poller
   │      │
   │      └─▶ up to maxThreads ── actively processing a request
   │             (the rest are idle keep-alive, or waiting for a worker)
   │
   ├─▶ beyond maxConnections  ── queued in the OS backlog, up to acceptCount
   │
   └─▶ beyond acceptCount     ── connection refused

The right way to size maxThreads is from measurement. Little's Law again: for 300 requests per second at 150 ms average service time you need about 45 concurrent workers, so something in the region of 100 gives you comfortable burst headroom. Then verify against the downstream pool — if the JDBC pool has 50 connections, 400 Tomcat threads simply create a queue inside the connection pool, invisible in Tomcat's own metrics.

acceptCount deserves a moment of thought too. A large queue converts a load spike into a latency spike: requests sit in the backlog, the client times out, and Tomcat then processes work nobody is waiting for any more. A small queue fails fast and lets a load balancer route elsewhere. Fast rejection is usually the better behaviour.

The pipeline and valves

Each container in the tree — Engine, Host, Context, Wrapper — owns a pipeline, an ordered chain of Valve objects ending in a mandatory basic valve that passes the request down to the next level. This is Tomcat's interception mechanism, and it sits below the Servlet API, so a valve sees requests that never reach any application.

Engine pipeline   → [ ErrorReportValve ] [ StandardEngineValve ]
  Host pipeline   → [ AccessLogValve ] [ RemoteIpValve ] [ StandardHostValve ]
    Context pipe. → [ AuthenticatorValve ] [ StandardContextValve ]
      Wrapper pipe→ [ StandardWrapperValve ] → filter chain → servlet

Valves you are likely to configure in production:

• RemoteIpValve — reads X-Forwarded-For and X-Forwarded-Proto so that behind Apache or a load balancer, request.getRemoteAddr() returns the real client and request.isSecure() reflects the original TLS. Without it, access logs record the proxy's IP and redirects can downgrade HTTPS to HTTP.
• AccessLogValve — the access log. Add %D to record request duration; a log without timings is far less useful during an incident.
• RemoteAddrValve — network-level allow/deny, typically to fence off the manager application.
• StuckThreadDetectionValve — logs a stack trace for any request exceeding a threshold. Effectively free, and it turns "the app hung last night" into an actual stack trace.

<Valve className="org.apache.catalina.valves.RemoteIpValve"
       internalProxies="10\.\d+\.\d+\.\d+"
       protocolHeader="X-Forwarded-Proto" />

<Valve className="org.apache.catalina.valves.AccessLogValve"
       directory="logs" prefix="access." suffix=".log"
       pattern="%h %l %u %t &quot;%r&quot; %s %b %D" />

<Valve className="org.apache.catalina.valves.StuckThreadDetectionValve"
       threshold="60" />

Finding the right servlet

The Mapper resolves a URI to a Wrapper in one pass, using rules fixed by the Servlet specification — not by declaration order in web.xml. The precedence is:

1. Exact match — /api/status
2. Longest path prefix — /api/* beats /*
3. Extension match — *.jsp
4. Default servlet — /, which serves static files

Two consequences worth internalising. First, mapping a servlet or framework dispatcher to / replaces the default servlet, so static resources stop being served unless the framework handles them. Second, /* and / are different mappings with different precedence — a mistake that produces "my filter runs but my servlet 404s" with impressive regularity.

The classloader hierarchy

Tomcat deliberately breaks the standard Java delegation model, and knowing where it breaks resolves a large class of ClassCastException and NoSuchMethodError mysteries.

Bootstrap (JVM)
   └── System  (CLASSPATH: bootstrap.jar, tomcat-juli.jar)
        └── Common  (${catalina.base}/lib — servlet-api.jar etc.)
             ├── WebappClassLoader  /app-one
             └── WebappClassLoader  /app-two

A normal Java classloader asks its parent first. Tomcat's WebappClassLoader instead looks in this order:

1. JVM bootstrap classes (always first — you cannot override java.lang.String)
2. /WEB-INF/classes of the application
3. /WEB-INF/lib/*.jar of the application
4. The Common classloader — Tomcat's lib/

So your application's own copy of a library wins over the container's. That is what makes two applications with incompatible versions of the same framework coexist in one JVM. The exception is the Servlet API itself: it must come from the container, which is why servlet-api.jar is marked provided in Maven builds. Bundling it in WEB-INF/lib gives you two incompatible copies of the same interface and a ClassCastException between types that look identical in the stack trace.

Sessions and clustering

By default each Context has a StandardManager holding sessions in the JVM heap, keyed by a JSESSIONID cookie. Two behaviours follow. Sessions do not survive a crash, and in a cluster a request that lands on a different node sees no session at all — hence sticky sessions in the load balancer, using jvmRoute to append a node identifier to the session ID.

<Engine name="Catalina" defaultHost="localhost" jvmRoute="node01">

// resulting cookie: JSESSIONID=A1B2C3D4E5F6.node01
// mod_jk / mod_proxy_balancer route on the suffix

Stickiness alone still loses sessions when a node dies. The options, in increasing order of robustness:

• PersistentManager — swaps idle sessions to disk or a database. Survives a graceful restart; cheap, and often enough.
• DeltaManager — Tomcat's built-in clustering, replicating session deltas to every node over multicast. Simple, but all-to-all replication limits it to small clusters (roughly four to six nodes).
• BackupManager — each session has one designated backup node rather than being copied everywhere. Scales considerably further.
• External store — Redis, Hazelcast or similar behind a custom manager. The right answer for containerised deployments, where pods are disposable and a canary or rolling update will move users between versions mid-session.

Whatever you choose, everything placed in a replicated session must be Serializable, and sessions should stay small. Replication cost is proportional to session size, and a session carrying a large object graph turns every request into a network event.

Tuning what you now understand

<Connector port="8080" protocol="HTTP/1.1"
           maxThreads="150"            // from measured concurrency, not folklore
           minSpareThreads="25"
           maxConnections="4096"       // sockets held, not requests processed
           acceptCount="100"           // fail fast rather than queue deeply
           connectionTimeout="20000"   // time allowed to send the request line
           keepAliveTimeout="15000"
           maxKeepAliveRequests="100"
           compression="on"
           compressibleMimeType="text/html,text/css,application/json"
           maxHttpHeaderSize="16384"
           server="Server"             // suppress the version banner
           URIEncoding="UTF-8" />

Alongside the connector, the JVM settings matter as much as anything in server.xml:

# setenv.sh — keep JVM options out of catalina.sh
CATALINA_OPTS="-Xms2g -Xmx2g \
  -XX:+UseG1GC -XX:MaxGCPauseMillis=200 \
  -XX:+HeapDumpOnOutOfMemoryError \
  -XX:HeapDumpPath=/var/log/tomcat/ \
  -Djava.security.egd=file:/dev/urandom"

• Set -Xms equal to -Xmx on a dedicated server. Heap resizing costs full collections at exactly the wrong moment.
• Always enable HeapDumpOnOutOfMemoryError. An OOM without a heap dump is an incident you will investigate twice.
• Account for thread stacks outside the heap. 200 threads at the default 1 MB stack is 200 MB of native memory the heap setting does not cover — relevant when Tomcat runs in a container with a hard memory limit.
• Match the front tier. If Apache HTTPD proxies to this Tomcat, its worker count and per-child connection pool must be sized against maxThreads, or excess load queues invisibly inside Tomcat instead of visibly in Apache.

← Back to home