Define the blast radius before tuning anything
When an alert says that “the database is slow,” first determine exactly what became slow and when. Check whether the degradation affects one API, one tenant, reads, writes, background jobs, or the entire database. Separate application response time into connection acquisition, network time, query execution, and downstream processing. If requests spend most of their time waiting for a pooled connection while queries reaching the database remain fast, changing SQL or adding an index will not address the immediate bottleneck.
Build a precise incident timeline and align it with deployments, imports, scheduled jobs, schema changes, parameter changes, backups, failovers, and traffic shifts. Capture evidence before restarting components because blocking may disappear and caches may begin warming again. Preserve slow-query data, wait events, lock relationships, active sessions, CPU, memory, disk latency, IOPS, network latency, and replication lag. A useful first-pass checklist is:
- Scope: Which services, query classes, tables, nodes, or regions are affected?
- Timing: Was the change abrupt, gradual, periodic, or correlated with a known event?
- Resources: Are compute, memory, storage, or connection limits being approached?
- Changes: Did code, data volume, statistics, maintenance work, or infrastructure change beforehand?
Use waits to distinguish busy work from blocked work
High CPU is a symptom, not a diagnosis. The database may be consuming CPU because of repeated scans, or sessions may be accumulating because storage, locks, or a remote dependency is slow. Start with what active sessions are waiting for, then correlate those waits with host and cloud metrics. High CPU with a growing runnable queue points toward expensive operators or excessive concurrency. Rising storage latency near a throughput or IOPS limit calls for identifying the queries generating the reads and writes; adding CPU will not make the storage path faster.
For lock contention, trace the blocking chain to its root. The longest-running visible query may simply be waiting behind an idle transaction, a batch update, or a schema operation. Inspect the oldest transaction, the locks it owns, and the application that opened it. Terminating that session may restore availability, but a large rollback can prolong pressure and retrying applications can recreate the same load. Before intervening, estimate the transaction size, understand the consistency impact, and decide whether retries must be throttled or temporarily disabled.
Compare against a healthy baseline and inspect plans
Sudden degradation often means that a threshold was crossed: data distribution changed, statistics became stale, an index stopped being useful, a parameter type changed, or table growth invalidated an earlier assumption. Compare healthy and degraded periods by total query time, mean latency, call count, rows examined, temporary-space usage, and returned errors. High total time with stable per-call latency usually indicates more executions. A sharp increase in per-call latency deserves closer inspection of the plan, cardinality estimates, and data distribution.
Do not reduce plan analysis to spotting full table scans. Compare estimated and actual row counts, join order, selected indexes, sort or hash spills, partition pruning, and implicit type conversions. Test with representative parameter values because a plan suitable for a common value may perform badly for a rare one, or vice versa. An index is also a trade-off: it consumes storage and increases write and maintenance cost. If a temporary batch is responsible, reducing its chunk size or moving its schedule may be better than creating a permanent index. For a recurring online query, updating statistics, rewriting the query, adding a targeted index, or applying a carefully tested plan-control feature may be justified.
Mitigate safely, then verify one hypothesis at a time
During an incident, favor reversible actions with a clear scope. Pause nonessential batches, cap concurrency for expensive work, reduce aggressive retries, remove a confirmed blocker, or route stale-tolerant reads to a healthy replica. Increasing the connection pool often makes matters worse by admitting more concurrent work and consuming additional database memory. Scaling the instance can buy time, but it may only hide a poor plan or an unbounded query. Make one principal change at a time, record its timestamp and expected effect, and watch both user-facing latency and the underlying wait or resource metric.
After recovery, turn the incident into durable controls. Retain query-level baselines, alert on connection-pool saturation and storage latency, set realistic statement and transaction timeouts, bound batch sizes, and review risky SQL and schema changes during deployment. The post-incident review should explain more than which query was slow: it should identify why the condition became possible, why detection was late, and which evidence was missing. The goal is to make the next diagnosis possible from prepared telemetry rather than improvised production access. Incidents spanning application code, cloud infrastructure, and database behavior are also where an experienced integration team can help establish clear ownership across those boundaries.