Build an ISP DDoS Response Playbook That Preserves Evidence
A practical DDoS response framework for ISP NOC and support teams: detection thresholds, containment steps, evidence capture, and recovery validation that survives review.
A practical DDoS response framework for ISP NOC and support teams: detection thresholds, containment steps, evidence capture, and recovery validation that survives review.
A volumetric or application-layer attack rarely announces itself cleanly. The first signal is usually a support queue filling with “internet is slow” tickets while an upstream link sits pinned at capacity. Without a rehearsed playbook, the NOC improvises under pressure, mitigation steps go unrecorded, and by the time service recovers there is no defensible account of what happened, who acted, or whether the fix will hold. This framework turns that chaos into an ordered workflow that contains the attack, preserves evidence, and validates recovery before the incident is closed.
Define What Triggers the Playbook
A response playbook is only useful if it starts at the right moment. Set objective triggers so the on-call engineer does not have to guess whether an event qualifies as an attack or a capacity problem.
Detection thresholds worth agreeing on
Agree in advance on the conditions that move an event from “watch” to “active incident.” Typical triggers include sustained inbound traffic above a defined percentage of upstream capacity, an abnormal ratio of new sessions to established sessions on your edge routers, a spike in packet-per-second rates that outpaces byte growth (a signature of small-packet floods), and a surge in identical requests hitting a single subscriber-facing service. Validate the exact rates and counters against your own baselines and the versions of the equipment you run.
Separating attack from congestion
Not every saturated link is hostile. Before escalating, confirm the pattern is anomalous relative to your normal peak. A predictable evening peak that clears on its own is congestion. A sudden, sustained flood from unusual source distributions, targeting a narrow set of destinations, is far more likely to be an attack. Device monitoring and alerts should surface both, but the human decision to invoke the playbook belongs to a named role, not an ambiguous group.
Assign Roles Before the Attack, Not During
Under load, unclear ownership costs minutes you do not have. Assign roles ahead of time and make them role-focused rather than person-specific, so the playbook works regardless of who is on shift.
Incident lead
Owns the decision to invoke and stand down the playbook, coordinates actions, and is the single point of record for the timeline. No mitigation ships without their awareness.
Edge engineer
Executes filtering, rate limiting, and upstream coordination on routers. Records every command and change with a timestamp against the incident.
Support liaison
Translates technical status into subscriber-facing updates, tracks affected accounts, and prevents duplicate escalations from flooding the queue.
Evidence keeper
Captures telemetry snapshots, logs, and screenshots at each phase so the post-incident review has a complete, tamper-evident record.
Contain the Attack in Ordered Steps
Containment should proceed from least disruptive to most disruptive, so you do not black-hole legitimate customers before trying targeted measures. Work the steps in order and validate each before moving on.
- Confirm and classify. Identify the target destinations, the traffic type, and the source distribution. Record the classification before you touch anything.
- Apply targeted filtering. Rate-limit or filter the specific malicious pattern at your edge where possible, protecting the affected service without dropping unrelated traffic. Validate configuration syntax against your RouterOS or platform version before committing.
- Engage upstream. If volume exceeds what your edge can absorb, request upstream filtering or remote-triggered blackholing for the targeted prefixes. Log the request time and the provider ticket reference.
- Isolate as a last resort. If a single subscriber prefix or service is the target and cannot be defended, isolate it to protect the wider network, and record the business decision and who approved it.
- Verify partial recovery. After each measure, confirm whether link utilization and session rates are returning toward baseline before escalating further.
Never delete or overwrite logs to “clean up” during an incident. Evidence removed under pressure cannot be recovered, and it undermines both your post-incident review and any obligation to report the event. Confirm what your local regulations and upstream contracts require you to retain.
Capture Evidence While the Incident Is Live
Evidence gathered after the fact is always incomplete. The evidence keeper should capture the picture at each phase, not reconstruct it afterward.
What to record
Snapshot interface counters and utilization graphs at the moment of detection and after each mitigation step. Preserve flow or session telemetry that shows source distribution and target concentration. Keep the exact commands issued on edge devices, with timestamps, so the change record and the observed effect can be lined up. Record upstream communications and their timing. Note every subscriber-facing status update and when it went out.
Why the timeline matters
A clean, ordered timeline answers the questions that always follow: when did it start, when did we notice, what did we do, and when did service recover? Operational audit records and subscriber change logs let you reconstruct this without relying on memory. The goal is a record that survives independent review, not a story assembled from recollection.
How ISPbills Supports This Workflow
ISPbills connects subscriber, network, support, and access-control workflows in one operational system, which matters during an incident when those functions otherwise fragment across separate tools. Its device monitoring and alerts help the NOC detect the abnormal utilization and session patterns that trigger the playbook, so escalation starts from a signal rather than a support-ticket backlog. Confirm which monitored counters and alert conditions are available on your plan and tune them to your own baselines.
During and after containment, the operational audit records and subscriber change logs give the evidence keeper a durable account of who did what and when, tying mitigation actions to the accounts and services they affected. Role-focused permissions keep the incident lead, edge engineer, and support liaison acting within defined scopes, so containment steps are attributable and financial-data isolation keeps billing information out of reach of roles that do not need it during a security event. The handoff that becomes simpler is the post-incident review: instead of stitching together exports from disconnected systems, the team works from a single record. Backup workflows support recovery if configuration or subscriber data needs to be restored after an incident. Verify the current behavior of these capabilities against ISPbills product documentation and your own configuration.
Validate Recovery Before You Stand Down
Declaring an incident over too early invites a second wave into an unprepared network. Recovery is a checklist, not a feeling.
Recovery validation checks
Confirm link utilization has returned to and held near baseline for a defined window, not just dipped momentarily. Verify session and packet-rate anomalies have cleared. Check that any temporary filters or blackholes are documented with an owner and a review time, so they do not silently become permanent policy. Confirm affected subscribers can reach the previously targeted service and that the support queue reflects that recovery. Only then does the incident lead stand down the playbook and record the closing timestamp.
What “done” looks like
The incident is done when service is verifiably stable, every temporary mitigation is either removed or formally adopted with an owner, the evidence record is complete and timestamped, and a short review has captured what to change before next time. Anything less leaves the network exposed to a repeat.
A Decision Standard You Can Adopt
Turn this framework into a standard your team can hold itself to. Before your next quiet week ends, agree on the specific detection thresholds that invoke the playbook, assign the four roles to shifts, and run a tabletop exercise against a realistic scenario. Judge readiness by three tests: can any on-call engineer invoke the correct first step from the trigger conditions alone; can the evidence keeper produce a complete timeline from your logs and audit records without guesswork; and can the incident lead prove that every temporary mitigation was either removed or adopted with an owner. If any test fails, you have found the gap to close before the next attack finds it for you. Validate every configuration, threshold, contract obligation, and regulatory requirement against the versions and jurisdictions that apply to your operation.
Research basis: ISPbills product documentation; MikroTik RouterOS documentation; Broadband Forum technical resources; OWASP guidance. Validate implementation details against the software releases, contracts, configurations, and local regulations governing your network.