Not a day goes when we don’t hear about someone getting hit by a distributed denial of service (DDoS) attack. More often than not, the attacks target large companies such as JP Morgan. But smaller startups are targeted as well, sometimes because the startup’s product elicited outrage. Other times, there are drive-by attacks that have no discernible rationale.
So what happens when a smaller company comes under attack? Is there a way to fight back against the DDoS? During my time as head of technical operations at Shopify and at Vox Media, I learned a few things, largely because I had to deal with a few of these attacks.
In a DDoS or “Distributed Denial of Service” attack, the attacker prevents access to a service by using many computers to overwhelm the target with traffic. The attacker focuses on the website (or service’s) weakest link. DDoS Attacks generally fall into three categories:
Overwhelm the CPU power: Find a resource-intensive function and then send many requests for it, until the server CPUs are overloaded.
Overwhelm the bandwidth: Find large pages and send many requests for them.
Hit software limits: Force the application to hit software limits. A classic example is opening and holding open connections to the server, using up the servers’ number of available network connections.
4 ways to prepare for a DDoS
Even the smallest startups can deflect DDoS attacks with a small investment. Here are four things you can do today:
1. If you are using dedicated servers, create a backup connection called an Out-of-Band (OOB) connection in addition to your regular network connection. If your regular network connection becomes inaccessible, you will be able to use your OOB connection to get onto the server, similar to having a VIP back entrance in case the front door is inaccessible. Most hosting providers can add one for you quickly, and it is inexpensive, on the order of one hundred dollars per month. If you are using cloud hosting, then this step can be skipped, since you can usually restart your cloud server instance.
2. Install an early warning system that will monitor the network and send an alert when it sees suspicious spikes. The alert can be sent to the email or phone of your system administrators and engineering team. You should get familiar with the pattern of your user traffic and set alerts that adjust for your traffic on each day of the week, as well as holidays and special events.
3. For pages and data that is frequency accessed, use a cache. The cache will store that data on a different server (sometimes even hosted by another company) to reduce the load on your servers, and make them more resilient to DDoS. This has the extra benefit of making your web pages load faster. CDNs and memcached are two popular types of caches.
4. Identify your weak points, so you can add extra monitors to them and also strengthen them over time. Use an application profiler to determine where your weak points are. One popular service is New Relic RPM. I’ve gone through multiple companies where the weak point started off as RAM and code inefficiency and changed into network hardware (routers and firewall) as the company grew. Once we identified our weak points, we added extra alerts for it, and then put in a plan to upgrade that hardware over time.
I am being DDoSed. Now what?
Let’s simulate how I would handle an attack.
It’s a sunny morning and I’m getting coffee at the coffee stand next to my house. Suddenly I receive three text messages on my cell phone that contain automated alerts about traffic levels.
I sprint home and try to connect to the server. It’s not responding, possibly due to being clogged from too much traffic. I use my Out-of-Band connection to sign in. Now I inspect my logs to determine which resources are under siege.
I look for a common pattern. Is the traffic originating from a particular set of IP addresses, or a particular country? Is it using the same browser user-agent to request the same URL?
In this scenario, I might find that all of the traffic is from a European country, directed to a particular webpage. I add a block in the firewall against a set of IP addresses that I identified from the log. I monitor the latency of the web site over the next hour to make sure that regular users are unaffected.
If the attack were more serious, I would phone my hosting provider.
When should I use third-party DDoS services?
Third-party DDoS services can provide expertise and scale beyond what you can do with your own hardware. One common solution is to permanently send traffic through their servers, which they will proxy. The benefit of these services is that they scale up to handle their biggest customers, which you can leverage since you’ll be on the same infrastructure.
However, there are side effects to consider. Adding third-party services as a permanent step between your users and your servers has a latency impact for every page. Many studies have shown that additional latency will result in a higher bounce rate, so this is a tradeoff that you must consider.
Life after DDoS
After a DDoS attack, identify what the bottlenecks ended up being, and work toward removing those bottlenecks.