Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.


Route Whitelist

Living in fear of a security scanning bot taking down your website? Fear no more. We've got the perfect solution for you.

Read post

Say you have a SilverStripe website. Then along comes a bot probing your security vulnerabilities on your site. This bot could be written by a Chinese hacker, the NSA or, most embarrassingly, by a security firm you hired to tell you if your website is vulnerable to attack.

The next thing you know: your server’s load goes through the roof; the server runs out of memory; and the website crashes, failing to respond to any requests until you do a hard-reboot.

What just happened?

The inner life of security bots
Security scanning bots work is by sending requests to your website trying to detect potentially exploitable code. The bot has a long list of possible URLs that indicate your website is running xyz software. If xyz software is detected, the bot can try various exploits on that software. Since you are running a SilverStripe website, most of the probing tests result in a 404 responses. There is, for example, no “/wp-admin” on your website.

SilverStripe and 404 Page Not Found requests
Here is what happens when you send request to a SilverStripe site, and that request results in a “Page not found” response:

The request goes to Apache and then Apache runs rules to check for a static file matching the request.

  1. Apache passes the request to SilverStripe.
  2. SilverStripe spins up, loads 1000s of files, checks for logged-in users, etc.
  3. SilverStripe checks for routes that match request.
  4. SilverStripe checks for controllers that match request.
  5. SilverStripe does a bunch of database queries to try and match the request to URLSegments of pages in the database.
  6. SilverStripe fetches the “Error page” from the database.
  7. SilverStripe asks the “Error page” to construct a response.
  8. The response object is rendered to HTML.
  9. The server passes the HTML response back to the web browser.

That whole process typically takes somewhere between 200 - 1000ms, depending on the complexity and page count of your website.

If the bot has no request throttling built-in, it will send a flood of probing requests to your site, all resulting in 404 responses. Each of these requests will trigger multiple database queries, tying up server threads and preventing your site responding to genuine user requests. If too many requests come in all at once, then your server experiences a Denial of Service (DoS) attack and goes down in flames.

I wrote a simple script to simulate a bot scanning for security vulnerabilities. The script sequentially queries 880 potentially vulnerable URLs and reports how long the scan takes. The longer this mock-scan takes, the worse your website will do targeted by a real scan. The result here:

  • Large fully-featured SilverStripe website with 2000 pages: 302 seconds
  • Base SilverStripe installation: 138 seconds
  • Large fully-featured SilverStripe website with Route Whitelist installed: 8 seconds

Route Whitelist
Now I bet you are wondering: what’s this Route Whitelist thing and how does it makes the security scanning problem go away?

Route Whitelist is SilverStripe module that generates a whitelist of potentially valid URLs. That is, a list of URLs that may result in a 200 response. Any request not matching a URL in the whitelist will definitely result in a 404.

It is nearly impossible to generate a list of every possible valid URL for a large complex site. So, instead, the Route Whitelist module generates a list of only all top-level routes, all top-level pages and all controllers. The whitelist will include, for example:


The module then adds a check to the very first step of the SilverStripe page serving process. That is, it adds an Apache htaccess rule to immediately serve a 404 if a request does not match a URL in the whitelist. That’s why it’s so fast. It cuts out 9 unnecessary steps.

But it only includes top-level items. What about subURLs like: “/who-we-are/our-team”?

Route Whitelist will match only on the first segment of a URL. A request for an invalid subURL (/who-we-are/zyx-cheap-viagra) still gets processed in the slow traditional way.

But that’s okay. We don’t need to speed up every possible “Page Not Found” request. Speeding up 99.9% of 404-requests is plenty good enough. It’s a pragmatic compromise to make this module work.

No fear of the big bad bot
In summary, by installing Route Whitelist you no longer need to fear that a security scanning bot might accidentally or intentionally take down your website.

If that sounds like something you’d like to have running on your website, then go ahead, download a copy of the Route Whitelist module.

About the author
Julian Seidenberg

Dr. Julian Seidenberg is a Solutions Architect at SilverStripe with a PhD in Computer Science from the University of Manchester. He is an expert in PHP, Javascript and AWS.

Julian mentors a number of SilverStripe’s delivery teams, making sure they have all the information and support they need to do their best work. He also helps to understand SilverStripe’s clients’ needs, making it possible for us to create awesome innovative web technology together.
To ensure that we are aware of the latest technology, Julian does technical research and builds prototypes.

He enjoys working at SilverStripe because we collaborate with interesting clients on ambitious projects. Additionally, he loves how he can be himself at SilverStripe, developing deep bonds of camaraderie with fellow employees and amicably collaborate with everyone.

Julian is technically an American citizen, but was born and grew up in Germany, and is now happily living in Auckland, New Zealand. He enjoys riding his scooter to and from work every day, and arriving home to his young son; who proceeds to run right past him, climb up onto scooter and pretend to ride it. You might also find Julian at The Loft giving talks on the Krishna knowledge of ancient India (ask him about self-realisation and whatnot). He also writes on his personal blog.

Post your comment


No one has commented on this page yet.

RSS feed for comments on this page | RSS feed for all comments