Speeding up SilverStripe with the Configuration System

Posted by Hamish Friedlander on 14 December 2011

One of our focuses in SS3 has been to improve on performance. All frameworks have to maintain a balance between features and performance, and generally SilverStripe has done that pretty well. In SS3 we haven't looked to adjust that balance, but instead looked at the features that cause the biggest performance issues, and see what we can do to refactor them so that we provide the same functionality in a more efficient manner.

Since I'm working on the configuration system this week to do just that, I thought it would be appropriate to talk a bit about it, to give you an idea about how we tackle the task of keeping or improving the feature set while still improving performance. Hold on, because this is likely to get a little technical.

The SilverStripe Framework provides a lot of tools for developers to enable developing sites easier, but a cost of that is that there is a lot of work the framework does before user code is called in order to set these tools up. Though we made improvements between 2.3 and 2.4, there can still be up to an 80ms delay between a request being received by the server and the framework handing control over to user code to respond. On high traffic sites, this adds up. It also means that low-latency controllers - such as auto-complete ajax handlers - have to be carefully written to be sufficiently fast.

One of the biggest offenders remaining in this startup code is the setting of configuration values. In 2.4, each module can provide a special piece of code, called _config.php, which is executed on every request in order to set up configuration. This has several disadvantages:

  • Each of these files executes every request, so a performance issue in any _config.php file affects every request
  • Much of the configuration set in this way isn't used for the current request, but they all get set anyway. Not only does this take time, it also causes all the code for all those features that configuration is set for to be included - which takes time and increases our memory usage.
  • Because these _config.php files can perform any operation, they are uncache-able
  • There's no way to set the order the _config.php files are executed in, making overriding configuration in other modules impossible.

However there is one significant advantage to the current approach that we didn't want to eliminate:

  • Configuration can be complex and intelligent, setting various configuration settings after probing the environment the code is executing on to determine the best value.

Because of all these issues, SS3 will include a complete configuration management API. Legacy _config.php files can continue to be used, but their use is deprecated for any configuration that can be done through a new system which uses several YAML files stored in a specially named _config directory. 

Each YAML file contains a sequence of header and configuration blocks. The header blocks can specify whether the following configuration block should be included based on several environmental checks, and also the order of the inclusion of configuration blocks. The configuration blocks specify the various configuration parameters for the classes in the framework. These configuration values are stored in a "Configuration" object rather than in the classes they are for, which means that setting configuration can happen without having to include the configured class's code.

As a result we solve all the issues above while still maintaining all the features that are present in 2.4. We also do so in a manner that significant improves startup performance.

Some examples will help illustrate the concepts.

In 2.4 system, because _config.php inclusion order is uncontrolled, when adding Routes (maps between URLs and the code that responds to those URLs) the system offers a method that takes a set of routes and an integer "priority" - the higher the priority, the more important the rule. Here are the portions of the cms and framework _config.php that configure these routes:

Not only does this call to addRules slow down startup, but reading those two files it's very hard to tell which rule would "win" in the case of a conflict.

Let's compare this to the same configuration in SS3.

It's immediately clear when looking at cms/_config/routes.yml that it overrides the settings in framework/_config/routes.yml

As an extension - what happens if we only want the admin/security area to be configured if the cms module isn't installed? For example, if the security admin in framework was bare-bones and the cms module provided a more fully featured version?

Then we could replace that second config file with

As a result of this new system we can still manage configuration in SS3 with as much (in fact, more) control and fluency as in 2.4, but we've managed to improve the start up speed. The system remains backwards-compatible for those migrating 2.4 applications, but similarly to the new DataList API, by putting a bit of effort into porting the configuration of a site to this new system, the developer will see immediate benefits.

Post your comment

Note: Comments are moderated and won't show until they are approved

Comments

  • I like both solutions, thank you.

    If you use an exact hostname for domain matching you have to allow more than one hostname:

    Host: ["example.com", "www.example.com"]

    or

    Host: "beta.example.com"

    =>

    if (!empty($cfg->Only->Host)) {
    if (is_array($cfg->Only->Host)) {
    return in_array($_SERVER['HTTP_HOST'], $cfg->Only->Host);
    } else {
    return $_SERVER['HTTP_HOST'] == $cfg->Only->Host;
    }
    }

    Another solution is drupal's folder hierachy: http://drupal.org/node/53705 (I know drupal ist evil :D)

    Or a RegExp witch will be mutch faster than parsing the url structure ... (because it is native C)

    Host: /^(www\.)?example\.(com|net)$/i


    > 3) Just keep using _config.php where you have to. There are cases where you need to make decisions that aren't one of the ones supported by the configuration system. I purposely only support the most common, again to balance features versus performance.

    I haven't found a Config class on GitHub yet ... so will it possible to relay on old _config.php syntax and write something like: Config::set('MyClass.Foobar', 'Baz', array('After' => 'mysite/#blar')); instead of using MyClass::set_option() ?

    Posted by Dominik Beerbohm, 2 years ago

  • Hi Hamish,

    A big "Thank You" for sharing your ideas.

    I am really happy to be updated on the improvements in Silverstripe 3 with some detailed technical/practical infos.

    What is the realistic ETA of 3.0?

    Greetings,
    Hendrik

    Posted by Hendrik Schaper, 2 years ago

  • Hi All,

    Couple of issues with the code images. Getting them sorted now.

    @Joel - hopefully the changed image will make things clearer? The "Before" line says "the block of configuration immediately below should come before the one I'm naming here"

    @Dominik - currently that's not supported. Three options:

    1) It's probably a pretty common use case, so I could add support like https://gist.github.com/1478799 - but for max efficiency that would only work for exact matches against the host name, not partial matches.

    2) You could configure like https://gist.github.com/1478812 - then write a method on OurWebServiceClass called getWebserviceBaseURL that does the match against HTTP_HOST and gives the right value.

    3) Just keep using _config.php where you have to. There are cases where you need to make decisions that aren't one of the ones supported by the configuration system. I purposely only support the most common, again to balance features versus performance.

    @Frank - sorry, there actually wasn't a difference because the same image was posted twice. Should be fixed soon.

    Posted by Hamish Friedlander, 2 years ago

  • Hi Hamish

    I'm really sorry, but it is really not clear to me that cms/_config/routes.yml overrides the settings in framework/_config/routes.yml!?

    I guess I'm stupied, but I've tried to read your explanation many times without luck.

    I really do like the fact though that SilverStripe 3 will become faster...

    Posted by Joel Gr√łndrup, 2 years ago

  • Sweet!

    Remind me: Why do we need the double slashes?

    Posted by Philipp, 2 years ago @xeraa

  • We frequently use something like this in our config files:

    if (strpos($_SERVER['HTTP_HOST'], 'localnet')) {
    OurWebServiceClass::set_base_url('http://sandbox.example-webservice.com/');
    } else {
    OurWebServiceClass::set_base_url('http://live.example-webservice.com/');
    }

    So we don't have to change the config files on upload to the live system.
    Relying on Director::isDev() is no solution here.


    How we can realize this with the new config system?

    Posted by Dominik Beerbohm, 2 years ago

  • I'm having trouble spotting the difference between the two framework routes.yml files...

    Posted by Frank, 2 years ago @frankmullenger

  • Hamish - I am really looking forward to using this new system.

    Posted by nicolaas, 2 years ago

RSS feed for comments on this page | RSS feed for all comments

Want to know more about the company that brought you SilverStripe? Then check out SilverStripe.com

Comments on this website? Please give feedback.