Hey, I've found how to get style_formats to work in SS 3.0; I am not sure about 2.4, it's possible that its TinyMCE version is not compatible.
Anyway, you need to pass parameter as actual nested PHP array, which then gets converted to JavaScript objects; if you pass the string, even TinyMCE gets the string and won't be happy.
In your case, it may look like this:
$formats = array(
array('title' => 'Red text', 'inline' => 'span', 'styles' => array('color' => '#ff0000'))
);
HtmlEditorConfig::get('cms')->setOption('style_formats',$formats);
Or this HTML5 example from TinyMCE – http://www.tinymce.com/tryit/html5_formats.php – may be implemented in SilverStripe like this:
$formats = array(
array('title' => 'section', 'block' => 'section', 'wrapper' => true, 'merge_siblings' => false),
array('title' => 'article', 'block' => 'article', 'wrapper' => true, 'merge_siblings' => false),
array('title' => 'hgroup', 'block' => 'hgroup', 'wrapper' => true),
array('title' => 'aside', 'block' => 'aside', 'wrapper' => true),
array('title' => 'figure', 'block' => 'figure', 'wrapper' => true),
);
HtmlEditorConfig::get('cms')->setOption('style_formats',$formats);
(Note that for HTML5 tags to work, you also need to add them to valid_elements.)