One of the key points you need to consider is the content model. WordPress and SilverStripe have pretty different content models where WP pretty much splits everything into Posts and Pages. If your site is mostly page based then making the switch shouldn't be all that difficult.
The main thing to consider in that case is how you're using custom fields today. SilverStripe doesn't have the notion of custom fields at all but instead works with different page types. Each page type can have different fields and a different template from all the rest. So if you're using custom fields extensively you need to go through your site and note those down and then create them as page types in SilverStripe.
After you've done this you should be able to write an import script that takes the xml file you get from the WordPress exporter and imports it to SilverStripe. I'm not sure about the limitations of the exporter when it comes to custom fields so you might run into issues there.
If your WordPress site is mostly post based then a switch will be harder. SilverStripe doesn't really have the notion of non hierarchical posts like WordPress does. DataObjects don't have a hierarchy but they're really not meant to be displayed in the same way pages are. There is a blog module that you can use but converting posts over to SilverStripes blog module would likely take some work.
One other key point is that working with SilverStripe means writing PHP code, at least initially. Every content type in SilverStripe is a class so you have to get your fingers dirty. For simpler WordPress sites you can sometimes get away with writing very little PHP code if any but in SilverStripe you're going to have to write some. That said, SilverStripe is very powerful and once you get used to the workflow you can often do pretty big changes to the content model alot quicker than you can in WordPress.