I am brand new to SilverStripe as of yesterday but faced this same question, and solved it, and thought I would share my learning for others.
Ok I got a site up quickly then installed a theme and wanted to get rid of the word "Home" in the title bar of the browser, and also the word "Home" that appeared on the top of my home page itself. It took me soooooo long to figure this out . the answer from the previous poster works perfect for changing the Title bar text (thanks!!). Now, as for word "Home" stuck on the top of my home page... you need to read the Basic Tutorial and follow each step about creating a home page that is different from the other pages. However since I had installed a theme, instead of creating HomePage.php in tutorial/code, I created it in mysite/code. And instead of creating HomePage.ss in tutorial/templates I created it in themes/mytheme/templates. But actually I didn't even do that. I just went into themes/mytheme/templates/layout and copied Page.ss to HomePage.ss, then commented out the lines that added the word "Home" like this:
<!-- <div class="header">
<h3>$Title</h3>
</div> -->
Sounds easy now and you experts are reading this and thinking what a goofball but it took a lot of searching to figure this out. Hope it benefits someone.