For the life of me, I can't figure out where you set the site's name and tagline. Do I have to edit a file outside of the CMS to do it or can I do it internally? I've gone through all of the options and don't see anywhere to enter it so my site is called "Your Site Name".
Have a look in mysite/templates under the directory where you have installed SS, you will find a template/view file called page.ss in this file you should find a div line <div id="Header"> in this section you should see the text for the site name and tag line. If you have installed the tutorial then there will be no site name but you should find the <div>.
I suggest you read through the tutorial http://doc.silverstripe.com/doku.php?id=tutorial:1-building-a-basic-site this will help you understand the role of the templates.
If you change the template you have to flush the cache that SS keeps by adding ?flush=1 onto the end of your url of your installation ie localhost/silverstripe?flush=1.
There isn't a site name and tag line within the cms as this depends how you want to design your layout as you may have this info within an image.
Which installation option did you use? If you choose the Black candy option when you install then the file you mention is put into the mysite/template/page.ss file and from here you can tailor to your requirements.
Read through the tutorial this is all explained and will help improve your understanding of how the information is presented.