You can start from scratch if you wish. Use the BlackCandy or "tutorial" themes as a guide for the template language expressions you'll need, such as <% base_tag %> and $MetaTags() constructs. In a default install, there is only a "Page" type, so you'll just have the Page.ss template.
As for seeing garbage, maybe it extracted poorly? I grabbed the 2.4.0 tar archive and checked blackcandy/templates/Page.ss and saw this:
As for opening a specific page template, well, I highly suggest doing a test install of SilverStripe and running through tutorial #1. If you can't I'll give a quick explanation of why you can't open a template for a specific page.
The first thing to understand is that each "page" in SilverStripe doesn't have a backing file. Rather, it's all about the type of page each item is.
Templates in SilverStripe are based on the type of each item. The default built-in type for free-form content pages is, simply, "Page". There are also redirector types, virtual types, and so on but that's not important for this. When you make a new page, it will have the type of "Page". This page type corresponds to the themes/*/templates/Page.ss and mysite/code/Page.php pages.
As such, even the homepage will just be another item of type Page unless you make a specific HomePage (or whatever you want to call it) type with associated template. Tutorial #1 explains how to split out things like the header and footer of a site so you don't end up repeating code. By creating different page types, you can make different template layouts for different types of pages on your website.
As an example, imagine I am building a website for a design studio. They want to have sections for the different work they do (branding, web design, typography, packaging design) and a page for each example. I can have a generic Page type that will hold common things like contact pages, legal notices. I can also have more specific types like a home page type with a design of a nice rotation of their work, a product showcase page type that will provide a layout for each example, and a category type designed to hold a grid with each item that is linked to the example page. I only have to create four templates in this case and only have four template files to manage for the whole site.
Yes, it's a bit overkill for simple sites, but for websites with multiple areas with different layout needs, it's a great system. It also is a good balance of flexibility and structure.