Is there a method that returns every page? Staticpublisher example seem to point to SiteTree::get(); but on my site (c. 2500 pages) when I run it it maxs out CPU for a while and doesn't generate any cached versions of pages. But if I isolate a single branch of the site it runs fine using SiteTree::get()->filter('ParentID', 54); for example. Any tips?
Try staticpublisherqueue (https://github.com/silverstripe-labs/silverstripe-staticpublishqueue) which will generate pages through an event system and perhaps better for larger sites (and going to be more scalable than the default static publisher).
Do you know what part is causing the delay - the fetch of pages or the actual processing of each page? If it's a fetch, then no nice way apart from querying for the page links in batches. The processing of each page can be batched down a bit so rather than calling RebuildStaticCacheTask and rebuilding all pages at once you can do something like