I THEN saw the warning on this page about setting the appropriate default language for your content BEFORE building the database with Translatable enabled. http://doc.silverstripe.org/sapphire/en/topics/translation
Have I screwed up massively?
In the CMS, the default language for all existing and new pages appears to be 'English (US)', which I don't want as the default.
OK, looks like it may be sorted now. I commented out the translatable extension, re-built, and then un-commented it. I also had to create NZ versions of all of my pages one by one (only about 35 pages). But now it seems to be defaulting to NZ at least.
One thing that made it seem worse than it actually was was that I had top-level redirect pages linking to sub pages. When I had created NZ language versions of a section, the redirector wasn't automatically going to the correct child page, it was still linking to the child page of the US version. Something to watch for.