The site builder never destroys data. It merely archives old versions away in the database. Handy in case you ever need to do an emergency restore of a page (had a client accidentally delete about 20 pages two weeks ago, emergency restore saved his ass). While it won't completely purge the database of old fields, this will clear old fields out of the current version: dev/build/?flush=1

Anyway, are you changing the theme via the Admin CMS or via the config file? If you're changing it via Admin, try changing it via the config file instead. Run a ?flush=all after changing the theme. If it's stuck, delete the silverstripe-cache folder.