I'm looking to get a hold of older releases of SilverStripe CMS, from 2.3.4 upwards.
The links that were published for those releases are broken now, like http://www.silverstripe.org/assets/downloads/SilverStripe-v2.3.4.tar.gz
The migrated tags on GitHub seem unreliable, with the tags for 2.3.5 and 2.3.6 pointing to the same commit, for example.
Do the original zip/tar.gz files exist anywhere?

Hey there, the release archive on the site seems to only go back to 2.4.x versions, I think you may have to use the tagged github repos. Possible you can use composer and the silverstripe-installer to get the versions you are after https://github.com/silverstripe/silverstripe-installer

@camfindlay thanks for the reply :) Unfortunately the tags on Github don't seem to be reliable, with some of them pointing to identical commits. Are the actual releases lost forever?

I think the pre-git svn site is still up which will have code for these older releases:
http://svn.silverstripe.com/open/

They'll be split up into the phpinstaller, sapphire module and cms module (which you'll need to checkout separately via svn). 2.3 is way past end of life, and 2.4 is going end of life in March next year. I suspect perhaps in the move to the new website some of the old release archives where dropped. I'll make a ticket in our internal project board to have a look into whether we still have those assets on backup.