After all that, we never ended up visiting the Yen Tu site. I was interested in visiting the Yen Tu Mountains, and could see some tours visiting from Hanoi. But upon arriving in Hanoi, I found that they were either not running at this time of year or we'd have to book the whole bus and it'd be expensive (we didn't even get to how much). The best time to visit the site is in January or February, in the lead up to Tet, when it's very popular as a pilgrimage site and there's regular trips running. Also more ideal weather conditions for climbing! The other alternative would be to stay in Halong, before or after a Halong Bay Cruise, as it's a lot closer and organise a driver from there. I also found out you can get dropped off by the buses from Hanoi to Halong, but it's 4 hours, so pushing it for a day trip.
We instead considered visiting the closer pagoda that Els visited, but weren't quite up to chancing it on public transport and walking in the 38C heat. Our hotel found a driver, but it would have been 88 USD, so we skipped. Couldn't see much progress yet being made with the listing (so who knows if or when), and we can always return to Hanoi if it is added.
A few more heads up on Vietnam sites:
* Thanks tonylag for the heads up for booking a driver to Ho Citadel via Thu Guest House. The current price is 50 USD. I organised it quickly and easy via Facebook messenger, and the driver was good and knew where to go.
* A great option we discovered for My Son only after visiting The Sinh Tourist office in Hoi An. They also do sunrise tours. 130k dong instead of the usual 99k dong, but our bus group were the only ones there apart from another small group and it was a lot cooler first thing in the morning. Pick up from hotel around 4:30, then you get back to Hoi An around 9:30. You don't actually get sun rise at the site though, it's probably during the bus trip there. It's about a 30 seater bus, the only downside the aircon wasn't working fully.