We need to be careful about assuming that long distance events going near to WHS will have actually gone "within" the boundaries.
Yep, taking the strictest definition then the Marathon wouldn't go through Westminster or Tower of London as the sites stop at the curbs next to the marathon routes, same with the Forbidden city, which is set back from the 2008 route as well.
I remember the Road Race going through the Great Wall in 2008 (UK's first Gold Medal was won there).
So as it stands we have
2012: London: Greenwich (equestrian) Dorset & Devon Coast (Sailing)
2008: Beijing: Temple of Heaven (marathon) & Great Wall (Cycling)
2000: Sydney: Opera House (may not be included if we have the strict definition)
1968: Mexico City: Historic Centre (marathon started on Zocalo) UNAM (main stadium)
1964: Rome: Historic centre (marathon went through at night lit by torches!)
1956: Melbourne: Royal Exhibition Building (Basketball, weightlifting, wrestling, fencing)
I'm trawling through the other host cities but can't find any concrete links. I guess Moscow, Paris and Amsterdam could well turn something up.
Also investigating Athens (doubt it), Antwerp, Berlin, Helsinki, Seoul and Barcelona, which all have sites that could conceivably be involved in the main events.
No real idea about Winter Olympics to be honest but there are a few that are near WHS (Turin)