After giving it a thought I think one can distinguish several categories of underwhelming/unworthy sites:
1) Sites that are worthy of inscription and impressive but are not necessarily inscribed for the right criteria - For me Quebrada de Humahuaca and Hallstatt-Dachstein are good examples. These are impressive and historically interesting sites, but a great part of what makes them exceptional is their natural settings. In both cases these are not included in the OUV.
2) Sites that I believe have OUV, but are underwhelming to visit. Some examples are Messel or the Carmel Caves, both of which are significant for what was found there and what might be found there in the future. There is not much to see though. Tel Aviv is another such example. Its architecture might have been influential, but it is not that appealing.
3) Sites that are rewarding to visit, but for which I am not that sure there is enough OUV. Some examples:
Trogir - A very nice medieval Venetian-Dalmatian coastal town. If you are in Split you should not miss it, for its picturesque architecture. However, I think there are tens such towns in Italy and even some others on the Croatian coast.
Horta - Interesting and beautiful for Art Nouveau lovers. However, considering artistic exceptionality I think other architects are by far more impressive owing to their individual note (Otto Wagner in Vienna, Gaudi), or to the fact represent better examples of the artistic movement (de Montaner in Barcelona, Eisenstein in Riga). But maybe it is just a matter of personal taste?
Lviv - Lviv is certainly one of the nicest cities in the Ukraine and the most visited site in Western Ukraine. However, compared to other Eastern European cities in general and Polish cities in particular it does not fare that exceptionally high to my taste. Travelling from Cracow to Lviv you pass through Przemysl, a typical Polish city with no aspirations of being an outstanding city, and yet it has impressive churches, a picturesque historic centre, a castle on a hill etc. I suspect that one of the reasons for Lviv to be on the list is that it is on the other side of the border.
Incense Route in the Negev - One of the best sites to visit when in the Negev desert. However, compared with contemporary Petra they are really nothing special. There are plenty of far better examples of desert agriculture, trade routes and Byzantine churches.
4) Sites that find underwhelming and unworthy of their inscription. Some examples:
Lorsch - Historically Lorsch was the largest monastery in Europe and it was groundbreaking in the development of modern medicine. However, barely anything remained there, almost all of the buildings being wiped out by the Bishop of Mainz in the 18th century. The only significant building still standing is impressive from the outside, unimpressive from the inside. I think other examples of Carolingian architecture like Aachen or Müstair are better. And even of Lorsch is important for having this small building, I do not think it is enough for an inscription.
Lednice-Valtice - Nice gardens to visit when in the area, but many other royal residences and parks are inscribed that are better (Schönbrunn, Potsdam, Versailles, Wörlitz).
Colonia - Often mentioned as an underwhelming site, it is the best historical site Uruguay has to offer, which is also the main reason for its inscription. On South American scale other colonial sites fare much higher (Lima, Quito, Cartagena for Spanish architecture; Bahia, Ouro Preto and Olinda for Portuguese).
Neusiedlersee - Not much to see, mostly reed and vineyards. The nicest historical town is Rust, which is nice, but nothing spectacular. Better examples of vineyards in Europe are abundant (Piedmont, Middle Rhine Valley, Loire Valley, Cinque Terre, Wachau). I personally cannot find any OUV in the lake.
Regensburg - Another case we discussed. I think there are plenty of European cities with similar qualities (medieval cathedrals, historic town centre). It is true it is the best example of tower houses in Europe north of the Alps, but is this enough to warrant OUV? Would we even consider the best example of tower houses north of the Asir mountains in Saudi Arabia? The best example of Inca architecture south of Cuzco? Or the best example of subtropical shrub south of the Rio Negro?
These are only a few examples based on my travels. Reading through guides and reviews I am positive many similar examples can be found in other parts of the world too.