The holiest site for Muslims. Remnant of pagan rituals probably starting already prior to the 2nd Century AD on this site.

Yes in "rational" circumstances Mecca would have to be on the list. But somehow I can't see Saudi Arabia reacting well to a group of ICOMOS bureaucrats questioning their management plan, buffer zones and "historical integrity"!! And in fact of course Mecca has undergone incredible and many would think, unsuitable, development very close to the Ka'bah. Saudi Arabia has wrestled for years with its conscience on far less significant heritage within its borders and has a record of almost wilful destruction of some of its physical heritage. Wahhabism fears the idolatry it sees as inherent in such objects. It will be interesting this summer to see if it gets any of its 3 relatively recently T listed sites inscribed. But I am hopeful I will get at least 1 addition to my visited list as the "modernisers" appear at the moment to have the upper hand on this matter against the issuers of Fatwa 16626.

However, for our purposes which are to identify the most significant non-inscribed heritage I support this suggestion!

Certainly would rank among the top 5 most significant locations not included but aside from the Ka'bah there is very little of significant age/ exceptional design, mostly for the reasons outlined above.
There are a few sites that are included for their significance rather than the monuments themselves (Luther memorials) and there is certainly no argument against Mecca being included on those grounds.