From the World Heritage inscription:
From the Louvre to the Eiffel Tower, from the Place de la Concorde to the Grand and Petit Palais, the evolution of Paris and its history can be seen from the River Seine. The Cathedral of Notre-Dame and the Sainte Chapelle are architectural masterpieces while Haussmann’s wide squares and boulevards influenced late 19th- and 20th-century town planning the world over.
There are certain cities which contain so much history, that the entire city becomes a world heritage site, even though there are multiple locations in the city which could warrant world heritage status on their own. Paris is one of those cities. Just walk around the Seine and you’ll bump into history. You can’t avoid it.