Market Square holds the Portsmouth Athenæum, an independent membership library, gallery, and museum at the city's heart. The building you see dates to 1805 and was designed by Bradbury Johnson (1766–1820) as the office for the New Hampshire Fire & Marine Insurance Company — a company that later went bankrupt. The Athenæum itself was incorporated in 1817 as the Portsmouth Athenæum Society and acquired this 1805 Federal style building in 1823.
Listed on the National Register of Historic Places since 1973, it preserves and provides access to an extensive collection of manuscripts, rare books, photographs, artworks, artifacts, and digital collections related to local history and genealogy, and it maintains a circulating library for its membership. As an intellectual center, it sponsors exhibitions, concerts, lectures, and other educational and cultural programs. In 1981 and again in 1995 the Athenæum expanded into the adjacent building at 6–8 Market Square to house the Shaw Research Library and the Randall Gallery.
Notice the building's Federal lines and the neighboring wing at 6–8 Market Square, home now to the Shaw Research Library and Randall Gallery.