Congress Hall fills this whole city block, with Beach Avenue to the south and the Washington Street Mall to the east. A contributing building in the Cape May National Historic District, it began in 1816 as a wooden boarding house for guests to the new seaside resort of Cape May. The proprietor, Thomas H. Hughes, christened it "The Big House"; locals, wary it was too grand, nicknamed it "Tommy's Folly." In 1828, when Hughes was elected to the House of Representatives, he changed the name to Congress Hall.
A dramatic chapter came in Cape May's Great Fire of 1878, when the original building burned to the ground. Within a year the owners rebuilt in brick, explicitly to convince leery prospective guests that the building was fireproof. Over time Congress Hall drew national attention: Franklin Pierce, James Buchanan, Ulysses S. Grant and Benjamin Harrison all vacationed here, and Harrison even made Congress Hall his official Summer White House, making it the center of state business for several months each year. John Philip Sousa regularly visited Congress Hall with the United States. Notice this block once welcomed presidents Pierce, Buchanan, Grant, and Harrison, who made Congress Hall his Summer White House for months each year.