In the 16th century London was still a medieval city with crooked, narrow lanes. Most houses were timber-framed. Only the churches were built of stone. The Tower is one of the few surviving buildings in London from the period in which the novel is set. The oldest part, the White Tower was built by William the Conqueror. In the course of its history the Tower served as a fortress, a royal castle and a gaol for high-ranking as well as political prisoners, who have left inscriptions in their cells. The Mint, the Armoury and the Ordnance, as well as the Menagerie, in which exotic animals were kept, were also lodged in the castle. The Garden Tower mentioned in the novel was renamed Bloody Tower after the alleged suicide of Henry Percy, eighth Earl of Northumberland in 1585. The moat was drained in 1843.

Elizabeth I's powerful courtiers had magnificent mansions built in the Renaissance style, but the typical manor house dated back to the later Middle Ages. In contrast to earlier centuries, defence had become less important, high Gothic windows let in more light and additional rooms provided more comfort and privacy. In the north near the Scottish border, however, moat and gatehouse, which could be barricaded in times of danger, were retained. There were two courts: the stables, barns and dog kennels were grouped around the entrance courtyard, which was accessed via the gatehouse, and protected the front of the house. The inner court was often used as a herb garden and was surrounded by the kitchen and other offices like bake- and brewhouses as well as the chapel. As the manor houses tended to be isolated from the outside world, particularly in winter, many households were self-sufficient up to the 19th century and only purchased goods that they could not manufacture themselves.

  The old City was totally destroyed in the Great Fire of 1666, the buildings which survived the fire were lost over the centuries through land speculation.