I’m not sure how I haven’t noticed this before on Hardings Way, but it’s the gateway to Whitefriars, a Carmelite Friary which stood here between around 1260 and the Dissolution of the Monasteries in the late 1530s. Although the rest of the monastery has now gone, the street names here reflect the past, there’s The Friars, Carmelite Terrace, Whitefriar’s Terrace, Whitefriar’s Road, All Saints’ Street and Friars Street.
Over the last couple of centuries, buildings have surrounded the gate, but these have mostly been cleared and the structure now looks quite exposed. This area around this gate, inside the former monastery, was also used as a graveyard and some local residents (as well as Anglian Water staff digging in the area) have uncovered bodies.
George Plunkett has a photo of how the gateway looked back in 1935 and there’s an etching of the gate by William Taylor and William Pickford.