Moderately intrigued following seeing the village sign, I went back to The Concise Oxford Dictionary Of English Placenames. I certainly know how to have an entertaining afternoon…. Anyway.
Hardley, Norfolk. Hardale in Domesday Book, Hardele in 1115, Hardeleygh in 1268. Hard clearing, perhaps referring to hard soil. Some forms have halh.
The ‘halh’ is the old English for cave, closet or corner, which has apparently slipped into the settlement’s name over time. Some of the area around Hardley is today marshland, so a hard clearing would make sense in this context.