The church in East Ruston isn’t in the geographical heart of the community, but it is the oldest building in the village. And, this is what The Concise Oxford Dictionary Of English Placenames has to say about it:
East Ruston, Norfolk. Ristuna in the Domesday Book, Ristone in 1129, Riston in 1198, Estriston in 1405. From Old English Hristun, meaning brushwood.
That might not be the most exciting of all name origins, a settlement in a bushy area. The evolution of the village name also explains why it’s called East Ruston, as there isn’t any such place as West Ruston. But, the old word was Riston and there is a place in Norfolk called West Riston.