A Herefordshire gamebird rearer has had his bid to live in an on-site caravan dismissed, meaning he will have to leave and remove it.