Owners of an Old Market cafe have said they are "gutted" as they revealed that their Hereford branch "hasn't worked out".