A new shop that will be "bursting" with vintage and quirky items will soon open in Ledbury.