More than two-thirds of people think the increase in the price of a first-class stamp over the past five years is unfair, a survey has found..