The Insititute for Fiscal Studies has said that the loss of the single market will hit British banks hard.