You could always
wear a D3o beanie instead. I feel the same way wearing a helmet, I can't even make it through doorways without smacking my head on them I just can't get used to the extra circumference around my head and everything feels off like you say, especially my hearing. I wear a d3o beanie though and it's saved my head numerous times, plus it's good for more than one impact.
Those of you saying to wear a beanie under your helmet; your helmet should be snug against your head if it fits properly you won't have room for a beanie under there.