You can try and just get the money out for things that aren't specifically school related. I remember being told that I would be able to use the college money to buy a car under the claim that I would need the car to drive to school.
And the bank wants to be able to hold onto your money as long as possible so you may be able to negotiate transfer to an RRSP or something- instead of removing it in the next couple years it means they get it for 40+ years so it benefits them as well as you