I got pretty into it last winter, then summer rolled around and I stopped, now its getting cold again and I've been going more. I think the cold makes me think I'm skiing so it's auto-fun. When I started last winter I kept track of my hours and I only saw improvements when I was logging like 2 hours a day, occasionally over 4 hours, then I splinted my shins from pushing which pretty much obliterated what momentum I'd built over a few months. Basically, I agree it's really hard.
Really important to just chill and progress naturally though. Don't expect to be doing anything besides riding for a long long time. Ride switch, push regular and mongo (helps with tendon/bone issues), practice any balancy thing that feels hard before you move into anything fancy. At least that's my plan and I found I didn't suffer all that bad by following it last time. I don't skate park, I go out at night and explore the city.
With skiing I used to aim for getting like 40 days in, so like 160-200 hours of pure skiing. Aim for triple that over your skate season, it makes you have to be consistent to keep up with the hours.
Pain... I find it hurts bad the first few times per spot then your nerves pretty quickly get ruined and falling doesn't hurt so much.
Most importantly you have to have fun most of the time, absolutely crucial, but frustration is also part of it. It's like life, you'll have good days and bad days, just keep going.