You’re driving down the road and, in a moment of inattention, you run a red light. In one universe a cop pulls you over and gives you a ticket. In another universe you hit a little old lady and kill her.
In the first universe you’re just an ordinary motorist. In the second you’re a shameful monster. But you had no control over the presence of the little old lady; the same (small) list of controllable actions were available to you in both universes.
If our moral responsibility extends only to our voluntary actions, then in both universes your only transgression lies in running the red light. Why then do we assign additional blame for hitting the lady, an outcome over which you had no control?