I would seriously doubt you could contract rabies just by touching the broom handle. If you washed your hands before eating or touching your mouth. I believe it has to enter your bloodstream to infect you. If you are worried you can call the ER and ask to speak with a nurse.
If the possum had rabies - he probably wouldn't have been as calm as you say he was and biting the broom handle was just his way of trying to defend himself.
Hope this helps. God bless you . . . and that mischievious possum.