Go get some emergency contraceptives from your dr or a clinic. You have a 72-hour window in which they are most effective, so get going. Regardless of if he pulled out before ejaculating (and you aren't even sure he did), there is still a risk. The dr will probably just ask you a few questions (no exam) and give you some free ones or a prescription. If you can't get any ECP - for whatever reason - but you CAN get regular birth control pills, they can be taken as ECP, if you follow the required dosages. (If you have any questions about that, get back to us.)
As for side effects, they do have them, and they can bring on spotting and your cycle may get messed up but if getting pregnant and having a baby is not an option, it is more than worth it.
Also, I noticed in your original question you added the phrase "Do I need to do something to make sure I didn't get pregnant the first time???" The odds of getting pregnant the first time are as good as with any subsequent time. There is no magic anti-pregnancy thing about the first time. People need to take precautions EVERY time, doesn't matter if it's the first.