It DOES matter when you start your pills. You can't just go and get a pack and start taking them whenever you want.
Doesn't your pack have instructions? Usually, the instructions will say start the pills the first Sunday after you start your period, but in your case, that would have been been on the 20th. Waiting another 7 days to have a Sunday start is not an option. I believe at this point your only option (other than waiting until next month) is to do a fifth-day start, which would be now. (And even though it's Wednesday already , it would still be okay to start today.)