yeah you should clean up, you don't want any of that nasty sperm getting into your partner! unless you want babies! dont worry though sperm normally dies with contact with air.
you don't have to start again, just wipe it and carry on from where you left off. unless you've lost all sexual desire then maybe you'll have to back track or just stop. but finish your partner off first. it's very rude to get pleasure and not give something back!!