Doing this yourself will only make things MUCH, MUCH worse. DON'T!
First, you need to know WHAT you have. A cyst is very different from a boil, though an infected cyst can be symptomatic in the same way as a boil. It is the underlying problem that needs to be determined.
A doc should lance this for you -- this will provide immediate relief and will be the most antiseptic way of going about it. Peroxide will NOT "remove the infection," sorry. These infections are usually far too deep for any topical applications to help at all.
To treat a boil at home, apply hot compresses to the area several times a day or take very warm baths with a small amount of water in the tub to soak the area. This will draw the infected fluid to the surface and it will ultimately drain by itself. This will NOT be pleasant and as the fluid builds up nearer to the surface of the skin, it can become extremely painful.
After it drains, keep the area very clean with soap and water and dry it very well. Cover with gauze until the drainage stops and the area heals naturally.
You can purchase drawing salve from any pharmacy -- this can help speed the process of drainage -- but not always.
The BEST and WISEST thing for you to do is see a doc. Poking it by yourself can drive the infection deeper and it can return even worse than it is now.
If you have a cyst that is infected, it's a whole new ball game. You MUST get a diagnosis to know. A cyst in that area complicates matters.
I just wrote about pilonidal cysts in another forum here. Check it out. Just don't assume you have one until you get a doc's diagnosis first.