About a month ago I discovered a recently inverted nipple and called my doctor. Right away she asked if anybody in my family had cancer (which terrified me). After a screening mammogram, diagnostic mammogram, and ultra-sound I appear to be in the clear.
You always hear about lumps, but there are other symptoms. If anything seems suspicious you need to have it checked out.