You are correct you can lose weight without excercise, but at some point you will reach a crossroads. You will get to a point where you plateau, your metabolism slows because of the calorie intake, and weight loss becomes more difficult.
I have become an excercise demon. I excercise 2+ hours daily. I am building muscle mass, my metabolism is much higher, and I can eat a good meal and still be great! (I always have an eye on fat content and carbs and sugars, got to keep the heart in good shape to excercise!
I guess the answer is excercise may not be "necessary" for weight loss, but is sure makes it alot easier!