Why Is Exercising And Diet Control Not Working For Me?

Ok, I am going crazy with this problem. I am 23 yrs old, 5′ 9″ and weight around 180 lbs. I am going to gym regularly for about 4 months now and lost only 6 lbs!! I do aerobics (cycling, treadmill, ellyptical) on an average 4 days a week. I work out atleast for 30 mins and sometimes for as long as 1 hr. I am a vegetarian and eat out only about 2-3 meals a week. That too is not a full meal like in a restaurant but just a quick fix meal like a sandwich. I also roughly calculated my calorie intake and believe me on average my food intake is around 1600-1800 calories. Most days even less than 1500.
I think one of the reasons I think is I have to sit at a place and work at job. I go mad when I see people around me (people I know) splurge on all the junk food that I usually avoid and still stay in good shape with no exercise. I miss eating good food and hate to go to gym. If I go to gym, I see marginal results and if i dont, I put on weight insanely even eating less. Any thoughts/comments?