I used to work in an office with a "NO PERFUME" policy. It was instituted before I started. I had no trouble at all with it until one of the women who initiated the policy started complaining about my hand lotion. My department handled paper all day we all used a lot of lotion because paper dries out your hands wicked fast. The boss said we had to use different hand lotion because our berry scented lotion was too strong. Ok, so I brought in something unscented and life went on. Then this woman complained about my lotion again on a day I wasn't wearing any yet. The boss came to talk to me again and the only thing we could figure out was she was smelling my shampoo or possibly my deodorant. Neither was strongly scented at all. He asked if I could switch to something else because she was "allergic to scents". 
Well as it turned out I had just heard an episode of Dr. Dean Edell where they talked about scent allergies and how incredibly rare an actual scent allergy was. Annoyed at my tattletale coworker, I told the boss that if this other employee could provide paperwork showing she was medically diagnosed with a scent allergy that I would be happy to switch despite the fact that I buy in bulk and it would be expensive for me to give away the products I already bought and buy more in the unscented variety. His other option was to start buying me approved shampoo, deodorant and lotion and I would use only that for the days I was working. He was annoyed at the suggestion that he should by my lotions and potions but saw my point and had a talk with her.
Funny, but neither she nor the other woman with "scent allergies" was ever able to provide paperwork showing an actual allergy. I also learned that one of the women with a "scent allergy" was dating a man who apparently swam in a vat of cologne every morning before work. The whole warehouse could smell him every time he picked her up for lunch. The other woman had an office close to a bathroom. She was constantly spraying her office with vanilla or strawberry air freshener. Not hypoallergenic or special air freshener, just the nasty smelling cheap stuff.
These two biddies finally pushed things too far when they started complaining about the food people were cooking in the lunch room. They wanted the boss to ban certain foods. He finally realized both of them were just bitter, manipulative wenches and he put a stop to their complaints. The no perfume policy remained, but everything else was none of their business and they were welcomed to quit if it was a problem.
The point of my rambling story is that if coworkers seem insensitive, they may have had experiences with people like my former coworkers. While people do have a legitimate cause to complain about the loads of perfume some use due to REAL allergy, asthma or migraine issues, some office workers are just lying sacks of crap and like to act like manipulative tools.