When I was brought up, my parents always got onto me if I didn't say yes ma'ma or sir. Since I used it so much thats how I always talk to my elders. I think it is a sign of respect, but yes if they wish not to be called ma'ma or sir, then respect them and don't say it. But never just say yeah...