The “tell me, woman” part is HILLARIOUS! 
). I am female too but I dont find any wrong with that kind of saying, I think it may? be just the differences in cultures. I mean it’s ok to say it in his country but not here.
Source:
http://demilovato.getstarbeauty.com/2013/10/28/simon-cowell-talks-demi-lovato-cant-shut-her-up-x-factor-season-3/