Married people expect others to help enforce their monogamy. Having single people available around already threatens that. Poly people have declared on top that they don't respect that institution and have opted out of that mutual obligation. It is a refusal to disarm.