The arguments in favor of sexual freedom rest on similar grounds as freedom of speech. Our love is part of our freedom, whether straight, gay, sapiosexual, one on one, or non monogamous.
Recommended reading, as pertinent today as in the past.
John Stuart Mill: On Liberty