Nope, it is not just about how much sex you’re having, often