Teens who go go Ibiza or any other such Mediterranean hotspot just for endless (hetero)sexual encounters are not labeled "sex tourists" even though that is the prime object of their trip. Olders guys who go to Thailand are.

The possible factors causing the second category to be condemned are: (1) absolute age of the tourist ["any older person wanting sex is a bad thing"], (2) the age differential ["any intergenerational relationships are bad"], (3) the element of payment that's usually involved ["any paying for sex is bad"].

And I suspect that there's a fourth factor too - envy on the part of many people who don't have the ability/opportunity/gumption/balls to do such a thing, while others of us do have and are going away and having a good time.