I have done the trip overland, second one below, although in the opposite direction.

First time (Bangkok==>Siem Reap), a few years ago:
Bangkok railway station to Aranya Prathet (6 hours) . About 50 baht at the time
Tuk tuk to border 60 baht
Then found a taxi on the street and shared it with a Polish tourist. $20 each, total $40. No scam.
[OK, this isn't strictly starting from Pattaya, however add in a bus to Bangkok and it's an option]

Second Time (Siem Reap ==>Pattaya)
Taxi from Siem Reap to Border $40
Bus from border market to Pattaya (cannot remember cost, but must be about 350 baht).

I've also been overland from Sihanoukville to Pattaya.

In all 3 cases the border crossing only took a few minutes. When entering Cambodia, I have always used an e-visa and the border control has always been very quick, whether it was at PoiPet or the airports.

The slowest part has generally been the rail or bus journey on the Thai side.

Somewhere I saw a taxi service recommended that could be pre-booked from PoiPet to Siem Reap. I've not tried that. Perhaps google might locate it for you.