Its been a couple of years since ive been, but the days of druggy riff raff on samui were long ago IMHO.
I love Samui, it was my stomping ground the first few times in Thailand, first couple with ex UK bird, then a few times after solo, including one unfortunate trip which was to change my life forever, fooking queers.
I wouldnt hesitate to stay on Chaweng or Lamai with my little un Si, the beaches are nice, a long way ahead of anything up Pattaya way, but i wouldnt go as far to say stunning, i would say the nicest beaches i have seen recently are kRABI,(last year).
The downside with Samui is the expense, getting there for a start is steep, then hotels are significantly more than what your used to in Pattaya, for a week or two stay compared with Pattaya with everything in your looking at hundreds more, probably running into over a thousand.
If youve got the dosh, and have the time why not tour the south, cheap flight to phuket for a few days, over to PHIPHI for a few more, boat and bus to Krabi for a stay then bus and boat to samui to finish up, great way to see the area if you dont mind the travelling,and you dont get board staying in the same place. did this a few times myself, albeit solo or with TG, no kids.
Plenty of family hotels on Samui, two years ago stayed in the Beachcomber Chaweng,right on the beach, nice enough, nipper loved it.
Motherfucking cock suckers!