Another site option is at Cwmdu, near Crickhowell. It has limited play facilities, but it's a nice quiet site, there's plenty of walking routes around it and there is a café within walking distance (and a pub).
In retrospect it might be a better option than Rhayader, which can be busy and (as Rob pointed out) is next to a major arterial route.
GC