I've had good luck with Seaworthy Small Ships wood model kits. They are free sailers, but are fun for adults and kids alike (provided you have a pond where you can get to all sides so as to recover and relaunch sailboat). When I can't get to all sides of a pond, I have sailed the schooner using a fishing line attached to the bow; tug the line to tack back to shore (like Vnkwi's video)
.
If you like kit-bashing: I converted the "coaster schooner" into a brigantine, and the "skipjack" into a sandbagger. Both sail nicely.
The pinewood model kits are good too; your 4 year old could help make them, I bet. I'll take 4 of them (completed models) with me to the pond for kids to sail/race. They are simple, robust, and easy to repair if a mast gets broken (just a 1/8" dowel).
http://www.seaworthysmallships.com/The website was down tonight, but it's been active as recently as Christmas 2014.