Moulds can stick even under the best circumstances, but I've never had one fail to that extent.
Your laminating system sounds okay to me. My technique with polyester resin is to use gelcoat, let that tack off for three hours or so until dragging a brush over it leaves no impression behind. Left overnight, things should still be fine, as the gelcoat is air inhibited. Could eb a problem if you left iot for several days. I apply glass tissue over the dry gelcoat with laminating resin, this conforms to the gelcoat better avoiding pockets of air. You can let that cure if you like, or immediately apply the layers of chopped matt.
The biggest mistake you've made, and it's common, is to start learning about GRP on an important master. You should have practiced GRP and release techniques on something small and non-critical first.
Sounds like the application of the release agent was insufficient. PVA is quite difficult to apply over wax, it tends to separate, as the there is no key for the fluid to cling to. I find a thin coat or two of hairspray (water soluable plastic like PVA) over the wax, then PVA works much better, as it gives it something to cling to. For some tools I use PVA on it's own.
As chunks of the mould broke away, you have probably got poor reinforcement in some areas, and possibly poor adhesion between gelcoat and subsequent layers. I also apply glass strands mixed with resin in corners before laying matt in, to make sure they have a nice smooth radius.
I can understand you maybe disheartened, but GRP work is a skill, not hugely difficult to master, but one which has a learning curve. Carve some small skiff/lifeboat plugs, 3" or 4" long, no bigger. Carve them from wood, primer them, then when dry, and practice making a tool from them.
Another tip, if you're using wood to make the plug, it pays to coat it in resin or paint it with a resin paint system (e.g. two pack paints). Normal lacquer type primers are porous and unfortunately even with careful application of release agent, some resin can get through, and stick to the surface. A plastic surface on the other hand is far more slippery, and by coating the wood in a resin, you are effectively turning it into a plastic piece.
Last point, although your tool maybe damaged, perhaps you can use some bodyfilller to repair some of the defects to the point where you pull a rough hull from it? Then use that hull as the basis for a new master. Also beware of undercuts.
