Hi Beeky
There is no formula, as such. You can have much larger than, or smaller than, standard sail areas.
However, there are other things to consider.
Do a search on this board (Dry dock, yachts and sail) and on the R&D board for yachts and sail
for 'centre of'. There are several consideration to do with 'centre of lateral resistance', 'centre of area' (a.k.a. 'centre of effort)'.
Also search for 'lee helm' and 'weather helm'.
Why no formula?
Because there are too many variables.
Hull shape, keel length, keel area, keel bulb mass, rudder size.
Masts, mast height, mast position (fore/aft), number of masts.
Bowsprit,
Sails : The ratio of canvas in front of the mast compared to to behind the mast.
You can buy books on this. But I recommend you go for trial and error.
So what can go wrong:
-Mast too high, the wind will knock the boat over or she will 'broach'.
-Mast too low, she won't reach her maximum speed (but you won't be racing).
Everything else is down to tuning.
NB every real boat changes its sail plan to match the weather, and so there is no best sail plan, they are changeable.
So, where do you start?
1/
Find a picture or silhouette of and similar boat and compare length of hull to hight of mast. Work out the ratio.
This gives you a mast height that won't be far off.
2/
Buy 1.99 shower curtain and cut out some rough sails based on the silhouette you used before.
3/ Put it on the water and tell us what happens.
4/ We can then suggest how to tune your rig.
Questions
Can you see where the mast should go?
If so, is it one hole or many?
Can you see if there used to be a bowsprit?
A photo of the deck would help.
PS
Sorry for the long delay in replying.