In runners are for direct RPM .. and also require you to cool the motor and the esc..
A near direct replacement for a 540 motor would be a 1000KV-1500kv Outrunner.. which only require the esc to be cooled.
Outrunners generally require a much smaller ESC then inrunners (which is also far cheaper) outrunners also give you the torque u need to throw larger props, inrunners in boats generally only can drive subsurface props fast and can struggle with overheating with submerged props, although not impossible to use effectively.
The best way to work out what size motor you need is to know roughly what sort of speed you want. to go, reasonable fast, you would want around 8-9,000 rpm on a x40-45 prop.
The maths bit is... the Motor KV x Voltage (battery u want to use) = the RPM
Some outrunners also DO not Require the use of lipos... some of them the motor draw is so small that you are able to use a NHMI 7.2 or 8.2 pack!
Ive other projects that use a 1100KV 28mm outrunners running off a 25 amp esc in a 7.2V Nhmi on a x35 prop and i get great performance in a boat that's 500mm long.
One of my current projects i'm using a 1500KV Overlander Thumper outrunner on a 30amp esc. (which can use front mounting brackets) which will give me around 11,100RPM on 7.4v 2S lipo
The boat is 800mm long. so a fair size ( i've used this motor before in another boat of similar size an it goes like a bat outta hell!)
First 2 photos are my current project. the last photo is a previous one.