I have no experience here, I am just thinking out of the box.
Paint the edge of the acrylic black. This could be done by using a thin coat of paint on card or dense foam, then run/rub the edge of the glass along the paint bearing surface. Do this for several coats.
Once done, a few coats of paint used on the superstructure to match.
You may be able to achieve similar, by using a goop made from black acrylic and solvent, instead of paint. Running the windows' edges across a very thin smear of goop could give you a consistent thickness along the edge. This would then fuse to the edge of the window. Maybe even a little ridge build up that may or may not be wanted, but can be trimmed with a fine blade and polished smooth later.
A similar effect could be done by wrapping the edge in black tape, then finely trimming. This would be fiddley, and paper tape may be easier to use, but care would be need in sealing it from water, before a final paint in the same color as the superstructure.
Maybe try cnc cut clear acrylic windows and cnc cut black acrylic frames. With the frames cut to fit around/over the glass.
The frames would be weak, but once glued to the windows, would be strong. The frames could be made from a slightly thicker material. But this probably won't work at 1/32.