Excessive hiss is usually the result of too much gain and not enough signal- ie. SNR, and another possibility is the input impedance isn't optimized for the impedance presented by the pickups.
If the pickups are passive, you'll want to aim for an input impedance of at least 500K to 1Meg, or higher. A fet input opamp would be the best choice here. If they are active pickups, a bipolar transistor input tailored for a low source impedance will yield the least noise. Should the pickups be active, any external preamp will now also be amplifying the self noise of the preamps built into the pickups. When too many gain stages need to be attenuated to keep the signal down to a usable level, the noise accumulates with every subsequent amplification stage.
The battery and caps have little to do with this noise, unless some of the coupling caps are completely open. I would be willing to bet it's just poor design or the wrong application for this preamp.