Yup,It doesn’t sound faulty though, I’m pretty sure a replacement would do the same. It doesnt automatically connect the ground of your audio signal as that would introduce a ground loop if using an amp, so instead you need to connect the ground to something else. This can be common in other devices that can be used with headphones, like the Strymon Iridium (they have a FAQ entry about it)
"Noise Performance
Iridium was designed as a an amp modeler solution in a pedal format that you can drop into any pedalboard. Like most other guitar pedals, it receives its ground connection via the 1/4″ output jacks. When listening to Iridium through the headphone output and using a wall adapter, there is no connection earth ground like there is with an amplifier. This causes some noise (hum) and is usually reduced when you ground the guitar by touching the strings. The adapter that is included with Iridium significantly reduces this hum when compared to similar parts. If an amplifier with an earth ground connection is available, then, the hum can be eliminated by connecting the LEFT or RIGHT OUT from Iridium to the input of the amplifier. The amplifier doesn’t need to be powered on for the ground to be provided. Power supplies like the Zuma and Ojai have much better filtering and isolation which will improve the noise performance"