Actually, in reality, I hear EMP blasts are more likely to affect low power devices, such as silicon electronics. There's a reason many vacuum tube computers were kept operational during the cold war, even well after silicon based ones where much smaller, faster, and cheaper to operate. The vacuum tubes operated at over 100V (from what I heard), and were less susceptible to EMP.
I believe the ENIAC was designed to operate at 200V, with a safety factor of 2 for all components. (So they would still function correctly if operated in the 100V - 400V range). The reason for the needed tolerance, was partly because the machine was going to be so much larger than anything previously built, and thus the probability of a single component failing (and bringing down the whole machine) was much higher. Some estimates at the time suggested the machine would have about a 50% uptime.
But yes, in short, higher voltage means less chance an EMP wave will affect it. The reason I believe was that the changing magnetic field passing over the circuitry would induce a voltage in the wiring, and if the device was designed to operate at low voltages, this would cause the components to burn out. If the device was designed for high voltage operation however, it would be less likely to cause damage. If the induced voltage is comparable to the normal operating voltage, then it might cause some 0s to jump to 1s, which may lead to incorrect results, or the machine crashing, but a simple reboot would fix that. If the induced voltage wasn't very comparable to the normale operating voltage, than it might not even interrupt the execution of the machine.