Ed Felten and Avi Rubin give more detail at Felten's blog, Freedom to Tinker, and question whether it makes sense to build voting machines based on commodity hardware and operating systems due to these risks. This certainly seems like an application where you'd want hardware-enforced verification of a stripped-down trusted computing platform.
Hursti's report says that there are three layers of software in the Diebold machines: a boot loader, an operating system (customized Windows CE), and an application program (the voting software). Each of the three layers has backdoors which allow bypassing security controls. The report states that "Different files on the system carry various subsets of the following features: Signature check, mode check, and integrity check. None of these can be considered security features against tampering. For example, the integrity check is [redacted]. This check can be equated to a very crude spell-checker. It is effective against accidental typing errors but not deliberate attacks."
The redacted portion, based on the description, is apparently a weak checksum such as CRC (cyclic redundancy check), rather than a cryptographically stronger checksum like MD5 or SHA1 (both of which have weaknesses of their own).
The Hursti report describes how an attacker could exploit the weaknesses at multiple levels to prevent the removal of malicious code. One such flaw (the details of which are redacted from the report) is that inserting a standard PCMCIA memory card into the machine containing a file with the appropriate name will cause the boot loader to reflash itself, installing the code in that file as the new boot loader on the system. As Hursti points out, "Due to the fact that the boot loader is the primary mechanism for its own reprogramming, if the boot loader is compromised with a deep attack, using the boot loader itself to install a known clean version of a boot loader is no longer a viable option as a recovery path to clean the system."
The report goes on to show similar flaws in replacing the operating system image, and points out a voter-accessible hidden button (labeled "battery test") that could be exploited by malicious code as a trigger for an attack.
The recommended defense against attacks is to physically protect the machines--as a machine can be compromised with less than five minutes of physical access, chain of custody evidence must be maintained from the machines' origin to final use, with no unsupervised access.