Presentation of threats used in these tests:
As said previously, we can't be as exhaustive as possibe: only samples of malwares and attacks are used.
It is really statistically enough to test the efficiency of an HIPS.
-adware/spyware: classical spywares like BHO are used, with some prevalent ones (180search assistant for instance).
Unfortunately, many pieces of code (like cookies) or flash advertising can be considered as spyware/adware.
-backdoor: this kind of malwares is more intended for testing firewalls or antivirus databases.
But as some personal HIPS provides firewall features (outbound connections control), we have included samples of backdoors in the methodology (usual protocols like TCP, unusual like ICMP), basic method (basicbackdoor) or more advanced ones (BackShell).
When the tested HIPS does not have connection filtring feature, the result are moderated; and not taken into consideration for the final estimation.
In the same way for attacks/zero days.
-keylogging/spying: here we often use classical methods for spying on a computer: commercial or free keyloggers.
Most of them use well known message hooks.
Server side keylogging via virtual keyloggers for instance is not integrated since most personal HIPS provides only a local/desktop protection.
Most of all, we can't cover all spying possibilities: keyloggers (softwares, hardwares, virtual), keyboard sniffing, via webcam or micro camera sloted in the keyboard, screen emanation (Tempest), keyboard sound emanation, VOIP recording ...
Searchers of the MIT are working on spying via smart dust (nanothechnologies): the limit of espionage is only the limit of the human genius.
-trojan/rootkit: here we use trojans which write the registry and rootkits which need to load a kernel driver.
An advanced POC of boot sector rootkit (more a backdoor) is also integrated.
-attack/zero day: most personal HIPS are not designed to prevent attacks (like buffer overflows) and zero days.
Here we use recent (this methodology has begun to be released in january of this year) vulnerabilities and some local (privilege/shatter) and server/side attack (Man in the middle, URL obfuscation).
It seems technically impossible to detect all known and unknown attacks: there's a frontier between the marketing and the limits of security.
Even if specialized products provides an interesting protection against these threats like CoreForce, SocketShield for home users environment or Sanctuary for a corporate one.
-system's integrity: here we reach the limit of HIPS (personal or corporate).
A system is like a human body: changes occur each day, normal or pathological; legitimate or malicious.
Since changes are necessary (updates, softwares installation etc), the integrity of the system depends also on the human factor: a screensaver may be innocent and legitimate, or a trojan, a spyware or anything else; an update can be corupted etc.
Before being a variable of product, the security is here almost a variable of environmnent where human factor is in the heart of the system.