DefenseWall Test -- Overall
OVERALL
________________________
Results and Ratings :
* First part : 94 %: Excellent.
* Second part : 71.5 %: Very good.
* Third part : 23.5 %: Not sufficient.
Rating threat by threat :
The result may often depend on the user's configuration : what is trusted, and what is untrusted. A simple screensaver or toolbar could be spywares/adwares as totally legitimate and innocent.
* Adware/spyware : The protection against these pests is really effective and sufficient. Specially against malwares which target browser (BHO, etc).
* Backdoors : DefenseWall does not control outbound connections.
But backdoors launched via CMD untrusted are able to open connections.
Consequently, the protection against backdoors is not sufficient.
* Keylogging/spying : The protection is good against common and well known windows hooks used by keyloggers (commercial or not).
But we have shown that some messages hooks are not intercepted (case of some tests also for System Safety Monitor, for instance).
* Trojans/rootkits : The effective registry protection and kernel guard provides a strong protection against common trojans and rootkits : Since the malware writes in the registry or needs to load a driver, it will be blocked and prevented by DefenseWall.
This is another story for server side trojans (DefenseWall provides only a protection of the local host/desktop) or possible boot sector rootkit like Eyee Boot.
* System's integrity : Since applications are run as untrusted, DefenseWall provides an effective integrity protection.
Unfortunately, there's no integrity checking of system files.
* Attacks/zero day : DefenseWall is mostly a prevention defence software. If exploits, attacks, remote code execution are committed, their impact on the system will be very limited.
On the other hand, we can't be sure of the impact since the exploit or the attack is able to run.
Assessment
DefenseWall was designed to provide a classical HIPS protection as a kernel guard, but without their traditional inconveniences and drawbacks : Popup syndrome (fatigue) and the serious knowledge often required (system's processes, malwares behaviour).
From this point of view, DefenseWall is really a success.
When some HIPS editors only release products which are clones of other ones (Prosecurity and SecurityTask 2005 are Process Guard clones, for instance), Ilya Rabinovitch, an ex nuclear physicist reconverted as a programmer, has taken the door of innovation : Policy and kernel restrictions, virtualization, and a kind of sandboxing are integrated into DefenseWall.
Moreover, the white list protection is really interesting from many criterias, and appears also as my favourite concept.
With DefenseWall, the user just needs to understand that trusted applications do not mean legitimate applications, and that all infection vectors programs (especially those which require a connection) should be configured as untrusted.
Once this idea is assimilated, the user can surf, play video games, communicate via chat and instant messaging, he's still protected by DefenseWall without being disturbed by an avalanche of popups : If an unknown application runs, its impact on the system will be limited.
And most of all, DefenseWall seems a good choice for P2P users : Since an application is downloaded from the P2P network, it can be run as untrusted, and the user has the ability to reverse the changes that may be done with the rollback option.
Therefore, DefenseWall can be recommended to beginners and classical users, who use their computer for entertainment.
On the other hand, advanced and experienced users will be a little bit frustrated by this product : As the configuration possibilities and execution protection are limited, the user can't get the control of activities and apply his own policy (like blocking an execution, instead of running it as untrusted).
But this little inconvenience can be easily reduced by combining DefenseWall (white list HIPS) with anomaly detection HIPS (application firewall/behaviour blocker) like System Safety Monitor, Antihook, Online Armor, Safe'n'Sec, AppDefend, etc.
Moreover, even if sufficient, the protection against keyloggers does not cover the largest possibilities.
And the protection during the boot appears very limited, since DefenseWall does only start with Windows.
Some restrictions regarding system's applications, like Cmd, should be hardened.
During these tests, I've noticed by some mail exchanges that Ilya Rabinovitch is little bit fanatic (very involved by contesting the failed results of part 3, for instance) about his product.
But this is here more a quality: firstly it's an indication of professional behaviour; and secondly, when most products (firewalls, antivirus, some HIPS) are redesigned once a year, DefenseWall is continuously updated.
The efficiency is more on the table draw than in marketing!
Finally, if we take into consideration the good support and value for money, the features of the version 2.0 and the warranty of constant improvements, we can consider DefenseWall as definitively excellent.
Kareldjag
_________________________________________________________________________
I've only few comments to add to this presentation, since I won't repeat what was said by Kareldjag.
Regarding the results of theses tests, DefenseWall did perform very well. The only area where this program had average results is keyloggers, with 4 tests passed on 7 tests.
And I must say the protection concept offered by DefenseWall was quite upset by the methodology used for some tests:
* Several tests were made with DefenseWall disabled. Or, the program doesn't pretend to protect from changes made during a time where it was disabled (incidentally, in previous versions, prior to 1.40, there was no "disable protection" option).
* DefenseWall doesn't claim to replace your antivirus and firewall, but to complete their protection. That's why the program had bad results in the tests involving connections : DefenseWall doesn't control connections, then tests about backdoors were lost in advance. The author of DefenseWall does consider it as the job of the firewall (no need to overlap with FW). Which is right, but this statement rests on the presupposition that the user has a good firewall, could we add.
* Finally, DefenseWall didn't perform very good in the tests about exploits. I've inserted a small note on this matter in part 3, explaining that running the exploits (I mean the parent process, and the files downloaded thanks to the exploits/the payload) as untrusted was indeed DefenseWall's way to protect against it. Then, this is not a flaw, but a feature : by design. Therefore, the "failed" status of theses tests about exploits must be appraised accordingly (see in particular the test on the "warez site infection" for a good illustration).
In the light of theses necessary comments, we can say the overall tests results of DefenseWall are very good. Especially for a program protecting the user without prompts or popups. The target publicc the author of DefenseWall had in mind when designing his program is typically the "average user", not only the "security geek". The goal is obviously reached.
Of course, there is always room for improvements, it could be even better - and some features announced for next versions will make it better without a doubt.
But the main quality of DefenseWall is for me the ratio between ease of use and efficiency of the protection, which does imply a clever balance : This program is establishing a very high standard on this point.
For example if some features like control of process executions and connections were added, the program would be more difficult to use for the "average user". Let's see how the program is improved in the next months : We can trust the author to preserve this fine balance I talked about between ease of use/efficiency.
Few improvements desirable :
* Something like an "expert/advanced mode" , with more user interactivity : For instance to be notified when a new/several program(s) are running untrusted (Useful to detect sudden malware activity).
* A way to check the status of Files/folders with the contextual right-clic menu (For now, we've to open the "Add/remove untrusted" panel to check if a file/folder is untrusted or not.
* A more convenient logging : During these tests, I've noticed that opening Internet Explorer was enough to fill up the log size limit. If this size limitation was done on purpose, I consider it can prevent the user from being informed about important events : For instance, if malware is or was running. Such events are "pushed out" from the logs by IE-related entries. That's a kind of information-loss.
For Kareldjag :
* An option to block creation of threads or executables (known or not, trusted or not) from running : This option can be helpful against infection via web sites and against zero day attacks.
* Boot start service/driver, instead of system start only.
* Integrity checking (MD5/SHA) of applications to prevent malicious changes (almost for trusted applications!).
* A sound which warns the user (like most AV, and System Safety Monitor) for an untrusted processes and events.
* The killing option should be enhanced (even if already effective).
nicM and Kareldjag
________________________________________________
Ilya Rabinovitch's comments :
First of all, I need to note that DefenseWall HIPS has been designed not as a standalone product. It has been designed as easy in use and strong in defense HIPS part for "protection-in-depth" conception for the amateur, non technical user. This conception means that anti-virus detects already known malware modules, firewall controls Internet connections and its trusted processes against browser hijack, for instance, HIPS blocks unknown by anti-viruses malware activities, prevent them from being correctly installed into the system and reduces its possible damages level.
So, according to this conception, DefenseWall doesn't need to control network/Internet connections, control taffic and protect against browser hijack :It is typical firewall's job. Anyway, most of the people whi use DefenseWall already have good firewalls purchased, and I see no reasons to make them buy network traffic control system one more time.
As the part of the layered defense system, DefenseWall is implemented to be simple and easy in use, strong protection against both known and unknown malwares. It is very important for the user-friendly defense to be balanced between being simple and non-irritating for the amateur, non-technical user, and the defense level itself. As for DefenseWall, there's no signature scan, no constant databases update, no popup windows with technical questions, no false positives. Just reliable, simple and transparent defense, very light on resources.
DefenseWall is 100% driver level defense, it uses no user mode hooks and then can not be stopped or bypassed from the user mode level. As you may see, according to the tests, it makes its part of the security job as good as it should be but keyloggers section. Unfortunately, Windows NT/2000/XP/2003 has a huge security design flaw against keylogging, which can not be compensated from the driver level (all the user level attempts to make it could be easily bypassed by malware). In fact, non of the HIPS will be able to protect you from being keylogged with this security flaw. The only salvation is termination of all the untrusted processe with DefenseWall's "big red button" before you go banking online, or type something important.
All these tests were made with DefenseWall 1.40. Now the current version is 1.60, there are lots of fixes and improvements here (WMI process termination/ creation control, additional defense against ransomware and so on). If you like DefenseWall and you have any ideas how to improve it - you are always welcome to share it with all the DefenseWall community. If you have found any bug - mail to support [at] softsphere [dot] com or post it to the support forum
( http://gladiator-antivirus.com/forum/index.php?showforum=192) immediately, all the bugs MUST be fixed as soon as possible, they are not in option!
________________________________________________________________________
Pros :
* Innovative product based white list concept and using virtualization technology (in vogue also in corporate environment)
* Ease of use : No popup fatigue and no advanced knowledge required
* Efficiency of the kernel (service/ driver installation, physical memory) and registry protection, which blocks most prevalent malwares (spywares, trojans, rootkits, etc)
* Rollback option which allows the user to reverse changes made by known or unknow applications
Nb : This feature seems unique in the market of HIPS for end-users, even if products like ViGuard can recover clean files after a viral/ malicious code infection.
* Integrated in Windwos explorer as a shell extension : The user can run any applications as trusted or untrusted, very useful while logged as an administrator or for suspect files downloaded from P2P networks
* Interesting account protection (not common on personal HIPS)
* Very reactive editor : I've reported a security issue in march, at the beginning of these tests, and this issue has been fixed the same day, and the product updated the day after
* Good support (forum available)
* No excessive marketing at all : The product does what it claims
* Good vaue for money (between 30 $ and 23 €)
* Editor is listening carefully to users feedback, and does add new features very quickly when users have legitimate suggestions (NicM)
Cons :
* Limited execution protection : Unknown applications are allowed to run as untrusted : Unfortunately, since a code is allowed to run, there will always be a security risk; even if its impact on the core of the system is limited by Defensewall's protection
Pure white list HIPS like Anti-Executable blocks any unknown file : No chance to run, no possibility of infection.
On the other hand, this kind of features is very restrictive, and instead, Defensewall provides more freedom but less possibilities of malicious impact.
* White list concept of trusted/ untrusted zone may be confused and difficult to understand for beginners : A trusted application does not mean a legitimate application (browsers, mail clients, etc) : More infection vectors are run as untrusted, highest is the level of security
* Only a system's start service/ driver
* Limited protection during the boot : DefenseWall works only with a system's start service/ driver, and does not provide protection when Windows starts
* English language only
__________________________________________________________________________
Acknowledgements :
- To nicM for his implication during these tests,
- To Ilya Rabinovitch for the compilation of 3 test files,
- To Todd, an english native language for his proof-read (for some typo and ortography mistakes).
- To Kareldjag for his help during these tests,
- To Ilya Rabinovitch for his helpful assistance during these tests (always replying to questions on some DefenseWall protection aspects, providing informations, etc),
- To Elodie for her help during translation.