Where do you stand on this issue? I really like the way CR did their testing and I think it benefits end users in the long run by showing how well/poor a given AV software performs. As long as they can keep the code private, which is questionable. It would also be a good idea to submit the code to any of the AV vendors that  didn't detect it, so they can update their detection libraries.


Consumer Reports creating viruses?
Posted: Thursday, August 17 at 08:42 pm CT by Bob Sullivan

Consumer Reports recently conducted one of the most thorough tests ever of antivirus programs. But to really put these security programs through the paces, the magazine hired a firm to create 5,500 new viruses, using them to test the antivirus software products for their ability to detect unexpected threats.

Now antivirus companies are crying foul, saying the magazine ignored a long-standing principle not to invent new viruses.

"Creating new viruses for the purpose of testing and education is generally not considered a good idea,” wrote Igor Muttik of McAfee's antivirus lab on a public company blog this week. “Viruses can leak and cause real trouble." The entry helped touch off a firestorm.

....
http://redtape.msnbc.com/2006/08/consumer_report.html#posts

I'm going to assume the 5,500 number is hyperbole, and that most of the viruses were variants of one another. If you create a new virus it will initally defeat signature based virus checkers, that's the nature of the beast.

I'm not a big fan of the heuristic approach to antivirus because I've yet to see it work effectively. Defensive software that limits the functionality of unknown executables I believe is much more effective but is inherently limited in that it alters the fabric of the operating environment.

I'd sooner see more lab research than waiting for the bad guy to raise their game.