Ok, before I start, I value SANS (GIAC) certifications a lot. I have learned an enormous amout of stuff while getting ready for GSEC and GPEN.

But, as some of you know, I am not a great expert at this. I have been studying non stop for about 2 years in IT security, but previous to that, I was just an humble web app developer.

So I manage to write two SANS certs WITHOUT taking their courses. In fact, other than for PWB, I have never taken a course in IT security. I did ok for GSEC and pretty good for GPEN.

I am posting on this topic because I was looking at certified GWAPT people. I was astonish by their marks (see attached picture)! Maybe a lot of them failed and we don't know about it, but the average mark seems to be around 90%!!

So the fact that I don't have much experience, I passed two of them without taking a course and that the average marks are pretty high make me wonder if it shouldn't be a close book exam...

But that being said, I worked pretty hard to get ready nevertheless...

What are your toughts on that?

Hi Dark Knight.
I think you're right; you can now do GIAC certifications without writing a paper, and that gives you the "Silver" type of certification.

Gold requires you do a paper - see http://www.giac.org/gold/

There's also Platinum/Expert - http://www.giac.org/gsx.php

The GIAC exams are not that difficult but part of it stems from the quality of the questions. Too many certification bodies think they need to be ambiguous or try to trip up the students with the wording of the question. GIAC doesnt do that. If you have a question with the output of a packet and the question asks you what the byte offset is for the beginning of the payload it should be pretty clear what the answer is if you know the material.

Also SANS does a really fantastic job at immersing the student in the technical concepts needed to succeed at the exam. I understand that you managed to self study and succeed, but how typical is that really? Making it a closed book exam would lower success rates, but when did you have to solve a real world problem and you didn't have google or at least man pages to help you out? Rote memorization does not prove anything at all beyond good memory, its the concepts that are important to understand and without that understanding the books won't help much.

The GSE does have a practical component and I don't think anyone can say that's an easy process to go through. I'd like to see GIAC adopt more practical components and more platinum level certifications. I think it would really add value. For instance, if you passed GPEN, GWAPT and GAWN, sit for a practical exam (GPWN maybe?) that requires blended attacks to succeed at a set of objectives and then write a report. Maybe include a scoping exercise in there as well. Pentesters should have to demonstrate that they can work with the target organization to define scope and help guide them when they don't know what they want (which is pretty common) by stepping back from the system level and focusing on critical or sensitive business processes and figuring out what systems support those processes directly or indirectly or factor into protection mechanisms. I don't know of any certifications out there that validate these skills. GPEN asks a few questions and covers this on day 1 of the course, but I don't know that those skills are really validated.

The problem with these certs is they test theoretical knowledge, and they test your ability to recognize when a technical answer is the right answer, but they don't test your ability to come up with a solution to a technical problem unless you have a practical exam or a paper. Anyone skilled at multiple choice exams with a understanding of the material can pass and succeed.

The Gold cert with a written paper is a great option but there's very little incentive for students to pursue that unless HR folks start asking for it. GIAC has changed recertification requirements in the last year to allow for an upgrade to Gold to allow for recertification (or take another SANS course) which is nice and provides some additional incentives. I think it was a mistake from a credibility standpoint for them to remove this as an option, but SANS/GIAC is a business, and the barrier for entry to their certifications was just too high before they removed that requirement. I don't know for certain that this was why they made the change but I suspect it was financially motivated. SANS is not cheap, but there's no questioning the quality of the training.

That's it right there Doing the GPEN AFTER the OSCP. So you essentially had a head start going in.

Another point to note is that SANS not only has very good training materials but the support is also top notch. In fact for want of a better phrase the student is 'almost' spoon fed. They go above and beyond to facilitate the student. So in the end you really have no excuse to fail or score low.(I need to remember this when I do the GCIA )

ON the issue of the grades you posted, the flip to that is we do not know how many people failed the exam or didn't bother taking it at all. Imagine not coming from a developer background and not having ANY experience in web app testing.

I would imagine that in that case the person would find the GWAPT a bit challenging no?

To be honest, from everything I hear OSCP sets the bar really high. I have not done OSCP yet so can't speak from experience but am very familiar with their "Try harder" mindset as I have seen it frequently on irc and in their forums. So to expend as much effort as you did on OSCP whether you passed or not meant you had to do a lot of research and learning on your own. You obtain a MUCH stronger command of the material when it isn't spoonfed to you. SANS doesn't make you work hard for the knowledge, and consequently if you don't start using it as soon as you get home your retention will probably not be that great. The bonus here for SANS training is there is such a tremendous amount of information and it's explained in such a way that you really gain an understanding of the underlying technologies. I feel both formats have tremendous value, and are very complementary.

Something else you have to consider when looking at those high scores is the caliber of students attempting the certification. SANS is very expensive and few people not already working in the field can afford to attend. The same cannot be said of OSCP since the financial barrier for entry is much lower. I'm not sure if that's good or bad but it is a possible variable when calculating these statistics. (I'm not suggesting that OSCP students are less capable, but I personally feel that many may come to the program with less experience and wind up attempting something that is probably a bit more difficult. I don't mean that in a bad way.) Also by GIAC giving everyone who attempts certification 2 practice tests, that's just additional preparation. Those practice tests are VERY representative of the test.

My thought...

I've taken the GSEC course (did not test) and self-studied for GWAPT (I'm somewhere in the stats you posted  ). Self-study works best for me (to constantly revisit concepts) to be successful in gaining the cert and applying methodologies.