Chapter 13
Don’t Fear the Assessor
Information in this chapter:
The title of this chapter might shock you a little bit. Why? Have you noticed that the words “audit” and “auditor” in reference to PCI DSS are copiously missing from this book? That’s because the correct terms are “assessment” and “assessor” when referring to PCI DSS. While your QSA may be a CPA, it is not a requirement, and most QSAs are not; instead more come from IT domain. The procedures an assessor uses to validate your compliance with PCI DSS are called the Security Assessment Procedures (not the Auditing Procedures). It’s amazing what the change of a word will do to get you a more complete assessment. Imagine if your Internal Audit group changed their name to the Primary Assessment Group, and everyone changed their title to Assessor.
Sure, it’s a psychology trick, but part of the goal here is to use the right terminology with sound advice to follow it. Your internal audit group should be involved in the PCI DSS program from a self-assessment perspective, but remember, PCI DSS is assessed, and you work with assessors.
Whether it’s your first on-site assessment or your first vulnerability scan, it’s pretty easy to find gaps to compliance. And while this may not be the case for you, you should have a plan in place to deal with this if it happens. This may happen because you interpreted a requirement slightly different from an assessor, or it may be that you simply missed something that an experienced assessor would catch. When things go wrong, it’s easy to blame the assessor. Having the right attitude came make all the difference. Generally, assessors should not allow you an easy pass unless your environment is truly one of the best managed and secure. If they do not properly report real gaps in your PCI DSS compliance they can lose their QSA status. In addition, your company will be dragged through the mud after the breach as yet another example showing how ignoring PCI and only paying lip service to it will get you not only breached, but also fined by the card brands.
Remember, Assessors Are There to Help
When dealing with on-site assessors or approved scanning vendors, most people fit into one of three groups.
1. Some people are intimidated by assessors. They see assessors as people with a lot of power, and they hope they will say and do the right things, avoiding the pain of a gap.
2. Some look at assessors as their enemy. They believe they must wrestle with the assessor and hopefully win in the end.
3. Some people treat the assessor like a consultant (“a mandatory consultant” as the case may be) they’ve brought in to help bring their company into compliance. They respect the assessor’s opinions and keep the assessor in the loop as they work out solutions.
While it might surprise you, the last group will get the most out of their assessor and will have the best overall experience. They will quickly be able to bring their company into compliance with the least amount of hassle, and will usually gain quick upper management support for funding required to resolve the issues identified during the assessment.
As hard as it might be to believe, assessors are there to help you. After all, you are the one paying for their services. It’s important to know how to work well with assessors so that your assessment will go smoothly and efficiently, and ensure that you get your money’s worth. A good assessor will go over your company’s systems, practices, and policies with a fine-toothed comb, and tell you what you can do to improve your security. Hopefully, your primary goal in becoming PCI compliant is to have your company become more secure and decrease the likelihood of the card data theft or loss. When you realize that assessors provide you with a valuable service and that you’re both on the same team working towards a common goal, you will have the right attitude. Remember that assessors have moral and professional obligations to follow the guidelines and procedures they’ve been given for the assessment, despite the fact that you are paying for their services. It is not appropriate to ask them to compromise those obligations, just for that reason. The assessor’s integrity is more important to them than your fee. Assessors are trained and likely have performed many assessments, and they can give you great advice on what you can do to bring yourself into compliance.
When you choose your assessor, interview them! This assessment is probably one of the most important security projects you will embark on this year. Getting out of the gates with the wrong assessor will make your pain so much greater.
First off, set up an interview with someone from your prospective assessor. Understand their methodology, and how they will strive to ensure you are not one of the companies that exited their assessment process thinking they were compliant, only to find out (sometimes DAYS later!) that they had been breached, and their assessor did a poor job (sadly, there are some example of this very event, which are well-covered in the media). Good assessors will bring a team of at least two QSAs to every engagement to make sure you get the most accurate result. Good assessors will come on-site and will not just interact with you via email. Have you ever tried to explain to someone how to build a Lego Millennium Falcon over the phone or, worse, email? Of course not. So don’t expect your assessor to get a good view of your infrastructure over the phone, either. Finally, you get what you pay for! You don’t always need to choose the most expensive bid, but if you get bids of $20K and $200K from the exact same scope, something is not right. Don’t get stuck in an apples to oranges comparison. You will end up with prune juice every time! In many cases, the cheapest assessor will end up being the least competent.
When you have the right attitude you will find ways to use your assessor to improve the security of your company. Seasoned assessors have a wealth of knowledge, even outside of payment security, and can be leveraged when bridging gaps in compliance. They have seen many technologies, policies, and practices others have put into place to mitigate risks, and should be able to give you choices to help you meet requirements that work best for your situation. For example, if cost is your main concern, an assessor may know of a low cost or open source tool that you can use to comply with certain requirements. On the other hand if time is more important, the assessor may know of a solution that is quick to set up that will bring you into compliance. As you work on your remediation, it’s important to keep your assessor in the loop. This way he can give opinions on what you’ve chosen to do and can give further advice. It will also likely make your next assessment much easier for both parties involved.
Don’t forget the old business adage… Pick any two of the following when asking someone to provide you with a good or service: good, fast, or cheap! For QSA services, picking “cheap” with either of the remaining two often leads to spectacular (and costly!) trouble later.
Balancing Remediation Needs
Do your homework when looking at ways to bridge compliance gaps. Depending on the problem you’re trying to solve, there may be open source tools, managed solutions, off-the-shelf software, or hardware appliances to consider. When looking at products and services that can help bring you into compliance, there are usually four main factors that you should consider.
• Effectiveness: Will the solution you’re looking at really solve the problem and allow you to pass your next assessment? If it won’t, it should be ignored.
• Cost: Normally cost is a factor in any decision made by a business. Sometimes decisions are based solely on initial cost, but costs of maintaining should also be considered. While one product or service is cheaper up front, it may end up costing your organization much more in the long run.
• Time to Install: You probably want to get into compliance quickly. If you’re not in compliance you may be facing fines, but will definitely have gaps in your security begging for a hacker to exploit.
• Time to Maintain: Many times, the time used to maintain a product will be the most expensive part of adding it to your organization. It may end up that a solution you choose will be more expensive in the long run, because it takes a lot of time to maintain (i.e. purchasing a log management solution instead of outsourcing the management to a managed service provider).
Depending on your exact situation, some of these may be more important than others, but they should all be considered when choosing a solution.
How FAIL == WIN
In some cases, failing an assessment ends up being a huge win for the security of your company, and not even for just payment security. In many organizations, the security staff (or security minded IT staff) would like to put certain security measures in place, but have been blocked by upper management because of cost or the notorious “other priorities.” Remember, upper management’s job is to help the company make money, not spend money. Even after you have done a careful cost-benefit analysis and have determined that the benefits outweigh the costs, upper management may still say “no.” A failed assessment may be the perfect time to help them to say “yes.” If the assessor is requiring that you add something to comply with PCI DSS, you can use that as leverage with upper management to get it put into place. Again, submit a cost-benefit analysis, adding the cost of noncompliance to the total cost. Let them know that the assessor says you will not be compliant without that measure.
Dealing With Assessors’ Mistakes
Assessors are human. Humans make mistakes. Thus, assessors make mistakes. While this does not happen often, there is a right way to deal with it when it does. The first thing to do is to talk to the assessor and have him explain how he came to his or her conclusion. Many times someone misunderstood a requirement or believed a compensating control mitigated a problem, but the assessor doesn’t agree. Having good open dialogue about what you believe is a mistake, will often solve the problem quickly.
Many assessors find their roots in security (some find it in auditing, which can, on occasion, make this more difficult). Sometimes an assessor will “make up” a requirement because it just makes good sense, but when you ask the assessor to show you where in the PCI DSS that requirement exists, they will realize their mistake. Notice how that last sentence was phrased. Ask them to “show you where the requirement is in PCI DSS,” don’t ask them to “PROVE IT!”
Before you go to the step in the next paragraph, consider why you think the assessor made a mistake. Is it because you personally have an attachment to a particular system or control? Was the assessor rude? Have you tried to fix the issue the assessor identified with no success? Or was he trying to sell you something that his company happens to offer? Assessors make mistakes, but don’t assume that because an assessor feels a certain way that he is alone on an island where no other assessor would dare sail. Before pushing back on the assessor, ask yourself this: “If I were breached tomorrow, could this be a cause?” If it is, don’t waste time bullying the assessor. Take some time to research and look up the issue; don’t argue because you happen to dislike his advice or don’t want to deploy a particular technology or change a certain “bad habit” of your organization. Use that energy to fix the issue and close the issue that an attacker may use to break into your system.
On the other hand, if you know that this does not increase your risk and do not seem to be expressly spelled out in PCI DSS or supplemental guidance documents, maybe the argument should be continued and you need to push back further.
NOTE
You may feel like you have compensating controls in place to solve a problem but the assessor doesn’t agree. If the assessor does not agree with the control and you are a merchant, try working with your acquirer (sorry service providers, you do not have this option). If your acquirer chooses to accept the risk, they can absolutely do so. Most acquirers will side with an assessor, so be sure if you want to go down this route that you present a good case. Most of the time it’s easier to follow the requirement exactly than to try to get a mitigating control to fix the problem.
In some cases, you may need to push back on your assessor. Pushing-back is when you challenge an assessor’s results. When you push back, be polite. Simply explain to the assessor your point of view and why you believe there was a mistake. If the assessor disagrees, ask him to explain his reasoning. If the assessor has explained why you didn’t pass and you don’t agree with his reasoning, you may need to talk to his manager or a practice lead about the situation. Explain your situation to the manager and why you think a mistake was made. Most of the time, the manager will talk to the assessor to get his side of the story before coming to any conclusions. If the assessor’s manager agrees with the assessor, you will need to fix the problem to be validated as compliant.
Sometimes an ASV scanning tool will report a “false positive.” This is when an assessment shows you have a vulnerability such as a missing patch or vulnerable system that really is not there. This seems to happen more with remote scans, since they have less access to systems. Any good assessor or ASV knows how to keep false positives to a minimum. When you do get a false positive, your ASV should be able to work through it with you; many have a mature, automated process for dealing with scan mistakes and other false positives. They may want to get more details from you so they can verify it as a false positive, and then fix the system so the error does not come back in the future.
WARNING
Don’t forget, you get what you pay for! Most ASV scanning engines today are based on either Qualys (http://www.qualys.com/), Nessus (http://www.tenablesecurity.com/), but not all ASVs will produce the same results. Interview your ASV and ensure they are involved in the process, not just setting you up in a database for scans. You may need help interpreting the results or addressing false positives, and those features may cost more.
Some approved scanning vendors only run automated tools with very little human checking. This generally works well most of the time, but sometimes the scans can be complicated, and a false positive may end up in a report. If you get a report listing a serious vulnerability, first act as if it’s true and see if there’s something you can do to remedy the problem quickly. Depending on the situation, it may be a good idea to do some tests on your own. You already have a scanning solution on-site for the internal scans mandated by Requirement 11.2, so use it against the target in the external scan report. Depending on the type of vulnerability that was reported, you may be able to do some manual testing. For example if your report says that a patch is missing, you may want to manually check the system to validate the finding. If you are unable to find the vulnerability after your testing, it may be time to challenge your scanning vendor’s findings and report it as a false positive. They should do additional tests to determine why the false positive happened, and fix the problem or remove this finding from your final PCI scan report.
Planning for Remediation
A good rule of thumb when doing remediation is that it should be as transparent as possible, so that it has a minimal impact on the business. Sometimes business or user impact is impossible to avoid. For example, implementing a much stricter password policy or disabling group accounts may have an effect on how people perform their jobs. For the most part, patches and system updates should be transparent to users. The more transparent your remediation, the fewer problems you’re likely to have implementing it. As you plan your remediation process, always keep transparency in mind.
The first thing you should do in planning for remediation is review your gap analysis with your assessor. Your gap analysis describes the difference from where you are now to where you should be to be compliant. Ask your assessor which risks he considers high priority. For example, if the assessor feels that you have urgent risks that could easily be exploited at any time, you would want to work to remedy those first. In a few cases, an assessor will find a risk that is being actively exploited. In this case, the assessor should let you know as soon as he finds the problem and not wait until the rest of the assessment is done. This would then become your top priority, and you should follow your company’s incident response plan. See Figure 13.1 for a visual representation of this process.
Figure 13.1 Remediation Process
Now that you have your results and understand what needs to be done to comply with PCI DSS, it’s time to prioritize your risk. With the help of your assessor, work to determine which problem can be exploited easiest and can cause the most damage. These are the ones that should be fixed first. If there are not any “gaping holes,” the conversation should turn to the items that you can address that will (1) get you some quick wins, and (2) give you the biggest bang for your compliance buck. There are many tools that can be used to help you classify risks, including the many vulnerability Web sites. Here are some that you might find useful
• Common Vulnerability and Exposures (CVE): This is the industry standard, and much referred to, listing of vulnerabilities in products. Many products use CVE numbers to reference vulnerabilities (http://cve.mitre.org/).
• National Vulnerability Database: Supported by the Department of Homeland Security and has a great database of many types of vulnerabilities (http://nvd.nist.gov/) with scored vulnerability severities using CVSS method.
• Open Source Vulnerability Database (OSVDB): This community run database of vulnerabilities will give you a lot of great information on a vulnerability, including references, ways to test your system, and how to mitigate the problem (www.osvdb.org).
• Security Focus Bugtraq: A well-organized site that will give you a lot of information including what versions are affected, an overview of the problem, and examples of exploits. It uses Bugtraq IDs (bids) which are supported in many products (www.securityfocus.com/bid/).
Fun Ways to Use CVSS
The Common Vulnerability Scoring System (CVSS) is a standard for scoring vulnerabilities that has become more widely used. Approved scanning vendors were mandated to use CVSS scores instead of PCI scores from June 30, 2007 for any vulnerabilities that have a CVSS score. Most vulnerability databases, such as NVD, will list CVSS scores, which are great in helping you determine the impact of a vulnerability. There are some vulnerabilities that may not have a CVSS score, but NIST provides a tool to help you calculate them, which can be found at http://nvd.nist.gov/cvss.cfm?calculator.
For example, let’s say that your report shows that you don’t have your credit card area physically secured. Since this is not a specific vulnerability with a specific system, there won’t be a CVSS score for it, but you can use CVSS to help you determine the priority.
In this example, we’ll use a physical security issue to show you how this works. While this system is mainly for computer security issues, it works pretty well for physical vulnerabilities as well.
Jeff is the afternoon manager for Teri’s Tapas To Go, a small tapas bar near midtown Manhattan. When Teri built out the location, she found certain constraints as to where electric and telecommunications wiring could be placed. Thus, she has a fax machine near the bathroom to receive faxes containing orders with cardholder data on them. Because the fax machine is not visible by Jeff (or any employee) unless he is in front of the counter, he cannot closely monitor it. There are often times when Teri’s staff is busy with customers and are not watching the fax machine. If they are too busy, they may not hear the fax machine and therefore delay checking for new orders. Anyone passing by the bathroom could easily grab a fax.
On the calculator page, Teri would start with the Base Scoring Metrics. This gives CVSS a base score to work of off for the vulnerability.
Related exploit range is where an attacker would have to be to be able to exploit this vulnerability. If an attacker can compromise the system over the Internet or some other remote means, then it would be remote. In our case, with the credit card area not being physically secured properly, it would be Local.
Attack complexity is how difficult the attack is to pull off once an attacker has found the vulnerable target. If the attack requires other factors to be in place for it to work, it may make it complex. In our case, we’ll say that this is Low complexity. Once an attacker knows where the credit card data is, it’s easy for them to get to it.
The level of authentication needed is if an attacker must be authenticated to pull off an attack. This means that there is some test to verify who the user is that must be bypassed to attack the system. An example would be something like a fake badge to get access to the fax machine. In this case, an attacker would not because the fax machine is in a public area, so the level will be Not Required.
Confidentiality impact describes how the exploit will affect the confidentiality of data in question. In our case, if they can access cardholder data by walking into a protected area and wheeling a file cabinet with all cardholder data in it out the door, it would be complete. Normally a heavy filing cabinet is pretty safe, but since Teri has faxes coming in with cardholder data and there is little to no protection of that data once it hits the fax machine. In this case the confidentiality impact could be Partial (as in you are not getting ALL of the cardholder data), or Complete (as in you did get the complete card number). For illustration purposes, we’ll choose Partial.
Integrity impact describes how the attack will impact the integrity of data. In our case, it’s not likely that integrity will be compromised, so we’ll use None.
Availability impact describes the measure of how the availability of systems and data is affected. Since the attacker can walk off with a fax, the data is no longer available, so we’ll mark that as Partial.
The Impact value weighting allows you to give more weight to confidentiality, integrity, or availability. In our case, the biggest problem will be confidentiality, because the attacker just walked off with cardholder data, so we will chose Weight confidentiality. At this point if we click Update Scores, we will get a base score of 3.7.
Next we will do the temporal score metrics.
Availability of an exploit lets you determine if an exploit is actually available or not. In our case, we’ll say that a functional exploit exists since the attack would work much of the time, but there may be times when one of Teri’s employees would catch somebody.
The type of fix available allows us to specify if there is currently any way to remediate the problem. We’ll say that Teri has asked employees to keep an eye on the fax machine, which is a Temporary fix until she finds a better home for the fax machine.
Level of verification that the vulnerability exists allows us to specify how sure we are the vulnerability is actually present in the system. In our case, we know that the vulnerability exists so we’ll choose Confirmed.
Finally, the environmental score metrics section. Here we will score the kind of damage that will happen.
Organization specific potential for loss allows you to specify the physical impact the attack could have on your systems. In our case, one credit card number stolen on a fax won’t bankrupt Teri, so we’ll say it has Low (light loss) potential for loss.
The percentage of vulnerable systems allows us to choose how many of our systems are vulnerable to this attack. In Teri’s case, this is her only fax machine so we’ll say all choose High (76–100%).
Now that we’re done, we click the Update Scores button and get an overall score of 3.9.
There are many ways to prioritize risks—more than we could review in the scope of this book. Don’t spend a huge amount of time and effort prioritizing risks, since in the end they all need to be fixed. But it’s good to have a general idea.
Planning for Reassessing
As you are working through your gap assessment, include your assessor! Not only can she give advice on how to mitigate some risks and bring yourself into compliance, she will can also help you set realistic completion dates. As you run into roadblocks, she can help you adjust the dates and remediation plan, and be there to support you through the process.
After this is done and everything is in place, plan to reassess yourself. We provide some self-assessment tips in Chapter 15: You’re Compliant, Now What? Validate that the gaps are closed to save time and money with your assessor. Provided you included your assessor during the remediation process, your reassessment should be quick and painless. Then you will finally be able to have your PCI DSS compliance party!
Summary
Don’t feel bad if your first assessment does not end in a compliant report. Instead, use it to your advantage to better your company’s security posture. Work with your assessors instead of against them. Remember you and your assessor are on the same team and the process of assessing should feel like a partnership. By following your assessor’s recommendations, your assessment should be less painful and go by quickly. You should involve your assessor as you work to bridge your compliance gaps. The more you involve your assessor, the easier your reassessment will be.