Thematic Review: Testing sanction screening systems

Here at AML Analytics, a Thematic Review refers to the testing, review and comparison process of the anti-financial crime systems used by regulated entities. We perform this type of testing on behalf of regulators all around the globe; in this instance we are concerned with the testing of sanction screening systems, a field in which we are world experts.

We are the only company in the world to work with regulators, governments, supervisors and central banks, using our advanced SupTech and unrivalled expertise to test the sanction screening systems of regulated entities as an integral part of a Thematic Review.

Our work with regulators has helped shape and develop new testing requirements and good practice guidance for regulated entities to help them meet ever-increasing regulatory expectation.

As it stands, we have worked with nearly 50 regulatory authorities all across the world and tested over 900 financial systems as part of the Thematic Review process.

What are the regulatory benefits of a Thematic Review?

• To raise the financial integrity of an entire jurisdiction
• To ensure regulated entities are meeting regulatory requirements in terms of sanction screening system performance
• To gain essential supervisory oversight about prevalent and emerging AML/CFT risks
• To bring about trust and transparency within a market
• To help implement international best-practice guidance
• To ensure sanction screening standards in an entire jurisdiction are in line with FATF Recommendations

Testing a sanction screening system

A Thematic Review uses our technology and expertise to create tests using our Sandbox solution for each regulated entity under a regulator’s jurisdiction. A test is created according to the specific instructions and testing requirements of the regulator and is sent to all regulated entities within scope to run through their sanction screening systems. The same test is used by all entities for benchmarking purposes.

The data

Thanks to our dedicated research team here at AML Analytics, our Sandbox solution creates tests from numerous publicly available, globally important sanctions lists and politically exposed persons (PEP) data, namely:

• HM Treasury Consolidated List of Financial Sanctions Targets (UK)
• OFAC Consolidated Sanctions List (US)
• OFAC Specially Designated Nationals List (US)
• Financial Sanctions Consolidated List (EU)

The test data is usually comprised of control, manipulated and clean ID data.

Control data are records exactly as they appear on sanctions lists.

Manipulated data are records that have been deliberately altered to test a sanction screening systems fuzzy logic matching capabilities. For example, changing the sanctioned name of Johnathan Jones to John Jones and testing if the system picks it up.

Clean IDs are records that do not appear on any sanction lists. These are included to see a sanction screening system’s tendency to pick up false positives and represent real-life customer data.

Onboarding and testing

We then provide these tests to the regulated entities. This is known as the onboarding process where we support each regulated entity through the ‘format verification’ process. This entails providing the correct test formats to each individual regulated entity to ensure full system compatibility.

The regulated entities then run the tests through their sanction screening systems to test all aspects of its performance. Expert guidance and dedicated support from AML Analytics team members for regulated entities during the entire testing process is provided.

Analysis and review

Once the regulated entities have uploaded their sanction screening test results into Analyser Online (our state-of-the-art communication hub), test results can be viewed instantly. Advanced analysis tools within our hub allows for granular-level detail about every sanction record that has been alerted or not.

For regulators concerned with jurisdictional oversight of their regulated entities, Thematic Review test results feed into ORBS (Online Risk Based Systems) for high-level oversight and absolute clarity of an entire market’s AML/CFT risk.

Feedback and follow-up

We provide feedback sessions to each regulated entity to ensure they have a full understanding of results and to suggest remediation steps with market-led peer review data. We then identify any overall risks that are emerging or prevalent within a market and set targets for improvement according to FATF Recommendations.

Our final step is to perform a follow-up Thematic Review to ensure targets for improvement set by the regulator are met. Repeat Thematic Reviews will drive system performance improvement across an entire market.

Our Thematic Review results

Now you have an idea of the process we follow to test sanction screening systems, here are some of the results of past Thematic Reviews.

During our first Thematic Review ever in 2014 with the South African Reserve Bank (SARB), we tested a total of 30 regulated entities. The results saw an average control score of 83.08% and an average manipulated score of 69.92%, with 20% of regulated entities in scope using manual means to test their sanction screening systems.

The follow-up review in 2017 saw a remarkable improvement. The average control and manipulated scores increased to 93.29% and 89.19% respectively, with only 3% sticking to manual tests.

The above are typical results of all the Thematic Reviews we’ve carried out on behalf of regulators around the world; we perform an initial one to diagnose the issues and provide crucial feedback on what needs to be addressed, and then perform a follow-up review to show regulators that their entities have taken our advice on board.

What we’ve learned

Having tested many hundreds of screening systems of regulated entities on behalf of many regulators, we’ve come to learn a lot of things about sanction screening systems and how they are being used.

• It’s how a sanction screening system is used that provides the results

A regulated entity could have just bought the most technically superior sanction screening system vendors have to offer, but if it is not being used optimally, then operationally it will make no difference to whether or not a regulated entity will meet regulatory requirements. It takes knowledge and expertise to tune a sanction screening system correctly in order for it to work as it is supposed to, drawing on as few resources as possible.

• There is a heavy reliance on manual processes

Not only is manually testing a sanction screening system an inaccurate, mistake-ridden process, but it draws upon valuable resource that most regulated entities can’t afford to give up. We’ve learnt over our time conducting Thematic Reviews that moving away from a manual testing process towards an automated one fixes a lot of problems that regulated entities have with their sanction screening systems.

• Thematic Reviews are always cost effective

Every piece of feedback we’ve ever received from the regulators we have worked with is just how cost effective a Thematic Review is. No expertise or resource is required from the regulator’s side; AML Analytics provides everything for clarity and peace of mind.

• The lack of internal resources is an unnecessary concern

A common pushback from regulators is the lack of resources available to them, and despite reassurance that no internal resource is required, there can often be concern. However, upon completion, all regulators have been exceptionally pleased with the Thematic Review process, and the dedicated work of the AML Analytics teams carried out on behalf of the regulator.

• A number of regulated entities do not have testing and auditing programmes in place

Many regulated entities have up-to-date sanction screening system technologies, but they do not know whether or not a system is working effectively and efficiently. This is because the systems in place have never been tested; they’ve merely been installed and expected to run optimally.