Improved Anomaly Detection By Leveraging ‘Grey Labels’

In one of my recent papers, titled “Belling Schrödinger’s cat“, I had picked the analogy of Schrödinger’s thought experiment around how it was not possible to conceive if the cat was alive or not, and how in a similar fashion in business we have ambiguity in data points, that could potentially be used to extract superior decisions.

Examples of ambiguity arising out of a decision process can be seen in the following situations:

Processing vendor payments – where the back-office associate suspects an invoice to be fraudulent or duplicate, but does not have hard evidence or the time to conduct the necessary checks
Insurance claims – where the claim examiner has a gut feel that the claim is dubious but does not have the ability to conduct a full investigation, for reasons of bandwidth, data not available in time or not fully investigated as it was a low value claim
Other instances where there is sensitivity around the individual concerned and the organization does not want to risk an adverse reaction.

These are scenarios that we commonly encounter, however today, advancements in technology is helping enterprises turn a different chapter. With today’s smart detection techniques, we have seen implementation moving from a rule-based regime to a probabilistic prediction of anomalies – as I spoke about earlier here.

In the above mentioned business scenarios, building an anomaly detection model that does not have a meaningful approach to handling the ‘unproven suspects‘ or ‘greys‘ could impact the predictive accuracy. We have seen that these ‘greys’ are typically handled in one of the following ways:

Ignored greys from the analysis
Assumed greys to be blacks (conservative stance – ‘Blacks’ denote the confirmed frauds)
Assumed greys to be whites (assigned as the majority label – ‘Whites’ denote clean transactions)

All these options lead to loss of vital information and hence inferior decision making. To overcome this, we explored if there is a way to leverage the “two-state” nature of the grey set to build a better anomaly detection model, compared to the above-mentioned options. Essentially what we sought to do was to run a series of iterations and assume that the ‘grey’ data points could either be ‘blacks’ or ‘whites’ in each of these iterations and build this into our prediction model and then consolidate the results. The individual predictors could be any of the traditional classifiers.

The approach was implemented on three different datasets across three very different business domains. All three datasets contained a small number of known frauds and greys (under 1% each) and the remaining were normal / clean records.

Improved Anomaly Detection By Leveraging ‘Grey Labels’

In all three scenarios, we saw an enhanced performance for the model that classifies the greys, compared to ignoring them or treating them as black / white as the default option. Performance is shown in terms of F5 score (The Fx score establishes the trade-off between precision and recall and enables us to compare multiple models) in the graph below. We have taken β=5.

The key findings from the results are as follows:

We found that tapping the inherent cues from the seemingly unusable ‘grey data points’ is of value in improving the predictive performance of anomaly detection models
Additionally, we observed that multiple iterations improve the performance, as various patterns get factored in; indicating that the models have the potential to become more robust through assignment of different combinations of black vs white to each grey. (However, as per intuition, law of diminishing returns kicks-in with limited predictive lift with increase in voting beyond 70-80 iterations)
We didn’t see any appreciable difference in predictive accuracy, by varying the underlying classifiers

As we deploy this approach in solving other compelling business problems, we are achieving more flawless detection models – with false positive rates dropping further and greater process velocity strengthening the organization’s risk climate.

About the Author

R. Guha heads Corporate Business Development at Wipro Technologies and is focused on two strategic themes at the intersection of technology and business: cybersecurity framed from a business risk perspective and how to leverage machine learning for business transformation.

Guha is currently leading the development and deployment of Apollo, an anomaly detection platform that seeks to mitigate risk and improve process velocity through smarter detection.

Service Offerings

Service Offerings

Client Themes

Client Themes

Improved Anomaly Detection By

Leveraging ‘Grey Labels’

About the Author

Service Offerings

Industries

Cloud

Cybersecurity

Digital

EngineeringNXT

Holmes

Contact Us

About Wipro

Careers

Location

Leadership

Investors

Innovation

News and Events

Insights

Analyst Speak

Products & Platforms

Partner Ecosystem

Sustainability

Contact Wipro