Scikit Learn - Anomaly Detection

Here, we will learn about what is anomaly detection in Sklearn and how it is used in identification of the data points.

Anomaly detection is a technique used to identify data points in dataset that does not fit well with the rest of the data. It has many apppcations in business such as fraud detection, intrusion detection, system health monitoring, surveillance, and predictive maintenance. Anomapes, which are also called outper, can be spanided into following three categories −

Point anomapes − It occurs when an inspanidual data instance is considered as anomalous w.r.t the rest of the data.

Contextual anomapes − Such kind of anomaly is context specific. It occurs if a data instance is anomalous in a specific context.

Collective anomapes − It occurs when a collection of related data instances is anomalous w.r.t entire dataset rather than inspanidual values.

Methods

Two methods namely outper detection and novelty detection can be used for anomaly detection. It’s necessary to see the distinction between them.

Outper detection

The training data contains outpers that are far from the rest of the data. Such outpers are defined as observations. That’s the reason, outper detection estimators always try to fit the region having most concentrated training data while ignoring the deviant observations. It is also known as unsupervised anomaly detection.

Novelty detection

It is concerned with detecting an unobserved pattern in new observations which is not included in training data. Here, the training data is not polluted by the outpers. It is also known as semi-supervised anomaly detection.

There are set of ML tools, provided by scikit-learn, which can be used for both outper detection as well novelty detection. These tools first implementing object learning from the data in an unsupervised by using fit () method as follows −

estimator.fit(X_train)

Now, the new observations would be sorted as inpers (labeled 1) or outpers (labeled -1) by using predict() method as follows −

estimator.fit(X_test)

The estimator will first compute the raw scoring function and then predict method will make use of threshold on that raw scoring function. We can access this raw scoring function with the help of score_sample method and can control the threshold by contamination parameter.

We can also define decision_function method that defines outpers as negative value and inpers as non-negative value.

estimator.decision_function(X_test)

Sklearn algorithms for Outper Detection

Let us begin by understanding what an elpptic envelop is.

Fitting an elpptic envelop

This algorithm assume that regular data comes from a known distribution such as Gaussian distribution. For outper detection, Scikit-learn provides an object named covariance.ElppticEnvelop.

This object fits a robust covariance estimate to the data, and thus, fits an elppse to the central data points. It ignores the points outside the central mode.

Parameters

Following table consist the parameters used by sklearn. covariance.ElppticEnvelop method −

Attributes

Following table consist the attributes used by sklearn. covariance.ElppticEnvelop method −

Implementation Example

import numpy as np^M
from sklearn.covariance import ElppticEnvelope^M
true_cov = np.array([[.5, .6],[.6, .4]])
X = np.random.RandomState(0).multivariate_normal(mean = [0, 0], cov=true_cov,size=500)
cov = ElppticEnvelope(random_state = 0).fit(X)^M
# Now we can use predict method. It will return 1 for an inper and -1 for an outper.
cov.predict([[0, 0],[2, 2]])

Output

array([ 1, -1])

Isolation Forest

In case of high-dimensional dataset, one efficient way for outper detection is to use random forests. The scikit-learn provides ensemble.IsolationForest method that isolates the observations by randomly selecting a feature. Afterwards, it randomly selects a value between the maximum and minimum values of the selected features.

Here, the number of spptting needed to isolate a sample is equivalent to path length from the root node to the terminating node.

Parameters

Followings table consist the parameters used by sklearn. ensemble.IsolationForest method −

Attributes

Following table consist the attributes used by sklearn. ensemble.IsolationForest method −

Implementation Example

The Python script below will use sklearn. ensemble.IsolationForest method to fit 10 trees on given data

from sklearn.ensemble import IsolationForest
import numpy as np
X = np.array([[-1, -2], [-3, -3], [-3, -4], [0, 0], [-50, 60]])
OUTDClf = IsolationForest(n_estimators = 10)
OUTDclf.fit(X)

Output

IsolationForest(
   behaviour =  old , bootstrap = False, contamination= legacy ,
   max_features = 1.0, max_samples =  auto , n_estimators = 10, n_jobs=None,
   random_state = None, verbose = 0
)

Local Outper Factor

Local Outper Factor (LOF) algorithm is another efficient algorithm to perform outper detection on high dimension data. The scikit-learn provides neighbors.LocalOutperFactor method that computes a score, called local outper factor, reflecting the degree of anomapty of the observations. The main logic of this algorithm is to detect the samples that have a substantially lower density than its neighbors. Thats why it measures the local density deviation of given data points w.r.t. their neighbors.

Parameters

Followings table consist the parameters used by sklearn. neighbors.LocalOutperFactor method

Attributes

Following table consist the attributes used by sklearn.neighbors.LocalOutperFactor method −

Implementation Example

The Python script given below will use sklearn.neighbors.LocalOutperFactor method to construct NeighborsClassifier class from any array corresponding our data set

from sklearn.neighbors import NearestNeighbors
samples = [[0., 0., 0.], [0., .5, 0.], [1., 1., .5]]
LOFneigh = NearestNeighbors(n_neighbors = 1, algorithm = "ball_tree",p=1)
LOFneigh.fit(samples)

Output

NearestNeighbors(
   algorithm =  ball_tree , leaf_size = 30, metric= minkowski ,
   metric_params = None, n_jobs = None, n_neighbors = 1, p = 1, radius = 1.0
)

Example

Now, we can ask from this constructed classifier is the closet point to [0.5, 1., 1.5] by using the following python script −

print(neigh.kneighbors([[.5, 1., 1.5]])

Output

(array([[1.7]]), array([[1]], dtype = int64))

One-Class SVM

The One-Class SVM, introduced by Schölkopf et al., is the unsupervised Outper Detection. It is also very efficient in high-dimensional data and estimates the support of a high-dimensional distribution. It is implemented in the Support Vector Machines module in the Sklearn.svm.OneClassSVM object. For defining a frontier, it requires a kernel (mostly used is RBF) and a scalar parameter.

For better understanding let s fit our data with svm.OneClassSVM object −

Example

from sklearn.svm import OneClassSVM
X = [[0], [0.89], [0.90], [0.91], [1]]
OSVMclf = OneClassSVM(gamma =  scale ).fit(X)

Now, we can get the score_samples for input data as follows −

OSVMclf.score_samples(X)

Output

array([1.12218594, 1.58645126, 1.58673086, 1.58645127, 1.55713767])