# Cohen Kappa

## A Simple Explanation - By Varsha Saini

In the classification model, let us assume that there are two classes: class 0 and class 1, and two raters who will rate each data point that belongs to a particular class. One of the raters can be considered as a classification model and the other as a real-world observer. Â The Cohen Kappa score can be used to compare the agreement of these raters on a variety of different tasks.

## What is Cohenâ€™s Kappa Score or Kappa Coefficient?

Cohen Kappa Score is a statistical measurement of the performance of the classification machine learning model. It is based on the agreement between the two raters. The agreement between the two raters (real-world observer and the classification model) can be perfect or it can be by chance.

## How to Calculate Cohenâ€™s Kappa Score?

Cohen’s Kappa Score can be calculated using the Confusion Matrix. Since the cohen kappa is based on the concept of raters. Let us assume actual values are values by rater 1 and predicted values are values by rater 2.

### 1. True Positive

These are the correctly predicted positive values i.e. the value which is predicted as positive and is true.

### 2. True Negative

These are the correctly predicted negative values i.e. the value which is predicted as negative and is true.

### 3. False Positive

These are the incorrectly predicted positive values i.e. the value which is predicted as positive and is false.

### 4. False Negative

These are the incorrectly predicted negative values i.e. the value which is predicted as negative and is false.

### Number of Agreements

The number of agreements (where both raters are in agreement with each other) is True Positive and True Negative.

### Number of Disagreements

The number of disagreements (where both raters are in disagreement with each other) is False Positive and False Negative.

## Cohenâ€™s Kappa Formula

The below formula represents Cohen’s Kappa:

where

• Po= observed probability, the probability that rater 1 and rater 2 are in perfect agreement.
• Pe = expected probability, probability (expected) both the raters are in agreement by chance or randomly.

## Interpretation of Kappa Score or Coefficient

The value of the Cohen Kappa Score or Kappa Coefficient can range between -1 and +1.

• Kappa’s score of -1 is considered to be the worst possible performance.
• Kappa’s score of +1 is considered to be the best possible performance.
• Kappa’s score of 0 means that the model is randomly guessing.

## Advantage of Cohen Kappa

Cohen Kappa or Kappa Coefficient is a very useful performance evaluation metric when dealing with imbalanced data. Therefore in highly imbalanced data where the Accuracy metric might fail, the Cohen Kappa coefficient can be considered.