How to Calculate Jaccard Similarity in Python?

The Jaccard similarity coefficient, also known as the Jaccard index, is a statistic used for comparing the similarity and diversity of sample sets. The coefficient measures similarity between finite sample sets and is defined as the size of the intersection divided by the size of the union of the sample sets.

Here, we will discuss how to calculate the Jaccard Similarity in Python using different methods.

Setting Up

To start, you will need to have Python installed on your system. Python 3.6 or newer is recommended. You will also need to install the NumPy, Scikit-Learn, and NLTK libraries if you haven’t already:

pip install numpy
pip install scikit-learn
pip install nltk

Calculating Jaccard Similarity for Binary Variables

For binary (0/1) variables, you can use the jaccard_score method provided by scikit-learn. Let’s start by importing the necessary modules:

from sklearn.metrics import jaccard_score

Now, suppose you have two binary vectors, a and b:

a = [0, 1, 1, 0, 1, 1, 0, 0, 1]
b = [1, 1, 0, 0, 1, 1, 0, 1, 0]

You can calculate the Jaccard Similarity as follows:

jaccard = jaccard_score(a, b)
print(f'Jaccard Similarity: {jaccard}')

The jaccard_score function computes the Jaccard similarity coefficient between the two binary vectors.

Calculating Jaccard Similarity for Sets

The Jaccard Similarity can be calculated for sets using Python’s built-in set operations. Here is an example:

# Define two sets
set1 = set(['dog', 'cat', 'bird', 'fish'])
set2 = set(['dog', 'cat', 'turtle', 'rabbit'])

# Calculate intersection
intersection = len(set1.intersection(set2))

# Calculate union
union = len(set1.union(set2))

# Calculate Jaccard Similarity
jaccard = intersection / union

print(f'Jaccard Similarity: {jaccard}')

Here, we calculate the Jaccard similarity by taking the size of the intersection of the sets (elements common to both sets) and dividing it by the size of the union of the sets (all distinct elements from both sets).

Calculating Jaccard Similarity for Text

When working with text data, you might want to calculate the Jaccard similarity between documents or sentences. Here, each document or sentence can be considered as a set of words.

from nltk import ngrams

# Define two sentences
sentence1 = "I feel the product is of high quality."
sentence2 = "The product I got was of very high quality."

# Tokenize sentences to get sets of n-grams
n = 3 # You can choose n as per your requirement
set1 = set(ngrams(sentence1.split(), n))
set2 = set(ngrams(sentence2.split(), n))

# Calculate intersection
intersection = len(set1.intersection(set2))

# Calculate union
union = len(set1.union(set2))

# Calculate Jaccard Similarity
jaccard = intersection / union

print(f'Jaccard Similarity: {jaccard}')

Here, we are using the ngrams function from NLTK to convert sentences into a set of n-grams. An n-gram is a contiguous sequence of n items from a given sample of text or speech. Then, we calculate the Jaccard similarity as before.

The Jaccard similarity is a simple yet powerful metric for understanding the similarity between different data points. While this guide focuses on binary vectors, sets, and text data, the concept can be extended to other data types as well.