Quality of Embeddings & Triplet Loss

Author: Atharv Katkar

Directed by: Sandeep Giri

OVERVIEW:

In Natural Language Processing (NLP), embeddings transform human language into numerical vectors. These are usually arrays of multiple dimensions & have schematic meaning based on their previous training text corpus The quality of these embeddings directly affects the performance of search engines, recommendation systems, chatbots, and more.

But here’s the problem:

Not all embeddings are created equal.

So how do we measure their quality?

To Identify the quality of embeddings i conducted one experiment:

I took 3 leading (Free) Text → Embedding pretrained models which worked differently & provided a set of triplets and found the triplets loss to compare the contextual  importance of each one.

1) Sentence-BERT (SBERT)

Transformer-based Captures deep sentence-level semantics:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(‘all-MiniLM-L6-v2’)

2) Universal Sentence Encoder (USE)

TensorFlow Encoder Good general-purpose semantic encoding:

import tensorflow_hub as hub

model2 = hub.load(“https://tfhub.dev/google/universal-sentence-encoder/4”)

embeddings = model2([“Cardiac arrest”])

3) FastText (by Facebook AI)

Word-based Lightweight, fast, but lacks context:

import fasttext.util

fasttext.util.download_model(‘en’, if_exists=’ignore’)

ft3 = fasttext.load_model(‘cc.en.300.bin’)

vec = ft3.get_word_vector(“Cardiac arrest”)

when i compared the sizes of output produced by them are different for each one

(384,), (1, 512), (300,)

GOALS
  1. To compare them using a triplet-based evaluation approach using triplet loss.
  2. Identify the Understanding of these around medical terminologies

CONCEPTS

What is Triplet Loss?

Triplet loss works with a 3-part input:

Anchor: The base sentence or phrase

Positive: A semantically similar phrase

Negative: A semantically absurd or unrelated phrase

AnchorPositiveNegative
tuberculosisLung infectiontest tube accident 
cardiac arrestheart attackcardi b arrest
asthmarespiratory conditionSpiritual awakening 
Samples From my test dataset

The goal is to push the anchor close to the positive and far from the negative in embedding space.

TripletLoss = max (d (a , p) − d (a , n) + margin , 0 )

a = anchor vector
p = positive vector (should be close to
anchor)
n = negative vector (should be far from
anchor)
d(x,y) = cosine distance
margin = a buffer that forces the negative to be not just farther, but significantly farther

What is Cosine Similarity?

Cosine similarity is a measure of how similar two vectors are — based on the angle between them rather than their magnitude. In the context of NLP, vectors represent words or sentences as embeddings.

  Cosine Similarity (A.B) =   A.B / ||A||.||B||

CosineDistance(A,B) = 1 − CosineSimilarity(A,B)

What is Margin?

The margin is a safety cushion.

If margin = 0.2, then even if the negative is slightly farther than the positive, the model still gets a penalty unless it’s at least 0.2 farther.

Testing The accuracy
TEST-SET(click on test set see set)

We ran each model over a set of ~50 curated triplets

Calculated:

Anchor–Positive distance (AP)

Anchor–Negative distance (AN)

Triplet loss

Visualized both individual performance per triplet and overall averages

(“asthma”, “respiratory condition”, “spiritual awakening”),

(“pneumonia”, “lung infection”, “foggy window”),

# General & Internal Medicine

(“diabetes”, “high blood sugar”, “candy addiction”),

(“arthritis”, “joint inflammation”, “rusty hinge”),

50+ such examples

Results:

less the cosine distance more accurate the understanding of the context AP need be less as we can see of SBERT & USE is less where FASTTEXT is keep messing up

Using PCA, we visualized where each model placed the anchor, positive, and negative in space.
Insert 2D scatter plot
You can actually see the anchor and positive clustering together especially in SBERT’s case while the negative floats far away. In below image

for better understanding you can see the Euclidean distance where anchor & positive are relatively closer than negative (SBERT)
The USE is also able to classify the hard medical terminology quite precisely like diabetes= high blood sugar not a candy addiction
On the other hand the Fasttext basically classified it at same place like cardiac arrest = heart attack = cardi b arrest which is wrong

Triplet Example wise Score :

the SBERT ,USE performed efficiently if we not consider 2-3 examples where fasttext not able to understand any of context (just understood characters hierarchy & similarity)

The SBERT & USE performed good as we can see using few interpretations and loss tracking of triplets

CONCLUSION:

What We Learned

SBERT is highly reliable for understanding sentence-level meaning

USE performs reasonably well and is easy to use with TensorFlow

FastText, while fast, struggles with context and full sentences

Visual Results

Triplet Loss (Lower = Better)

SBERT     : 0.0381

USE       : 0.0320

FastText  : 0.2175

If you’re building Search engines , Recommendation systems , Chatbots …or anything involving meaning, good embeddings are key. Triplet loss is a simple yet powerful way to test how smart your model really is. I insist all of the preprocessing stages of making NLP models or context based systems the Triplet Loss needs to be used to select optimal pretrained or trained model.

The source code if want to conduct the experiments. Good Luck..!

Revolutionizing Mental Health Care with AI and AI-Powered Chatbots

Mental health care is an essential component of overall well-being, yet it remains one of the most underserved areas of medicine. The stigma surrounding mental health issues, coupled with limited access to qualified professionals, has created barriers to effective care for millions worldwide. AI-powered chatbots are emerging as a promising solution to bridge these gaps, providing accessible, scalable, and cost-effective mental health support. This blog explores how these innovative tools revolutionize mental health care, their challenges, and their potential future impact.

History of AI in Mental Health Care

The integration of artificial intelligence into mental health care has a rich and evolving history. The journey began in the mid-20th century with the development of early AI programs designed to simulate human conversation. One of the earliest examples was ELIZA, created in the 1960s by computer scientist Joseph Weizenbaum. ELIZA was a rudimentary chatbot that used pattern matching and substitution methodology to simulate a psychotherapist’s responses. While basic by today’s standards, ELIZA demonstrated the potential of conversational AI in providing mental health support.

Continue reading “Revolutionizing Mental Health Care with AI and AI-Powered Chatbots”

GPT 4 and its advancements over GPT 3

The field of natural language processing has witnessed remarkable advancements over the years, with the development of cutting-edge language models such as GPT-3 and the recent release of GPT-4. These models have revolutionized the way we interact with language and have opened up new possibilities for applications in various domains, including chatbots, virtual assistants, and automated content creation.

What is GPT?

GPT is a natural language processing (NLP) model developed by OpenAI that utilizes the transformer model. Transformer is a type of Deep Learning model, best known for its ability to process sequential data, such as text, by attending to different parts of the input sequence and using this information to generate context-aware representations of the text.

What makes transformers special is that they can understand the meaning of the text, instead of just recognizing patterns in the words. They can do this by “attending” to different parts of the text and figuring out which parts are most important to understanding the meaning of the whole.

For example, imagine you’re reading a book and come across the sentence “The cat sat on the mat.” A transformer would be able to understand that this sentence is about a cat and a mat and that the cat is sitting on the mat. It would also be able to use this understanding to generate new sentences that are related to the original one.

GPT is pre-trained on a large dataset, which consists of:

Continue reading “GPT 4 and its advancements over GPT 3”

Scholarship Test for PG Certificate in Data Science, AI/ML from IIT Roorkee. Earn Rs 75,000 Discount in One Hour.

We all know what’s ruling technology right now.

Yes, it is Artificial Intelligence, Machine Learning, Data Science, and Data Engineering. 

Therefore, now is the time to propel your Data Science career. Look no further because you can enroll for a PG Certificate Course in Data Science from IIT Roorkee. To make enrolment easy for you, here’s a Free Scholarship Test you can take and earn discounts up to Rs.75,000!

The Scholarship Test is a great opportunity for you to earn discounts. There are 50 questions that you have to attempt in one hour.
Each question you answer correctly earns you a discount of Rs 1000, and you can earn a maximum discount of Rs 75,000! (50/50 rewards you with an additional 25000 scholarship)

This Scholarship Test for the Data Science course is a great way to challenge yourself in basic aptitude and basic programming questions and to earn a massive discount on the course fees. 

The PG Certificate course from IIT Roorkee covers all that you need to know in technology right now. You will learn the architecture of ChatGPT, Stable Diffusion, Machine Learning, Artificial Intelligence, Data Science, Data Engineering and more! The course will be delivered by Professors from IIT Roorkee and industry experts and follows a blended mode of learning. Learners will also get 365 days of access to cloud labs for hands-on practice in a gamified learning environment. 

Data Scientists, Data Engineers, Data Architects are some of the highly sought after professionals today. With businesses and life-changing innovations being data driven in every domain, the demand for expertise in Deep Learning, Machine Learning is on the rise. This PG Certificate Course gives you the skills and knowledge required for a propelling career in Data Science. 

So what are you waiting for? Seats to the PG Certificate Course in Data Science from IIT Roorkee are limited. Take the Scholarship Test, earn discounts, and enroll now.

Link to the Scholarship Test is here.

Details about the PG Certificate Course in AI, Machine Learning, and Data Science are here.

Impact of AI on various industries in 2024

Artificial intelligence (AI) is having a profound impact on many different industries and is transforming the way businesses and organizations operate and serve their customers. With the help of AI, organizations are able to automate complex processes, make better predictions and decisions, and provide more personalized and efficient services to their customers.

One of the key areas where AI is making a big difference is in the field of healthcare. AI algorithms are being used to analyze medical data, such as images, records, and biomarkers, and to make more accurate predictions about the likelihood of diseases and the effectiveness of treatments. This can help healthcare providers to diagnose and treat patients more effectively, and improve the overall quality of care.

Continue reading “Impact of AI on various industries in 2024”

I’m from the telecom industry, should I learn Data Science and AI?

In the telecom industry, the use of AI and data science is becoming increasingly important for companies that want to stay competitive and deliver the best possible services to their customers.

Only by leveraging the power of AI and data science, telecom companies can gain valuable insights into their operations and make data-driven decisions that can help them improve efficiency, reduce costs, and develop new products and services.

One key area where AI and data science can help telecom companies is in network optimization. By analyzing vast amounts of data from network sensors and other sources, AI algorithms can identify patterns and anomalies that can indicate where the network is underperforming or prone to failure. This can help telecom companies take proactive steps to improve network reliability and reduce downtime, leading to a better overall customer experience.

Continue reading “I’m from the telecom industry, should I learn Data Science and AI?”

I’m from the manufacturing industry, should I learn Data Science and AI?

In today’s competitive manufacturing landscape, companies that want to stay ahead of the curve are turning to AI and data science to improve efficiency and drive innovation. By harnessing the power of AI and data science, manufacturing companies can gain valuable insights into their operations and make data-driven decisions that can help them improve productivity, reduce costs, and develop new products and services.

One key area where AI and data science can help manufacturing companies is in the realm of predictive maintenance. By analyzing vast amounts of data from sensors and other sources, AI algorithms can identify patterns and anomalies that can indicate when equipment is likely to fail. This can help companies schedule maintenance and repairs at the optimal time, reducing downtime and improving overall equipment reliability.

Continue reading “I’m from the manufacturing industry, should I learn Data Science and AI?”

Jobs that will emerge/stay relevant in the near future.

As AI and other technologies continue to advance, it is likely that many jobs that are currently considered essential will become obsolete, while new job opportunities will emerge in areas related to AI and other emerging technologies.

Continue reading “Jobs that will emerge/stay relevant in the near future.”

I’m from the banking industry, should I learn Data Science and AI

If you work in the banking industry, learning about data science, machine learning, and AI could be a valuable investment in your career. These fields are rapidly growing and are expected to play an increasingly important role in the banking industry in the coming years.

Here are a few reasons why learning about data science, machine learning, and AI could be beneficial for individuals in the banking industry:

Continue reading “I’m from the banking industry, should I learn Data Science and AI”

How to Crack Machine Learning Interviews with Top Interview Questions(2024)

Machine Learning is the most rapidly growing domain in the software industry. More and more sectors are using concepts of Machine Learning to enhance their businesses. It is now not an add-on but has become a necessity for businesses to use ML algorithms for optimizing their businesses and to offer a personalised user experience.

This demand for Machine Learning in the industry has directly increased the demand for Machine Learning Engineers, the ones who unload this magic in reality. According to a survey conducted by LinkedIn, Machine Learning Engineer is the most emerging job role in the current industry with nearly 10 times growth.

But, even this high demand doesn’t make getting a job in ML any easier. ML interviews are tough regardless of your seniority level. But as said, with the right knowledge and preparation, interviews become a lot easier to crack.

In this blog, I will walk you through the interview process for an ML job role and will pass on some tips and tactics on how to crack one. We will also discuss the skills required in accordance with each round of the process.

Continue reading “How to Crack Machine Learning Interviews with Top Interview Questions(2024)”