Quality of Embeddings & Triplet Loss

Author: Atharv Katkar

Directed by: Sandeep Giri

OVERVIEW:

In Natural Language Processing (NLP), embeddings transform human language into numerical vectors. These are usually arrays of multiple dimensions & have schematic meaning based on their previous training text corpus The quality of these embeddings directly affects the performance of search engines, recommendation systems, chatbots, and more.

But here’s the problem:

Not all embeddings are created equal.

So how do we measure their quality?

To Identify the quality of embeddings i conducted one experiment:

I took 3 leading (Free) Text → Embedding pretrained models which worked differently & provided a set of triplets and found the triplets loss to compare the contextual  importance of each one.

1) Sentence-BERT (SBERT)

Transformer-based Captures deep sentence-level semantics:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(‘all-MiniLM-L6-v2’)

2) Universal Sentence Encoder (USE)

TensorFlow Encoder Good general-purpose semantic encoding:

import tensorflow_hub as hub

model2 = hub.load(“https://tfhub.dev/google/universal-sentence-encoder/4”)

embeddings = model2([“Cardiac arrest”])

3) FastText (by Facebook AI)

Word-based Lightweight, fast, but lacks context:

import fasttext.util

fasttext.util.download_model(‘en’, if_exists=’ignore’)

ft3 = fasttext.load_model(‘cc.en.300.bin’)

vec = ft3.get_word_vector(“Cardiac arrest”)

when i compared the sizes of output produced by them are different for each one

(384,), (1, 512), (300,)

GOALS
  1. To compare them using a triplet-based evaluation approach using triplet loss.
  2. Identify the Understanding of these around medical terminologies

CONCEPTS

What is Triplet Loss?

Triplet loss works with a 3-part input:

Anchor: The base sentence or phrase

Positive: A semantically similar phrase

Negative: A semantically absurd or unrelated phrase

AnchorPositiveNegative
tuberculosisLung infectiontest tube accident 
cardiac arrestheart attackcardi b arrest
asthmarespiratory conditionSpiritual awakening 
Samples From my test dataset

The goal is to push the anchor close to the positive and far from the negative in embedding space.

TripletLoss = max (d (a , p) − d (a , n) + margin , 0 )

a = anchor vector
p = positive vector (should be close to
anchor)
n = negative vector (should be far from
anchor)
d(x,y) = cosine distance
margin = a buffer that forces the negative to be not just farther, but significantly farther

What is Cosine Similarity?

Cosine similarity is a measure of how similar two vectors are — based on the angle between them rather than their magnitude. In the context of NLP, vectors represent words or sentences as embeddings.

  Cosine Similarity (A.B) =   A.B / ||A||.||B||

CosineDistance(A,B) = 1 − CosineSimilarity(A,B)

What is Margin?

The margin is a safety cushion.

If margin = 0.2, then even if the negative is slightly farther than the positive, the model still gets a penalty unless it’s at least 0.2 farther.

Testing The accuracy
TEST-SET(click on test set see set)

We ran each model over a set of ~50 curated triplets

Calculated:

Anchor–Positive distance (AP)

Anchor–Negative distance (AN)

Triplet loss

Visualized both individual performance per triplet and overall averages

(“asthma”, “respiratory condition”, “spiritual awakening”),

(“pneumonia”, “lung infection”, “foggy window”),

# General & Internal Medicine

(“diabetes”, “high blood sugar”, “candy addiction”),

(“arthritis”, “joint inflammation”, “rusty hinge”),

50+ such examples

Results:

less the cosine distance more accurate the understanding of the context AP need be less as we can see of SBERT & USE is less where FASTTEXT is keep messing up

Using PCA, we visualized where each model placed the anchor, positive, and negative in space.
Insert 2D scatter plot
You can actually see the anchor and positive clustering together especially in SBERT’s case while the negative floats far away. In below image

for better understanding you can see the Euclidean distance where anchor & positive are relatively closer than negative (SBERT)
The USE is also able to classify the hard medical terminology quite precisely like diabetes= high blood sugar not a candy addiction
On the other hand the Fasttext basically classified it at same place like cardiac arrest = heart attack = cardi b arrest which is wrong

Triplet Example wise Score :

the SBERT ,USE performed efficiently if we not consider 2-3 examples where fasttext not able to understand any of context (just understood characters hierarchy & similarity)

The SBERT & USE performed good as we can see using few interpretations and loss tracking of triplets

CONCLUSION:

What We Learned

SBERT is highly reliable for understanding sentence-level meaning

USE performs reasonably well and is easy to use with TensorFlow

FastText, while fast, struggles with context and full sentences

Visual Results

Triplet Loss (Lower = Better)

SBERT     : 0.0381

USE       : 0.0320

FastText  : 0.2175

If you’re building Search engines , Recommendation systems , Chatbots …or anything involving meaning, good embeddings are key. Triplet loss is a simple yet powerful way to test how smart your model really is. I insist all of the preprocessing stages of making NLP models or context based systems the Triplet Loss needs to be used to select optimal pretrained or trained model.

The source code if want to conduct the experiments. Good Luck..!

Revolutionizing Mental Health Care with AI and AI-Powered Chatbots

Mental health care is an essential component of overall well-being, yet it remains one of the most underserved areas of medicine. The stigma surrounding mental health issues, coupled with limited access to qualified professionals, has created barriers to effective care for millions worldwide. AI-powered chatbots are emerging as a promising solution to bridge these gaps, providing accessible, scalable, and cost-effective mental health support. This blog explores how these innovative tools revolutionize mental health care, their challenges, and their potential future impact.

History of AI in Mental Health Care

The integration of artificial intelligence into mental health care has a rich and evolving history. The journey began in the mid-20th century with the development of early AI programs designed to simulate human conversation. One of the earliest examples was ELIZA, created in the 1960s by computer scientist Joseph Weizenbaum. ELIZA was a rudimentary chatbot that used pattern matching and substitution methodology to simulate a psychotherapist’s responses. While basic by today’s standards, ELIZA demonstrated the potential of conversational AI in providing mental health support.

Continue reading “Revolutionizing Mental Health Care with AI and AI-Powered Chatbots”

GPT 4 and its advancements over GPT 3

The field of natural language processing has witnessed remarkable advancements over the years, with the development of cutting-edge language models such as GPT-3 and the recent release of GPT-4. These models have revolutionized the way we interact with language and have opened up new possibilities for applications in various domains, including chatbots, virtual assistants, and automated content creation.

What is GPT?

GPT is a natural language processing (NLP) model developed by OpenAI that utilizes the transformer model. Transformer is a type of Deep Learning model, best known for its ability to process sequential data, such as text, by attending to different parts of the input sequence and using this information to generate context-aware representations of the text.

What makes transformers special is that they can understand the meaning of the text, instead of just recognizing patterns in the words. They can do this by “attending” to different parts of the text and figuring out which parts are most important to understanding the meaning of the whole.

For example, imagine you’re reading a book and come across the sentence “The cat sat on the mat.” A transformer would be able to understand that this sentence is about a cat and a mat and that the cat is sitting on the mat. It would also be able to use this understanding to generate new sentences that are related to the original one.

GPT is pre-trained on a large dataset, which consists of:

Continue reading “GPT 4 and its advancements over GPT 3”

Scholarship Test for PG Certificate in Data Science, AI/ML from IIT Roorkee. Earn Rs 75,000 Discount in One Hour.

We all know what’s ruling technology right now.

Yes, it is Artificial Intelligence, Machine Learning, Data Science, and Data Engineering. 

Therefore, now is the time to propel your Data Science career. Look no further because you can enroll for a PG Certificate Course in Data Science from IIT Roorkee. To make enrolment easy for you, here’s a Free Scholarship Test you can take and earn discounts up to Rs.75,000!

The Scholarship Test is a great opportunity for you to earn discounts. There are 50 questions that you have to attempt in one hour.
Each question you answer correctly earns you a discount of Rs 1000, and you can earn a maximum discount of Rs 75,000! (50/50 rewards you with an additional 25000 scholarship)

This Scholarship Test for the Data Science course is a great way to challenge yourself in basic aptitude and basic programming questions and to earn a massive discount on the course fees. 

The PG Certificate course from IIT Roorkee covers all that you need to know in technology right now. You will learn the architecture of ChatGPT, Stable Diffusion, Machine Learning, Artificial Intelligence, Data Science, Data Engineering and more! The course will be delivered by Professors from IIT Roorkee and industry experts and follows a blended mode of learning. Learners will also get 365 days of access to cloud labs for hands-on practice in a gamified learning environment. 

Data Scientists, Data Engineers, Data Architects are some of the highly sought after professionals today. With businesses and life-changing innovations being data driven in every domain, the demand for expertise in Deep Learning, Machine Learning is on the rise. This PG Certificate Course gives you the skills and knowledge required for a propelling career in Data Science. 

So what are you waiting for? Seats to the PG Certificate Course in Data Science from IIT Roorkee are limited. Take the Scholarship Test, earn discounts, and enroll now.

Link to the Scholarship Test is here.

Details about the PG Certificate Course in AI, Machine Learning, and Data Science are here.

Impact of AI on various industries in 2024

Artificial intelligence (AI) is having a profound impact on many different industries and is transforming the way businesses and organizations operate and serve their customers. With the help of AI, organizations are able to automate complex processes, make better predictions and decisions, and provide more personalized and efficient services to their customers.

One of the key areas where AI is making a big difference is in the field of healthcare. AI algorithms are being used to analyze medical data, such as images, records, and biomarkers, and to make more accurate predictions about the likelihood of diseases and the effectiveness of treatments. This can help healthcare providers to diagnose and treat patients more effectively, and improve the overall quality of care.

Continue reading “Impact of AI on various industries in 2024”

I’m from the telecom industry, should I learn Data Science and AI?

In the telecom industry, the use of AI and data science is becoming increasingly important for companies that want to stay competitive and deliver the best possible services to their customers.

Only by leveraging the power of AI and data science, telecom companies can gain valuable insights into their operations and make data-driven decisions that can help them improve efficiency, reduce costs, and develop new products and services.

One key area where AI and data science can help telecom companies is in network optimization. By analyzing vast amounts of data from network sensors and other sources, AI algorithms can identify patterns and anomalies that can indicate where the network is underperforming or prone to failure. This can help telecom companies take proactive steps to improve network reliability and reduce downtime, leading to a better overall customer experience.

Continue reading “I’m from the telecom industry, should I learn Data Science and AI?”

I’m from the manufacturing industry, should I learn Data Science and AI?

In today’s competitive manufacturing landscape, companies that want to stay ahead of the curve are turning to AI and data science to improve efficiency and drive innovation. By harnessing the power of AI and data science, manufacturing companies can gain valuable insights into their operations and make data-driven decisions that can help them improve productivity, reduce costs, and develop new products and services.

One key area where AI and data science can help manufacturing companies is in the realm of predictive maintenance. By analyzing vast amounts of data from sensors and other sources, AI algorithms can identify patterns and anomalies that can indicate when equipment is likely to fail. This can help companies schedule maintenance and repairs at the optimal time, reducing downtime and improving overall equipment reliability.

Continue reading “I’m from the manufacturing industry, should I learn Data Science and AI?”

Jobs that will emerge/stay relevant in the near future.

As AI and other technologies continue to advance, it is likely that many jobs that are currently considered essential will become obsolete, while new job opportunities will emerge in areas related to AI and other emerging technologies.

Continue reading “Jobs that will emerge/stay relevant in the near future.”

I’m from the banking industry, should I learn Data Science and AI

If you work in the banking industry, learning about data science, machine learning, and AI could be a valuable investment in your career. These fields are rapidly growing and are expected to play an increasingly important role in the banking industry in the coming years.

Here are a few reasons why learning about data science, machine learning, and AI could be beneficial for individuals in the banking industry:

Continue reading “I’m from the banking industry, should I learn Data Science and AI”

Starting Machine Learning with an End-to-End Project

When you are learning about Machine Learning, it is best to experiment with real-world data alongside learning concepts. It is even more beneficial to start Machine Learning with a project including end-to-end model building, rather than going for conceptual knowledge first.

Benefits of Project-Based Learning

  1. You get to know about real-world projects which in a way prepares you for real-time jobs.
  2. Encourages critical thinking and problem-solving skills in learners.
  3. Gives an idea of the end-to-end process of building a project.
  4. Gives an idea of tools and technologies used in the industry.
  5. Learners get an in-depth understanding of the concepts which directly boosts their self-confidence.
  6. It is a more fun way to learn things rather than traditional methods of learning.

What is an End-to-End project?

End-to-end refers to a full process from start to finish. In an ML end-to-end project, you have to perform every task from first to last by yourself. That includes getting the data, processing it, preparing data for the model, building the model, and at last finalizing it.

Ideology to start with End to End project

It is much more beneficial to start learning Machine Learning with an end-to-end project rather than diving down deep into the vast ocean of Machine Learning concepts. But, what will be the benefit of practicing concepts without even understanding them properly? How to implement concepts when we don’t understand them properly?

There are not one but several benefits of starting your ML journey with a project. Some of them are:

Continue reading “Starting Machine Learning with an End-to-End Project”