In this step, we will create l1, l2 and l3 views which will help us in calculating sentiment of each tweet.
Create view l1. l1 view converts each tweet into lower case and explodes it into a list of words. Run below command in Hive query editor in Hue.
create view l1 as select id, words from tweets_raw lateral view explode(sentences(lower(text))) dummy as words;
Sample rows of view l1 are
Create view l2. l2 view stores every word of a tweet in a new row.
Run below command in Hive query editor in Hue.
create view l2 as select id, word from l1 lateral view explode( words ) dummy as word ;
Sample rows of view l2 are
Create view l3. l3 view joins l2 view with dictionary table and stores polarity of each word. Run below command in Hive query editor in Hue.
create view l3 as select id, l2.word, case d.polarity when 'negative' then -1 when 'positive' then 1 else 0 end as polarity from l2 left outer join dictionary d on l2.word = d.word;
Sample rows of view l3 are
What is the polarity of word "crushes"?