%% Analyze Text Data Containing Emojis
% This example shows how to analyze text data containing emojis.
% Emojis are pictorial symbols that appear inline in text. When writing text on mobile devices such as smartphones and tablets, people use emojis to keep the text short and convey emotion and feelings.
% You also can use emojis to analyze text data. For example, use them to identify relevant strings of text or to visualize the sentiment or emotion of the text.
% When working with text data, emojis can behave unpredictably. Depending on your system fonts, your system might not display some emojis correctly. Therefore, if an emoji is not displayed correctly, then the data is not necessarily missing. Your system might be unable to display the emoji in the current font.
%
%% Composing Emojis
% In most cases, you can read emojis from a file (for example, by using extractFileText, extractHTMLText, or readtable) or by copying and pasting them directly into MATLAB®. Otherwise, you must compose the emoji using Unicode UTF16 code units.
% Some emojis consist of multiple Unicode UTF16 code units. For example, the "smiling face with sunglasses" emoji (😎 with code point U+1F60E) is a single glyph but comprises two UTF16 code units "D83D" and "DE0E". Create a string containing this emoji using the compose function, and specify the two code units with the prefix "\x".

emoji = compose("\xD83D\xDE0E")

% First get the Unicode UTF16 code units of an emoji. Use char to get the numeric representation of the emoji, and then use dec2hex to get the corresponding hex value.

codeUnits = dec2hex(char(emoji))

% Reconstruct the composition string using the strjoin function with the empty delimiter "".

formatSpec = strjoin("\x" + codeUnits,"")
emoji = compose(formatSpec)

%
%% Import Text Data
% Extract the text data in the file weekendUpdates.xlsx using readtable. The file weekendUpdates.xlsx contains status updates containing the hashtags "#weekend" and "#vacation".

filename = "weekendUpdates.xlsx";
tbl = readtable(filename,'TextType','string');
head(tbl)

% Extract the text data from the field TextData and view the first few status updates.

textData = tbl.TextData;
textData(1:5)

% Visualize the text data in a word cloud.

figure
wordcloud(textData);

%
%% Filter Text Data by Emoji
% Identify the status updates containing a particular emoji using the contains function. Find the indices of the documents containing the "smiling face with sunglasses" emoji (😎 with code U+1F60E). This emoji comprises the two Unicode UTF16 code units "D83D" and "DE0E".

emoji = compose("\xD83D\xDE0E");
idx = contains(textData,emoji);
textDataSunglasses = textData(idx);
textDataSunglasses(1:5)

% Visualize the extracted text data in a word cloud.

figure
wordcloud(textDataSunglasses);

%
%% Extract and Visualize Emojis
% Visualize all the emojis in text data using a word cloud.
% Extract the emojis. First tokenize the text using tokenizedDocument, and then view the first few documents.

documents = tokenizedDocument(textData);
documents(1:5)

% The tokenizedDocument function automatically detects emoji and assigns the token type "emoji". View the first few token details of the documents using the tokenDetails function.

tdetails = tokenDetails(documents);
head(tdetails)

% Visualize the emojis in a word cloud by extracting the tokens with token type "emoji" and inputting them into the wordcloud function.

idx = tdetails.Type == "emoji";
tokens = tdetails.Token(idx);
figure
wordcloud(tokens);
title("Emojis")

% Copyright 2018 The MathWorks, Inc.