%% Remove Custom List of Stop Words from Documents
% To remove the default list of stop words using the language details of documents, use removeStopWords.
% To remove a custom list of stop words, use the removeWords function. You can use the stop word list returned by the stopWords function as a starting point.
% Load the example data. The file sonnetsPreprocessed.txt contains preprocessed versions of Shakespeare's sonnets. The file contains one sonnet per line, with words separated by a space. Extract the text from sonnetsPreprocessed.txt, split the text into documents at newline characters, and then tokenize the documents.

filename = "sonnetsPreprocessed.txt";
str = extractFileText(filename);
textData = split(str,newline);
documents = tokenizedDocument(textData);

% View the first few documents.

documents(1:5)

% Create a list of stop words starting with the output of the stopWords function.

customStopWords = [stopWords "thy" "thee" "thou" "dost" "doth"];

% Remove the custom stop words from the documents and view the first few documents.

documents = removeWords(documents,customStopWords);
documents(1:5)

%
% Copyright 2018 The MathWorks, Inc.