textrankScores

Document scoring with TextRank algorithm

collapse all in page

Syntax

scores = textrankScores(documents)
scores = textrankScores(bag)

Description

example

scores = textrankScores(documents) scores documents for importance according to pairwise similarity values using the TextRank algorithm. To compute similarities and importance scores, the function uses the BM25 and PageRank algorithms, respectively.

example

scores = textrankScores(bag) scores documents encoded by a bag-of-words or bag-of-n-grams model bag.

Examples

collapse all

Open Live Script

Create an array of tokenized documents.

str = [
    "the quick brown fox jumped over the lazy dog"
    "the fast brown fox jumped over the lazy dog"
    "the lazy dog sat there and did nothing"
    "the other animals sat there watching"];
documents = tokenizedDocument(str)
documents = 
  4x1 tokenizedDocument:

    9 tokens: the quick brown fox jumped over the lazy dog
    9 tokens: the fast brown fox jumped over the lazy dog
    8 tokens: the lazy dog sat there and did nothing
    6 tokens: the other animals sat there watching

Calculate the TextRank scores.

scores = textrankScores(documents);

Visualize the scores in a bar chart.

figure
bar(scores)
xlabel("Document")
ylabel("Score")
title("TextRank Scores")

Open Live Script

Create a bag-of-words model from the text data in sonnets.csv.

filename = "sonnets.csv";
tbl = readtable(filename,'TextType','string');
textData = tbl.Sonnet;
documents = tokenizedDocument(textData);
bag = bagOfWords(documents)
bag = 
  bagOfWords with properties:

          Counts: [154x3527 double]
      Vocabulary: [1x3527 string]
        NumWords: 3527
    NumDocuments: 154

Calculate the TextRank scores.

scores = textrankScores(bag);

Visualize the scores in a bar chart.

figure
bar(scores)
xlabel("Document")
ylabel("Score")
title("TextRank Scores")

Input Arguments

collapse all

Input documents, specified as a tokenizedDocument array, a string array of words, or a cell array of character vectors. If documents is not a tokenizedDocument array, then it must be a row vector representing a single document, where each element is a word. To specify multiple documents, use a tokenizedDocument array.

Input bag-of-words or bag-of-n-grams model, specified as a bagOfWords object or a bagOfNgrams object. If bag is a bagOfNgrams object, then the function treats each n-gram as a single word.

Output Arguments

collapse all

TextRank scores, returned as a N-by-1 vector, where scores(i) corresponds to the score for the ith input document and N is the number of input documents.

References

[1] Mihalcea, Rada, and Paul Tarau. "Textrank: Bringing order into text." In Proceedings of the 2004 conference on empirical methods in natural language processing, pp. 404-411. 2004.

See Also

| | | | | | |

Topics

Introduced in R2020a

Text Analytics Toolbox Documentation

Support