findThoughts: Find Thoughts

Description

Outputs most representative documents for a particular topic. Use this in order to get a better sense of the content of actual documents with a high topical content.

Usage

findThoughts(
  model,
  texts = NULL,
  topics = NULL,
  n = 3,
  thresh = NULL,
  where = NULL,
  meta = NULL
)

Value

A findThoughts object:

index: List with one entry per topic. Each entry is a vector of document indices.
docs: List with one entry per topic. Each entry is a character vector of the corresponding texts.

Arguments

model: Model object created by stm.
texts: A character vector where each entry contains the text of a document. Must be in the same order as the documents object. NOTE: This is not the documents which are passed to stm and come out of prepDocuments, this is the actual text of the document.
topics: The topic number or vector of topic numbers for which you want to find thoughts. Defaults to all topics.
n: The number of desired documents to be displayed per topic.
thresh: Sets a minimum threshold for the estimated topic proportion for displayed documents. It defaults to imposing no restrictions.
where: An expression in the form of a data.table query. This is passed to the i argument in data.table and a custom query is passed to j. This cannot be used with thresh. See below for more details.
meta: The meta data object to be used with where.

Details

Returns the top n documents ranked by the MAP estimate of the topic's theta value (which captures the modal estimate of the proportion of word tokens assigned to the topic under the model). Setting the thresh argument allows the user to specify a minimal value of theta for returned documents. Returns document indices and top thoughts.

Sometimes you may want to find thoughts which have more conditions than simply a minimum threshold. For example, you may want to grab all documents which satisfy certain conditions on the metadata or other topics. You can supply a query in the style of data.table to the where argument. Note that in data.table variables are referenced by their names in the data.table object. The topics themselves are labeled Topic1, Topic2 etc. If you supply the metadata to the meta argument, you can also query based on any available metadata. See below for examples.

If you want to pass even more complicated queries, you can use the function make.dt to generate a data.table object where you can write your own queries.

The plot.findThoughts function is a shortcut for the plotQuote function.

Examples

Run this code

findThoughts(gadarianFit, texts=gadarian$open.ended.response, topics=c(1,2), n=3)

#We can plot findThoughts objects using plot() or plotQuote
thought <- findThoughts(gadarianFit, texts=gadarian$open.ended.response, topics=1, n=3)

#plotQuote takes a set of sentences
plotQuote(thought$docs[[1]])

#we can use the generic plot as a shorthand which will make one plot per topic
plot(thought)

#we can select a subset of examples as well using either approach
plot(thought,2:3)
plotQuote(thought$docs[[1]][2:3])


#gather thoughts for only treated documents
thought <- findThoughts(gadarianFit, texts=gadarian$open.ended.response, topics=c(1,2), n=3, 
                       where = treatment==1, meta=gadarian)
plot(thought)
#you can also query in terms of other topics
thought <- findThoughts(gadarianFit, texts=gadarian$open.ended.response, topics=c(1), n=3, 
                        where = treatment==1 & Topic2>.2, meta=gadarian)
plot(thought)         
#these queries can be really complex if you like
thought <- findThoughts(gadarianFit, texts=gadarian$open.ended.response, topics=c(1), n=3, 
                       where = (treatment==1 | pid_rep > .5) & Topic3>.2, meta=gadarian)
plot(thought)