{ "$schema": "folio-library-document/v1", "id": "landauer-foltz-laham-1998-intro-lsa", "title": "An Introduction to Latent Semantic Analysis", "variant": "pdf", "pages": null, "tocSource": "none", "tocMethod": null, "toc": [], "sections": [ { "id": "page-001", "number": null, "title": "Page 1", "level": 1, "pageStart": 1, "pageEnd": 1, "labelStart": null, "labelEnd": null, "words": 64, "summary": null, "keywords": [ { "term": "latent semantic", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "psychology", "heading": false } ], "extract": "1 Running head: INTRODUCTION TO LATENT SEMANTIC ANALYSIS An Introduction to Latent Semantic Analysis Thomas K Landauer Department of Psychology University of Colorado at Boulder, Peter W. Foltz Department of Psychology New Mexico State University Darrell Laham Department of Psychology University of Colorado at Boulder, Landauer, T. K., Foltz, P. W., & Laham, D. (1998). Introduction to Latent Semantic Analysis.", "extractCut": true }, { "id": "page-002", "number": null, "title": "Page 2", "level": 1, "pageStart": 2, "pageEnd": 2, "labelStart": null, "labelEnd": null, "words": 195, "summary": null, "keywords": [ { "term": "latent semantic", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "word", "heading": false }, { "term": "mimics", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 2 Abstract Latent Semantic Analysis (LSA) is a theory and method for extracting and representing the contextual-usage meaning of words by statistical computations applied to a large corpus of text (Landauer and Dumais, 1997). The underlying idea is that the aggregate of all the word contexts in which a given word does and does not appear provides a set of mutual constraints", "extractCut": true }, { "id": "page-003", "number": null, "title": "Page 3", "level": 1, "pageStart": 3, "pageEnd": 3, "labelStart": null, "labelEnd": null, "words": 294, "summary": null, "keywords": [ { "term": "latent semantic", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "text corpora", "heading": false }, { "term": "lsa", "heading": false }, { "term": "landauer", "heading": false }, { "term": "kintsch", "heading": false }, { "term": "issue", "heading": false }, { "term": "articles", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 3 An Introduction to Latent Semantic Analysis Research reported in the three articles that followãFoltz, Kintsch & Landauer (1998/this issue), Rehder, et al. (1998/this issue), and Wolfe, et al. (1998/this issue)ãexploits a new theory of knowledge induction and representation (Landauer and Dumais, 1996, 1997) that provides a method for determining the similarity of meaning of", "extractCut": true }, { "id": "page-004", "number": null, "title": "Page 4", "level": 1, "pageStart": 4, "pageEnd": 4, "labelStart": null, "labelEnd": null, "words": 384, "summary": null, "keywords": [ { "term": "writers", "heading": false }, { "term": "similarity", "heading": false }, { "term": "lsa", "heading": false }, { "term": "correlations", "heading": false }, { "term": "meaning", "heading": false }, { "term": "phenomena", "heading": false }, { "term": "passages", "heading": false }, { "term": "representations", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 4 words and text segments that such relations may reflect, or (2) as a model of the computational processes and representations underlying substantial portions of the acquisition and utilization of knowledge. We next sketch both views. As a practical method for the characterization of word meaning, we know that LSA produces measures of word-word, word-passage and", "extractCut": true }, { "id": "page-005", "number": null, "title": "Page 5", "level": 1, "pageStart": 5, "pageEnd": 5, "labelStart": null, "labelEnd": null, "words": 418, "summary": null, "keywords": [ { "term": "lsa", "heading": false }, { "term": "words", "heading": false }, { "term": "unitary", "heading": false }, { "term": "meanings", "heading": false }, { "term": "passage", "heading": false }, { "term": "knowledge", "heading": false }, { "term": "differences", "heading": false }, { "term": "representations", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 5 words, or that their words have reflected such matters unintentionally, LSA has at least potential access to knowledge about them. The representations of passages that LSA forms \u0003\u0002\u0001\u0004$1+ \u0000\u0011# $\u0015\b \u0000\u0004 !2 !\u0000\u0002\b \u0000\u0004\u00121\u0001\u0002\u001e \u0001\u0002+ \u001e-\b4 0\u0001\u0002\u0003\u0002\b # \u0006 $ \u001e8\u0006 \u0017 \u0019\u0000\u00022 # \u001e-\u0006\u0015\u0012 \u0000\u0002\u001e \u000f\u00197 \u001e-\u0006\u0015%&\u0000\u0004\b4# %&\u0000\u0004\u001e \u0006\u0015\u0017\u0019\u0000\u00022 #4\u001e-\u0006 \u0012 \u0000\u0002\u001e8\u0006 \u0017 2 \u0014 !\u0000\u0002\" \u0016 \u0001 \u0000\u0004 !+ \u0001\u0002\" content such as philosophical arguments, and", "extractCut": true }, { "id": "page-006", "number": null, "title": "Page 6", "level": 1, "pageStart": 6, "pageEnd": 6, "labelStart": null, "labelEnd": null, "words": 372, "summary": null, "keywords": [ { "term": "lsa", "heading": false }, { "term": "dimensionality", "heading": false }, { "term": "word", "heading": false }, { "term": "meaning", "heading": false }, { "term": "relations", "heading": false }, { "term": "passages", "heading": false }, { "term": "think", "heading": false }, { "term": "kind", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 6 Another way to think of this is that LSA represents the meaning of a word as a kind of average of the meaning of all the passages in which it appears, and the meaning of a passage as a kind of average of the meaning of all the words it contains. LSA's ability to simultaneouslyãconjointlyãderive representations of these two interrelated kinds of meaning depends on an aspect", "extractCut": true }, { "id": "page-007", "number": null, "title": "Page 7", "level": 1, "pageStart": 7, "pageEnd": 7, "labelStart": null, "labelEnd": null, "words": 360, "summary": null, "keywords": [ { "term": "lsa", "heading": false }, { "term": "dimensionality", "heading": false }, { "term": "observational", "heading": false }, { "term": "space", "heading": false }, { "term": "dimensions", "heading": false }, { "term": "source", "heading": false }, { "term": "humans", "heading": false }, { "term": "meaning", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 7 than appears to be available in experience, the infamous problem of the \"insufficiency of evidence\" or \"poverty of the stimulus.\" The LSA mechanism that solves the problem consists simply of accommodating a very large number of local co-occurrence relations (between the right kinds of observational units) simultaneously in a space of the right dimensionality.", "extractCut": true }, { "id": "page-008", "number": null, "title": "Page 8", "level": 1, "pageStart": 8, "pageEnd": 8, "labelStart": null, "labelEnd": null, "words": 363, "summary": null, "keywords": [ { "term": "factor analysis", "heading": false }, { "term": "cell", "heading": false }, { "term": "discourse", "heading": false }, { "term": "stands", "heading": false }, { "term": "lsa", "heading": false }, { "term": "decomposed", "heading": false }, { "term": "row", "heading": false }, { "term": "column", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 8 knowledge.) Thus, we propose to researchers in discourse processing not only that they use LSA to expedite their investigations, but that they join in the project of testing, developing and exploring its fundamental theoretical implications and limits. What is LSA? LSA is a fully automatic mathematical/statistical technique for extracting and inferring relations of expected", "extractCut": true }, { "id": "page-009", "number": null, "title": "Page 9", "level": 1, "pageStart": 9, "pageEnd": 9, "labelStart": null, "labelEnd": null, "words": 329, "summary": null, "keywords": [ { "term": "original matrix", "heading": false }, { "term": "nine", "heading": false }, { "term": "reconstruction", "heading": false }, { "term": "titles", "heading": false }, { "term": "smallest", "heading": false }, { "term": "dimensions", "heading": false }, { "term": "cell", "heading": false }, { "term": "linear", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 9 than the smallest dimension of the original matrix. When fewer than the necessary number of factors are used, the reconstructed matrix is a least-squares best fit. One can reduce the dimensionality of the solution simply by deleting coefficients in the diagonal matrix, ordinarily starting with the smallest. (In practice, for computational reasons, for very large corpora", "extractCut": true }, { "id": "page-010", "number": null, "title": "Page 10", "level": 1, "pageStart": 10, "pageEnd": 10, "labelStart": null, "labelEnd": null, "words": 295, "summary": null, "keywords": [ { "term": "graph minors", "heading": false }, { "term": "eps", "heading": false }, { "term": "trees", "heading": false }, { "term": "response time", "heading": false }, { "term": "system response", "heading": false }, { "term": "survey", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 10 might be changed everywhere; this is the mathematical sense in which LSA performs inference or induction. Example of text data: Titles of Some Technical Memos c1: Human machine interface for ABC computer applications c2: A survey of user opinion of computer system response time c3: The EPS user interface management system c4: System and human system engineering testing of", "extractCut": true }, { "id": "page-011", "number": null, "title": "Page 11", "level": 1, "pageStart": 11, "pageEnd": 11, "labelStart": null, "labelEnd": null, "words": 278, "summary": null, "keywords": [ { "term": "appear", "heading": false }, { "term": "occurred", "heading": false }, { "term": "greater", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 11 The dimension reduction step has collapsed the component matrices in such a way that words that occurred in some contexts now appear with greater or lesser estimated frequency, and some that did not appear originally now do appear, at least fractionally. X W S P { } = { }{ }{ } ' W { } = 0.22 -0.11 0.29 -0.41 -0.11 -0.34 0.52 -0.06 -0.41 0.20 -0.07 0.14 -0.55 0.28 0.50", "extractCut": true }, { "id": "page-012", "number": null, "title": "Page 12", "level": 1, "pageStart": 12, "pageEnd": 12, "labelStart": null, "labelEnd": null, "words": 357, "summary": null, "keywords": [ { "term": "minors", "heading": false }, { "term": "shaded", "heading": false }, { "term": "cells", "heading": false }, { "term": "survey", "heading": false }, { "term": "graph", "heading": false }, { "term": "trees", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 12 ˆX { } = c1 c2 c3 c4 c5 m1 m2 m3 m4 human 0.16 0.40 0.38 0.47 0.18 -0.05 -0.12 -0.16 -0.09 interface 0.14 0.37 0.33 0.40 0.16 -0.03 -0.07 -0.10 -0.04 computer 0.15 0.51 0.36 0.41 0.24 0.02 0.06 0.09 0.12 user 0.26 0.84 0.61 0.70 0.39 0.03 0.08 0.12 0.19 system 0.45 1.23 1.05 1.27 0.56 -0.07 -0.15 -0.21 -0.05 response 0.16 0.58 0.38 0.42 0.28 0.06 0.13 0.19 0.22 time 0.16", "extractCut": true }, { "id": "page-013", "number": null, "title": "Page 13", "level": 1, "pageStart": 13, "pageEnd": 13, "labelStart": null, "labelEnd": null, "words": 406, "summary": null, "keywords": [ { "term": "correlations among", "heading": false }, { "term": "theory papers", "heading": false }, { "term": "raw co-occurrence", "heading": false }, { "term": "graph theory", "heading": false }, { "term": "minors", "heading": false }, { "term": "abstract concept", "heading": false }, { "term": "co-occurrence data", "heading": false }, { "term": "reconstructed", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 13 This text segment is best described as having so much of abstract concept one and so much of abstract concept two, and this word has so much of concept one and so much of concept two, and combining those two pieces of information (by vector arithmetic), my best guess is that word X actually appeared 0.6 times in context Y. Now let us consider what such changes may do to", "extractCut": true }, { "id": "page-014", "number": null, "title": "Page 14", "level": 1, "pageStart": 14, "pageEnd": 14, "labelStart": null, "labelEnd": null, "words": 276, "summary": null, "keywords": [ { "term": "hci titles", "heading": false }, { "term": "correlations", "heading": false }, { "term": "raw data", "heading": false }, { "term": "reconstruction", "heading": false }, { "term": "dimensional", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 14 Correlations between titles in raw data: c1 c2 c3 c4 c5 m1 m2 m3 c2 -0.19 c3 0.00 0.00 c4 0.00 0.00 0.47 c5 -0.33 0.58 0.00 -0.31 m1 -0.17 -0.30 -0.21 -0.16 -0.17 m2 -0.26 -0.45 -0.32 -0.24 -0.26 0.67 m3 -0.33 -0.58 -0.41 -0.31 -0.33 0.52 0.77 m4 -0.33 -0.19 -0.41 -0.31 -0.33 -0.17 0.26 0.56 0.02 -0.30 0.44 Correlations in two dimensional space: c2 0.91 c3 1.00 0.91 c4", "extractCut": true }, { "id": "page-015", "number": null, "title": "Page 15", "level": 1, "pageStart": 15, "pageEnd": 15, "labelStart": null, "labelEnd": null, "words": 320, "summary": null, "keywords": [ { "term": "inferences", "heading": false }, { "term": "ann", "heading": false }, { "term": "mother", "heading": false }, { "term": "mary", "heading": false }, { "term": "bob", "heading": false }, { "term": "son", "heading": false }, { "term": "john", "heading": false }, { "term": "relations", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 15 data are to be accommodated in two dimensions. In this case, the inferences appear to be intuitively sensible. Note that much of the information that LSA used to infer relations among words and passages is in data about passages in which particular words did not occur. Indeed, Landauer and Dumais (1997) found that in LSA simulations of schoolchild word knowledge", "extractCut": true }, { "id": "page-016", "number": null, "title": "Page 16", "level": 1, "pageStart": 16, "pageEnd": 16, "labelStart": null, "labelEnd": null, "words": 339, "summary": null, "keywords": [ { "term": "nearby locations", "heading": false }, { "term": "meanings even", "heading": false }, { "term": "even though", "heading": false }, { "term": "dimensionality", "heading": false }, { "term": "dimensions", "heading": false }, { "term": "infer", "heading": false }, { "term": "original data", "heading": false }, { "term": "space", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 16 generated from a source of the same dimensionality and general structure as the reconstruction. Suppose, for example, that speakers or writers generate paragraphs by choosing words from a k-dimensional space in such a way that words in the same paragraph tend to be selected from nearby locations. If listeners or readers try to infer the similarity of meaning from these", "extractCut": true }, { "id": "page-017", "number": null, "title": "Page 17", "level": 1, "pageStart": 17, "pageEnd": 17, "labelStart": null, "labelEnd": null, "words": 364, "summary": null, "keywords": [ { "term": "training corpora", "heading": false }, { "term": "lsa", "heading": false }, { "term": "passages", "heading": false }, { "term": "entropy", "heading": false }, { "term": "investigators", "heading": false }, { "term": "transformation", "heading": false }, { "term": "words", "heading": false }, { "term": "cell", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 17 Additional detail about LSA As mentioned, one additional part of the analysis, the data preprocessing transformation, needs to be described more fully. Before the SVD is computed, it is customary in LSA to subject the data in the raw word-by-context matrix to a two-part transformation. First, the word frequency (+ 1) in each cell is converted to its log. Second, the", "extractCut": true }, { "id": "page-018", "number": null, "title": "Page 18", "level": 1, "pageStart": 18, "pageEnd": 18, "labelStart": null, "labelEnd": null, "words": 257, "summary": null, "keywords": [ { "term": "assessed", "heading": false }, { "term": "predictor", "heading": false }, { "term": "lsa", "heading": false }, { "term": "simulation", "heading": false }, { "term": "replicate", "heading": false }, { "term": "word-word", "heading": false }, { "term": "synonym", "heading": false }, { "term": "human", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 18 and are necessarily noisy. Therefore, studies using them require the use of replicate cases and statistical treatment in a manner similar to human data. \u0000\u0002\u0001\u0004\u0003\u0006\u0005\u0007\u0003 \b\u0004 \u000f\u000e\u0006 \u000f\u0010\u0006\u0011\u0012\u0010\u0014\u0013\u0004\u0015\u0016 \u0018\u0017\u0018\u0019\u0004\u001a\u001c\u001b\u001e\u001d \u001f \u0010\u001e\u001d\u0004!\"\u0015\"#\u0004 $\u0019\u0004\u001b\u001e \u0018%\u0018\u001d\u0004\u0010\u001e&' \u0015\"\u0013\u0004(\u001e\u0015 How well does LSA actually work as a representational model and measure of human verbal concepts? Its performance has been assessed more or less", "extractCut": true }, { "id": "page-019", "number": null, "title": "Page 19", "level": 1, "pageStart": 19, "pageEnd": 19, "labelStart": null, "labelEnd": null, "words": 426, "summary": null, "keywords": [ { "term": "weighted average", "heading": false }, { "term": "standard collections", "heading": false }, { "term": "query", "heading": false }, { "term": "information retrieval", "heading": false }, { "term": "documents", "heading": false }, { "term": "lsi", "heading": false }, { "term": "meaning", "heading": false }, { "term": "queries", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 19 metaphor and context effects in decision making. We will take space here to review only some of the most systematic and pertinent of these results. LSA and information retrieval J. R. Anderson (1990) has called attention to the analogy between information retrieval and human semantic memory processes. One way of expressing their commonality is to think of a searcher as", "extractCut": true }, { "id": "page-020", "number": null, "title": "Page 20", "level": 1, "pageStart": 20, "pageEnd": 20, "labelStart": null, "labelEnd": null, "words": 385, "summary": null, "keywords": [ { "term": "synonym", "heading": false }, { "term": "lsi", "heading": false }, { "term": "antonym", "heading": false }, { "term": "words", "heading": false }, { "term": "dumais", "heading": false }, { "term": "near", "heading": false }, { "term": "somewhat", "heading": false }, { "term": "papers", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 20 performance ranged from just equivalent to the best prior methods up to about 30% better. In a recent project sponsored by the National Institute of Standards and Technology, LSI was compared with a large number of other research prototypes and commercial retrieval schemes. Direct quantitative comparisons among the many systems were somewhat muddied by the use of varying", "extractCut": true }, { "id": "page-021", "number": null, "title": "Page 21", "level": 1, "pageStart": 21, "pageEnd": 21, "labelStart": null, "labelEnd": null, "words": 461, "summary": null, "keywords": [ { "term": "ivens", "heading": false }, { "term": "lsa", "heading": false }, { "term": "ets", "heading": false }, { "term": "grade", "heading": false }, { "term": "encyclopedia", "heading": false }, { "term": "similarities", "heading": false }, { "term": "mind", "heading": false }, { "term": "word", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 21 Nonetheless, the relationship between some close neighbors in LSA space can \u0006\u0015\u0003\u0004\u0003\u0002\u0001\u0002\u001e,# \u0006\u0015$ \u0001\u0002\"4\" \u00161+\u0015\u0000\u0011%&\u0016\u0015\u001e-\b4\u0000\u0002 !# \u0006 \u0014\u0015\u001e \u0002!\u00005 * 7\u0007 \u0001 \u0000\u0002 !+ \u0001\u0002\"4\" \u0016\u0007\u000f \u0001\u0002$\u0015\u0012 \u0019\u001e-\u0001\u0002\u0012\u0015\u0006 % \u0001\u0004\u001e,\u0006 \u0003\u0002\u0005\u0015# \u001e,% \u000f ()#4\b \u00051\u0001 \u0003\u0002\u0006 \u001e,# $\u0015\u0000\u0011\u0006\u0015\u0017 \u0000from the encyclopedia space), and some pairs that should be close are not. It's impossible to say exactly why these oddities occur, but it is plausible that some", "extractCut": true }, { "id": "page-022", "number": null, "title": "Page 22", "level": 1, "pageStart": 22, "pageEnd": 22, "labelStart": null, "labelEnd": null, "words": 429, "summary": null, "keywords": [ { "term": "correct alternative", "heading": false }, { "term": "average correlation", "heading": false }, { "term": "lsa", "heading": false }, { "term": "students", "heading": false }, { "term": "stem", "heading": false }, { "term": "chose", "heading": false }, { "term": "word", "heading": false }, { "term": "million", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 22 been used. In one experiment, an SVD was performed on text segments consisting of 500 \u0003\u0002\u0005 \u0001\u0002 !\u0001\u0002\u0003\u0002\b \u0000\u0002 !\u001e8\u0006 \" \u0000\u0002\u001e-\u001e \u0002!\u0006\u0015$1\u0001 \u0001 \u0000\u0002 !\u0001\u0002* \u0000 91(.\u0006 !\u0012\u0015\u001e-7 \u0001\u0002+ \u0006\u0015\u0014 \b5\u0001 2 \u0001\u0002 !\u0001\u0002* !\u0001\u00022\u0015\u00051\u001e8()\u0006\u0015 !\b \u0005 \b \u0001\u0002' \u0000\u0002$1\u0017! !\u0006 % +\u0015\u0000\u0002* # $\u0015$\u0015# $ * portions of each of 30,473 articles in the encyclopedia, a total of 4.5 million words of text, roughly equivalent to what a child would have read by", "extractCut": true }, { "id": "page-023", "number": null, "title": "Page 23", "level": 1, "pageStart": 23, "pageEnd": 23, "labelStart": null, "labelEnd": null, "words": 195, "summary": null, "keywords": [ { "term": "dimension reduction", "heading": false }, { "term": "guessing", "heading": false }, { "term": "co-occurrence", "heading": false }, { "term": "correct", "heading": false }, { "term": "correlation", "heading": false }, { "term": "lsa", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 23 To assess the role of dimension reduction, the number of dimensions was varied from 2 to 1,032 (the largest number for which SVD was computationally feasible.) On log- linear coordinates, the TOEFL test results showed a very sharp and highly significant peak (Figure 5). Corrected for guessing by the standard formula ((correct - chance)/(1- chance)), LSA got 52.7% correct", "extractCut": true }, { "id": "page-024", "number": null, "title": "Page 24", "level": 1, "pageStart": 24, "pageEnd": 24, "labelStart": null, "labelEnd": null, "words": 480, "summary": null, "keywords": [ { "term": "word sorting", "heading": false }, { "term": "primary school", "heading": false }, { "term": "anglin", "heading": false }, { "term": "landauer", "heading": false }, { "term": "simulating", "heading": false }, { "term": "paragraph", "heading": false }, { "term": "lsa", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 24 relation. This demonstrates conclusively that the LSA dimension reduction technique captures much more than mere co-occurrence (simply choosing the alternative that co- occurs with the stem in the largest number of corpus paragraphs gets only 11% right when corrected for guessing). More importantly for our argument, it implies that indirect associations or structural", "extractCut": true }, { "id": "page-025", "number": null, "title": "Page 25", "level": 1, "pageStart": 25, "pageEnd": 25, "labelStart": null, "labelEnd": null, "words": 361, "summary": null, "keywords": [ { "term": "grade scaling", "heading": false }, { "term": "lsa estimates", "heading": false }, { "term": "level scaling", "heading": false }, { "term": "college level", "heading": false }, { "term": "child data", "heading": false }, { "term": "both child", "heading": false }, { "term": "adults", "heading": false }, { "term": "part-of-speech", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 25 versus concrete similarity relations. Anglin measured the semantic similarity of every pair of words by the proportion of subjects who put them in the same pile. He found that parts of speech clustered moderately in both child and adult sets, and, confirming the hypothesis behind the study, that adults showed more evidence of use of abstract categories than did children.", "extractCut": true }, { "id": "page-026", "number": null, "title": "Page 26", "level": 1, "pageStart": 26, "pageEnd": 26, "labelStart": null, "labelEnd": null, "words": 425, "summary": null, "keywords": [ { "term": "part-of-speech cosine", "heading": false }, { "term": "introductory psychology", "heading": false }, { "term": "foltz", "heading": false }, { "term": "lsa", "heading": false }, { "term": "university", "heading": false }, { "term": "laham", "heading": false }, { "term": "landauer", "heading": false }, { "term": "documents", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 26 average cosine =.19, the average within part-of-speech cosine = .23 and the average between part-of-speech cosine = .17. As in the vocabulary acquisition simulations, it appears that the relations obtained \u00170 0\u0006 % \u0001 \u0003\u0004\u0006 !2 \u0014 \u001e \u0006 \u0017\u0019\u001e-%&\u0001\u0004\" \"\u0015\u001e-#\u0001\u0000\u0004\u0000 !\u0000\u0004\" \u0001\u0004\b # \u0001 \u0000\b \u0006\u0007\u0001 \b \u0016 2 # \u0003\u0002\u0001\u0004\" \u0001\u0004\u0012 \u0014 \" \b \u001e \u0003\u0004\u0014 %&\u0014 \" \u0001\u0002\b # \u0001 \u0000\" \u0001\u0004$ * \u0014 \u0001\u0004* \u0000\u0000\u00023 2 \u0006 \u001e-\u0014 0\u0000 !\u0000\u0004\u001e-\u0000\u0002%&+ \"4\u0000\u0011\u0003\u0004\u0005\u0015# \" \u0012", "extractCut": true }, { "id": "page-027", "number": null, "title": "Page 27", "level": 1, "pageStart": 27, "pageEnd": 27, "labelStart": null, "labelEnd": null, "words": 408, "summary": null, "keywords": [ { "term": "lsa predictions", "heading": false }, { "term": "polysemous word", "heading": false }, { "term": "semantic priming", "heading": false }, { "term": "target words", "heading": false }, { "term": "control word", "heading": false }, { "term": "mint", "heading": false }, { "term": "passages", "heading": false }, { "term": "correlated", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 27 relatedness of the concept pairs. The LSA predictions correlated significantly with the subjects, with the correlation stronger to that of the experts in the domain ( r = 0.41) than that of the novices ( r = 0.36). (Note again that two human ratings would also not correlated perfectly.) An analysis of where LSA's predictions deviated greatly from that of the humans", "extractCut": true }, { "id": "page-028", "number": null, "title": "Page 28", "level": 1, "pageStart": 28, "pageEnd": 28, "labelStart": null, "labelEnd": null, "words": 346, "summary": null, "keywords": [ { "term": "inference words", "heading": false }, { "term": "target words", "heading": false }, { "term": "priming", "heading": false }, { "term": "passages", "heading": false }, { "term": "mimic", "heading": false }, { "term": "cosines", "heading": false }, { "term": "lsa", "heading": false }, { "term": "associates", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 28 In the Till et al. study, target words related to both senses of the homographic words were correctly responded to faster than unrelated control words if presented within 100 ms after the homograph. If delayed by 300 ms, only the context-appropriate associate was primed. At a one second delay, the so-called inference words were also primed. In the LSA simulation, the", "extractCut": true }, { "id": "page-029", "number": null, "title": "Page 29", "level": 1, "pageStart": 29, "pageEnd": 29, "labelStart": null, "labelEnd": null, "words": 234, "summary": null, "keywords": [ { "term": "combining mechanism", "heading": false }, { "term": "candy", "heading": false }, { "term": "latent semantic", "heading": false }, { "term": "lsa", "heading": false }, { "term": "till", "heading": false }, { "term": "priming", "heading": false }, { "term": "semantic analysis", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 29 Table 1 LSA Simulation of Till, Mross, & Kintsch (1988) Priming Study. Mint: Money Candy Ground .21 .20 .07 Thinking amount garlic dinner guest asked: Money Candy .15 .21 Ground Earthquake Breath .15 .14 .21 Note. LSA = Latent Semantic Analysis. Of course, LSA as currently constituted contains no model of the temporal dynamics of discourse comprehension. To fit the", "extractCut": true }, { "id": "page-030", "number": null, "title": "Page 30", "level": 1, "pageStart": 30, "pageEnd": 30, "labelStart": null, "labelEnd": null, "words": 374, "summary": null, "keywords": [ { "term": "student essay", "heading": false }, { "term": "holistic quality", "heading": false }, { "term": "quality scores", "heading": false }, { "term": "cosine", "heading": false }, { "term": "distances", "heading": false }, { "term": "dimension", "heading": false }, { "term": "sentences", "heading": false }, { "term": "lsa", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 30 satisfaction mechanisms, or even to models including hierarchical syntactic structure (presumably, automatically induced), only if and as found necessary. Assigning holistic quality scores to essay test answers In another set of studies to be published elsewhere by Landauer, Laham and Foltz (1998), LSA has been used to assign holistic quality scores to written answers to", "extractCut": true }, { "id": "page-031", "number": null, "title": "Page 31", "level": 1, "pageStart": 31, "pageEnd": 31, "labelStart": null, "labelEnd": null, "words": 376, "summary": null, "keywords": [ { "term": "kintsch", "heading": false }, { "term": "correlated", "heading": false }, { "term": "coherence", "heading": false }, { "term": "comprehension", "heading": false }, { "term": "expert", "heading": false }, { "term": "lsa", "heading": false }, { "term": "discourse", "heading": false }, { "term": "scores", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 31 All five methods provided the basis of scores that correlated approximately as well with expert assigned scores as such scores correlated with each other, sometimes slightly less well, on average somewhat better. In one set of studies (Laham, 1997a), method one was applied to a total of eight exams ranging in topic from heart anatomy and physiology, through psychological", "extractCut": true }, { "id": "page-032", "number": null, "title": "Page 32", "level": 1, "pageStart": 32, "pageEnd": 32, "labelStart": null, "labelEnd": null, "words": 389, "summary": null, "keywords": [ { "term": "kintsch", "heading": false }, { "term": "coherence", "heading": false }, { "term": "lsa", "heading": false }, { "term": "student", "heading": false }, { "term": "text", "heading": false }, { "term": "issue", "heading": false }, { "term": "foltz", "heading": false }, { "term": "articles", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 32 propositionalized by hand. This has limited research to small samples of text and has inhibited its practical application to composition and instruction. Foltz, Kintsch, and Landauer (1993, this issue; Foltz, 1996) have applied LSA to the task. LSA can make automatic coherence judgments by computing the cosine from one sentence or passage and the following one. In one", "extractCut": true }, { "id": "page-033", "number": null, "title": "Page 33", "level": 1, "pageStart": 33, "pageEnd": 33, "labelStart": null, "labelEnd": null, "words": 351, "summary": null, "keywords": [ { "term": "human cognitive", "heading": false }, { "term": "detailed processes", "heading": false }, { "term": "lsa", "heading": false }, { "term": "lacks", "heading": false }, { "term": "phenomena", "heading": false }, { "term": "theory", "heading": false }, { "term": "lexical", "heading": false }, { "term": "ability", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 33 conceptual continuum, such as level of sophistication or relevance to a particular topic, can be measured. Summary and some caveats It is clear enough from the conjunction of all these formal and informal results that LSA is able to capture and represent significant components of the lexical and passage meanings evinced in judgment and behavior by humans. The following", "extractCut": true }, { "id": "page-034", "number": null, "title": "Page 34", "level": 1, "pageStart": 34, "pageEnd": 34, "labelStart": null, "labelEnd": null, "words": 384, "summary": null, "keywords": [ { "term": "discourse comprehension", "heading": false }, { "term": "corpora", "heading": false }, { "term": "brain", "heading": false }, { "term": "lsa", "heading": false }, { "term": "representative", "heading": false }, { "term": "kintsch", "heading": false }, { "term": "seems", "heading": false }, { "term": "mind", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 34 rigorous computational simulation that takes in the very same data from which humans learn about words and passages and produces a representation that gives veridical simulations of a wide range of human judgments and behavior. While it seems highly doubtful that the human brain uses the same mathematical algorithms as LSA/SVD, it seems almost certain that the brain uses", "extractCut": true }, { "id": "page-035", "number": null, "title": "Page 35", "level": 1, "pageStart": 35, "pageEnd": 35, "labelStart": null, "labelEnd": null, "words": 185, "summary": null, "keywords": [ { "term": "lsa", "heading": false }, { "term": "thousands", "heading": false }, { "term": "undergo", "heading": false }, { "term": "hundreds", "heading": false }, { "term": "millions", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 35 unavailability and difficulty of constructing large general or topically relevant text samples that approximate what a variety of individual learners would have met. But another is due to current computational limitations. LSA became practical only when computational power and algorithm efficiency improved sufficiently to support SVD of thousands of words-by-thousands of", "extractCut": true }, { "id": "page-036", "number": null, "title": "Page 36", "level": 1, "pageStart": 36, "pageEnd": 36, "labelStart": null, "labelEnd": null, "words": 258, "summary": null, "keywords": [ { "term": "latent semantic", "heading": false }, { "term": "dumais", "heading": false }, { "term": "information retrieval", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "anglin", "heading": false }, { "term": "discourse", "heading": false }, { "term": "berry", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 36 REFERENCES Anderson, J. R. (1990). The adaptive character of thought . Hillsdale, NJ: Lawrence Erlbaum Associates. Anglin, J. M. (1970) The growth of word meaning. Cambridge, MA.: MIT Press. Anglin, J. M., Alexander, T. M., & Johnson, C. J. (1996). Word learning and the growth of potentially knowable vocabulary. Submitted for publication. Berry, M. W. (1992). Large scale", "extractCut": true }, { "id": "page-037", "number": null, "title": "Page 37", "level": 1, "pageStart": 37, "pageEnd": 37, "labelStart": null, "labelEnd": null, "words": 254, "summary": null, "keywords": [ { "term": "latent semantic", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "computing machinery", "heading": false }, { "term": "kintsch", "heading": false }, { "term": "foltz", "heading": false }, { "term": "comprehension", "heading": false }, { "term": "erlbaum", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 37 Foltz, P. W. (1996) Latent Semantic Analysis for text-based research. Behavior Research Methods, Instruments and Computers. 28, 197-202. Foltz, P. W., Britt, M. A., & Perfetti, C. A. (1996) Reasoning from multiple texts: An automatic analysis of readers' situation models. In G. Cottrell (Ed.) Proceedings of the 18th Annual Cognitive Science Conference. Hillsdale, NJ:", "extractCut": true }, { "id": "page-038", "number": null, "title": "Page 38", "level": 1, "pageStart": 38, "pageEnd": 38, "labelStart": null, "labelEnd": null, "words": 295, "summary": null, "keywords": [ { "term": "lawrence erlbaum", "heading": false }, { "term": "erlbaum associates", "heading": false }, { "term": "latent semantic", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "unpublished manuscript", "heading": false }, { "term": "educational testing", "heading": false }, { "term": "science society", "heading": false }, { "term": "cognitive science", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 38 Laham, D. (1997b). Latent Semantic Analysis Approaches to Categorization. In M. G. Shafto & P. Langley (Eds.), Proceedings of the 19th Annual Conference of the Cognitive Science Society (p. 979). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Landauer, T.K. and Dumais, S.T. (1994). Latent semantic analysis and the measurement of knowledge. In R. M. Kaplan and J. C.", "extractCut": true }, { "id": "page-039", "number": null, "title": "Page 39", "level": 1, "pageStart": 39, "pageEnd": 39, "labelStart": null, "labelEnd": null, "words": 219, "summary": null, "keywords": [ { "term": "discourse processes", "heading": false }, { "term": "kintsch", "heading": false }, { "term": "latent semantic", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "schreiner", "heading": false }, { "term": "cognition", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 39 McNamara, D. S., Kintsch, E., Songer B. N., & Kintsch, W. (1996) Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction , 14 , 1, 1-43. Rehder, B., Schreiner, M. E., Wolfe, B. W., Laham, D., Landauer, T. K., & Kintsch, W. (1998/this issue). Using Latent Semantic", "extractCut": true }, { "id": "page-040", "number": null, "title": "Page 40", "level": 1, "pageStart": 40, "pageEnd": 40, "labelStart": null, "labelEnd": null, "words": 206, "summary": null, "keywords": [ { "term": "lsa", "heading": false }, { "term": "essay", "heading": false }, { "term": "demonstrations", "heading": false }, { "term": "save", "heading": false }, { "term": "website", "heading": false }, { "term": "page", "heading": true } ], "extract": "Introduction to Latent Semantic Analysis 40 Appendix The latest information and applications of LSA can be found at our website: http://LSA.colorado.edu/ This website is organized into three content areas, Information, Demonstrations, and Applications. The Information section contains additional papers, links, and other pertinent information on LSA. The Demonstrations section currently includes examples of essay", "extractCut": true }, { "id": "page-041", "number": null, "title": "Page 41", "level": 1, "pageStart": 41, "pageEnd": 41, "labelStart": null, "labelEnd": null, "words": 99, "summary": null, "keywords": [ { "term": "landauer", "heading": false }, { "term": "colorado", "heading": false }, { "term": "latent semantic", "heading": false } ], "extract": "Introduction to Latent Semantic Analysis 41 Author Note Darrell Laham and Thomas K Landauer, Department of Psychology, University of Colorado, Boulder. Peter W. Foltz, Department of Psychology, New Mexico State University. This research was supported in part by a contract from ARPA-CAETI to T. Landauer and W. Kintsch. Correspondence concerning this article should be addressed to Thomas K Landauer, Department of", "extractCut": true } ], "checks": {}, "withheld": false, "pagesCovered": 41, "keywords": [ { "term": "latent semantic", "heading": false }, { "term": "semantic analysis", "heading": false }, { "term": "lsa vectors", "heading": false }, { "term": "lsa space", "heading": false }, { "term": "lsa measure", "heading": false }, { "term": "lsa simulation", "heading": false }, { "term": "analysis words", "heading": false }, { "term": "lsa estimates", "heading": false }, { "term": "lsa induces", "heading": false }, { "term": "lsa chose", "heading": false }, { "term": "lsa got", "heading": false }, { "term": "lsa performs", "heading": false } ], "links": { "repo": "litlfred/cat-harness", "branch": "main", "dir": "library/landauer-foltz-laham-1998-intro-lsa" } }