{ "$schema": "folio-library-entry/v1", "id": "landauer-foltz-laham-1998-intro-lsa", "blocks": [ { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-001", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 1", "pageStart": 1, "pageEnd": 1, "target": "../sections/page-001.md", "narrative": null, "content": "1\nRunning head: INTRODUCTION TO LATENT SEMANTIC ANALYSIS\nAn Introduction to Latent Semantic Analysis\nThomas K Landauer\nDepartment of Psychology\nUniversity of Colorado at Boulder,\nPeter W. Foltz\nDepartment of Psychology\nNew Mexico State University\nDarrell Laham\nDepartment of Psychology\nUniversity of Colorado at Boulder,\nLandauer, T. K., Foltz, P. W., & Laham, D. (1998).\n Introduction to Latent Semantic Analysis.\n Discourse Processes, 25, 259-284.", "truncated": false, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-002", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 2", "pageStart": 2, "pageEnd": 2, "target": "../sections/page-002.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n2\nAbstract\nLatent Semantic Analysis (LSA) is a theory and method for extracting and representing the\ncontextual-usage meaning of words by statistical computations applied to a large corpus of\ntext (Landauer and Dumais, 1997). The underlying idea is that the aggregate of all the word\ncontexts in which a given word does and does not appear provides a set of mutual\nconstraints that largely determines the similarity of meaning of words and sets of words to\n\u0000\u0002\u0001\u0004\u0003\u0002\u0005\u0007\u0006\t\b\n\u0005\t\u0000\u0002\u000b\r\f\u000f\u000e\u0010\u0005\t\u0000\u0011\u0001\u0002\u0012\t\u0000\u0004\u0013\u0015\u0014\t\u0001\u0004\u0003\u0002\u0016\u0007\u0006\u0015\u0017\u0019\u0018\u001b\u001a\u001d\u001c\u001f\u001e \u000b!\u0000\u0004\u0017!\"\n\u0000\u0004\u0003\u0002\b\n#\n\u0006\t$\u0007\u0006\u0015\u0017\u0019\u0005\t\u0014\u0015%&\u0001\u0004$\u0007'\u0015$\t\u0006\t()\"\n\u0000\u0002\u0012", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-003", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 3", "pageStart": 3, "pageEnd": 3, "target": "../sections/page-003.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n3\nAn Introduction to Latent Semantic Analysis\nResearch reported in the three articles that followãFoltz, Kintsch & Landauer (1998/this\nissue), Rehder, et al. (1998/this issue), and Wolfe, et al. (1998/this issue)ãexploits a new\ntheory of knowledge induction and representation (Landauer and Dumais, 1996, 1997) that\nprovides a method for determining the similarity of meaning of words and passages by\nanalysis of large text corpora. After processing a large sample of machine-readable\nlanguage, Latent Semantic Analysis (LSA) represents the words used in it,", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-004", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 4", "pageStart": 4, "pageEnd": 4, "target": "../sections/page-004.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n4\nwords and text segments that such relations may reflect, or (2) as a model of the\ncomputational processes and representations underlying substantial portions of the\nacquisition and utilization of knowledge. We next sketch both views.\nAs a practical method for the characterization of word meaning, we know that LSA\nproduces measures of word-word, word-passage and passage-passage relations that are\nwell correlated with several human cognitive phenomena involving association or semantic\nsimilarity. Empirical evidence of this will be reviewed shortly. The", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-005", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 5", "pageStart": 5, "pageEnd": 5, "target": "../sections/page-005.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n5\nwords, or that their words have reflected such matters unintentionally, LSA has at least\npotential access to knowledge about them. The representations of passages that LSA forms\n\u0003\u0002\u0001\u0004$1+\t\u0000\u0011#\n$\u0015\b\n\u0000\u0004\u000b!2\t\u000b!\u0000\u0002\b\n\u0000\u0004\u00121\u0001\u0002\u001e \u0001\u0002+\t\u001e-\b4\u000b0\u0001\u0002\u0003\u0002\b\n#\n\u0006\t$\t\u001e8\u0006\t\u0017\n\r\u0019\u0000\u00022\t#\n\u001e-\u0006\u0015\u0012\t\u0000\u0002\u001e\n\u000f\u00197\n\u001e-\u0006\u0015%&\u0000\u0004\b4#\n%&\u0000\u0004\u001e\n\u0006\u0015\u0017\u0019\u0000\u00022\t#4\u001e-\u0006\t\u0012\t\u0000\u0002\u001e8\u0006\t\u0017\n2\t\u0014\t\u000b!\u0000\u0002\"\n\u0016\n\u0001\n\u0000\u0004\u000b!+\t\u0001\u0002\"\ncontent such as philosophical arguments, and sometimes episodes from real or imagined\nlife coded into verbal descriptions. Its representation of words, in turn, is intertwined with\n\u0001\u0002$\t\u0012\u0007%&\u0014\u0015\b\n\u0014\t\u0001\u0004\"\n\"\n\u00161#\n$\t\b\n\u0000\u0002\u000b0\u0012\u0015\u0000\u00042\t\u0000\u0002$\t\u0012\t\u0000", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-006", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 6", "pageStart": 6, "pageEnd": 6, "target": "../sections/page-006.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n6\nAnother way to think of this is that LSA represents the meaning of a word as a kind\nof average of the meaning of all the passages in which it appears, and the meaning of a\npassage as a kind of average of the meaning of all the words it contains. LSA's ability to\nsimultaneouslyãconjointlyãderive representations of these two interrelated kinds of\nmeaning depends on an aspect of its mathematical machinery that is its second important\nproperty. LSA assumes that the choice of dimensionality in which all of the local word-\ncontext relations are simultaneo", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-007", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 7", "pageStart": 7, "pageEnd": 7, "target": "../sections/page-007.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n7\nthan appears to be available in experience, the infamous problem of the \"insufficiency of\nevidence\" or \"poverty of the stimulus.\" The LSA mechanism that solves the problem\nconsists simply of accommodating a very large number of local co-occurrence relations\n(between the right kinds of observational units) simultaneously in a space of the right\ndimensionality. Hypothetically, the optimal space for the reconstruction has the same\ndimensionality as the source that generates discourse, that is, the human speaker or writer's\nsemantic space. Naturally obser", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-008", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 8", "pageStart": 8, "pageEnd": 8, "target": "../sections/page-008.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n8\nknowledge.) Thus, we propose to researchers in discourse processing not only that they\nuse LSA to expedite their investigations, but that they join in the project of testing,\ndeveloping and exploring its fundamental theoretical implications and limits.\n \nWhat is LSA? \nLSA is a fully automatic mathematical/statistical technique for extracting and inferring\nrelations of expected contextual usage of words in passages of discourse. It is not a\ntraditional natural language processing or artificial intelligence program; it uses no humanly\nconstructed dict", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-009", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 9", "pageStart": 9, "pageEnd": 9, "target": "../sections/page-009.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n9\nthan the smallest dimension of the original matrix. When fewer than the necessary number\nof factors are used, the reconstructed matrix is a least-squares best fit. One can reduce the\ndimensionality of the solution simply by deleting coefficients in the diagonal matrix,\nordinarily starting with the smallest. (In practice, for computational reasons, for very large\ncorpora only a limited number of dimensionsãcurrently a few thousandã can be\nconstructed.)\nHere is a small example that gives the flavor of the analysis and demonstrates what\nthe technique acc", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-010", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 10", "pageStart": 10, "pageEnd": 10, "target": "../sections/page-010.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n10\nmight be changed everywhere; this is the mathematical sense in which LSA performs\ninference or induction.\nExample of text data: Titles of Some Technical Memos\nc1:\nHuman machine interface for ABC computer applications\nc2:\nA survey of user opinion of computer system response time\nc3:\nThe EPS user interface management system\nc4:\nSystem and human system engineering testing of EPS\nc5:\nRelation of user perceived response time to error measurement\nm1: \nThe generation of random, binary, ordered trees\nm2:\nThe intersection graph of paths in trees\nm3:\nGraph min", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-011", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 11", "pageStart": 11, "pageEnd": 11, "target": "../sections/page-011.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n11\nThe dimension reduction step has collapsed the component matrices in such a way\nthat words that occurred in some contexts now appear with greater or lesser estimated\nfrequency, and some that did not appear originally now do appear, at least fractionally.\nX\nW\nS\nP\n{ } = { }{ }{ }\n'\nW\n{\n} =\n 0.22\n-0.11\n 0.29\n-0.41\n-0.11\n-0.34\n 0.52\n-0.06\n-0.41\n 0.20\n-0.07\n 0.14\n-0.55\n 0.28\n 0.50\n-0.07\n-0.01\n-0.11\n 0.24\n 0.04\n-0.16\n-0.59\n-0.11\n-0.25\n-0.30\n 0.06\n 0.49\n 0.40\n 0.06\n-0.34\n 0.10\n 0.33\n 0.38\n 0.00\n 0.00\n 0.01\n 0.64\n-0.17\n 0.36\n 0.33\n-0.16\n-0.21\n-0.17\n 0.03\n 0.", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-012", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 12", "pageStart": 12, "pageEnd": 12, "target": "../sections/page-012.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n12\nˆX\n{ } =\nc1\nc2\nc3\nc4\nc5\nm1\nm2\nm3\nm4\nhuman\n 0.16\n 0.40\n 0.38\n 0.47\n 0.18\n-0.05\n-0.12\n-0.16\n-0.09\ninterface\n 0.14\n 0.37\n 0.33\n 0.40\n 0.16\n-0.03\n-0.07\n-0.10\n-0.04\ncomputer\n 0.15\n 0.51\n 0.36\n 0.41\n 0.24\n 0.02\n 0.06\n 0.09\n 0.12\nuser\n 0.26\n 0.84\n 0.61\n 0.70\n 0.39\n 0.03\n 0.08\n 0.12\n 0.19\nsystem\n 0.45\n 1.23\n 1.05\n 1.27\n 0.56\n-0.07\n-0.15\n-0.21\n-0.05\nresponse\n 0.16\n 0.58\n 0.38\n 0.42\n 0.28\n 0.06\n 0.13\n 0.19\n 0.22\ntime\n 0.16\n 0.58\n 0.38\n 0.42\n 0.28\n 0.06\n 0.13\n 0.19\n 0.22\nEPS\n 0.22\n 0.55\n 0.51\n 0.63\n 0.24\n-0.07\n-0.14\n-0.20\n-0.11\nsurvey\n 0.10\n 0.53\n 0.23\n 0.21\n 0", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-013", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 13", "pageStart": 13, "pageEnd": 13, "target": "../sections/page-013.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n13\nThis text segment is best described as having so much of abstract concept one and\nso much of abstract concept two, and this word has so much of concept one and so\nmuch of concept two, and combining those two pieces of information (by vector\narithmetic), my best guess is that word X actually appeared 0.6 times in context Y.\nNow let us consider what such changes may do to the imputed relations between\nwords or between multi-word textual passages. For two examples of word-word relations,\ncompare the shaded and/or boxed rows for the words human , us", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-014", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 14", "pageStart": 14, "pageEnd": 14, "target": "../sections/page-014.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n14\nCorrelations between titles in raw data:\nc1\nc2\nc3\nc4\nc5\nm1\nm2\nm3\nc2\n-0.19\nc3\n0.00\n0.00\nc4\n0.00\n0.00\n0.47\nc5\n-0.33\n0.58\n0.00\n-0.31\nm1\n-0.17\n-0.30\n-0.21\n-0.16\n-0.17\nm2\n-0.26\n-0.45\n-0.32\n-0.24\n-0.26\n0.67\nm3\n-0.33\n-0.58\n-0.41\n-0.31\n-0.33\n0.52\n0.77\nm4\n-0.33\n-0.19\n-0.41\n-0.31\n-0.33\n-0.17\n0.26\n0.56\n 0.02\n-0.30\n0.44\nCorrelations in two dimensional space:\nc2\n0.91\nc3\n1.00\n0.91\nc4\n1.00\n0.88\n1.00\nc5\n0.85\n0.99\n0.85\n0.81\nm1\n-0.85\n-0.56\n-0.85\n-0.88\n-0.45\nm2\n-0.85\n-0.56\n-0.85\n-0.88\n-0.44\n1.00\nm3\n-0.85\n-0.56\n-0.85\n-0.88\n-0.44\n1.00\n1.00\nm4\n-0.81\n-0.50\n-0.81\n-0.84\n-0.3", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-015", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 15", "pageStart": 15, "pageEnd": 15, "target": "../sections/page-015.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n15\ndata are to be accommodated in two dimensions. In this case, the inferences appear to be\nintuitively sensible. Note that much of the information that LSA used to infer relations\namong words and passages is in data about passages in which particular words did not\noccur. Indeed, Landauer and Dumais (1997) found that in LSA simulations of schoolchild\nword knowledge acquisition, about three-fourths of the gain in total comprehension\nvocabulary that results from reading a paragraph is indirectly inferred knowledge about\nwords not in the paragraph at all", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-016", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 16", "pageStart": 16, "pageEnd": 16, "target": "../sections/page-016.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n16\ngenerated from a source of the same dimensionality and general structure as the\nreconstruction. Suppose, for example, that speakers or writers generate paragraphs by\nchoosing words from a k-dimensional space in such a way that words in the same\nparagraph tend to be selected from nearby locations. If listeners or readers try to infer the\nsimilarity of meaning from these data, they will do better if they reconstruct the full set of\nrelations in the same number of dimensions as the source. Among other things, given the\nright analysis, this will allow th", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-017", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 17", "pageStart": 17, "pageEnd": 17, "target": "../sections/page-017.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n17\n Additional detail about LSA \nAs mentioned, one additional part of the analysis, the data preprocessing transformation,\nneeds to be described more fully. Before the SVD is computed, it is customary in LSA to\nsubject the data in the raw word-by-context matrix to a two-part transformation. First, the\nword frequency (+ 1) in each cell is converted to its log. Second, the information-theoretic\nmeasure, entropy , of each word is computed as -Ö p log p over all entries in its row, and\neach cell entry then divided by the row entropy value. The effect", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-018", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 18", "pageStart": 18, "pageEnd": 18, "target": "../sections/page-018.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n18\nand are necessarily noisy. Therefore, studies using them require the use of replicate cases\nand statistical treatment in a manner similar to human data.\n \n\u0000\u0002\u0001\u0004\u0003\u0006\u0005\u0007\u0003\t\b\u0004\n\f\u000b\f\n\f\r\u000f\u000e\u0006\r\u000f\u0010\u0006\u0011\u0012\u0010\u0014\u0013\u0004\u0015\u0016\u000b\u0018\u0017\u0018\u0019\u0004\u001a\u001c\u001b\u001e\u001d \u001f\t\u0010\u001e\u001d\u0004!\"\u0015\"#\u0004\r$\u0019\u0004\u001b\u001e\u000b\u0018%\u0018\u001d\u0004\u0010\u001e&'\u000b\f\u0015\"\u0013\u0004(\u001e\u0015 \nHow well does LSA actually work as a representational model and measure of human\nverbal concepts? Its performance has been assessed more or less rigorously in several\nways. We give eight examples:\n(1) LSA was assessed as a predictor of query-document topic similarity judgments.\n(2) LSA was assessed as a simulation", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-019", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 19", "pageStart": 19, "pageEnd": 19, "target": "../sections/page-019.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n19\nmetaphor and context effects in decision making. We will take space here to review only\nsome of the most systematic and pertinent of these results.\n LSA and information retrieval \nJ. R. Anderson (1990) has called attention to the analogy between information retrieval and\nhuman semantic memory processes. One way of expressing their commonality is to think\nof a searcher as having in mind a certain meaning, which he or she expresses in words, and\nthe system as trying to find a text with the same meaning. Success, then, depends on the\nsystem representing", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-020", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 20", "pageStart": 20, "pageEnd": 20, "target": "../sections/page-020.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n20\nperformance ranged from just equivalent to the best prior methods up to about 30% better.\nIn a recent project sponsored by the National Institute of Standards and Technology, LSI\nwas compared with a large number of other research prototypes and commercial retrieval\nschemes. Direct quantitative comparisons among the many systems were somewhat\nmuddied by the use of varying amounts of preprocessingãthings like getting rid of\ntypographical errors, identifying proper nouns as special, differences in stop lists, and the\namount of tuning that systems were", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-021", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 21", "pageStart": 21, "pageEnd": 21, "target": "../sections/page-021.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n21\nNonetheless, the relationship between some close neighbors in LSA space can\n\u0006\u0015\u0003\u0004\u0003\u0002\u0001\u0002\u001e,#\n\u0006\u0015$\t\u0001\u0002\"4\"\n\u00161+\u0015\u0000\u0011%&\u0016\u0015\u001e-\b4\u0000\u0002\u000b!#\n\u0006\t\u0014\u0015\u001e\n\u0002!\u00005\f\n*\n\f\n7\u0007\r\n\u0001\n\u0000\u0002\u000b!+\t\u0001\u0002\"4\"\n\u0016\u0007\u000f\n\u0001\u0002$\u0015\u0012\n\r\u0019\u001e-\u0001\u0002\u0012\u0015\u0006\t%\n\u0001\u0004\u001e,\u0006\t\u0003\u0002\u0005\u0015#\n\u001e,%\n\u000f\n()#4\b\n\u00051\u0001\n\u0003\u0002\u0006\t\u001e,#\n$\u0015\u0000\u0011\u0006\u0015\u0017\n\f\n\u0000from the encyclopedia space), and some pairs that should be close are not. It's impossible\nto say exactly why these oddities occur, but it is plausible that some words that have more\nthan one contextual meaning receive a sort of average high-dimensional placement that out\nof context signifies nothing, and that many words are sampled", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-022", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 22", "pageStart": 22, "pageEnd": 22, "target": "../sections/page-022.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n22\nbeen used. In one experiment, an SVD was performed on text segments consisting of 500\n\u0003\u0002\u0005\t\u0001\u0002\u000b!\u0001\u0002\u0003\u0002\b\n\u0000\u0002\u000b!\u001e8\u0006\t\u000b\n\"\n\u0000\u0002\u001e-\u001e\n\u0002!\u0006\u0015$1\u0001\n\u0001\n\u0000\u0002\u000b!\u0001\u0002*\t\u0000\f\t91(.\u0006\t\u000b!\u0012\u0015\u001e-7\t\u0001\u0002+\t\u0006\u0015\u0014\t\b5\u0001\n2\t\u0001\u0002\u000b!\u0001\u0002*\t\u000b!\u0001\u00022\u0015\u00051\u001e8()\u0006\u0015\u000b!\b\n\u0005\t\f\n\b\n\u0001\u0002'\t\u0000\u0002$1\u0017!\u000b!\u0006\t%\n+\u0015\u0000\u0002*\t#\n$\u0015$\u0015#\n$\t*\nportions of each of 30,473 articles in the encyclopedia, a total of 4.5 million words of text,\nroughly equivalent to what a child would have read by the end of eighth grade. This\nresulted in a vector for each of 60 thousand words.\nThe TOEFL vocabulary test consists of items in which the question part is usually a\nsingl", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-023", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 23", "pageStart": 23, "pageEnd": 23, "target": "../sections/page-023.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n23\nTo assess the role of dimension reduction, the number of dimensions was varied\nfrom 2 to 1,032 (the largest number for which SVD was computationally feasible.) On log-\nlinear coordinates, the TOEFL test results showed a very sharp and highly significant peak\n(Figure 5). Corrected for guessing by the standard formula ((correct - chance)/(1-\nchance)), LSA got 52.7% correct with 300 and 325 dimensions, 13.5% correct with just\ntwo or three dimensions. When there was no dimension reduction at all (equivalent to\nchoosing correct answers by the correlation", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-024", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 24", "pageStart": 24, "pageEnd": 24, "target": "../sections/page-024.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n24\nrelation. This demonstrates conclusively that the LSA dimension reduction technique\ncaptures much more than mere co-occurrence (simply choosing the alternative that co-\noccurs with the stem in the largest number of corpus paragraphs gets only 11% right when\ncorrected for guessing). More importantly for our argument, it implies that indirect\nassociations or structural relations induced by analysis of the whole corpus are involved in\n\u0018\u001b\u001a\n\u001c\n\u001e8\u001e-\u0014\u0015\u0003\u0002\u0003\u0002\u0000\u0002\u001e,\u001e8(.#\n\b4\u00051#4$\t\u0012\u0015#\n\u0001\n#4\u0012\t\u0014\u0015\u0001\u0002\"5(.\u0006\u0015\u000b!\u0012\u0015\u001e\n\f\n\u000e\u001b\u0005\u0015\u0014\t\u001e,7\t\u0003\u0002\u0006\u0015\u000b!\u000b!\u0000\u0002\u0003\u0002\b5\u000b!\u0000\u00022\u0015\u000b!\u0000\u0002\u001e,\u0000\u0002$\t\b4\u0001\u0002\b\n#4\u0006\t$\n\u0006\t\u0017\n\u0001\u0002", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-025", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 25", "pageStart": 25, "pageEnd": 25, "target": "../sections/page-025.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n25\nversus concrete similarity relations. Anglin measured the semantic similarity of every pair\nof words by the proportion of subjects who put them in the same pile. He found that parts\nof speech clustered moderately in both child and adult sets, and, confirming the hypothesis\nbehind the study, that adults showed more evidence of use of abstract categories than did\nchildren.\nLaham and Landauer measured the similarity between the same word pairs by\ncosines based on 5 grade-partitioned scalings of samples of schoolchild readingã3rd, 6th,\n9th, 12th grade a", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-026", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 26", "pageStart": 26, "pageEnd": 26, "target": "../sections/page-026.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n26\naverage cosine =.19, the average within part-of-speech cosine = .23 and the average\nbetween part-of-speech cosine = .17.\nAs in the vocabulary acquisition simulations, it appears that the relations obtained\n\u00170\u000b0\u0006\t%\n\u0001\n\u0003\u0004\u0006\t\u000b!2\n\u0014\t\u001e \u0006\t\u0017\u0019\u001e-%&\u0001\u0004\"\n\"\u0015\u001e-#\u0001\u0000\u0004\u0000\u000b!\u0000\u0004\"\n\u0001\u0004\b\n#\n\u0001\n\u0000\b\n\u0006\u0007\u0001\n\b\n\u0016\t2\t#\n\u0003\u0002\u0001\u0004\"\n\u0001\u0004\u0012\t\u0014\t\"\n\b\n\u001e \u0003\u0004\u0014\t%&\u0014\t\"\n\u0001\u0002\b\n#\n\u0001\n\u0000\"\n\u0001\u0004$\t*\t\u0014\t\u0001\u0004*\t\u0000\u0000\u00023\t2\n\u0006\t\u001e-\u0014\t\u000b0\u0000\u000b!\u0000\u0004\u001e-\u0000\u0002%&+\t\"4\u0000\u0011\u0003\u0004\u0005\u0015#\n\"\n\u0012\t\u000b!\u0000\u0004$1\u001e-\u0006\t%&\u0000\u0002(.\u0005\t\u0001\u0004\b5%&\u0006\t\u000b!\u0000\u0011\b\n\u0005\t\u0001\u0002$\u0007\u0001\u0002\u0012\t\u0014\u0015\"\n\b\n\u001e\n\f\r\u0018\u0010\u001a\n\u001c\u001f\u001e (.\u0000\u0004\u0001\u0002'\u0007\u000b!\u0000\u0002\u00170\"\n\u0000\u0002\u0003\u0002\b\n#\n\u0006\t$1\u0006\t\u0017\n()\u0006\u0015\u000b0\u00121\u0003\u0002\"\n\u0001\u0004\u001e-\u001e8#\n$1\b\n\u0005\t#\n\u001e\nrather small sample of data would appear to confirm the expec", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-027", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 27", "pageStart": 27, "pageEnd": 27, "target": "../sections/page-027.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n27\nrelatedness of the concept pairs. The LSA predictions correlated significantly with the\nsubjects, with the correlation stronger to that of the experts in the domain ( r = 0.41) than\nthat of the novices ( r = 0.36). (Note again that two human ratings would also not correlated\nperfectly.) An analysis of where LSA's predictions deviated greatly from that of the\nhumans indicated that LSA tended to underpredict more global or situational relationships\nthat were not directly discussed in the text but would be common historical knowledge of\nany undergradu", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-028", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 28", "pageStart": 28, "pageEnd": 28, "target": "../sections/page-028.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n28\nIn the Till et al. study, target words related to both senses of the homographic words were\ncorrectly responded to faster than unrelated control words if presented within 100 ms after\nthe homograph. If delayed by 300 ms, only the context-appropriate associate was primed.\nAt a one second delay, the so-called inference words were also primed. In the LSA\nsimulation, the cosines between the polysemic word and its two associates were computed\nto mimic the expected initial priming. The cosine between the two associates of the\npolysemic word and the sentenc", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-029", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 29", "pageStart": 29, "pageEnd": 29, "target": "../sections/page-029.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n29\nTable 1\n LSA Simulation of Till, Mross, & Kintsch (1988) Priming Study.\nMint:\nMoney\nCandy\nGround\n .21\n .20\n .07\nThinking amount garlic dinner guest asked:\nMoney\nCandy\n .15\n .21\n Ground\nEarthquake\nBreath\n .15\n .14\n .21\nNote. LSA = Latent Semantic Analysis.\nOf course, LSA as currently constituted contains no model of the temporal\ndynamics of discourse comprehension. To fit the temporal findings of the Till et al.\nexperiment one would need to assume that the combining (averaging) of word vectors into\na single vector to represent the whole passage takes", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-030", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 30", "pageStart": 30, "pageEnd": 30, "target": "../sections/page-030.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n30\nsatisfaction mechanisms, or even to models including hierarchical syntactic structure\n(presumably, automatically induced), only if and as found necessary.\n Assigning holistic quality scores to essay test answers \nIn another set of studies to be published elsewhere by Landauer, Laham and Foltz (1998),\nLSA has been used to assign holistic quality scores to written answers to essay questions.\nFive different methods have been tried, all with good success. In all cases an LSA space\nwas first constructed based either on the instructional text read by stud", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-031", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 31", "pageStart": 31, "pageEnd": 31, "target": "../sections/page-031.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n31\nAll five methods provided the basis of scores that correlated approximately as well\nwith expert assigned scores as such scores correlated with each other, sometimes slightly\nless well, on average somewhat better. In one set of studies (Laham, 1997a), method one\nwas applied to a total of eight exams ranging in topic from heart anatomy and physiology,\nthrough psychological concepts, to American history, current social issues and marketing\nproblems. A meta-analysis found that LSA correlated significantly better with individual\nexpert graders (from ETS", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-032", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 32", "pageStart": 32, "pageEnd": 32, "target": "../sections/page-032.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n32\npropositionalized by hand. This has limited research to small samples of text and has\ninhibited its practical application to composition and instruction. Foltz, Kintsch, and\nLandauer (1993, this issue; Foltz, 1996) have applied LSA to the task. LSA can make\nautomatic coherence judgments by computing the cosine from one sentence or passage and\nthe following one. In one case, analysis of the coherence between a set of sentences about\nthe heart, the LSA measure predicted comprehension scores extremely well, r = .93. As will\nbe discussed in the artic", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-033", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 33", "pageStart": 33, "pageEnd": 33, "target": "../sections/page-033.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n33\nconceptual continuum, such as level of sophistication or relevance to a particular topic, can\nbe measured.\n Summary and some caveats \nIt is clear enough from the conjunction of all these formal and informal results that LSA is\nable to capture and represent significant components of the lexical and passage meanings\nevinced in judgment and behavior by humans. The following papers exploit this ability in\ninteresting and potentially useful ways that simultaneously provide additional\ndemonstrations and tests of the method and its underlying theory. Howev", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-034", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 34", "pageStart": 34, "pageEnd": 34, "target": "../sections/page-034.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n34\nrigorous computational simulation that takes in the very same data from which humans\nlearn about words and passages and produces a representation that gives veridical\nsimulations of a wide range of human judgments and behavior. While it seems highly\ndoubtful that the human brain uses the same mathematical algorithms as LSA/SVD, it\nseems almost certain that the brain uses as much analytic power as LSA to transform its\ntemporally local experiences into global knowledge. The present theory clearly does not\naccount for all aspects of knowledge and cognit", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-035", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 35", "pageStart": 35, "pageEnd": 35, "target": "../sections/page-035.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n35\nunavailability and difficulty of constructing large general or topically relevant text samples\nthat approximate what a variety of individual learners would have met. But another is due\nto current computational limitations. LSA became practical only when computational\npower and algorithm efficiency improved sufficiently to support SVD of thousands of\nwords-by-thousands of contexts matrices; it is still impossible to perform SVD on the\nhundreds of thousands by tens of millions matrices that would be needed to truly represent\n\b\n\u0005\u0015\u0000\u0011\u001e-\u0014\t%\n\u0006\t\u0017\n\u0001\u0002$1\u0001\u0004\u0012\u0015\u0014", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-036", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 36", "pageStart": 36, "pageEnd": 36, "target": "../sections/page-036.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n36\nREFERENCES\nAnderson, J. R. (1990). \nThe adaptive character of thought . Hillsdale, NJ: Lawrence\nErlbaum Associates.\nAnglin, J. M. (1970) \nThe growth of word meaning. Cambridge, MA.: MIT Press.\nAnglin, J. M., Alexander, T. M., & Johnson, C. J. (1996). Word learning and the\ngrowth of potentially knowable vocabulary. Submitted for publication.\nBerry, M. W. (1992). Large scale singular value computations. International Journal\n of Supercomputer Applications , 6 , 13-49.\nBerry, M. W., Dumais, S. T. and O'Brien, G.W. (1995) Using linear algebr", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-037", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 37", "pageStart": 37, "pageEnd": 37, "target": "../sections/page-037.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n37\nFoltz, P. W. (1996) Latent Semantic Analysis for text-based research. Behavior\n \nResearch \nMethods, Instruments and \nComputers. 28, 197-202.\nFoltz, P. W., Britt, M. A., & Perfetti, C. A. (1996) Reasoning from multiple texts:\nAn automatic analysis of readers' situation models. In G. Cottrell (Ed.) \nProceedings of the\n 18th \nAnnual Cognitive Science \nConference. Hillsdale, NJ: Lawrence Erlbaum Associates.\nFoltz, P. W. & Dumais, S. T. (1992). Personalized information delivery: An\nanalysis of information filtering m", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-038", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 38", "pageStart": 38, "pageEnd": 38, "target": "../sections/page-038.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n38\nLaham, D. (1997b). Latent Semantic Analysis Approaches to Categorization. In M.\nG. Shafto & P. Langley (Eds.), \nProceedings of the 19th Annual Conference of the\n \nCognitive Science Society (p. 979). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.\nLandauer, T.K. and Dumais, S.T. (1994). Latent semantic analysis and the\nmeasurement of knowledge. In R. M. Kaplan and J. C. Burstein (Eds) \nEducational Testing\n Service Conference on Natural Language Processing Techniques and Technology in\n \nAssessment and Education. Princeton, Educational Test", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-039", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 39", "pageStart": 39, "pageEnd": 39, "target": "../sections/page-039.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n39\nMcNamara, D. S., Kintsch, E., Songer B. N., & Kintsch, W. (1996) Are good texts\nalways better? Interactions of text coherence, background knowledge, and levels of\nunderstanding in learning from text. Cognition and Instruction , 14 , 1, 1-43.\nRehder, B., Schreiner, M. E., Wolfe, B. W., Laham, D., Landauer, T. K., &\nKintsch, W. (1998/this issue). Using Latent Semantic Analysis to assess knowledge:\nSome technical considerations. Discourse Processes , 25 , 337-354.\nSteinhart, D. J. (1996). \nResolving Lexical ambiguity: Does context pla", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-040", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 40", "pageStart": 40, "pageEnd": 40, "target": "../sections/page-040.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n40\nAppendix\nThe latest information and applications of LSA can be found at our website:\n http://LSA.colorado.edu/\nThis website is organized into three content areas, Information,\nDemonstrations, and Applications. The Information section contains additional papers,\nlinks, and other pertinent information on LSA.\nThe Demonstrations section currently includes examples of essay scoring and\nmatching learners to text. The matching application allows you to explore the use of LSA\nas a tool for selecting texts that will augment learning. The demonstration shows", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } }, { "id": "library/landauer-foltz-laham-1998-intro-lsa/blocks/prose-page-041", "types": [ "folio-assistant-core:Prose", "doco:Section" ], "kind": "prose", "title": "Page 41", "pageStart": 41, "pageEnd": 41, "target": "../sections/page-041.md", "narrative": null, "content": "Introduction to Latent Semantic Analysis\n41\nAuthor Note\nDarrell Laham and Thomas K Landauer, Department of Psychology, University of\nColorado, Boulder. Peter W. Foltz, Department of Psychology, New Mexico State\nUniversity.\nThis research was supported in part by a contract from ARPA-CAETI to T.\nLandauer and W. Kintsch.\nCorrespondence concerning this article should be addressed to Thomas K\nLandauer, Department of Psychology, Campus Box 345, University of Colorado,\nBoulder, CO, 80309. Electronic mail may be sent via Internet to\nlandauer@psych.colorado.edu.\nLandauer, T. K., Foltz, P. W., & Laham,", "truncated": true, "provenance": "ingested", "summary": { "status": "not-summarised" } } ] }