RetrievalEnvironment

java.lang.Object
- ivory.core.RetrievalEnvironment

public class RetrievalEnvironment
extends Object

Author:: Don Metzler, Jimmy Lin

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`dataCollection`
`static long`	`defaultCf`
`static int`	`defaultDf`
`static int`	`documentCount`
`static boolean`	`mIsNewModel`
`static int`	`topK`

Constructor Summary

Constructors
Constructor and Description

RetrievalEnvironment(String indexPath, FileSystem fs)

Constructors
Constructor and Description
`RetrievalEnvironment(String indexPath, FileSystem fs)`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`addImportanceModel(String key, ConceptImportanceModel m)`
`static String`	`appendPath(String base, String file)`
`void`	`clearPostingsReaderCache()`
`long`	`collectionFrequency(String expression)` Returns the collection frequency of a particular expression.
`static Path`	`createPath(String base, String file)`
`int`	`documentFrequency(String expression)` Returns the document frequency of a particular expression.
`IntDocVector[]`	`documentVectors(int[] docSet)`
`String`	`getCfByIntData()` Returns file that contains a list of collection frequencies sorted by term id.
`String`	`getCfByTermData()` Returns file that contains a list of collection frequencies sorted by the alphabetical order of terms.
`long`	`getCollectionSize()`
`long`	`getDefaultCf()` Returns the default collection frequency.
`int`	`getDefaultDf()` Returns the default document frequency.
`String`	`getDfByIntData()` Returns file that contains a list of document frequencies sorted by term id.
`String`	`getDfByTermData()` Returns file that contains a list of document frequencies sorted by the alphabetical order of terms.
`Path`	`getDoclengthsData()` Returns file that contains the document length data.
`Path`	`getDoclengthsDirectory()` Returns directory that contains the document length data.
`DocnoMapping`	`getDocnoMapping()`
`Path`	`getDocnoMappingData()`
`Path`	`getDocnoMappingDirectory()`
`float`	`getDocScore(String type, int docno)`
`long`	`getDocumentCount()`
`int`	`getDocumentLength(int docid)`
`int`	`getIdFromTerm(String term)`
`ConceptImportanceModel`	`getImportanceModel(String id)`
`Collection<ConceptImportanceModel>`	`getImportanceModels()`
`String`	`getIndexTermIdMappingData()` Returns file that contains an index of term ids into the array of terms.
`String`	`getIndexTermIdsData()` Returns file that contains term ids sorted by the alphabetical order of terms.
`String`	`getIndexTermsData()` Returns file that contains the list of terms in the collection.
`String`	`getIntDocVectorsDirectory()` Returns directory that contains the `IntDocVector` representation of the collection.
`String`	`getIntDocVectorsForwardIndex()` Returns file that contains an index into the `IntDocVector` representation of the collection.
`String`	`getNonPositionalPostingsDirectory()`
`String`	`getPostingsDirectory()`
`String`	`getPostingsIndexData()` Returns file that contains an index into the postings.
`PostingsList`	`getPostingsList(String term)`
`PostingsReader`	`getPostingsReader(Expression exp)`
`String`	`getTempDirectory()`
`String`	`getTermDfCfDirectory()`
`String`	`getTermDocVectorsDirectory()` Returns directory that contains the `TermDocVector` representation of the collection.
`String`	`getTermDocVectorsForwardIndex()` Returns file that contains an index into the `TermDocVector` representation of the collection.
`String`	`getTermFromId(int termid)`
`String`	`getWeightedIntDocVectorsDirectory()`
`String`	`getWeightedIntDocVectorsForwardIndex()`
`String`	`getWeightedTermDocVectorsDirectory()`
`void`	`initialize(boolean loadDoclengths)`
`static DocnoMapping`	`loadDocnoMapping(String indexPath, FileSystem fs)`
`void`	`loadDocScore(String type, String provider, String path)`
`static void`	`main(String[] args)`
`float`	`readCollectionAverageDocumentLength()`
`int`	`readCollectionDocumentCount()`
`long`	`readCollectionLength()`
`String`	`readCollectionName()`
`String`	`readCollectionPath()`
`int`	`readCollectionTermCount()`
`String`	`readDocnoMappingClass()`
`int`	`readDocnoOffset()`
`String`	`readInputFormat()`
`String`	`readPostingsType()`
`String`	`readTokenizerClass()`
`static void`	`setIsNew(boolean isNewModel)`
`String[]`	`tokenize(String text)` Tokenizes text according to the tokenizer used to process the document collection.
`void`	`writeCollectionAverageDocumentLength(float n)`
`void`	`writeCollectionDocumentCount(int n)`
`void`	`writeCollectionLength(long cnt)`
`void`	`writeCollectionName(String s)`
`void`	`writeCollectionPath(String s)`
`void`	`writeCollectionTermCount(int cnt)`
`void`	`writeDocnoMappingClass(String s)`
`void`	`writeDocnoOffset(int n)`
`void`	`writeInputFormat(String s)`
`void`	`writePostingsType(String type)`
`void`	`writeTokenizerClass(String s)`

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - dataCollection
```
public static String dataCollection
```
  - defaultCf
```
public static long defaultCf
```
  - defaultDf
```
public static int defaultDf
```
  - documentCount
```
public static int documentCount
```
  - mIsNewModel
```
public static boolean mIsNewModel
```
  - topK
```
public static int topK
```
- Constructor Detail
  - RetrievalEnvironment
```
public RetrievalEnvironment(String indexPath,
                    FileSystem fs)
                     throws IOException
```
    Throws:
    
    IOException
- Method Detail
  - addImportanceModel
```
public void addImportanceModel(String key,
                      ConceptImportanceModel m)
```
  - appendPath
```
public static String appendPath(String base,
                String file)
```
  - clearPostingsReaderCache
```
public void clearPostingsReaderCache()
```
  - collectionFrequency
```
public long collectionFrequency(String expression)
```
    Returns the collection frequency of a particular expression.
  - createPath
```
public static Path createPath(String base,
              String file)
```
  - documentFrequency
```
public int documentFrequency(String expression)
```
    Returns the document frequency of a particular expression.
  - documentVectors
```
public IntDocVector[] documentVectors(int[] docSet)
```
  - getCfByIntData
```
public String getCfByIntData()
```
    Returns file that contains a list of collection frequencies sorted by term id. The file consists of a stream of ints, read into an array. The array index position corresponds to the term id. This is used to retrieve the cf of a term given its term id.
  - getCfByTermData
```
public String getCfByTermData()
```
    Returns file that contains a list of collection frequencies sorted by the alphabetical order of terms. The file consists of a stream of ints, read into an array. The array index position corresponds to alphabetical sort order of the term. This is used to retrieve the cf of a term given its position in the getIndexTermsData() data.
  - getCollectionSize
```
public long getCollectionSize()
```
  - getDefaultCf
```
public long getDefaultCf()
```
    Returns the default collection frequency.
  - getDefaultDf
```
public int getDefaultDf()
```
    Returns the default document frequency.
  - getDfByIntData
```
public String getDfByIntData()
```
    Returns file that contains a list of document frequencies sorted by term id. The file consists of a stream of ints, read into an array. The array index position corresponds to the term id. Note that the term ids are assigned in descending order of document frequency. This is used to retrieve the df of a term given its term id.
  - getDfByTermData
```
public String getDfByTermData()
```
    Returns file that contains a list of document frequencies sorted by the alphabetical order of terms. The file consists of a stream of ints, read into an array. The array index position corresponds to alphabetical sort order of the term. This is used to retrieve the df of a term given its position in the getIndexTermsData() data.
  - getDoclengthsData
```
public Path getDoclengthsData()
```
    Returns file that contains the document length data. This file serves as input to DocLengthTable, which provides random access to document lengths.
  - getDoclengthsDirectory
```
public Path getDoclengthsDirectory()
```
    Returns directory that contains the document length data. Data in this directory is compiled into the denoted by getDoclengthsData().
  - getDocnoMapping
```
public DocnoMapping getDocnoMapping()
                             throws IOException
```
    Throws:
    
    IOException
  - getDocnoMappingData
```
public Path getDocnoMappingData()
```
  - getDocnoMappingDirectory
```
public Path getDocnoMappingDirectory()
```
  - getDocScore
```
public float getDocScore(String type,
                int docno)
```
  - getDocumentCount
```
public long getDocumentCount()
```
  - getDocumentLength
```
public int getDocumentLength(int docid)
```
  - getIdFromTerm
```
public int getIdFromTerm(String term)
```
  - getImportanceModel
```
public ConceptImportanceModel getImportanceModel(String id)
```
  - getImportanceModels
```
public Collection<ConceptImportanceModel> getImportanceModels()
```
  - getIndexTermIdMappingData
```
public String getIndexTermIdMappingData()
```
    Returns file that contains an index of term ids into the array of terms. The file consists of a stream of ints, read into an array. The array index position corresponds to the term ids. This is used to retrieve the index position into the getIndexTermsData() file to map from term ids back to terms.
  - getIndexTermIdsData
```
public String getIndexTermIdsData()
```
    Returns file that contains term ids sorted by the alphabetical order of terms. The file consists of a stream of ints, read into an array. The array index position corresponds to alphabetical sort order of the term. This is used to retrieve the term id of a term given its position in the getIndexTermsData() data.
  - getIndexTermsData
```
public String getIndexTermsData()
```
    Returns file that contains the list of terms in the collection. The file consists of a stream of bytes, read into an array, representing the terms, which are alphabetically sorted and prefix-coded.
  - getIntDocVectorsDirectory
```
public String getIntDocVectorsDirectory()
```
    Returns directory that contains the IntDocVector representation of the collection.
  - getIntDocVectorsForwardIndex
```
public String getIntDocVectorsForwardIndex()
```
    Returns file that contains an index into the IntDocVector representation of the collection. This file serves as input to IntDocVectorsForwardIndex, which provides random access to the document vectors.
  - getNonPositionalPostingsDirectory
```
public String getNonPositionalPostingsDirectory()
```
  - getPostingsDirectory
```
public String getPostingsDirectory()
```
  - getPostingsIndexData
```
public String getPostingsIndexData()
```
    Returns file that contains an index into the postings. This file serves as input to IntPostingsForwardIndex, which provides random access to postings lists.
  - getPostingsList
```
public PostingsList getPostingsList(String term)
```
  - getPostingsReader
```
public PostingsReader getPostingsReader(Expression exp)
```
  - getTempDirectory
```
public String getTempDirectory()
```
  - getTermDfCfDirectory
```
public String getTermDfCfDirectory()
```
  - getTermDocVectorsDirectory
```
public String getTermDocVectorsDirectory()
```
    Returns directory that contains the TermDocVector representation of the collection.
  - getTermDocVectorsForwardIndex
```
public String getTermDocVectorsForwardIndex()
```
    Returns file that contains an index into the TermDocVector representation of the collection. This file serves as input to TermDocVectorsForwardIndex, which provides random access to the document vectors.
  - getTermFromId
```
public String getTermFromId(int termid)
```
  - getWeightedIntDocVectorsDirectory
```
public String getWeightedIntDocVectorsDirectory()
```
  - getWeightedIntDocVectorsForwardIndex
```
public String getWeightedIntDocVectorsForwardIndex()
```
  - getWeightedTermDocVectorsDirectory
```
public String getWeightedTermDocVectorsDirectory()
```
  - initialize
```
public void initialize(boolean loadDoclengths)
                throws IOException,
                       ConfigurationException
```
    Throws:
    
    IOException
    
    ConfigurationException
  - loadDocnoMapping
```
public static DocnoMapping loadDocnoMapping(String indexPath,
                            FileSystem fs)
                                     throws IOException
```
    Throws:
    
    IOException
  - loadDocScore
```
public void loadDocScore(String type,
                String provider,
                String path)
```
  - main
```
public static void main(String[] args)
                 throws Exception
```
    Throws:
    
    Exception
  - readCollectionAverageDocumentLength
```
public float readCollectionAverageDocumentLength()
```
  - readCollectionDocumentCount
```
public int readCollectionDocumentCount()
```
  - readCollectionLength
```
public long readCollectionLength()
```
  - readCollectionName
```
public String readCollectionName()
```
  - readCollectionPath
```
public String readCollectionPath()
```
  - readCollectionTermCount
```
public int readCollectionTermCount()
```
  - readDocnoMappingClass
```
public String readDocnoMappingClass()
```
  - readDocnoOffset
```
public int readDocnoOffset()
```
  - readInputFormat
```
public String readInputFormat()
```
  - readPostingsType
```
public String readPostingsType()
```
  - readTokenizerClass
```
public String readTokenizerClass()
```
  - setIsNew
```
public static void setIsNew(boolean isNewModel)
```
  - tokenize
```
public String[] tokenize(String text)
```
    Tokenizes text according to the tokenizer used to process the document collection. This typically includes stopwords filtering, stemming, etc.
    
    Parameters:
    text - text to tokenize
    
    Returns:
    array of tokens
  - writeCollectionAverageDocumentLength
```
public void writeCollectionAverageDocumentLength(float n)
```
  - writeCollectionDocumentCount
```
public void writeCollectionDocumentCount(int n)
```
  - writeCollectionLength
```
public void writeCollectionLength(long cnt)
```
  - writeCollectionName
```
public void writeCollectionName(String s)
```
  - writeCollectionPath
```
public void writeCollectionPath(String s)
```
  - writeCollectionTermCount
```
public void writeCollectionTermCount(int cnt)
```
  - writeDocnoMappingClass
```
public void writeDocnoMappingClass(String s)
```
  - writeDocnoOffset
```
public void writeDocnoOffset(int n)
```
  - writeInputFormat
```
public void writeInputFormat(String s)
```
  - writePostingsType
```
public void writePostingsType(String type)
```
  - writeTokenizerClass
```
public void writeTokenizerClass(String s)
```

Class RetrievalEnvironment

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

dataCollection

defaultCf

defaultDf

documentCount

mIsNewModel

topK

Constructor Detail

RetrievalEnvironment

Method Detail

addImportanceModel

appendPath

clearPostingsReaderCache

collectionFrequency

createPath

documentFrequency

documentVectors

getCfByIntData

getCfByTermData

getCollectionSize

getDefaultCf

getDefaultDf

getDfByIntData

getDfByTermData

getDoclengthsData

getDoclengthsDirectory

getDocnoMapping

getDocnoMappingData

getDocnoMappingDirectory

getDocScore

getDocumentCount

getDocumentLength

getIdFromTerm

getImportanceModel

getImportanceModels

getIndexTermIdMappingData

getIndexTermIdsData

getIndexTermsData

getIntDocVectorsDirectory

getIntDocVectorsForwardIndex

getNonPositionalPostingsDirectory

getPostingsDirectory

getPostingsIndexData

getPostingsList

getPostingsReader

getTempDirectory

getTermDfCfDirectory

getTermDocVectorsDirectory

getTermDocVectorsForwardIndex

getTermFromId

getWeightedIntDocVectorsDirectory

getWeightedIntDocVectorsForwardIndex

getWeightedTermDocVectorsDirectory

initialize

loadDocnoMapping

loadDocScore

main

readCollectionAverageDocumentLength

readCollectionDocumentCount

readCollectionLength

readCollectionName

readCollectionPath

readCollectionTermCount

readDocnoMappingClass

readDocnoOffset

readInputFormat

readPostingsType

readTokenizerClass

setIsNew

tokenize

writeCollectionAverageDocumentLength

writeCollectionDocumentCount

writeCollectionLength

writeCollectionName

writeCollectionPath