org.apache.lucene.index.memory
Class SynonymTokenFilter
java.lang.Object
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.index.memory.SynonymTokenFilter
public class SynonymTokenFilter
- extends TokenFilter
Injects additional tokens for synonyms of token terms fetched from the
underlying child stream; the child stream must deliver lowercase tokens
for synonyms to be found.
- Author:
- whoschek.AT.lbl.DOT.gov
Method Summary |
protected Token |
createToken(String synonym,
Token current)
Creates and returns a token for the given synonym of the current input
token; Override for custom (stateless or stateful) behaviour, if desired. |
Token |
next()
Returns the next token in the stream, or null at EOS. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
SYNONYM_TOKEN_TYPE
public static final String SYNONYM_TOKEN_TYPE
- The Token.type used to indicate a synonym to higher level filters.
- See Also:
- Constant Field Values
SynonymTokenFilter
public SynonymTokenFilter(TokenStream input,
SynonymMap synonyms,
int maxSynonyms)
- Creates an instance for the given underlying stream and synonym table.
- Parameters:
input
- the underlying child token streamsynonyms
- the map used to extract synonyms for termsmaxSynonyms
- the maximum number of synonym tokens to return per underlying
token word (a value of Integer.MAX_VALUE indicates unlimited)
next
public Token next()
throws IOException
- Returns the next token in the stream, or null at EOS.
- Specified by:
next
in class TokenStream
- Throws:
IOException
createToken
protected Token createToken(String synonym,
Token current)
- Creates and returns a token for the given synonym of the current input
token; Override for custom (stateless or stateful) behaviour, if desired.
- Parameters:
synonym
- a synonym for the current token's termcurrent
- the current token from the underlying child stream
- Returns:
- a new token, or null to indicate that the given synonym should be
ignored