TokenStreampublic abstract class TokenStream extends Object | A TokenStream enumerates the sequence of tokens, either from
fields of a document or from query text.
This is an abstract class. Concrete subclasses are:
- {@link Tokenizer}, a TokenStream
whose input is a Reader; and
- {@link TokenFilter}, a TokenStream
whose input is another TokenStream.
|
| Methods Summary |
|---|
| public void | close()Releases resources associated with this stream.
| | public abstract org.apache.lucene.analysis.Token | next()Returns the next token in the stream, or null at EOS.
| | public void | reset()Resets this stream to the beginning. This is an
optional operation, so subclasses may or may not
implement this method. Reset() is not needed for
the standard indexing process. However, if the Tokens
of a TokenStream are intended to be consumed more than
once, it is neccessary to implement reset().
|
|
