FileDocCategorySizeDatePackage
TokenStream.javaAPI DocApache Lucene 1.4.31275Tue Mar 30 00:48:00 BST 2004org.apache.lucene.analysis

TokenStream

public abstract class TokenStream extends Object
A TokenStream enumerates the sequence of tokens, either from fields of a document or from query text.

This is an abstract class. Concrete subclasses are:

  • {@link Tokenizer}, a TokenStream whose input is a Reader; and
  • {@link TokenFilter}, a TokenStream whose input is another TokenStream.

Fields Summary
Constructors Summary
Methods Summary
public voidclose()
Releases resources associated with this stream.

public abstract org.apache.lucene.analysis.Tokennext()
Returns the next token in the stream, or null at EOS.