File Doc Category Size Date Package
TokenStream.java API Doc Apache Lucene 2.1.0 1463 Wed Feb 14 10:46:38 GMT 2007 org.apache.lucene.analysis

TokenStream

java.lang.Object

public abstract class TokenStream extends Object

A TokenStream enumerates the sequence of tokens, either from fields of a document or from query text.

This is an abstract class. Concrete subclasses are:

{@link Tokenizer}, a TokenStream whose input is a Reader; and
{@link TokenFilter}, a TokenStream whose input is another TokenStream.

Fields Summary
Constructors Summary
Methods Summary
public void close()
Releases resources associated with this stream.
public abstract org.apache.lucene.analysis.Token next()
Returns the next token in the stream, or null at EOS.