File Doc Category Size Date Package
TokenStream.java API Doc Apache Lucene 1.4.3 1275 Tue Mar 30 00:48:00 BST 2004 org.apache.lucene.analysis

TokenStream

java.lang.Object

public abstract class TokenStream extends Object

A TokenStream enumerates the sequence of tokens, either from fields of a document or from query text.

This is an abstract class. Concrete subclasses are:

{@link Tokenizer}, a TokenStream whose input is a Reader; and
{@link TokenFilter}, a TokenStream whose input is another TokenStream.

Fields Summary
Constructors Summary
Methods Summary
public void close()
Releases resources associated with this stream.
public abstract org.apache.lucene.analysis.Token next()
Returns the next token in the stream, or null at EOS.