FileDocCategorySizeDatePackage
BreakIterator.javaAPI DocJava SE 6 API26537Tue Jun 10 00:25:50 BST 2008java.text

BreakIterator

public abstract class BreakIterator extends Object implements Cloneable
The BreakIterator class implements methods for finding the location of boundaries in text. Instances of BreakIterator maintain a current position and scan over text returning the index of characters where boundaries occur. Internally, BreakIterator scans text using a CharacterIterator, and is thus able to scan text held by any object implementing that protocol. A StringCharacterIterator is used to scan String objects passed to setText.

You use the factory methods provided by this class to create instances of various types of break iterators. In particular, use getWordIterator, getLineIterator, getSentenceIterator, and getCharacterIterator to create BreakIterators that perform word, line, sentence, and character boundary analysis respectively. A single BreakIterator can work only on one unit (word, line, sentence, and so on). You must use a different iterator for each unit boundary analysis you wish to perform.

Line boundary analysis determines where a text string can be broken when line-wrapping. The mechanism correctly handles punctuation and hyphenated words. Actual line breaking needs to also consider the available line width and is handled by higher-level software.

Sentence boundary analysis allows selection with correct interpretation of periods within numbers and abbreviations, and trailing punctuation marks such as quotation marks and parentheses.

Word boundary analysis is used by search and replace functions, as well as within text editing applications that allow the user to select words with a double click. Word selection provides correct interpretation of punctuation marks within and following words. Characters that are not part of a word, such as symbols or punctuation marks, have word-breaks on both sides.

Character boundary analysis allows users to interact with characters as they expect to, for example, when moving the cursor through a text string. Character boundary analysis provides correct navigation through character strings, regardless of how the character is stored. The boundaries returned may be those of supplementary characters, combining character sequences, or ligature clusters. For example, an accented character might be stored as a base character and a diacritical mark. What users consider to be a character can differ between languages.

The BreakIterator instances returned by the factory methods of this class are intended for use with natural languages only, not for programming language text. It is however possible to define subclasses that tokenize a programming language.

Examples:

Creating and using text boundaries: 

public static void main(String args[]) {
if (args.length == 1) {
String stringToExamine = args[0];
//print each word in order
BreakIterator boundary = BreakIterator.getWordInstance();
boundary.setText(stringToExamine);
printEachForward(boundary, stringToExamine);
//print each sentence in reverse order
boundary = BreakIterator.getSentenceInstance(Locale.US);
boundary.setText(stringToExamine);
printEachBackward(boundary, stringToExamine);
printFirst(boundary, stringToExamine);
printLast(boundary, stringToExamine);
}
}
Print each element in order: 
public static void printEachForward(BreakIterator boundary, String source) {
int start = boundary.first();
for (int end = boundary.next();
end != BreakIterator.DONE;
start = end, end = boundary.next()) {
System.out.println(source.substring(start,end));
}
}
Print each element in reverse order: 
public static void printEachBackward(BreakIterator boundary, String source) {
int end = boundary.last();
for (int start = boundary.previous();
start != BreakIterator.DONE;
end = start, start = boundary.previous()) {
System.out.println(source.substring(start,end));
}
}
Print first element: 
public static void printFirst(BreakIterator boundary, String source) {
int start = boundary.first();
int end = boundary.next();
System.out.println(source.substring(start,end));
}
Print last element: 
public static void printLast(BreakIterator boundary, String source) {
int end = boundary.last();
int start = boundary.previous();
System.out.println(source.substring(start,end));
}
Print the element at a specified position: 
public static void printAt(BreakIterator boundary, int pos, String source) {
int end = boundary.following(pos);
int start = boundary.previous();
System.out.println(source.substring(start,end));
}
Find the next word: 
public static int nextWordStartAfter(int pos, String text) {
BreakIterator wb = BreakIterator.getWordInstance();
wb.setText(text);
int last = wb.following(pos);
int current = wb.next();
while (current != BreakIterator.DONE) {
for (int p = last; p < current; p++) {
if (Character.isLetter(text.codePointAt(p)))
return last;
}
last = current;
current = wb.next();
}
return BreakIterator.DONE;
}
(The iterator returned by BreakIterator.getWordInstance() is unique in that the break positions it returns don't represent both the start and end of the thing being iterated over. That is, a sentence-break iterator returns breaks that each represent the end of one sentence and the beginning of the next. With the word-break iterator, the characters between two boundaries might be a word, or they might be the punctuation or whitespace between two words. The above code uses a simple heuristic to determine which boundary is the beginning of a word: If the characters between this boundary and the next boundary include at least one letter (this can be an alphabetical letter, a CJK ideograph, a Hangul syllable, a Kana character, etc.), then the text between this boundary and the next is a word; otherwise, it's the material between words.)
see
CharacterIterator

Fields Summary
public static final int
DONE
DONE is returned by previous(), next(), next(int), preceding(int) and following(int) when either the first or last text boundary has been reached.
private static final int
CHARACTER_INDEX
private static final int
WORD_INDEX
private static final int
LINE_INDEX
private static final int
SENTENCE_INDEX
private static final SoftReference[]
iterCache
Constructors Summary
protected BreakIterator()
Constructor. BreakIterator is stateless and has no default behavior.
Methods Summary
public java.lang.Objectclone()
Create a copy of this iterator

return
A copy of this

        try {
            return super.clone();
        }
        catch (CloneNotSupportedException e) {
            throw new InternalError();
        }
private static java.text.BreakIteratorcreateBreakInstance(java.util.Locale locale, int type, java.lang.String dataName, java.lang.String dictionaryName)


        // Check whether a provider can provide an implementation that's closer 
        // to the requested locale than what the Java runtime itself can provide.
        LocaleServiceProviderPool pool =
            LocaleServiceProviderPool.getPool(BreakIteratorProvider.class);
        if (pool.hasProviders()) {
            BreakIterator providersInstance = pool.getLocalizedObject(
                                                    BreakIteratorGetter.INSTANCE, 
                                                    locale, type);
            if (providersInstance != null) {
                return providersInstance;
            }
        }

        ResourceBundle bundle = getBundle(
                        "sun.text.resources.BreakIteratorInfo", locale);
        String[] classNames = bundle.getStringArray("BreakIteratorClasses");
        
        String dataFile = bundle.getString(dataName);

        try {
            if (classNames[type].equals("RuleBasedBreakIterator")) {
                return new RuleBasedBreakIterator(dataFile);
            }
            else if (classNames[type].equals("DictionaryBasedBreakIterator")) {
                String dictionaryFile = bundle.getString(dictionaryName);
                return new DictionaryBasedBreakIterator(dataFile, dictionaryFile);
            }
            else {
                throw new IllegalArgumentException("Invalid break iterator class \"" +
                                classNames[type] + "\"");
            }
        }
        catch (Exception e) {
            throw new InternalError(e.toString()); 
        }
public abstract intcurrent()
Returns character index of the text boundary that was most recently returned by next(), next(int), previous(), first(), last(), following(int) or preceding(int). If any of these methods returns BreakIterator.DONE because either first or last text boundary has been reached, it returns the first or last text boundary depending on which one is reached.

return
The text boundary returned from the above methods, first or last text boundary.
see
#next()
see
#next(int)
see
#previous()
see
#first()
see
#last()
see
#following(int)
see
#preceding(int)

public abstract intfirst()
Returns the first boundary. The iterator's current position is set
to the first text boundary.
return
The character index of the first text boundary.
public abstract intfollowing(int offset)
Returns the first boundary following the specified character offset. If the
specified offset equals to the last text boundary, it returns
BreakIterator.DONE and the iterator's current position is unchanged.
Otherwise, the iterator's current position is set to the returned boundary.
The value returned is always greater than the offset or the value
BreakIterator.DONE.
param
offset the character offset to begin scanning.
return
The first boundary after the specified offset or
BreakIterator.DONE if the last text boundary is passed in
as the offset.
exception
IllegalArgumentException if the specified offset is less than
the first text boundary or greater than the last text boundary.
public static synchronized java.util.Locale[]getAvailableLocales()
Returns an array of all locales for which the
get*Instance methods of this class can return
localized instances.
The returned array represents the union of locales supported by the Java
runtime and by installed
{@link java.text.spi.BreakIteratorProvider BreakIteratorProvider} implementations.
It must contain at least a Locale
instance equal to {@link java.util.Locale#US Locale.US}.
return
An array of locales for which localized
BreakIterator instances are available.
        LocaleServiceProviderPool pool =
            LocaleServiceProviderPool.getPool(BreakIteratorProvider.class);
        return pool.getAvailableLocales();
    
private static java.text.BreakIteratorgetBreakInstance(java.util.Locale locale, int type, java.lang.String dataName, java.lang.String dictionaryName)
        if (iterCache[type] != null) {
            BreakIteratorCache cache = (BreakIteratorCache) iterCache[type].get();
            if (cache != null) {
                if (cache.getLocale().equals(locale)) {
                    return cache.createBreakInstance();
                }
            }
        }

        BreakIterator result = createBreakInstance(locale,
                                                   type,
                                                   dataName,
                                                   dictionaryName);
        BreakIteratorCache cache = new BreakIteratorCache(locale, result);
        iterCache[type] = new SoftReference(cache);
        return result;
    
private static java.util.ResourceBundlegetBundle(java.lang.String baseName, java.util.Locale locale)
         return (ResourceBundle) AccessController.doPrivileged(new PrivilegedAction() {
            public Object run() {
                return ResourceBundle.getBundle(baseName, locale);
            }
        });
    
public static java.text.BreakIteratorgetCharacterInstance()
Returns a new BreakIterator instance
for character breaks
for the {@linkplain Locale#getDefault() default locale}.
return
A break iterator for character breaks
        return getCharacterInstance(Locale.getDefault());
    
public static java.text.BreakIteratorgetCharacterInstance(java.util.Locale locale)
Returns a new BreakIterator instance
for character breaks
for the given locale.
param
locale the desired locale
return
A break iterator for character breaks
exception
NullPointerException if locale is null
        return getBreakInstance(locale,
                                CHARACTER_INDEX,
                                "CharacterData",
                                "CharacterDictionary");
    
static intgetInt(byte[] buf, int offset)
        int num = buf[offset]&0xFF;
        for (int i = 1; i < 4; i++) {
            num = num<<8 | (buf[offset+i]&0xFF);
        }
        return num;
    
public static java.text.BreakIteratorgetLineInstance()
Returns a new BreakIterator instance
for line breaks
for the {@linkplain Locale#getDefault() default locale}.
return
A break iterator for line breaks
        return getLineInstance(Locale.getDefault());
    
public static java.text.BreakIteratorgetLineInstance(java.util.Locale locale)
Returns a new BreakIterator instance
for line breaks
for the given locale.
param
locale the desired locale
return
A break iterator for line breaks
exception
NullPointerException if locale is null
        return getBreakInstance(locale,
                                LINE_INDEX,
                                "LineData",
                                "LineDictionary");
    
static longgetLong(byte[] buf, int offset)
        long num = buf[offset]&0xFF;
        for (int i = 1; i < 8; i++) {
            num = num<<8 | (buf[offset+i]&0xFF);
        }
        return num;
    
public static java.text.BreakIteratorgetSentenceInstance()
Returns a new BreakIterator instance
for sentence breaks
for the {@linkplain Locale#getDefault() default locale}.
return
A break iterator for sentence breaks
        return getSentenceInstance(Locale.getDefault());
    
public static java.text.BreakIteratorgetSentenceInstance(java.util.Locale locale)
Returns a new BreakIterator instance
for sentence breaks
for the given locale.
param
locale the desired locale
return
A break iterator for sentence breaks
exception
NullPointerException if locale is null
        return getBreakInstance(locale,
                                SENTENCE_INDEX,
                                "SentenceData",
                                "SentenceDictionary");
    
static shortgetShort(byte[] buf, int offset)
        short num = (short)(buf[offset]&0xFF);
        num = (short)(num<<8 | (buf[offset+1]&0xFF));
        return num;
    
public abstract java.text.CharacterIteratorgetText()
Get the text being scanned
return
the text being scanned
public static java.text.BreakIteratorgetWordInstance()
Returns a new BreakIterator instance
for word breaks
for the {@linkplain Locale#getDefault() default locale}.
return
A break iterator for word breaks

                               
       
    
        return getWordInstance(Locale.getDefault());
    
public static java.text.BreakIteratorgetWordInstance(java.util.Locale locale)
Returns a new BreakIterator instance
for word breaks
for the given locale.
param
locale the desired locale
return
A break iterator for word breaks
exception
NullPointerException if locale is null
        return getBreakInstance(locale,
                                WORD_INDEX,
                                "WordData",
                                "WordDictionary");
    
public booleanisBoundary(int offset)
Returns true if the specified character offset is a text boundary.
param
offset the character offset to check.
return
true if "offset" is a boundary position,
false otherwise.
since
1.2
        // NOTE: This implementation probably is wrong for most situations
        // because it fails to take into account the possibility that a
        // CharacterIterator passed to setText() may not have a begin offset
        // of 0.  But since the abstract BreakIterator doesn't have that
        // knowledge, it assumes the begin offset is 0.  If you subclass
        // BreakIterator, copy the SimpleTextBoundary implementation of this
        // function into your subclass.  [This should have been abstract at
        // this level, but it's too late to fix that now.]
        if (offset == 0)
            return true;
        else
            return following(offset - 1) == offset;
    
public abstract intlast()
Returns the last boundary. The iterator's current position is set
to the last text boundary.
return
The character index of the last text boundary.
public abstract intnext(int n)
Returns the nth boundary from the current boundary. If either
the first or last text boundary has been reached, it returns
BreakIterator.DONE and the current position is set to either
the first or last text boundary depending on which one is reached. Otherwise,
the iterator's current position is set to the new boundary.
For example, if the iterator's current position is the mth text boundary
and three more boundaries exist from the current boundary to the last text
boundary, the next(2) call will return m + 2. The new text position is set
to the (m + 2)th text boundary. A next(4) call would return
BreakIterator.DONE and the last text boundary would become the
new text position.
param
n which boundary to return. A value of 0
does nothing. Negative values move to previous boundaries
and positive values move to later boundaries.
return
The character index of the nth boundary from the current position
or BreakIterator.DONE if either first or last text boundary
has been reached.
public abstract intnext()
Returns the boundary following the current boundary. If the current boundary
is the last text boundary, it returns BreakIterator.DONE and
the iterator's current position is unchanged. Otherwise, the iterator's
current position is set to the boundary following the current boundary.
return
The character index of the next text boundary or
BreakIterator.DONE if the current boundary is the last text
boundary.
Equivalent to next(1).
see
#next(int)
public intpreceding(int offset)
Returns the last boundary preceding the specified character offset. If the
specified offset equals to the first text boundary, it returns
BreakIterator.DONE and the iterator's current position is unchanged.
Otherwise, the iterator's current position is set to the returned boundary.
The value returned is always less than the offset or the value
BreakIterator.DONE.
param
offset the characater offset to begin scanning.
return
The last boundary before the specified offset or
BreakIterator.DONE if the first text boundary is passed in
as the offset.
exception
IllegalArgumentException if the specified offset is less than
the first text boundary or greater than the last text boundary.
since
1.2

                                 
       

                                 
       

                                                                                                                                                                                          
        

                                                                                 
       

                                                                         
       

                                                                                                                       
        

                                                                                                                            
        
        // NOTE:  This implementation is here solely because we can't add new
        // abstract methods to an existing class.  There is almost ALWAYS a
        // better, faster way to do this.
        int pos = following(offset);
        while (pos >= offset && pos != DONE)
            pos = previous();
        return pos;
    
public abstract intprevious()
Returns the boundary preceding the current boundary. If the current boundary
is the first text boundary, it returns BreakIterator.DONE and
the iterator's current position is unchanged. Otherwise, the iterator's
current position is set to the boundary preceding the current boundary.
return
The character index of the previous text boundary or
BreakIterator.DONE if the current boundary is the first text
boundary.
public voidsetText(java.lang.String newText)
Set a new text string to be scanned. The current scan
position is reset to first().
param
newText new text to scan.
        setText(new StringCharacterIterator(newText));
    
public abstract voidsetText(java.text.CharacterIterator newText)
Set a new text for scanning. The current scan
position is reset to first().
param
newText new text to scan.