StreamTokenizer
class takes an input stream and
parses it into "tokens", allowing the tokens to be
read one at a time. The parsing process is controlled by a table
and a number of flags that can be set to various states. The
stream tokenizer can recognize identifiers, numbers, quoted
strings, and various comment styles.
Each byte read from the input stream is regarded as a character
in the range '\u0000'
through '\u00FF'
.
The character value is used to look up five possible attributes of
the character: white space, alphabetic,
numeric, string quote, and comment character.
Each character can have zero or more of these attributes.
In addition, an instance has four flags. These flags indicate:
A typical application first constructs an instance of this class,
sets up the syntax tables, and then repeatedly loops calling the
nextToken
method in each iteration of the loop until
it returns the value TT_EOF
.
'A'
through 'Z'
,
'a'
through 'z'
, and
'\u00A0'
through '\u00FF'
are
considered to be alphabetic.
'\u0000'
through
'\u0020'
are considered to be white space.
'/'
is a comment character.
'\''
and double quote '"'
are string quote characters.
ttype
field is TT_NUMBER
.
The initial value of this field is 0.0.
The current token is a word when the value of the
ttype
field is TT_WORD
. The current token is
a quoted string token when the value of the ttype
field is
a quote character.
The initial value of this field is null.
nextToken
method, this field
contains the type of the token just read. For a single character
token, its value is the single character, converted to an integer.
For a quoted string token (see , its value is the quote character.
Otherwise, its value is one of the following:
TT_WORD
indicates that the token is a word.
TT_NUMBER
indicates that the token is a number.
TT_EOL
indicates that the end of line has been read.
The field can only have this value if the
eolIsSignificant
method has been called with the
argument true
.
TT_EOF
indicates that the end of the input stream
has been reached.
The initial value of this field is -4.
Any other attribute settings for the specified character are cleared.
nextToken
method returns
TT_EOL
and also sets the ttype
field to
this value when an end of line is read.
A line is a sequence of characters ending with either a
carriage-return character ('\r'
) or a newline
character ('\n'
). In addition, a carriage-return
character followed immediately by a newline character is treated
as a single end-of-line token.
If the flag
is false, end-of-line characters are
treated as white space and serve only to separate tokens.
The equals
method implements an equivalence relation
on non-null object references:
x
, x.equals(x)
should return
true
.
x
and y
, x.equals(y)
should return true
if and only if
y.equals(x)
returns true
.
x
, y
, and z
, if
x.equals(y)
returns true
and
y.equals(z)
returns true
, then
x.equals(z)
should return true
.
x
and y
, multiple invocations of
x.equals(y) consistently return true
or consistently return false
, provided no
information used in equals
comparisons on the
objects is modified.
x
,
x.equals(null)
should return false
.
The equals method for class Object
implements
the most discriminating possible equivalence relation on objects;
that is, for any non-null reference values x
and
y
, this method returns true
if and only
if x
and y
refer to the same object
(x == y
has the value true
).
Note that it is generally necessary to override the hashCode method whenever this method is overridden, so as to maintain the general contract for the hashCode method, which states that equal objects must have equal hash codes.
java.util.Hashtable
.
The general contract of hashCode
is:
hashCode
method on each of
the two objects must produce the same integer result.
As much as is reasonably practical, the hashCode method defined by class Object does return distinct integers for distinct objects. (This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.)
true
, then the value in the
sval
field is lowercased whenever a word token is
returned (the ttype
field has the
value TT_WORD
by the nextToken
method
of this tokenizer.
If the flag argument is false
, then the
sval
field is not modified.
ttype
field. Additional information about the token may be in the
nval
field or the sval
field of this
tokenizer.
Typical clients of this class first set up the syntax tables and then sit in a loop calling nextToken to parse successive tokens until TT_EOF is returned.
wait
methods.
The awakened thread will not be able to proceed until the current thread relinquishes the lock on this object. The awakened thread will compete in the usual manner with any other threads that might be actively competing to synchronize on this object; for example, the awakened thread enjoys no reliable privilege or disadvantage in being the next thread to lock this object.
This method should only be called by a thread that is the owner of this object's monitor. A thread becomes the owner of the object's monitor in one of three ways:
synchronized
statement
that synchronizes on the object.
Class,
by executing a
synchronized static method of that class.
Only one thread at a time can own an object's monitor.
wait
methods.
The awakened threads will not be able to proceed until the current thread relinquishes the lock on this object. The awakened threads will compete in the usual manner with any other threads that might be actively competing to synchronize on this object; for example, the awakened threads enjoy no reliable privilege or disadvantage in being the next thread to lock this object.
This method should only be called by a thread that is the owner
of this object's monitor. See the notify
method for a
description of the ways in which a thread can become the owner of
a monitor.
ttype
field to the
character value.
Making a line terminator character "ordinary" may interfere
with the ability of a StreamTokenizer
to count
lines. The lineno
method may no longer reflect
the presence of such terminator characters in its line count.
low <= c <= high
are "ordinary" in this tokenizer. See the
ordinaryChar
method for more information on a
character being ordinary.0 1 2 3 4 5 6 7 8 9 . -
has the "numeric" attribute.
When the parser encounters a word token that has the format of a
double precision floating-point number, it treats the token as a
number rather than a word, by setting the ttype
field to the value TT_NUMBER
and putting the numeric
value of the token into the nval
field.
nextToken
method of this
tokenizer to return the current value in the ttype
field, and not to modify the value in the nval
or
sval
field.
When the nextToken
method encounters a string
constant, the ttype
field is set to the string
delimiter and the sval
field is set to the body of
the string.
If a string quote character is encountered, then a string is
recognized, consisting of all characters after (but not including)
the string quote character, up to (but not including) the next
occurrence of that same string quote character, or a line
terminator, or end of file. The usual escape sequences such as
"\n"
and "\t"
are recognized and
converted to single characters as the string is parsed.
Any other attribute settings for the specified character are cleared.
ordinaryChar
method
for more information on a character being ordinary.true
, this stream tokenizer
recognizes C++-style comments. Any occurrence of two consecutive
slash characters ('/'
) is treated as the beginning of
a comment that extends to the end of the line.
If the flag argument is false
, then C++-style
comments are not treated specially.
true
, this stream tokenizer
recognizes C-style comments. All text between successive
occurrences of /*
and */
are discarded.
If the flag argument is false
, then C-style comments
are not treated specially.
The precise string returned is unspecified, although the following example can be considered typical:
Token['a'], line 10
The current thread must own this object's monitor. The thread
releases ownership of this monitor and waits until another thread
notifies threads waiting on this object's monitor to wake up
either through a call to the notify
method or the
notifyAll
method. The thread then waits until it can
re-obtain ownership of the monitor and resumes execution.
As in the one argument version, interrupts and spurious wakeups are possible, and this method should always be used in a loop:
synchronized (obj) { while (<condition does not hold>) obj.wait(); ... // Perform action appropriate to condition }This method should only be called by a thread that is the owner of this object's monitor. See the
notify
method for a
description of the ways in which a thread can become the owner of
a monitor.The current thread must own this object's monitor.
This method causes the current thread (call it T) to place itself in the wait set for this object and then to relinquish any and all synchronization claims on this object. Thread T becomes disabled for thread scheduling purposes and lies dormant until one of four things happens:
A thread can also wake up without being notified, interrupted, or timing out, a so-called spurious wakeup. While this will rarely occur in practice, applications must guard against it by testing for the condition that should have caused the thread to be awakened, and continuing to wait if the condition is not satisfied. In other words, waits should always occur in loops, like this one:
synchronized (obj) { while (<condition does not hold>) obj.wait(timeout); ... // Perform action appropriate to condition }(For more information on this topic, see Section 3.2.3 in Doug Lea's "Concurrent Programming in Java (Second Edition)" (Addison-Wesley, 2000), or Item 50 in Joshua Bloch's "Effective Java Programming Language Guide" (Addison-Wesley, 2001).
If the current thread is interrupted by another thread while it is waiting, then an InterruptedException is thrown. This exception is not thrown until the lock status of this object has been restored as described above.
Note that the wait method, as it places the current thread into the wait set for this object, unlocks only this object; any other objects on which the current thread may be synchronized remain locked while the thread waits.
This method should only be called by a thread that is the owner
of this object's monitor. See the notify
method for a
description of the ways in which a thread can become the owner of
a monitor.
This method is similar to the wait
method of one
argument, but it allows finer control over the amount of time to
wait for a notification before giving up. The amount of real time,
measured in nanoseconds, is given by:
1000000*timeout+nanos
In all other respects, this method does the same thing as the method of one argument. In particular, wait(0, 0) means the same thing as wait(0).
The current thread must own this object's monitor. The thread releases ownership of this monitor and waits until either of the following two conditions has occurred:
notify
method
or the notifyAll
method.
timeout
milliseconds plus nanos
nanoseconds arguments, has
elapsed.
The thread then waits until it can re-obtain ownership of the monitor and resumes execution.
As in the one argument version, interrupts and spurious wakeups are possible, and this method should always be used in a loop:
synchronized (obj) { while (<condition does not hold>) obj.wait(timeout, nanos); ... // Perform action appropriate to condition }This method should only be called by a thread that is the owner of this object's monitor. See the
notify
method for a
description of the ways in which a thread can become the owner of
a monitor.low <= c <= high
are white space characters. White space characters serve only to
separate tokens in the input stream.
Any other attribute settings for the characters in the specified range are cleared.
low <= c <= high
are word constituents. A word token consists of a word constituent
followed by zero or more word constituents or number constituents.