Uses of Class
gate.creole.annic.apache.lucene.analysis.standard.Token

Packages that use Token

gate.creole.annic.apache.lucene.analysis.standard

A grammar-based tokenizer constructed with JavaCC.

Uses of Token in gate.creole.annic.apache.lucene.analysis.standard

Fields in gate.creole.annic.apache.lucene.analysis.standard declared as Token

Token	ParseException.currentToken This is the last token that has been consumed successfully.
Token	StandardTokenizer.jj_nt
Token	Token.next A reference to the next regular (non-special) token from the input stream.
Token	Token.specialToken This field is used to access special tokens that occur prior to this token, but after the immediately preceding regular (non-special) token.
Token	StandardTokenizer.token

Methods in gate.creole.annic.apache.lucene.analysis.standard that return Token

Token	StandardTokenizer.getNextToken()
Token	StandardTokenizerTokenManager.getNextToken()
Token	StandardTokenizer.getToken(int index)
private Token	StandardTokenizer.jj_consume_token(int kind)
protected Token	StandardTokenizerTokenManager.jjFillToken()
static Token	Token.newToken(int ofKind) Returns a new Token object, by default.

Constructors in gate.creole.annic.apache.lucene.analysis.standard with parameters of type Token

ParseException(Token currentTokenVal, int[][] expectedTokenSequencesVal, String[] tokenImageVal)
This constructor is used by the method "generateParseException" in the generated parser.