ANTLR3_MIN_TOKEN_TYPE (3) - Linux Manuals

NAME

include/antlr3commontoken.h -

Defines the interface for a common token.

SYNOPSIS

#include <antlr3defs.h>

Data Structures

struct ANTLR3_COMMON_TOKEN_struct
The definition of an ANTLR3 common token structure, which all implementations of a token stream should provide, installing any further structures in the custom pointer element of this structure.
struct ANTLR3_TOKEN_FACTORY_struct
ANTLR3 Token factory interface to create lots of tokens efficiently rather than creating and freeing lots of little bits of memory.

Defines

#define ANTLR3_EOR_TOKEN_TYPE 1

#define ANTLR3_FACTORY_POOL_SIZE 1024
How many tokens to allocate at once in the token factory.
#define ANTLR3_MIN_TOKEN_TYPE ANTLR3_TOKEN_UP + 1
First token that can be used by users/generated code.
#define ANTLR3_TEXT_CHARP 1

#define ANTLR3_TEXT_NONE 0

#define ANTLR3_TEXT_STRING 2

#define ANTLR3_TOKEN_DEFAULT_CHANNEL 0
Default channel for a token.
#define ANTLR3_TOKEN_DOWN 2
Imaginary token type to cause a traversal of child nodes in a tree parser.
#define ANTLR3_TOKEN_EOF (ANTLR3_CHARSTREAM_EOF & 0xFFFFFFFF)
End of file token.
#define ANTLR3_TOKEN_INVALID 0
Indicator of an invalid token.
#define ANTLR3_TOKEN_UP 3
Imaginary token type to signal the end of a stream of child nodes.
#define HIDDEN 99
Reserved channel number for a HIDDEN token - a token that is hidden from the parser.

Typedefs

typedef struct ANTLR3_COMMON_TOKEN_struct ANTLR3_COMMON_TOKEN
The definition of an ANTLR3 common token structure, which all implementations of a token stream should provide, installing any further structures in the custom pointer element of this structure.
typedef struct ANTLR3_TOKEN_FACTORY_struct ANTLR3_TOKEN_FACTORY
ANTLR3 Token factory interface to create lots of tokens efficiently rather than creating and freeing lots of little bits of memory.

Detailed Description

Defines the interface for a common token.

All token streams should provide their tokens using an instance of this common token. A custom pointer is provided, wher you may attach a further structure to enhance the common token if you feel the need to do so. The C runtime will assume that a token provides implementations of the interface functions, but all of them may be rplaced by your own implementation if you require it.

Define Documentation

#define ANTLR3_EOR_TOKEN_TYPE 1

Referenced by combineFollows(), mismatchIsMissingToken(), and recoverFromMismatchedElement().

#define ANTLR3_FACTORY_POOL_SIZE 1024

How many tokens to allocate at once in the token factory.

Referenced by factoryClose(), newPool(), newPoolToken(), and newPoolTree().

#define ANTLR3_MIN_TOKEN_TYPE ANTLR3_TOKEN_UP + 1

First token that can be used by users/generated code.

#define ANTLR3_TEXT_CHARP 1

Referenced by antlr3CommonTreeNodeStreamNew(), createToken(), createTokenFromToken(), getText(), and setText8().

#define ANTLR3_TEXT_NONE 0

Referenced by antlr3SetTokenAPI(), emit(), and setText8().

#define ANTLR3_TEXT_STRING 2

Referenced by createTokenFromToken(), emit(), getText(), and setText().

#define ANTLR3_TOKEN_DEFAULT_CHANNEL 0

Default channel for a token.

Referenced by antlr3CommonTokenStreamSourceNew(), antlr3SetTokenAPI(), getMissingSymbol(), nextTokenStr(), reset(), and toString().

#define ANTLR3_TOKEN_DOWN 2

Imaginary token type to cause a traversal of child nodes in a tree parser.

Referenced by antlr3CommonTreeNodeStreamNew(), and fillBuffer().

#define ANTLR3_TOKEN_EOF (ANTLR3_CHARSTREAM_EOF & 0xFFFFFFFF)

End of file token.

Referenced by antlr3CommonTreeNodeStreamNew(), antlr3dfapredict(), antlr3LexerNew(), antlr3RecognitionExceptionNew(), consumeUntil(), consumeUntilSet(), displayRecognitionError(), getMissingSymbol(), getText(), and nextToken().

#define ANTLR3_TOKEN_INVALID 0

Indicator of an invalid token.

Referenced by antlr3CommonTreeNodeStreamNew(), antlr3LexerNew(), antlr3SetTokenAPI(), errorNode(), recoverFromMismatchedSet(), and reset().

#define ANTLR3_TOKEN_UP 3

Imaginary token type to signal the end of a stream of child nodes.

Referenced by antlr3CommonTreeNodeStreamNew(), and fillBuffer().

#define HIDDEN 99

Reserved channel number for a HIDDEN token - a token that is hidden from the parser.

Typedef Documentation

typedef struct ANTLR3_COMMON_TOKEN_struct

The definition of an ANTLR3 common token structure, which all implementations of a token stream should provide, installing any further structures in the custom pointer element of this structure. Remarks:

Token streams are in essence provided by lexers or other programs that serve as lexers.

typedef struct ANTLR3_TOKEN_FACTORY_struct

ANTLR3 Token factory interface to create lots of tokens efficiently rather than creating and freeing lots of little bits of memory.

Author

Generated automatically by Doxygen for ANTLR3C from the source code.