#include <TokenBuffer.hpp>
Public Member Functions | |
TokenBuffer (TokenStream &input_) | |
virtual | ~TokenBuffer () |
void | reset (void) |
Reset the input buffer to empty state. | |
int | LA (unsigned int i) |
RefToken | LT (unsigned int i) |
unsigned int | mark () |
void | rewind (unsigned int mark) |
void | consume () |
virtual unsigned int | entries () const |
Return the number of entries in the TokenBuffer. | |
Protected Attributes | |
TokenStream & | input |
Token source. | |
unsigned int | nMarkers |
Number of active markers. | |
unsigned int | markerOffset |
Additional offset used when markers are active. | |
unsigned int | numToConsume |
Number of calls to consume() since last LA() or LT() call. | |
CircularQueue< RefToken > | queue |
Circular queue with Tokens. | |
Private Member Functions | |
void | fill (unsigned int amount) |
void | syncConsume () |
TokenBuffer (const TokenBuffer &other) | |
const TokenBuffer & | operator= (const TokenBuffer &other) |
A Stream of Token objects fed to the parser from a TokenStream that can be rewound via mark()/rewind() methods.
A dynamic array is used to buffer up all the input tokens. Normally, "k" tokens are stored in the buffer. More tokens may be stored during guess mode (testing syntactic predicate), or when LT(i>k) is referenced. Consumption of tokens is deferred. In other words, reading the next token is not done by conume(), but deferred until needed by LA or LT.
antlr::TokenBuffer::TokenBuffer | ( | TokenStream & | inp | ) |
Create a token buffer
A Stream of Token objects fed to the parser from a TokenStream that can be rewound via mark()/rewind() methods.
A dynamic array is used to buffer up all the input tokens. Normally, "k" tokens are stored in the buffer. More tokens may be stored during guess mode (testing syntactic predicate), or when LT(i>k) is referenced. Consumption of tokens is deferred. In other words, reading the next token is not done by conume(), but deferred until needed by LA or LT.
antlr::TokenBuffer::~TokenBuffer | ( | void | ) | [virtual] |
antlr::TokenBuffer::TokenBuffer | ( | const TokenBuffer & | other | ) | [private] |
void antlr::TokenBuffer::consume | ( | ) | [inline] |
Mark another token for deferred consumption
unsigned int antlr::TokenBuffer::entries | ( | ) | const [virtual] |
Return the number of entries in the TokenBuffer.
Get number of non-consumed tokens.
void antlr::TokenBuffer::fill | ( | unsigned int | amount | ) | [private] |
Ensure that the token buffer is sufficiently full
int antlr::TokenBuffer::LA | ( | unsigned int | i | ) |
Get a lookahead token value
RefToken antlr::TokenBuffer::LT | ( | unsigned int | i | ) |
Get a lookahead token
unsigned int antlr::TokenBuffer::mark | ( | ) |
Return an integer marker that can be used to rewind the buffer to its current state.
const TokenBuffer& antlr::TokenBuffer::operator= | ( | const TokenBuffer & | other | ) | [private] |
void antlr::TokenBuffer::reset | ( | void | ) | [inline] |
Reset the input buffer to empty state.
void antlr::TokenBuffer::rewind | ( | unsigned int | mark | ) |
Rewind the token buffer to a marker.
mark | Marker returned previously from mark() |
void antlr::TokenBuffer::syncConsume | ( | ) | [inline, private] |
Sync up deferred consumption
TokenStream& antlr::TokenBuffer::input [protected] |
Token source.
unsigned int antlr::TokenBuffer::markerOffset [protected] |
Additional offset used when markers are active.
unsigned int antlr::TokenBuffer::nMarkers [protected] |
Number of active markers.
unsigned int antlr::TokenBuffer::numToConsume [protected] |
CircularQueue<RefToken> antlr::TokenBuffer::queue [protected] |
Circular queue with Tokens.