antlr::TokenBuffer Class Reference

#include <TokenBuffer.hpp>

Collaboration diagram for antlr::TokenBuffer:
[legend]

List of all members.

Public Member Functions

 TokenBuffer (TokenStream &input_)
virtual ~TokenBuffer ()
void reset (void)
 Reset the input buffer to empty state.
int LA (unsigned int i)
RefToken LT (unsigned int i)
unsigned int mark ()
void rewind (unsigned int mark)
void consume ()
virtual unsigned int entries () const
 Return the number of entries in the TokenBuffer.

Protected Attributes

TokenStreaminput
 Token source.
unsigned int nMarkers
 Number of active markers.
unsigned int markerOffset
 Additional offset used when markers are active.
unsigned int numToConsume
 Number of calls to consume() since last LA() or LT() call.
CircularQueue< RefTokenqueue
 Circular queue with Tokens.

Private Member Functions

void fill (unsigned int amount)
void syncConsume ()
 TokenBuffer (const TokenBuffer &other)
const TokenBufferoperator= (const TokenBuffer &other)

Detailed Description

A Stream of Token objects fed to the parser from a TokenStream that can be rewound via mark()/rewind() methods.

A dynamic array is used to buffer up all the input tokens. Normally, "k" tokens are stored in the buffer. More tokens may be stored during guess mode (testing syntactic predicate), or when LT(i>k) is referenced. Consumption of tokens is deferred. In other words, reading the next token is not done by conume(), but deferred until needed by LA or LT.

Todo:
: see if we can integrate this one with InputBuffer into one template or so.
See also:
antlr.Token
antlr.TokenStream
antlr.TokenQueue

Constructor & Destructor Documentation

antlr::TokenBuffer::TokenBuffer ( TokenStream inp  ) 

Create a token buffer

A Stream of Token objects fed to the parser from a TokenStream that can be rewound via mark()/rewind() methods.

A dynamic array is used to buffer up all the input tokens. Normally, "k" tokens are stored in the buffer. More tokens may be stored during guess mode (testing syntactic predicate), or when LT(i>k) is referenced. Consumption of tokens is deferred. In other words, reading the next token is not done by conume(), but deferred until needed by LA or LT.

See also:
antlr.Token
antlr.TokenStream
antlr.TokenQueue Create a token buffer
antlr::TokenBuffer::~TokenBuffer ( void   )  [virtual]
antlr::TokenBuffer::TokenBuffer ( const TokenBuffer other  )  [private]

Member Function Documentation

void antlr::TokenBuffer::consume (  )  [inline]

Mark another token for deferred consumption

unsigned int antlr::TokenBuffer::entries (  )  const [virtual]

Return the number of entries in the TokenBuffer.

Get number of non-consumed tokens.

void antlr::TokenBuffer::fill ( unsigned int  amount  )  [private]

Ensure that the token buffer is sufficiently full

int antlr::TokenBuffer::LA ( unsigned int  i  ) 

Get a lookahead token value

RefToken antlr::TokenBuffer::LT ( unsigned int  i  ) 

Get a lookahead token

unsigned int antlr::TokenBuffer::mark (  ) 

Return an integer marker that can be used to rewind the buffer to its current state.

const TokenBuffer& antlr::TokenBuffer::operator= ( const TokenBuffer other  )  [private]
void antlr::TokenBuffer::reset ( void   )  [inline]

Reset the input buffer to empty state.

void antlr::TokenBuffer::rewind ( unsigned int  mark  ) 

Rewind the token buffer to a marker.

Parameters:
mark Marker returned previously from mark()
void antlr::TokenBuffer::syncConsume (  )  [inline, private]

Sync up deferred consumption


Member Data Documentation

Token source.

unsigned int antlr::TokenBuffer::markerOffset [protected]

Additional offset used when markers are active.

unsigned int antlr::TokenBuffer::nMarkers [protected]

Number of active markers.

unsigned int antlr::TokenBuffer::numToConsume [protected]

Number of calls to consume() since last LA() or LT() call.

Circular queue with Tokens.


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Friends Defines

Generated on 16 Nov 2015 for ANTLR Support Libraries 2.7.1+ by  doxygen 1.6.1