Module protogen :: Class protogentokenizer
[hide private]
[frames] | no frames]

Class protogentokenizer

source code

Instance Methods [hide private]
 
__init__(self, tokenizer, autogennamebase) source code
 
_getautogenname(self, line) source code
 
_lookahead(self, howfar=1)
Returns a token howfar ahead
source code
 
_realnext(self)
Gets the next token from our input, ignoring the pushback list
source code
 
_nextignorenl(self)
Gets next token ignoring newlines
source code
 
_next(self)
Gets next token from our input, looking in pushback list
source code
 
_consumenl(self)
consumes any newlines
source code
 
_getuptoeol(self)
Returns everything up to newline as a string.
source code
 
_getline(self, line) source code
 
__iter__(self) source code
 
next(self) source code
 
_getliteral(self)
Returns the section enclosed in %{ ...
source code
 
_getdict(self)
Returns a text string representing a dict.
source code
 
_processpacketheader(self) source code
 
_processpacketfield(self)
Read in one packet field
source code
Class Variables [hide private]
  LITERAL = 'LITERAL'
  PACKETSTART = 'PACKETSTART'
  PACKETEND = 'PACKETEND'
  CONDITIONALSTART = 'CONDITIONALSTART'
  CONDITIONALRESTART = 'CONDITIONALRESTART'
  CONDITIONALEND = 'CONDITIONALEND'
  FIELD = 'FIELD'
  CODE = 'CODE'
  ASSERTION = 'ASSERTION'
  STATE_TOPLEVEL = 'STATE_TOPLEVEL'
  STATE_PACKET = 'STATE_PACKET'
  STATE_CONDITIONAL = 'STATE_CONDITIONAL'
Method Details [hide private]

_getuptoeol(self)

source code 

Returns everything up to newline as a string. If end of line has backslash before it then next line is returned as well

_getliteral(self)

source code 

Returns the section enclosed in %{ ... }%. The %{ and }% must be on lines by themselves.

_getdict(self)

source code 

Returns a text string representing a dict. If the next token is not a dict start then None is returned