protogen.protogentokenizer

Module protogen :: Class protogentokenizer

[show private | hide private]

Class protogentokenizer

Method Summary
	`__init__(self, tokenizer, autogennamebase)`
	`__iter__(self)`
	`next(self)`
	`_consumenl(self)` consumes any newlines
	`_getautogenname(self, line)`
	`_getdict(self)` Returns a text string representing a dict.
	`_getline(self, line)`
	`_getliteral(self)` Returns the section enclosed in %{ ...
	`_getuptoeol(self)` Returns everything up to newline as a string.
	`_lookahead(self, howfar)` Returns a token howfar ahead
	`_next(self)` Gets next token from our input, looking in pushback list
	`_nextignorenl(self)` Gets next token ignoring newlines
	`_processpacketfield(self)` Read in one packet field
	`_processpacketheader(self)`
	`_realnext(self)` Gets the next token from our input, ignoring the pushback list

Class Variable Summary
`str`	`ASSERTION` = `'ASSERTION'`
`str`	`CODE` = `'CODE'`
`str`	`CONDITIONALEND` = `'CONDITIONALEND'`
`str`	`CONDITIONALRESTART` = `'CONDITIONALRESTART'`
`str`	`CONDITIONALSTART` = `'CONDITIONALSTART'`
`str`	`FIELD` = `'FIELD'`
`str`	`LITERAL` = `'LITERAL'`
`str`	`PACKETEND` = `'PACKETEND'`
`str`	`PACKETSTART` = `'PACKETSTART'`
`str`	`STATE_CONDITIONAL` = `'STATE_CONDITIONAL'`
`str`	`STATE_PACKET` = `'STATE_PACKET'`
`str`	`STATE_TOPLEVEL` = `'STATE_TOPLEVEL'`

Method Details

_consumenl(self)

consumes any newlines

_getdict(self)

Returns a text string representing a dict. If the next token is not a dict start then None is returned

_getliteral(self)

Returns the section enclosed in %{ ... }%. The %{ and }% must be on lines by themselves.

_getuptoeol(self)

Returns everything up to newline as a string. If end of line has backslash before it then next line is returned as well

_lookahead(self, howfar=1)

Returns a token howfar ahead

_next(self)

Gets next token from our input, looking in pushback list

_nextignorenl(self)

Gets next token ignoring newlines

_processpacketfield(self)

Read in one packet field

_realnext(self)

Gets the next token from our input, ignoring the pushback list

Class Variable Details

ASSERTION

Type:: str
Value:: 'ASSERTION'

CODE

Type:: str
Value:: 'CODE'

CONDITIONALEND

Type:: str
Value:: 'CONDITIONALEND'

CONDITIONALRESTART

Type:: str
Value:: 'CONDITIONALRESTART'

CONDITIONALSTART

Type:: str
Value:: 'CONDITIONALSTART'

FIELD

Type:: str
Value:: 'FIELD'

LITERAL

Type:: str
Value:: 'LITERAL'

PACKETEND

Type:: str
Value:: 'PACKETEND'

PACKETSTART

Type:: str
Value:: 'PACKETSTART'

STATE_CONDITIONAL

Type:: str
Value:: 'STATE_CONDITIONAL'

STATE_PACKET

Type:: str
Value:: 'STATE_PACKET'

STATE_TOPLEVEL

Type:: str
Value:: 'STATE_TOPLEVEL'

Generated by Epydoc 2.1 on Fri Aug 15 18:58:23 2008

http://epydoc.sf.net