This is a simplistic C/C++-like preprocessor. More...

#include <Preprocessor.h>

Classes
class	Macro
	A macro definition. More...

class	Token
	A input token. More...

Public Types
typedef void(*	ErrorHandlerFunc )(void iData, int iLine, const char iError, const char *iToken, size_t iTokenLen)
	An error handler function type. More...

Public Member Functions
	CPreprocessor ()
	Create an empty preprocessor object. More...

virtual	~CPreprocessor ()
	Destroy the preprocessor object. More...

void	Define (const char iMacroName, size_t iMacroNameLen, const char iMacroValue, size_t iMacroValueLen)
	Define a macro without parameters. More...

void	Define (const char *iMacroName, size_t iMacroNameLen, long iMacroValue)
	Define a numerical macro. More...

void	Define (const char iMacroName, const char iMacroValue)
	Define a macro without parameters. More...

void	Define (const char *iMacroName, long iMacroValue)
	Define a numerical macro. More...

bool	Undef (const char *iMacroName, size_t iMacroNameLen)
	Undefine a macro. More...

char *	Parse (const char *iSource, size_t iLength, size_t &oLength)
	Parse the input string and return a newly-allocated output string. More...

Public Attributes
void *	ErrorData
	User-specific storage, passed to Error() More...

Static Public Attributes
static ErrorHandlerFunc	ErrorHandler = DefaultError
	A pointer to the preprocessor's error handler. More...

Private Member Functions
	CPreprocessor (const Token &iToken, int iLine)
	Private constructor to re-parse a single token. More...

Token	GetToken (bool iExpand)
	Stateless tokenizer: Parse the input text and return the next token. More...

Token	HandleDirective (Token &iToken, int iLine)
	Handle a preprocessor directive. More...

bool	HandleDefine (Token &iBody, int iLine)
	Handle a #define directive. More...

bool	HandleUnDef (Token &iBody, int iLine)
	Undefine a previously defined macro. More...

bool	HandleIfDef (Token &iBody, int iLine)
	Handle an #ifdef directive. More...

bool	HandleIf (Token &iBody, int iLine)
	Handle an #if directive. More...

bool	HandleElse (Token &iBody, int iLine)
	Handle an #else directive. More...

bool	HandleEndIf (Token &iBody, int iLine)
	Handle an #endif directive. More...

Token	GetArgument (Token &oArg, bool iExpand)
	Get a single function argument until next ',' or ')'. More...

Token	GetArguments (int &oNumArgs, Token *&oArgs, bool iExpand)
	Get all the arguments of a macro: '(' arg1 { ',' arg2 { ',' ... More...

Token	GetExpression (Token &oResult, int iLine, int iOpPriority=0)
	Parse an expression, compute it and return the result. More...

bool	GetValue (const Token &iToken, long &oValue, int iLine)
	Get the numeric value of a token. More...

Token	ExpandMacro (const Token &iToken)
	Expand the given macro, if it exists. More...

Macro *	IsDefined (const Token &iToken)
	Check if a macro is defined, and if so, return it. More...

Token	Parse (const Token &iSource)
	Parse the input string and return a token containing the whole output. More...

void	Error (int iLine, const char iError, const Token iToken=NULL)
	Call the error handler. More...

Static Private Member Functions
static Token	ExpandDefined (CPreprocessor iParent, int iNumArgs, Token iArgs)
	The implementation of the defined() preprocessor function. More...

Private Attributes
const char *	Source
	The current source text input. More...

const char *	SourceEnd
	The end of the source text. More...

int	Line
	Current line number. More...

bool	BOL
	True if we are at beginning of line. More...

unsigned	EnableOutput
	A stack of 32 booleans packed into one value :) More...

Macro *	MacroList
	The list of macros defined so far. More...

Friends
class	CPreprocessor::Macro

Detailed Description

This is a simplistic C/C++-like preprocessor.

It takes an non-zero-terminated string on input and outputs a non-zero-terminated string buffer.

This preprocessor was designed specifically for GLSL shaders, so if you want to use it for other purposes you might want to check if the feature set it provides is enough for you.

Here's a list of supported features:

Fast memory allocation-less operation (mostly).
Line continuation (backslash-newline) is swallowed.
Line numeration is fully preserved by inserting empty lines where required. This is crucial if, say, GLSL compiler reports you an error with a line number.
#define: Parametrized and non-parametrized macros. Invoking a macro with less arguments than it takes assignes empty values to missing arguments.
#undef: Forget defined macros
#ifdef/#ifndef/#else/#endif: Conditional suppression of parts of code.
#if: Supports numeric expression of any complexity, also supports the defined() pseudo-function.

Definition at line 62 of file Preprocessor.h.

Member Typedef Documentation

typedef void(* CPreprocessor::ErrorHandlerFunc)(void *iData, int iLine, const char *iError, const char *iToken, size_t iTokenLen)

An error handler function type.

The default implementation just drops a note to stderr and then the parser ends, returning NULL.

Parameters

iData	User-specific pointer from the corresponding CPreprocessor object.
iLine	The line at which the error happened.
iError	The error string.
iToken	If not NULL contains the erroneous token
iTokenLen	The length of iToken. iToken is never zero-terminated!

Definition at line 526 of file Preprocessor.h.

Constructor & Destructor Documentation

CPreprocessor::CPreprocessor	(	const Token &	iToken,
		int	iLine
	)

private

Private constructor to re-parse a single token.

Definition at line 244 of file Preprocessor.cpp.

CPreprocessor::CPreprocessor ( )

inline

Create an empty preprocessor object.

Definition at line 427 of file Preprocessor.h.

CPreprocessor::~CPreprocessor ( )

virtual

Destroy the preprocessor object.

Definition at line 253 of file Preprocessor.cpp.

Member Function Documentation

void CPreprocessor::Define	(	const char *	iMacroName,
		size_t	iMacroNameLen,
		const char *	iMacroValue,
		size_t	iMacroValueLen
	)

Define a macro without parameters.

Parameters

iMacroName	The name of the defined macro
iMacroNameLen	The length of the name of the defined macro
iMacroValue	The value of the defined macro
iMacroValueLen	The length of the value of the defined macro

Definition at line 1159 of file Preprocessor.cpp.

void CPreprocessor::Define	(	const char *	iMacroName,
		size_t	iMacroNameLen,
		long	iMacroValue
	)

Define a numerical macro.

Parameters

iMacroName	The name of the defined macro
iMacroNameLen	The length of the name of the defined macro
iMacroValue	The value of the defined macro

Definition at line 1168 of file Preprocessor.cpp.

void CPreprocessor::Define	(	const char *	iMacroName,
		const char *	iMacroValue
	)

Define a macro without parameters.

Parameters

iMacroName	The name of the defined macro
iMacroValue	The value of the defined macro

Definition at line 1177 of file Preprocessor.cpp.

void CPreprocessor::Define	(	const char *	iMacroName,
		long	iMacroValue
	)

Define a numerical macro.

Parameters

iMacroName	The name of the defined macro
iMacroValue	The value of the defined macro

Definition at line 1182 of file Preprocessor.cpp.

void CPreprocessor::Error	(	int	iLine,
		const char *	iError,
		const Token *	iToken = `NULL`
	)

private

Call the error handler.

Parameters

iLine	The line at which the error happened.
iError	The error string.
iToken	If not NULL contains the erroneous token

Definition at line 258 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::ExpandDefined	(	CPreprocessor *	iParent,
		int	iNumArgs,
		Token *	iArgs
	)

staticprivate

The implementation of the defined() preprocessor function.

Parameters

iParent	The parent preprocessor object
iNumArgs	Number of arguments
iArgs	The arguments themselves

Returns: The return value encapsulated in a token

Definition at line 982 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::ExpandMacro ( const Token & iToken )

private

Expand the given macro, if it exists.

If macro has arguments, they are collected from source stream.

Parameters

iToken A KEYWORD token containing the (possible) macro name.

Returns: The expanded token or iToken if it is not a macro

Definition at line 407 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::GetArgument	(	Token &	oArg,
		bool	iExpand
	)

private

Get a single function argument until next ',' or ')'.

Parameters

oArg	The argument is returned in this variable.
iExpand	If false, parameters are not expanded and no expressions are allowed; only a single keyword is expected per argument.

Returns: The first unhandled token after argument.

Definition at line 736 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::GetArguments	(	int &	oNumArgs,
		Token *&	oArgs,
		bool	iExpand
	)

private

Get all the arguments of a macro: '(' arg1 { ',' arg2 { ',' ...

}} ')'

Parameters

oNumArgs	Number of parsed arguments is stored into this variable.
oArgs	This is set to a pointer to an array of parsed arguments.
iExpand	If false, parameters are not expanded and no expressions are allowed; only a single keyword is expected per argument.

Definition at line 808 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::GetExpression	(	Token &	oResult,
		int	iLine,
		int	iOpPriority = `0`
	)

private

Parse an expression, compute it and return the result.

Operator priority: 0: Whole expression 1: '(' ')' 2: || 3: && 4: | 5: ^ 6: & 7: '==' '!=' 8: '<' '<=' '>' '>=' 9: '<<' '>>' 10: '+' '-' 11: '*' '/' '' 12: unary '+' '-' '!' '~'.

Parameters

oResult	A token containing the result of expression
iLine	The line at which the expression starts (for error reports)
iOpPriority	Operator priority (at which operator we will stop if proceeding recursively – used internally. Parser stops when it encounters an operator with higher or equal priority).

Returns: The last unhandled token after the expression

Definition at line 474 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::GetToken ( bool iExpand )

private

Stateless tokenizer: Parse the input text and return the next token.

Parameters

iExpand If true, macros will be expanded to their values

Returns: The next token from the input stream

Definition at line 266 of file Preprocessor.cpp.

bool CPreprocessor::GetValue	(	const Token &	iToken,
		long &	oValue,
		int	iLine
	)

private

Get the numeric value of a token.

If the token was produced by expanding a macro, we will get an TEXT token which can contain a whole expression; in this case we will call GetExpression to parse it. Otherwise we just call the token's GetValue() method.

Parameters

iToken	The token to get the numeric value of
oValue	The variable to put the value into
iLine	The line where the directive begins (for error reports)

Returns: true if ok, false if not

Definition at line 660 of file Preprocessor.cpp.

bool CPreprocessor::HandleDefine	(	Token &	iBody,
		int	iLine
	)

private

Handle a #define directive.

Parameters

iBody	The body of the directive (everything after the directive until end of line).
iLine	The line where the directive begins (for error reports)

Returns: true if everything went ok, false if not

Definition at line 872 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::HandleDirective	(	Token &	iToken,
		int	iLine
	)

private

Handle a preprocessor directive.

Parameters

iToken	The whole preprocessor directive line (until EOL)
iLine	The line where the directive begins (for error reports)

Returns: The last input token that was not proceeded.

Definition at line 1053 of file Preprocessor.cpp.

bool CPreprocessor::HandleElse	(	Token &	iBody,
		int	iLine
	)

private

Handle an #else directive.

Parameters

iBody	The body of the directive (everything after the directive until end of line).
iLine	The line where the directive begins (for error reports)

Returns: true if everything went ok, false if not

Definition at line 1021 of file Preprocessor.cpp.

bool CPreprocessor::HandleEndIf	(	Token &	iBody,
		int	iLine
	)

private

Handle an #endif directive.

Parameters

iBody	The body of the directive (everything after the directive until end of line).
iLine	The line where the directive begins (for error reports)

Returns: true if everything went ok, false if not

Definition at line 1038 of file Preprocessor.cpp.

bool CPreprocessor::HandleIf	(	Token &	iBody,
		int	iLine
	)

private

Handle an #if directive.

Parameters

iBody	The body of the directive (everything after the directive until end of line).
iLine	The line where the directive begins (for error reports)

Returns: true if everything went ok, false if not

Definition at line 994 of file Preprocessor.cpp.

bool CPreprocessor::HandleIfDef	(	Token &	iBody,
		int	iLine
	)

private

Handle an #ifdef directive.

Parameters

iBody	The body of the directive (everything after the directive until end of line).
iLine	The line where the directive begins (for error reports)

Returns: true if everything went ok, false if not

Definition at line 947 of file Preprocessor.cpp.

bool CPreprocessor::HandleUnDef	(	Token &	iBody,
		int	iLine
	)

private

Undefine a previously defined macro.

Parameters

iBody	The body of the directive (everything after the directive until end of line).
iLine	The line where the directive begins (for error reports)

Returns: true if everything went ok, false if not

Definition at line 919 of file Preprocessor.cpp.

CPreprocessor::Macro * CPreprocessor::IsDefined ( const Token & iToken )

private

Check if a macro is defined, and if so, return it.

Parameters

iToken Macro name

Returns: The macro object or NULL if a macro with this name does not exist

Definition at line 398 of file Preprocessor.cpp.

CPreprocessor::Token CPreprocessor::Parse ( const Token & iSource )

private

Parse the input string and return a token containing the whole output.

Parameters

iSource The source text enclosed in a token

Returns: The output text enclosed in a token

Definition at line 1208 of file Preprocessor.cpp.

char * CPreprocessor::Parse	(	const char *	iSource,
		size_t	iLength,
		size_t &	oLength
	)

Parse the input string and return a newly-allocated output string.

Note: The returned preprocessed string is NOT zero-terminated (just like the input string).

Parameters

iSource	The source text
iLength	The length of the source text in characters
oLength	The length of the output string.

Returns: The output from preprocessor, allocated with malloc(). The parser can actually allocate more than needed for performance reasons, but this should not be a problem unless you will want to store the returned pointer for long time in which case you might want to realloc() it. If an error has been encountered, the function returns NULL. In some cases the function may return an unallocated address that's inside the source buffer. You must free() the result string only if the returned address is not inside the source text.

Definition at line 1304 of file Preprocessor.cpp.

bool CPreprocessor::Undef	(	const char *	iMacroName,
		size_t	iMacroNameLen
	)

Undefine a macro.

Parameters

iMacroName	The name of the macro to undefine
iMacroNameLen	The length of the name of the macro to undefine

Returns: true if the macro has been undefined, false if macro doesn't exist

Definition at line 1187 of file Preprocessor.cpp.

Friends And Related Function Documentation

friend class CPreprocessor::Macro

friend

Definition at line 205 of file Preprocessor.h.

Member Data Documentation

bool CPreprocessor::BOL

private

True if we are at beginning of line.

Definition at line 214 of file Preprocessor.h.

unsigned CPreprocessor::EnableOutput

private

A stack of 32 booleans packed into one value :)

Definition at line 216 of file Preprocessor.h.

void* CPreprocessor::ErrorData

User-specific storage, passed to Error()

Definition at line 538 of file Preprocessor.h.

CPreprocessor::ErrorHandlerFunc CPreprocessor::ErrorHandler = DefaultError

static

A pointer to the preprocessor's error handler.

You can assign the address of your own function to this variable and implement your own error handling (e.g. throwing an exception etc).

Definition at line 535 of file Preprocessor.h.

int CPreprocessor::Line

private

Current line number.

Definition at line 212 of file Preprocessor.h.

Macro* CPreprocessor::MacroList

private

The list of macros defined so far.

Definition at line 218 of file Preprocessor.h.

const char* CPreprocessor::Source

private

The current source text input.

Definition at line 208 of file Preprocessor.h.

const char* CPreprocessor::SourceEnd

private

The end of the source text.

Definition at line 210 of file Preprocessor.h.

The documentation for this class was generated from the following files:

/home/sathyam/0ad.svn/source/ps/Preprocessor.h
/home/sathyam/0ad.svn/source/ps/Preprocessor.cpp

Classes

Public Types

Public Member Functions

Public Attributes

Static Public Attributes

Private Member Functions

Static Private Member Functions

Private Attributes

Friends

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Friends And Related Function Documentation

Member Data Documentation