CPreprocessor::Token Class Reference

A input token. More...

Public Types
enum	Kind {   TK_EOS, TK_ERROR, TK_WHITESPACE, TK_NEWLINE,   TK_LINECONT, TK_NUMBER, TK_KEYWORD, TK_PUNCTUATION,   TK_DIRECTIVE, TK_STRING, TK_COMMENT, TK_LINECOMMENT,   TK_TEXT }

Public Member Functions
	Token ()
	Token (Kind iType)
	Token (Kind iType, const char *iString, size_t iLength)
	Token (const Token &iOther)
	~Token ()
Token &	operator= (const Token &iOther)
	Assignment operator. More...
void	Append (const char *iString, size_t iLength)
	Append a string to this token. More...
void	Append (const Token &iOther)
	Append a token to this token. More...
void	AppendNL (int iCount)
	Append given number of newlines to this token. More...
int	CountNL ()
	Count number of newlines in this token. More...
bool	GetValue (long &oValue) const
	Get the numeric value of the token. More...
void	SetValue (long iValue)
	Set the numeric value of the token. More...
bool	operator== (const Token &iOther)
	Test two tokens for equality. More...

Public Attributes
Kind	Type
	Token type. More...
size_t	Allocated
	True if string was allocated (and must be freed) More...
union {
const char *   String
	A pointer somewhere into the input buffer. More...
char *   Buffer
	A memory-allocated string. More...
};
size_t	Length
	Token length in bytes. More...

Detailed Description

A input token.

For performance reasons most tokens will point to portions of the input stream, so no unneeded memory allocation is done. However, in some cases we must allocate different memory for token storage, in this case this is signalled by setting the Allocated member to non-zero in which case the destructor will know that it must free memory on object destruction.

Again for performance reasons we use malloc/realloc/free here because C++-style new[] lacks the realloc() counterpart.

Definition at line 77 of file Preprocessor.h.

Member Enumeration Documentation

enum CPreprocessor::Token::Kind

Enumerator
TK_EOS
TK_ERROR
TK_WHITESPACE
TK_NEWLINE
TK_LINECONT
TK_NUMBER
TK_KEYWORD
TK_PUNCTUATION
TK_DIRECTIVE
TK_STRING
TK_COMMENT
TK_LINECOMMENT
TK_TEXT

Definition at line 80 of file Preprocessor.h.

Constructor & Destructor Documentation

CPreprocessor::Token::Token ( )

inline

Definition at line 111 of file Preprocessor.h.

CPreprocessor::Token::Token ( Kind  iType )

inline

Definition at line 114 of file Preprocessor.h.

CPreprocessor::Token::Token ( Kind  iType, const char *  iString, size_t  iLength  )

inline

Definition at line 117 of file Preprocessor.h.

CPreprocessor::Token::Token ( const Token &  iOther )

inline

Definition at line 121 of file Preprocessor.h.

CPreprocessor::Token::~Token ( )

inline

Definition at line 130 of file Preprocessor.h.

Member Function Documentation

void CPreprocessor::Token::Append	(	const char *	iString,
		size_t	iLength
	)

Append a string to this token.

Definition at line 56 of file Preprocessor.cpp.

void CPreprocessor::Token::Append

(

const Token &

iOther

)

Append a token to this token.

Definition at line 62 of file Preprocessor.cpp.

void CPreprocessor::Token::AppendNL

(

int

iCount

)

Append given number of newlines to this token.

Definition at line 160 of file Preprocessor.cpp.

int CPreprocessor::Token::CountNL

(

)

Count number of newlines in this token.

Definition at line 174 of file Preprocessor.cpp.

bool CPreprocessor::Token::GetValue

(

long &

oValue

)

const

Get the numeric value of the token.

Definition at line 102 of file Preprocessor.cpp.

Token& CPreprocessor::Token::operator= ( const Token &  iOther )

inline

Assignment operator.

Definition at line 134 of file Preprocessor.h.

bool CPreprocessor::Token::operator== ( const Token &  iOther )

inline

Test two tokens for equality.

Definition at line 164 of file Preprocessor.h.

void CPreprocessor::Token::SetValue

(

long

iValue

)

Set the numeric value of the token.

Definition at line 151 of file Preprocessor.cpp.

Member Data Documentation

union { ... }

size_t CPreprocessor::Token::Allocated

mutable

True if string was allocated (and must be freed)

Definition at line 100 of file Preprocessor.h.

char* CPreprocessor::Token::Buffer

A memory-allocated string.

Definition at line 106 of file Preprocessor.h.

size_t CPreprocessor::Token::Length

Token length in bytes.

Definition at line 109 of file Preprocessor.h.

const char* CPreprocessor::Token::String

A pointer somewhere into the input buffer.

Definition at line 104 of file Preprocessor.h.

Kind CPreprocessor::Token::Type

Token type.

Definition at line 98 of file Preprocessor.h.

---

The documentation for this class was generated from the following files:

• /home/sathyam/0ad.svn/source/ps/Preprocessor.h
• /home/sathyam/0ad.svn/source/ps/Preprocessor.cpp