KiCad PCB EDA Suite
DSNLEXER Class Reference

Implement a lexical analyzer for the SPECCTRA DSN file format. More...

#include <dsnlexer.h>

Public Member Functions

 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, FILE *aFile, const wxString &aFileName)
 Initialize a DSN lexer and prepares to read from aFile which is already open and has aFilename. More...
 
 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, const std::string &aSExpression, const wxString &aSource=wxEmptyString)
 Initialize a DSN lexer and prepares to read from aSExpression. More...
 
 DSNLEXER (const std::string &aSExpression, const wxString &aSource=wxEmptyString)
 Initialize a DSN lexer and prepares to read from aSExpression. More...
 
 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, LINE_READER *aLineReader=NULL)
 Initialize a DSN lexer and prepares to read from aLineReader which is already open, and may be in use by other DSNLEXERs also. More...
 
virtual ~DSNLEXER ()
 
bool SyncLineReaderWith (DSNLEXER &aLexer)
 Usable only for DSN lexers which share the same LINE_READER. More...
 
void SetSpecctraMode (bool aMode)
 Change the behavior of this lexer into or out of "specctra mode". More...
 
void PushReader (LINE_READER *aLineReader)
 Manage a stack of LINE_READERs in order to handle nested file inclusion. More...
 
LINE_READERPopReader ()
 Delete the top most LINE_READER from an internal stack of LINE_READERs and in the case of FILE_LINE_READER this means the associated FILE is closed. More...
 
int NextTok ()
 Return the next token found in the input file or DSN_EOF when reaching the end of file. More...
 
int NeedSYMBOL ()
 Call NextTok() and then verifies that the token read in satisfies IsSymbol(). More...
 
int NeedSYMBOLorNUMBER ()
 Call NextTok() and then verifies that the token read in satisfies bool IsSymbol() or the next token is DSN_NUMBER. More...
 
int NeedNUMBER (const char *aExpectation)
 Call NextTok() and then verifies that the token read is type DSN_NUMBER. More...
 
int CurTok () const
 Return whatever NextTok() returned the last time it was called. More...
 
int PrevTok () const
 Return whatever NextTok() returned the 2nd to last time it was called. More...
 
int GetCurStrAsToken () const
 Used to support "loose" matches (quoted tokens). More...
 
char SetStringDelimiter (char aStringDelimiter)
 Change the string delimiter from the default " to some other character and return the old value. More...
 
bool SetSpaceInQuotedTokens (bool val)
 Change the setting controlling whether a space in a quoted string isa terminator. More...
 
bool SetCommentsAreTokens (bool val)
 Change the handling of comments. More...
 
wxArrayString * ReadCommentLines ()
 Check the next sequence of tokens and reads them into a wxArrayString if they are comments. More...
 
void Expecting (int aTok) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Expecting (const char *aTokenList) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Unexpected (int aTok) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Unexpected (const char *aToken) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Duplicate (int aTok)
 Throw an IO_ERROR exception with a message saying specifically that aTok is a duplicate of one already seen in current context. More...
 
void NeedLEFT ()
 Call NextTok() and then verifies that the token read in is a DSN_LEFT. More...
 
void NeedRIGHT ()
 Call NextTok() and then verifies that the token read in is a DSN_RIGHT. More...
 
const char * GetTokenText (int aTok) const
 Return the C string representation of a DSN_T value. More...
 
wxString GetTokenString (int aTok) const
 Return a quote wrapped wxString representation of a token value. More...
 
const char * CurText () const
 Return a pointer to the current token's text. More...
 
const std::string & CurStr () const
 Return a reference to current token in std::string form. More...
 
wxString FromUTF8 () const
 Return the current token text as a wxString, assuming that the input byte stream is UTF8 encoded. More...
 
int CurLineNumber () const
 Return the current line number within my LINE_READER. More...
 
const char * CurLine () const
 Return the current line of text from which the CurText() would return its token. More...
 
const wxString & CurSource () const
 Return the current LINE_READER source. More...
 
int CurOffset () const
 Return the byte offset within the current line, using a 1 based index. More...
 

Static Public Member Functions

static bool IsSymbol (int aTok)
 Test a token to see if it is a symbol. More...
 
static const char * Syntax (int aTok)
 

Protected Types

typedef std::vector< LINE_READER * > READER_STACK
 

Protected Member Functions

void init ()
 
int readLine ()
 
int findToken (const std::string &aToken) const
 Take aToken string and looks up the string in the keywords table. More...
 
bool isStringTerminator (char cc) const
 

Protected Attributes

bool iOwnReaders
 on readerStack, should I delete them? More...
 
const char * start
 
const char * next
 
const char * limit
 
char dummy [1]
 when there is no reader. More...
 
READER_STACK readerStack
 all the LINE_READERs by pointer. More...
 
LINE_READERreader
 
bool specctraMode
 if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_in_quoted_tokens is functional else not. More...
 
char stringDelimiter
 
bool space_in_quoted_tokens
 blank spaces within quoted strings More...
 
bool commentsAreTokens
 true if should return comments as tokens More...
 
int prevTok
 curTok from previous NextTok() call. More...
 
int curOffset
 offset within current line of the current token More...
 
int curTok
 the current token obtained on last NextTok() More...
 
std::string curText
 the text of the current token More...
 
const KEYWORDkeywords
 table sorted by CMake for bsearch() More...
 
unsigned keywordCount
 count of keywords table More...
 
KEYWORD_MAP keyword_hash
 fast, specialized "C string" hashtable More...
 

Detailed Description

Implement a lexical analyzer for the SPECCTRA DSN file format.

It reads lexical tokens from the current LINE_READER through the NextTok() function.

Definition at line 78 of file dsnlexer.h.

Member Typedef Documentation

◆ READER_STACK

typedef std::vector<LINE_READER*> DSNLEXER::READER_STACK
protected

Definition at line 500 of file dsnlexer.h.

Constructor & Destructor Documentation

◆ DSNLEXER() [1/4]

DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
FILE *  aFile,
const wxString &  aFileName 
)

Initialize a DSN lexer and prepares to read from aFile which is already open and has aFilename.

Parameters
aKeywordTableis an array of KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aFileis an open file, which will be closed when this is destructed.
aFileNameis the name of the file

Definition at line 71 of file dsnlexer.cpp.

72  :
73  iOwnReaders( true ),
74  start( NULL ),
75  next( NULL ),
76  limit( NULL ),
77  reader( NULL ),
78  keywords( aKeywordTable ),
79  keywordCount( aKeywordCount )
80 {
81  FILE_LINE_READER* fileReader = new FILE_LINE_READER( aFile, aFilename );
82  PushReader( fileReader );
83  init();
84 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:187
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
A LINE_READER that reads from an open file.
Definition: richio.h:172
const char * limit
Definition: dsnlexer.h:497
#define NULL
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
void init()
Definition: dsnlexer.cpp:39

References init(), and PushReader().

◆ DSNLEXER() [2/4]

DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
const std::string &  aSExpression,
const wxString &  aSource = wxEmptyString 
)

Initialize a DSN lexer and prepares to read from aSExpression.

Parameters
aKeywordTableis an array of KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aSExpressionis text to feed through a STRING_LINE_READER
aSourceis a description of aSExpression, used for error reporting.

Definition at line 87 of file dsnlexer.cpp.

88  :
89  iOwnReaders( true ),
90  start( NULL ),
91  next( NULL ),
92  limit( NULL ),
93  reader( NULL ),
94  keywords( aKeywordTable ),
95  keywordCount( aKeywordCount )
96 {
97  STRING_LINE_READER* stringReader = new STRING_LINE_READER( aClipboardTxt, aSource.IsEmpty() ?
98  wxString( FMT_CLIPBOARD ) : aSource );
99  PushReader( stringReader );
100  init();
101 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:187
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
#define NULL
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
#define FMT_CLIPBOARD
Definition: dsnlexer.cpp:34
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
Is a LINE_READER that reads from a multiline 8 bit wide std::string.
Definition: richio.h:237
void init()
Definition: dsnlexer.cpp:39

References FMT_CLIPBOARD, init(), and PushReader().

◆ DSNLEXER() [3/4]

DSNLEXER::DSNLEXER ( const std::string &  aSExpression,
const wxString &  aSource = wxEmptyString 
)

Initialize a DSN lexer and prepares to read from aSExpression.

Use this one without a keyword table with the DOM parser in ptree.h.

Parameters
aSExpressionis text to feed through a STRING_LINE_READER
aSourceis a description of aSExpression, used for error reporting.

Definition at line 122 of file dsnlexer.cpp.

122  :
123  iOwnReaders( true ),
124  start( NULL ),
125  next( NULL ),
126  limit( NULL ),
127  reader( NULL ),
129  keywordCount( 0 )
130 {
131  STRING_LINE_READER* stringReader = new STRING_LINE_READER( aSExpression, aSource.IsEmpty() ?
132  wxString( FMT_CLIPBOARD ) : aSource );
133  PushReader( stringReader );
134  init();
135 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:187
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
#define NULL
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
static const KEYWORD empty_keywords[1]
Definition: dsnlexer.cpp:120
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
#define FMT_CLIPBOARD
Definition: dsnlexer.cpp:34
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
Is a LINE_READER that reads from a multiline 8 bit wide std::string.
Definition: richio.h:237
void init()
Definition: dsnlexer.cpp:39

References FMT_CLIPBOARD, init(), and PushReader().

◆ DSNLEXER() [4/4]

DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
LINE_READER aLineReader = NULL 
)

Initialize a DSN lexer and prepares to read from aLineReader which is already open, and may be in use by other DSNLEXERs also.

No ownership is taken of aLineReader. This enables it to be used by other DSNLEXERs.

Parameters
aKeywordTableis an array of #KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aLineReaderis any subclassed instance of LINE_READER, such as STRING_LINE_READER or FILE_LINE_READER. No ownership is taken.

Definition at line 104 of file dsnlexer.cpp.

105  :
106  iOwnReaders( false ),
107  start( NULL ),
108  next( NULL ),
109  limit( NULL ),
110  reader( NULL ),
111  keywords( aKeywordTable ),
112  keywordCount( aKeywordCount )
113 {
114  if( aLineReader )
115  PushReader( aLineReader );
116  init();
117 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:187
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
#define NULL
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
void init()
Definition: dsnlexer.cpp:39

References init(), and PushReader().

◆ ~DSNLEXER()

DSNLEXER::~DSNLEXER ( )
virtual

Definition at line 138 of file dsnlexer.cpp.

139 {
140  if( iOwnReaders )
141  {
142  // delete the LINE_READERs from the stack, since I own them.
143  for( READER_STACK::iterator it = readerStack.begin(); it!=readerStack.end(); ++it )
144  delete *it;
145  }
146 }
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:502

References iOwnReaders, and readerStack.

Member Function Documentation

◆ CurLine()

const char* DSNLEXER::CurLine ( ) const
inline

Return the current line of text from which the CurText() would return its token.

Definition at line 424 of file dsnlexer.h.

425  {
426  return (const char*)(*reader);
427  }

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurLineNumber()

int DSNLEXER::CurLineNumber ( ) const
inline

Return the current line number within my LINE_READER.

Definition at line 416 of file dsnlexer.h.

417  {
418  return reader->LineNumber();
419  }
virtual unsigned LineNumber() const
Return the line number of the last line read from this LINE_READER.
Definition: richio.h:135
LINE_READER * reader
Definition: dsnlexer.h:505

References LINE_READER::LineNumber(), and reader.

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurOffset()

int DSNLEXER::CurOffset ( ) const
inline

Return the byte offset within the current line, using a 1 based index.

Returns
a one based index into the current line.

Definition at line 444 of file dsnlexer.h.

445  {
446  return curOffset + 1;
447  }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519

References curOffset.

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurSource()

const wxString& DSNLEXER::CurSource ( ) const
inline

Return the current LINE_READER source.

Returns
source of the lines of text, e.g. a filename or "clipboard".

Definition at line 434 of file dsnlexer.h.

435  {
436  return reader->GetSource();
437  }
virtual const wxString & GetSource() const
Returns the name of the source of the lines in an abstract sense.
Definition: richio.h:109
LINE_READER * reader
Definition: dsnlexer.h:505

References LINE_READER::GetSource(), and reader.

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurStr()

const std::string& DSNLEXER::CurStr ( ) const
inline

Return a reference to current token in std::string form.

Definition at line 399 of file dsnlexer.h.

400  {
401  return curText;
402  }
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText.

◆ CurText()

const char* DSNLEXER::CurText ( ) const
inline

Return a pointer to the current token's text.

Definition at line 391 of file dsnlexer.h.

392  {
393  return curText.c_str();
394  }
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText.

Referenced by CADSTAR_ARCHIVE_PARSER::LoadArchiveFile(), PCAD2KICAD::LoadInputFile(), scanAtom(), and scanList().

◆ CurTok()

int DSNLEXER::CurTok ( ) const
inline

Return whatever NextTok() returned the last time it was called.

Definition at line 230 of file dsnlexer.h.

231  {
232  return curTok;
233  }
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:521

References curTok.

Referenced by Scan(), and scanList().

◆ Duplicate()

void DSNLEXER::Duplicate ( int  aTok)

Throw an IO_ERROR exception with a message saying specifically that aTok is a duplicate of one already seen in current context.

Parameters
aTokis the token/keyword type which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 345 of file dsnlexer.cpp.

346 {
347  wxString errText = wxString::Format(
348  _("%s is a duplicate"), GetTokenString( aTok ).GetData() );
350 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
wxString GetTokenString(int aTok) const
Return a quote wrapped wxString representation of a token value.
Definition: dsnlexer.cpp:302
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
#define _(s)
Definition: 3d_actions.cpp:33
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), GetTokenString(), and THROW_PARSE_ERROR.

◆ Expecting() [1/2]

void DSNLEXER::Expecting ( int  aTok) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokis the token/keyword type which was expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 321 of file dsnlexer.cpp.

322 {
323  wxString errText = wxString::Format(
324  _( "Expecting %s" ), GetTokenString( aTok ) );
326 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
wxString GetTokenString(int aTok) const
Return a quote wrapped wxString representation of a token value.
Definition: dsnlexer.cpp:302
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
#define _(s)
Definition: 3d_actions.cpp:33
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), GetTokenString(), and THROW_PARSE_ERROR.

Referenced by NeedLEFT(), NeedRIGHT(), NeedSYMBOL(), and NeedSYMBOLorNUMBER().

◆ Expecting() [2/2]

void DSNLEXER::Expecting ( const char *  aTokenList) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokenListis the token/keyword type which was expected at the current input location, e.g. "pin|graphic|property".
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 329 of file dsnlexer.cpp.

330 {
331  wxString errText = wxString::Format(
332  _( "Expecting '%s'" ), wxString::FromUTF8( text ) );
334 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
#define _(s)
Definition: 3d_actions.cpp:33
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), and THROW_PARSE_ERROR.

◆ findToken()

int DSNLEXER::findToken ( const std::string &  aToken) const
protected

Take aToken string and looks up the string in the keywords table.

Parameters
aTokenis a string to lookup in the keywords table.
Returns
with a value from the enum DSN_T matching the keyword text, or DSN_SYMBOL if aToken is not in the keywords table.

Definition at line 228 of file dsnlexer.cpp.

229 {
230  KEYWORD_MAP::const_iterator it = keyword_hash.find( tok.c_str() );
231 
232  if( it != keyword_hash.end() )
233  return it->second;
234 
235  return DSN_SYMBOL; // not a keyword, some arbitrary symbol.
236 }
KEYWORD_MAP keyword_hash
fast, specialized "C string" hashtable
Definition: dsnlexer.h:526

References DSN_SYMBOL, and keyword_hash.

Referenced by GetCurStrAsToken(), and NextTok().

◆ FromUTF8()

wxString DSNLEXER::FromUTF8 ( ) const
inline

Return the current token text as a wxString, assuming that the input byte stream is UTF8 encoded.

Definition at line 408 of file dsnlexer.h.

409  {
410  return wxString::FromUTF8( curText.c_str() );
411  }
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText.

Referenced by ReadCommentLines().

◆ GetCurStrAsToken()

int DSNLEXER::GetCurStrAsToken ( ) const
inline

Used to support "loose" matches (quoted tokens).

Definition at line 246 of file dsnlexer.h.

247  {
248  return findToken( curText );
249  }
int findToken(const std::string &aToken) const
Take aToken string and looks up the string in the keywords table.
Definition: dsnlexer.cpp:228
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText, and findToken().

◆ GetTokenString()

wxString DSNLEXER::GetTokenString ( int  aTok) const

Return a quote wrapped wxString representation of a token value.

Definition at line 302 of file dsnlexer.cpp.

303 {
304  wxString ret;
305 
306  ret << wxT("'") << wxString::FromUTF8( GetTokenText(aTok) ) << wxT("'");
307 
308  return ret;
309 }
const char * GetTokenText(int aTok) const
Return the C string representation of a DSN_T value.
Definition: dsnlexer.cpp:283

References GetTokenText().

Referenced by Duplicate(), Expecting(), and Unexpected().

◆ GetTokenText()

const char * DSNLEXER::GetTokenText ( int  aTok) const

Return the C string representation of a DSN_T value.

Definition at line 283 of file dsnlexer.cpp.

284 {
285  const char* ret;
286 
287  if( aTok < 0 )
288  {
289  return Syntax( aTok );
290  }
291  else if( (unsigned) aTok < keywordCount )
292  {
293  ret = keywords[aTok].name;
294  }
295  else
296  ret = "token too big";
297 
298  return ret;
299 }
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * name
unique keyword.
Definition: dsnlexer.h:41
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
static const char * Syntax(int aTok)
Definition: dsnlexer.cpp:239

References keywordCount, keywords, KEYWORD::name, and Syntax().

Referenced by GetTokenString().

◆ init()

void DSNLEXER::init ( )
protected

Definition at line 39 of file dsnlexer.cpp.

40 {
41  curTok = DSN_NONE;
42  prevTok = DSN_NONE;
43 
44  stringDelimiter = '"';
45 
46  specctraMode = false;
47  space_in_quoted_tokens = false;
48  commentsAreTokens = false;
49 
50  curOffset = 0;
51 
52 #if 1
53  if( keywordCount > 11 )
54  {
55  // resize the hashtable bucket count
56  keyword_hash.reserve( keywordCount );
57  }
58 
59  // fill the specialized "C string" hashtable from keywords[]
60  const KEYWORD* it = keywords;
61  const KEYWORD* end = it + keywordCount;
62 
63  for( ; it < end; ++it )
64  {
65  keyword_hash[it->name] = it->token;
66  }
67 #endif
68 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:516
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * name
unique keyword.
Definition: dsnlexer.h:41
KEYWORD_MAP keyword_hash
fast, specialized "C string" hashtable
Definition: dsnlexer.h:526
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:518
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:521
Hold a keyword string and its unique integer token.
Definition: dsnlexer.h:39
char stringDelimiter
Definition: dsnlexer.h:513
int token
a zero based index into an array of KEYWORDs
Definition: dsnlexer.h:42

References commentsAreTokens, curOffset, curTok, DSN_NONE, keyword_hash, keywordCount, keywords, KEYWORD::name, prevTok, space_in_quoted_tokens, specctraMode, stringDelimiter, and KEYWORD::token.

Referenced by DSNLEXER().

◆ isStringTerminator()

bool DSNLEXER::isStringTerminator ( char  cc) const
inlineprotected

Definition at line 483 of file dsnlexer.h.

484  {
485  if( !space_in_quoted_tokens && cc == ' ' )
486  return true;
487 
488  if( cc == stringDelimiter )
489  return true;
490 
491  return false;
492  }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
char stringDelimiter
Definition: dsnlexer.h:513

References space_in_quoted_tokens, and stringDelimiter.

Referenced by NextTok().

◆ IsSymbol()

bool DSNLEXER::IsSymbol ( int  aTok)
static

Test a token to see if it is a symbol.

This means it cannot be a special delimiter character such as DSN_LEFT, DSN_RIGHT, #DSN_QUOTE, etc. It may however, coincidentally match a keyword and still be a symbol.

Definition at line 312 of file dsnlexer.cpp.

313 {
314  // This is static and not inline to reduce code space.
315 
316  // if aTok is >= 0, then it is a coincidental match to a keyword.
317  return aTok == DSN_SYMBOL || aTok == DSN_STRING || aTok >= 0;
318 }

References DSN_STRING, and DSN_SYMBOL.

Referenced by NeedSYMBOL(), and NeedSYMBOLorNUMBER().

◆ NeedLEFT()

void DSNLEXER::NeedLEFT ( )

Call NextTok() and then verifies that the token read in is a DSN_LEFT.

Exceptions
IO_ERRORif the next token is not a DSN_LEFT

Definition at line 361 of file dsnlexer.cpp.

362 {
363  int tok = NextTok();
364  if( tok != DSN_LEFT )
365  Expecting( DSN_LEFT );
366 }
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:508
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:321

References DSN_LEFT, Expecting(), and NextTok().

◆ NeedNUMBER()

int DSNLEXER::NeedNUMBER ( const char *  aExpectation)

Call NextTok() and then verifies that the token read is type DSN_NUMBER.

Returns
the actual token read in.
Exceptions
IO_ERRORif the next token does not satisfy the above test.

Definition at line 395 of file dsnlexer.cpp.

396 {
397  int tok = NextTok();
398  if( tok != DSN_NUMBER )
399  {
400  wxString errText = wxString::Format(
401  _( "need a number for '%s'" ), wxString::FromUTF8( aExpectation ).GetData() );
403  }
404  return tok;
405 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
#define _(s)
Definition: 3d_actions.cpp:33
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:508
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), DSN_NUMBER, Format(), NextTok(), and THROW_PARSE_ERROR.

◆ NeedRIGHT()

void DSNLEXER::NeedRIGHT ( )

Call NextTok() and then verifies that the token read in is a DSN_RIGHT.

Exceptions
IO_ERRORif the next token is not a DSN_RIGHT

Definition at line 369 of file dsnlexer.cpp.

370 {
371  int tok = NextTok();
372  if( tok != DSN_RIGHT )
373  Expecting( DSN_RIGHT );
374 }
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:508
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:321

References DSN_RIGHT, Expecting(), and NextTok().

◆ NeedSYMBOL()

int DSNLEXER::NeedSYMBOL ( )

Call NextTok() and then verifies that the token read in satisfies IsSymbol().

Returns
the actual token read in.
Exceptions
IO_ERRORif the next token does not satisfy IsSymbol().

Definition at line 377 of file dsnlexer.cpp.

378 {
379  int tok = NextTok();
380  if( !IsSymbol( tok ) )
382  return tok;
383 }
static bool IsSymbol(int aTok)
Test a token to see if it is a symbol.
Definition: dsnlexer.cpp:312
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:508
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:321

References DSN_SYMBOL, Expecting(), IsSymbol(), and NextTok().

◆ NeedSYMBOLorNUMBER()

int DSNLEXER::NeedSYMBOLorNUMBER ( )

Call NextTok() and then verifies that the token read in satisfies bool IsSymbol() or the next token is DSN_NUMBER.

Returns
the actual token read in.
Exceptions
IO_ERRORif the next token does not satisfy the above test.

Definition at line 386 of file dsnlexer.cpp.

387 {
388  int tok = NextTok();
389  if( !IsSymbol( tok ) && tok!=DSN_NUMBER )
390  Expecting( "a symbol or number" );
391  return tok;
392 }
static bool IsSymbol(int aTok)
Test a token to see if it is a symbol.
Definition: dsnlexer.cpp:312
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:508
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:321

References DSN_NUMBER, Expecting(), IsSymbol(), and NextTok().

◆ NextTok()

int DSNLEXER::NextTok ( )

Return the next token found in the input file or DSN_EOF when reaching the end of file.

Users should wrap this function to return an enum to aid in grammar debugging while running under a debugger, but leave this lower level function returning an int (so the enum does not collide with another usage).

Returns
the type of token found next.
Exceptions
IO_ERRORonly if the LINE_READER throws it.

Definition at line 508 of file dsnlexer.cpp.

509 {
510  const char* cur = next;
511  const char* head = cur;
512 
513  prevTok = curTok;
514 
515  if( curTok == DSN_EOF )
516  goto exit;
517 
518  if( cur >= limit )
519  {
520 L_read:
521  // blank lines are returned as "\n" and will have a len of 1.
522  // EOF will have a len of 0 and so is detectable.
523  int len = readLine();
524  if( len == 0 )
525  {
526  cur = start; // after readLine(), since start can change, set cur offset to start
527  curTok = DSN_EOF;
528  goto exit;
529  }
530 
531  cur = start; // after readLine() since start can change.
532 
533  // skip leading whitespace
534  while( cur<limit && isSpace( *cur ) )
535  ++cur;
536 
537  // If the first non-blank character is #, this line is a comment.
538  // Comments cannot follow any other token on the same line.
539  if( cur<limit && *cur=='#' )
540  {
541  if( commentsAreTokens )
542  {
543  // Grab the entire current line [excluding end of line char(s)] as the
544  // current token. The '#' character may not be at offset zero.
545 
546  while( limit[-1] == '\n' || limit[-1] == '\r' )
547  --limit;
548 
549  curText.clear();
550  curText.append( start, limit );
551 
552  cur = start; // ensure a good curOffset below
554  head = limit; // do a readLine() on next call in here.
555  goto exit;
556  }
557  else
558  goto L_read;
559  }
560  }
561  else
562  {
563  // skip leading whitespace
564  while( cur<limit && isSpace( *cur ) )
565  ++cur;
566  }
567 
568  if( cur >= limit )
569  goto L_read;
570 
571  if( *cur == '(' )
572  {
573  curText = *cur;
574  curTok = DSN_LEFT;
575  head = cur+1;
576  goto exit;
577  }
578 
579  if( *cur == ')' )
580  {
581  curText = *cur;
582  curTok = DSN_RIGHT;
583  head = cur+1;
584  goto exit;
585  }
586 
587  // Non-specctraMode, understands and deciphers escaped \, \r, \n, and \".
588  // Strips off leading and trailing double quotes
589  if( !specctraMode )
590  {
591  // a quoted string, will return DSN_STRING
592  if( *cur == stringDelimiter )
593  {
594  // copy the token, character by character so we can remove doubled up quotes.
595  curText.clear();
596 
597  ++cur; // skip over the leading delimiter, which is always " in non-specctraMode
598 
599  head = cur;
600 
601  while( head<limit )
602  {
603  // ESCAPE SEQUENCES:
604  if( *head =='\\' )
605  {
606  char tbuf[8];
607  char c;
608  int i;
609 
610  if( ++head >= limit )
611  break; // throw exception at L_unterminated
612 
613  switch( *head++ )
614  {
615  case '"':
616  case '\\': c = head[-1]; break;
617  case 'a': c = '\x07'; break;
618  case 'b': c = '\x08'; break;
619  case 'f': c = '\x0c'; break;
620  case 'n': c = '\n'; break;
621  case 'r': c = '\r'; break;
622  case 't': c = '\x09'; break;
623  case 'v': c = '\x0b'; break;
624 
625  case 'x': // 1 or 2 byte hex escape sequence
626  for( i=0; i<2; ++i )
627  {
628  if( !isxdigit( head[i] ) )
629  break;
630  tbuf[i] = head[i];
631  }
632  tbuf[i] = '\0';
633  if( i > 0 )
634  c = (char) strtoul( tbuf, NULL, 16 );
635  else
636  c = 'x'; // a goofed hex escape sequence, interpret as 'x'
637  head += i;
638  break;
639 
640  default: // 1-3 byte octal escape sequence
641  --head;
642  for( i=0; i<3; ++i )
643  {
644  if( head[i] < '0' || head[i] > '7' )
645  break;
646  tbuf[i] = head[i];
647  }
648  tbuf[i] = '\0';
649  if( i > 0 )
650  c = (char) strtoul( tbuf, NULL, 8 );
651  else
652  c = '\\'; // a goofed octal escape sequence, interpret as '\'
653  head += i;
654  break;
655  }
656 
657  curText += c;
658  }
659 
660  else if( *head == '"' ) // end of the non-specctraMode DSN_STRING
661  {
662  curTok = DSN_STRING;
663  ++head; // omit this trailing double quote
664  goto exit;
665  }
666 
667  else
668  curText += *head++;
669 
670  } // while
671 
672  // L_unterminated:
673  wxString errtxt( _( "Un-terminated delimited string" ) );
675  cur - start + curText.length() );
676  }
677  }
678 
679  else // is specctraMode, tests in this block should not occur in KiCad mode.
680  {
681  /* get the dash out of a <pin_reference> which is embedded for example
682  like: U2-14 or "U2"-"14"
683  This is detectable by a non-space immediately preceeding the dash.
684  */
685  if( *cur == '-' && cur>start && !isSpace( cur[-1] ) )
686  {
687  curText = '-';
688  curTok = DSN_DASH;
689  head = cur+1;
690  goto exit;
691  }
692 
693  // switching the string_quote character
694  if( prevTok == DSN_STRING_QUOTE )
695  {
696  static const wxString errtxt( _("String delimiter must be a single character of ', \", or $"));
697 
698  char cc = *cur;
699  switch( cc )
700  {
701  case '\'':
702  case '$':
703  case '"':
704  break;
705  default:
707  }
708 
709  curText = cc;
710 
711  head = cur+1;
712 
713  if( head<limit && !isSep( *head ) )
714  {
716  }
717 
719  goto exit;
720  }
721 
722  // specctraMode DSN_STRING
723  if( *cur == stringDelimiter )
724  {
725  ++cur; // skip over the leading delimiter: ",', or $
726 
727  head = cur;
728 
729  while( head<limit && !isStringTerminator( *head ) )
730  ++head;
731 
732  if( head >= limit )
733  {
734  wxString errtxt( _( "Un-terminated delimited string" ) );
736  }
737 
738  curText.clear();
739  curText.append( cur, head );
740 
741  ++head; // skip over the trailing delimiter
742 
743  curTok = DSN_STRING;
744  goto exit;
745  }
746  } // specctraMode
747 
748  // non-quoted token, read it into curText.
749  curText.clear();
750 
751  head = cur;
752  while( head<limit && !isSep( *head ) )
753  curText += *head++;
754 
755  if( isNumber( curText.c_str(), curText.c_str() + curText.size() ) )
756  {
757  curTok = DSN_NUMBER;
758  goto exit;
759  }
760 
761  if( specctraMode && curText == "string_quote" )
762  {
764  goto exit;
765  }
766 
767  curTok = findToken( curText );
768 
769 exit: // single point of exit, no returns elsewhere please.
770 
771  curOffset = cur - start;
772 
773  next = head;
774 
775  return curTok;
776 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:516
bool isSep(char cc)
return true if cc is an s-expression separator character
Definition: dsnlexer.cpp:441
bool isStringTerminator(char cc) const
Definition: dsnlexer.h:483
static bool isNumber(const char *cp, const char *limit)
Function isNumber returns true if the next sequence of text is a number: either an integer,...
Definition: dsnlexer.cpp:458
int findToken(const std::string &aToken) const
Take aToken string and looks up the string in the keywords table.
Definition: dsnlexer.cpp:228
const char * next
Definition: dsnlexer.h:496
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
const char * limit
Definition: dsnlexer.h:497
#define NULL
static bool isSpace(char cc)
Function isSpace tests for whitespace.
Definition: dsnlexer.cpp:414
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:518
int readLine()
Definition: dsnlexer.h:454
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
const char * start
Definition: dsnlexer.h:495
#define _(s)
Definition: 3d_actions.cpp:33
std::string curText
the text of the current token
Definition: dsnlexer.h:522
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:521
char stringDelimiter
Definition: dsnlexer.h:513

References _, commentsAreTokens, CurLine(), CurLineNumber(), CurOffset(), curOffset, CurSource(), curText, curTok, DSN_COMMENT, DSN_DASH, DSN_EOF, DSN_LEFT, DSN_NUMBER, DSN_QUOTE_DEF, DSN_RIGHT, DSN_STRING, DSN_STRING_QUOTE, findToken(), isNumber(), isSep(), isSpace(), isStringTerminator(), limit, next, NULL, prevTok, readLine(), specctraMode, start, stringDelimiter, and THROW_PARSE_ERROR.

Referenced by CADSTAR_ARCHIVE_PARSER::LoadArchiveFile(), PCAD2KICAD::LoadInputFile(), NeedLEFT(), NeedNUMBER(), NeedRIGHT(), NeedSYMBOL(), NeedSYMBOLorNUMBER(), ReadCommentLines(), Scan(), and scanList().

◆ PopReader()

LINE_READER * DSNLEXER::PopReader ( )

Delete the top most LINE_READER from an internal stack of LINE_READERs and in the case of FILE_LINE_READER this means the associated FILE is closed.

The most recently used former LINE_READER on the stack becomes the current LINE_READER and its previous position in its input stream and the its latest line number should pertain. PopReader always starts reading from a new line upon returning to the previous LINE_READER. A pop is only possible if there are at least 2 #LINE_READERs on the stack, since popping the last one is not supported.

Returns
the LINE_READER that was in use before the pop, or NULL if there was not at least two readers on the stack and therefore the pop failed.

Definition at line 199 of file dsnlexer.cpp.

200 {
201  LINE_READER* ret = 0;
202 
203  if( readerStack.size() )
204  {
205  ret = reader;
206  readerStack.pop_back();
207 
208  if( readerStack.size() )
209  {
210  reader = readerStack.back();
211  start = reader->Line();
212 
213  // force a new readLine() as first thing.
214  limit = start;
215  next = start;
216  }
217  else
218  {
219  reader = 0;
220  start = dummy;
221  limit = dummy;
222  }
223  }
224  return ret;
225 }
An abstract class from which implementation specific LINE_READERs may be derived to read single lines...
Definition: richio.h:80
char * Line() const
Return a pointer to the last line that was read in.
Definition: richio.h:117
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:502
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
char dummy[1]
when there is no reader.
Definition: dsnlexer.h:498

References dummy, limit, LINE_READER::Line(), next, reader, readerStack, and start.

◆ PrevTok()

int DSNLEXER::PrevTok ( ) const
inline

Return whatever NextTok() returned the 2nd to last time it was called.

Definition at line 238 of file dsnlexer.h.

239  {
240  return prevTok;
241  }
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:518

References prevTok.

◆ PushReader()

void DSNLEXER::PushReader ( LINE_READER aLineReader)

Manage a stack of LINE_READERs in order to handle nested file inclusion.

This function pushes aLineReader onto the top of a stack of LINE_READERs and makes it the current LINE_READER with its own #GetSource(), line number and line text. A grammar must be designed such that the "include" token (whatever its various names), and any of its parameters are not followed by anything on that same line, because PopReader always starts reading from a new line upon returning to the original LINE_READER.

Definition at line 187 of file dsnlexer.cpp.

188 {
189  readerStack.push_back( aLineReader );
190  reader = aLineReader;
191  start = (const char*) (*reader);
192 
193  // force a new readLine() as first thing.
194  limit = start;
195  next = start;
196 }
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:502
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495

References limit, next, reader, readerStack, and start.

Referenced by DSNLEXER().

◆ ReadCommentLines()

wxArrayString * DSNLEXER::ReadCommentLines ( )

Check the next sequence of tokens and reads them into a wxArrayString if they are comments.

Reading continues until a non-comment token is encountered, and such last read token remains as CurTok() and as CurText(). No push back or "un get" mechanism is used for this support. Upon return you simply avoid calling NextTok() for the next token, but rather CurTok().

Returns
Heap allocated block of comments or NULL if none. The caller owns the allocation and must delete if not NULL.

Definition at line 779 of file dsnlexer.cpp.

780 {
781  wxArrayString* ret = 0;
782  bool cmt_setting = SetCommentsAreTokens( true );
783  int tok = NextTok();
784 
785  if( tok == DSN_COMMENT )
786  {
787  ret = new wxArrayString();
788 
789  do
790  {
791  ret->Add( FromUTF8() );
792  }
793  while( ( tok = NextTok() ) == DSN_COMMENT );
794  }
795 
796  SetCommentsAreTokens( cmt_setting );
797 
798  return ret;
799 }
bool SetCommentsAreTokens(bool val)
Change the handling of comments.
Definition: dsnlexer.h:289
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:508
wxString FromUTF8() const
Return the current token text as a wxString, assuming that the input byte stream is UTF8 encoded.
Definition: dsnlexer.h:408

References DSN_COMMENT, FromUTF8(), NextTok(), and SetCommentsAreTokens().

◆ readLine()

int DSNLEXER::readLine ( )
inlineprotected

Definition at line 454 of file dsnlexer.h.

455  {
456  if( reader )
457  {
458  reader->ReadLine();
459 
460  unsigned len = reader->Length();
461 
462  // start may have changed in ReadLine(), which can resize and
463  // relocate reader's line buffer.
464  start = reader->Line();
465 
466  next = start;
467  limit = next + len;
468 
469  return len;
470  }
471  return 0;
472  }
char * Line() const
Return a pointer to the last line that was read in.
Definition: richio.h:117
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
LINE_READER * reader
Definition: dsnlexer.h:505
unsigned Length() const
Return the number of bytes in the last line read from this LINE_READER.
Definition: richio.h:143
const char * start
Definition: dsnlexer.h:495
virtual char * ReadLine()=0
Read a line of text into the buffer and increments the line number counter.

References LINE_READER::Length(), limit, LINE_READER::Line(), next, reader, LINE_READER::ReadLine(), and start.

Referenced by NextTok().

◆ SetCommentsAreTokens()

bool DSNLEXER::SetCommentsAreTokens ( bool  val)
inline

Change the handling of comments.

If set true, comments are returned as single line strings with a terminating newline. Otherwise they are consumed by the lexer and not returned.

Definition at line 289 of file dsnlexer.h.

290  {
291  bool old = commentsAreTokens;
292  commentsAreTokens = val;
293  return old;
294  }
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:516

References commentsAreTokens.

Referenced by ReadCommentLines().

◆ SetSpaceInQuotedTokens()

bool DSNLEXER::SetSpaceInQuotedTokens ( bool  val)
inline

Change the setting controlling whether a space in a quoted string isa terminator.

Parameters
valIf true, means

Definition at line 273 of file dsnlexer.h.

274  {
275  bool old = space_in_quoted_tokens;
276 
277  if( specctraMode )
279 
280  return old;
281  }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507

References space_in_quoted_tokens, and specctraMode.

◆ SetSpecctraMode()

void DSNLEXER::SetSpecctraMode ( bool  aMode)

Change the behavior of this lexer into or out of "specctra mode".

If specctra mode, then: -#) stringDelimiter can be changed. -#) KiCad quoting protocol is not in effect. -#) space_in_quoted_tokens is functional else none of the above are true.

The default mode is non-specctra mode, meaning: -#) stringDelimiter cannot be changed. -#) KiCad quoting protocol is in effect. -#) space_in_quoted_tokens is not functional.

Definition at line 148 of file dsnlexer.cpp.

149 {
150  specctraMode = aMode;
151  if( aMode )
152  {
153  // specctra mode defaults, some of which can still be changed in this mode.
154  space_in_quoted_tokens = true;
155  }
156  else
157  {
158  space_in_quoted_tokens = false;
159  stringDelimiter = '"';
160  }
161 }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
char stringDelimiter
Definition: dsnlexer.h:513

References space_in_quoted_tokens, specctraMode, and stringDelimiter.

◆ SetStringDelimiter()

char DSNLEXER::SetStringDelimiter ( char  aStringDelimiter)
inline

Change the string delimiter from the default " to some other character and return the old value.

Parameters
aStringDelimiterThe character in lowest 8 bits.
Returns
The old delimiter in the lowest 8 bits.

Definition at line 258 of file dsnlexer.h.

259  {
260  int old = stringDelimiter;
261 
262  if( specctraMode )
263  stringDelimiter = aStringDelimiter;
264 
265  return old;
266  }
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
char stringDelimiter
Definition: dsnlexer.h:513

References specctraMode, and stringDelimiter.

◆ SyncLineReaderWith()

bool DSNLEXER::SyncLineReaderWith ( DSNLEXER aLexer)

Usable only for DSN lexers which share the same LINE_READER.

Synchronizes the pointers handling the data read by the LINE_READER. Allows 2 #DNSLEXER objects to share the same current line, when switching from a #DNSLEXER to another #DNSLEXER

Parameters
aLexerthe model.
Returns
true if the sync can be made ( at least the same line reader ).

Definition at line 164 of file dsnlexer.cpp.

165 {
166  // Synchronize the pointers handling the data read by the LINE_READER
167  // only if aLexer shares the same LINE_READER, because only in this case
168  // the char buffer is be common
169 
170  if( reader != aLexer.reader )
171  return false;
172 
173  // We can synchronize the pointers which handle the data currently read
174  start = aLexer.start;
175  next = aLexer.next;
176  limit = aLexer.limit;
177 
178  // Sync these parameters is not mandatory, but could help
179  // for instance in debug
180  curText = aLexer.curText;
181  curOffset = aLexer.curOffset;
182 
183  return true;
184 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curOffset, curText, limit, next, reader, and start.

◆ Syntax()

const char * DSNLEXER::Syntax ( int  aTok)
static

Definition at line 239 of file dsnlexer.cpp.

240 {
241  const char* ret;
242 
243  switch( aTok )
244  {
245  case DSN_NONE:
246  ret = "NONE";
247  break;
248  case DSN_STRING_QUOTE:
249  ret = "string_quote"; // a special DSN syntax token, see specctra spec.
250  break;
251  case DSN_QUOTE_DEF:
252  ret = "quoted text delimiter";
253  break;
254  case DSN_DASH:
255  ret = "-";
256  break;
257  case DSN_SYMBOL:
258  ret = "symbol";
259  break;
260  case DSN_NUMBER:
261  ret = "number";
262  break;
263  case DSN_RIGHT:
264  ret = ")";
265  break;
266  case DSN_LEFT:
267  ret = "(";
268  break;
269  case DSN_STRING:
270  ret = "quoted string";
271  break;
272  case DSN_EOF:
273  ret = "end of input";
274  break;
275  default:
276  ret = "???";
277  }
278 
279  return ret;
280 }

References DSN_DASH, DSN_EOF, DSN_LEFT, DSN_NONE, DSN_NUMBER, DSN_QUOTE_DEF, DSN_RIGHT, DSN_STRING, DSN_STRING_QUOTE, and DSN_SYMBOL.

Referenced by GetTokenText().

◆ Unexpected() [1/2]

void DSNLEXER::Unexpected ( int  aTok) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokis the token/keyword type which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 337 of file dsnlexer.cpp.

338 {
339  wxString errText = wxString::Format(
340  _( "Unexpected %s" ), GetTokenString( aTok ) );
342 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
wxString GetTokenString(int aTok) const
Return a quote wrapped wxString representation of a token value.
Definition: dsnlexer.cpp:302
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
#define _(s)
Definition: 3d_actions.cpp:33
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), GetTokenString(), and THROW_PARSE_ERROR.

Referenced by Scan(), and scanList().

◆ Unexpected() [2/2]

void DSNLEXER::Unexpected ( const char *  aToken) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokenis the token which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 353 of file dsnlexer.cpp.

354 {
355  wxString errText = wxString::Format(
356  _( "Unexpected '%s'" ), wxString::FromUTF8( text ) );
358 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
#define _(s)
Definition: 3d_actions.cpp:33
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), and THROW_PARSE_ERROR.

Member Data Documentation

◆ commentsAreTokens

bool DSNLEXER::commentsAreTokens
protected

true if should return comments as tokens

Definition at line 516 of file dsnlexer.h.

Referenced by init(), NextTok(), and SetCommentsAreTokens().

◆ curOffset

int DSNLEXER::curOffset
protected

offset within current line of the current token

Definition at line 519 of file dsnlexer.h.

Referenced by CurOffset(), init(), NextTok(), and SyncLineReaderWith().

◆ curText

std::string DSNLEXER::curText
protected

the text of the current token

Definition at line 522 of file dsnlexer.h.

Referenced by CurStr(), CurText(), FromUTF8(), GetCurStrAsToken(), NextTok(), and SyncLineReaderWith().

◆ curTok

int DSNLEXER::curTok
protected

the current token obtained on last NextTok()

Definition at line 521 of file dsnlexer.h.

Referenced by CurTok(), init(), and NextTok().

◆ dummy

char DSNLEXER::dummy[1]
protected

when there is no reader.

Definition at line 498 of file dsnlexer.h.

Referenced by PopReader().

◆ iOwnReaders

bool DSNLEXER::iOwnReaders
protected

on readerStack, should I delete them?

Definition at line 494 of file dsnlexer.h.

Referenced by ~DSNLEXER().

◆ keyword_hash

KEYWORD_MAP DSNLEXER::keyword_hash
protected

fast, specialized "C string" hashtable

Definition at line 526 of file dsnlexer.h.

Referenced by findToken(), and init().

◆ keywordCount

unsigned DSNLEXER::keywordCount
protected

count of keywords table

Definition at line 525 of file dsnlexer.h.

Referenced by GetTokenText(), and init().

◆ keywords

const KEYWORD* DSNLEXER::keywords
protected

table sorted by CMake for bsearch()

Definition at line 524 of file dsnlexer.h.

Referenced by GetTokenText(), and init().

◆ limit

const char* DSNLEXER::limit
protected

Definition at line 497 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), readLine(), and SyncLineReaderWith().

◆ next

const char* DSNLEXER::next
protected

Definition at line 496 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), readLine(), and SyncLineReaderWith().

◆ prevTok

int DSNLEXER::prevTok
protected

curTok from previous NextTok() call.

Definition at line 518 of file dsnlexer.h.

Referenced by init(), NextTok(), and PrevTok().

◆ reader

LINE_READER* DSNLEXER::reader
protected

◆ readerStack

READER_STACK DSNLEXER::readerStack
protected

all the LINE_READERs by pointer.

no ownership. ownership is via readerStack, maybe, if iOwnReaders

Definition at line 502 of file dsnlexer.h.

Referenced by PopReader(), PushReader(), and ~DSNLEXER().

◆ space_in_quoted_tokens

bool DSNLEXER::space_in_quoted_tokens
protected

blank spaces within quoted strings

Definition at line 514 of file dsnlexer.h.

Referenced by init(), isStringTerminator(), SetSpaceInQuotedTokens(), and SetSpecctraMode().

◆ specctraMode

bool DSNLEXER::specctraMode
protected

if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_in_quoted_tokens is functional else not.

Definition at line 507 of file dsnlexer.h.

Referenced by init(), NextTok(), SetSpaceInQuotedTokens(), SetSpecctraMode(), and SetStringDelimiter().

◆ start

const char* DSNLEXER::start
protected

Definition at line 495 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), readLine(), and SyncLineReaderWith().

◆ stringDelimiter

char DSNLEXER::stringDelimiter
protected

Definition at line 513 of file dsnlexer.h.

Referenced by init(), isStringTerminator(), NextTok(), SetSpecctraMode(), and SetStringDelimiter().


The documentation for this class was generated from the following files: