KiCad PCB EDA Suite
DSNLEXER Class Reference

Implement a lexical analyzer for the SPECCTRA DSN file format. More...

#include <dsnlexer.h>

Public Member Functions

 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, FILE *aFile, const wxString &aFileName)
 Initialize a DSN lexer and prepares to read from aFile which is already open and has aFilename. More...
 
 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, const std::string &aSExpression, const wxString &aSource=wxEmptyString)
 Initialize a DSN lexer and prepares to read from aSExpression. More...
 
 DSNLEXER (const std::string &aSExpression, const wxString &aSource=wxEmptyString)
 Initialize a DSN lexer and prepares to read from aSExpression. More...
 
 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, LINE_READER *aLineReader=NULL)
 Initialize a DSN lexer and prepares to read from aLineReader which is already open, and may be in use by other DSNLEXERs also. More...
 
virtual ~DSNLEXER ()
 
bool SyncLineReaderWith (DSNLEXER &aLexer)
 Usable only for DSN lexers which share the same LINE_READER. More...
 
void SetSpecctraMode (bool aMode)
 Change the behavior of this lexer into or out of "specctra mode". More...
 
void PushReader (LINE_READER *aLineReader)
 Manage a stack of LINE_READERs in order to handle nested file inclusion. More...
 
LINE_READERPopReader ()
 Delete the top most LINE_READER from an internal stack of LINE_READERs and in the case of FILE_LINE_READER this means the associated FILE is closed. More...
 
int NextTok ()
 Return the next token found in the input file or DSN_EOF when reaching the end of file. More...
 
int NeedSYMBOL ()
 Call NextTok() and then verifies that the token read in satisfies IsSymbol(). More...
 
int NeedSYMBOLorNUMBER ()
 Call NextTok() and then verifies that the token read in satisfies bool IsSymbol() or the next token is DSN_NUMBER. More...
 
int NeedNUMBER (const char *aExpectation)
 Call NextTok() and then verifies that the token read is type DSN_NUMBER. More...
 
int CurTok () const
 Return whatever NextTok() returned the last time it was called. More...
 
int PrevTok () const
 Return whatever NextTok() returned the 2nd to last time it was called. More...
 
int GetCurStrAsToken () const
 Used to support "loose" matches (quoted tokens). More...
 
char SetStringDelimiter (char aStringDelimiter)
 Change the string delimiter from the default " to some other character and return the old value. More...
 
bool SetSpaceInQuotedTokens (bool val)
 Change the setting controlling whether a space in a quoted string isa terminator. More...
 
bool SetCommentsAreTokens (bool val)
 Change the handling of comments. More...
 
wxArrayString * ReadCommentLines ()
 Check the next sequence of tokens and reads them into a wxArrayString if they are comments. More...
 
void Expecting (int aTok) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Expecting (const char *aTokenList) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Unexpected (int aTok) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Unexpected (const char *aToken) const
 Throw an IO_ERROR exception with an input file specific error message. More...
 
void Duplicate (int aTok)
 Throw an IO_ERROR exception with a message saying specifically that aTok is a duplicate of one already seen in current context. More...
 
void NeedLEFT ()
 Call NextTok() and then verifies that the token read in is a DSN_LEFT. More...
 
void NeedRIGHT ()
 Call NextTok() and then verifies that the token read in is a DSN_RIGHT. More...
 
const char * GetTokenText (int aTok) const
 Return the C string representation of a DSN_T value. More...
 
wxString GetTokenString (int aTok) const
 Return a quote wrapped wxString representation of a token value. More...
 
const char * CurText () const
 Return a pointer to the current token's text. More...
 
const std::string & CurStr () const
 Return a reference to current token in std::string form. More...
 
wxString FromUTF8 () const
 Return the current token text as a wxString, assuming that the input byte stream is UTF8 encoded. More...
 
int CurLineNumber () const
 Return the current line number within my LINE_READER. More...
 
const char * CurLine () const
 Return the current line of text from which the CurText() would return its token. More...
 
const wxString & CurSource () const
 Return the current LINE_READER source. More...
 
int CurOffset () const
 Return the byte offset within the current line, using a 1 based index. More...
 

Static Public Member Functions

static bool IsSymbol (int aTok)
 Test a token to see if it is a symbol. More...
 
static const char * Syntax (int aTok)
 

Protected Types

typedef std::vector< LINE_READER * > READER_STACK
 

Protected Member Functions

void init ()
 
int readLine ()
 
int findToken (const std::string &aToken) const
 Take aToken string and looks up the string in the keywords table. More...
 
bool isStringTerminator (char cc) const
 

Protected Attributes

bool iOwnReaders
 on readerStack, should I delete them? More...
 
const char * start
 
const char * next
 
const char * limit
 
char dummy [1]
 when there is no reader. More...
 
READER_STACK readerStack
 all the LINE_READERs by pointer. More...
 
LINE_READERreader
 
bool specctraMode
 if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_in_quoted_tokens is functional else not. More...
 
char stringDelimiter
 
bool space_in_quoted_tokens
 blank spaces within quoted strings More...
 
bool commentsAreTokens
 true if should return comments as tokens More...
 
int prevTok
 curTok from previous NextTok() call. More...
 
int curOffset
 offset within current line of the current token More...
 
int curTok
 the current token obtained on last NextTok() More...
 
std::string curText
 the text of the current token More...
 
const KEYWORDkeywords
 table sorted by CMake for bsearch() More...
 
unsigned keywordCount
 count of keywords table More...
 
KEYWORD_MAP keyword_hash
 fast, specialized "C string" hashtable More...
 

Detailed Description

Implement a lexical analyzer for the SPECCTRA DSN file format.

It reads lexical tokens from the current LINE_READER through the NextTok() function.

Definition at line 78 of file dsnlexer.h.

Member Typedef Documentation

◆ READER_STACK

typedef std::vector<LINE_READER*> DSNLEXER::READER_STACK
protected

Definition at line 500 of file dsnlexer.h.

Constructor & Destructor Documentation

◆ DSNLEXER() [1/4]

DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
FILE *  aFile,
const wxString &  aFileName 
)

Initialize a DSN lexer and prepares to read from aFile which is already open and has aFilename.

Parameters
aKeywordTableis an array of KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aFileis an open file, which will be closed when this is destructed.
aFileNameis the name of the file

Definition at line 72 of file dsnlexer.cpp.

73  :
74  iOwnReaders( true ),
75  start( nullptr ),
76  next( nullptr ),
77  limit( nullptr ),
78  reader( nullptr ),
79  keywords( aKeywordTable ),
80  keywordCount( aKeywordCount )
81 {
82  FILE_LINE_READER* fileReader = new FILE_LINE_READER( aFile, aFilename );
83  PushReader( fileReader );
84  init();
85 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:188
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
A LINE_READER that reads from an open file.
Definition: richio.h:172
const char * limit
Definition: dsnlexer.h:497
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
void init()
Definition: dsnlexer.cpp:40

References init(), and PushReader().

◆ DSNLEXER() [2/4]

DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
const std::string &  aSExpression,
const wxString &  aSource = wxEmptyString 
)

Initialize a DSN lexer and prepares to read from aSExpression.

Parameters
aKeywordTableis an array of KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aSExpressionis text to feed through a STRING_LINE_READER
aSourceis a description of aSExpression, used for error reporting.

Definition at line 88 of file dsnlexer.cpp.

89  :
90  iOwnReaders( true ),
91  start( nullptr ),
92  next( nullptr ),
93  limit( nullptr ),
94  reader( nullptr ),
95  keywords( aKeywordTable ),
96  keywordCount( aKeywordCount )
97 {
98  STRING_LINE_READER* stringReader = new STRING_LINE_READER( aClipboardTxt, aSource.IsEmpty() ?
99  wxString( FMT_CLIPBOARD ) : aSource );
100  PushReader( stringReader );
101  init();
102 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:188
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
#define FMT_CLIPBOARD
Definition: dsnlexer.cpp:35
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
Is a LINE_READER that reads from a multiline 8 bit wide std::string.
Definition: richio.h:237
void init()
Definition: dsnlexer.cpp:40

References FMT_CLIPBOARD, init(), and PushReader().

◆ DSNLEXER() [3/4]

DSNLEXER::DSNLEXER ( const std::string &  aSExpression,
const wxString &  aSource = wxEmptyString 
)

Initialize a DSN lexer and prepares to read from aSExpression.

Use this one without a keyword table with the DOM parser in ptree.h.

Parameters
aSExpressionis text to feed through a STRING_LINE_READER
aSourceis a description of aSExpression, used for error reporting.

Definition at line 123 of file dsnlexer.cpp.

123  :
124  iOwnReaders( true ),
125  start( nullptr ),
126  next( nullptr ),
127  limit( nullptr ),
128  reader( nullptr ),
130  keywordCount( 0 )
131 {
132  STRING_LINE_READER* stringReader = new STRING_LINE_READER( aSExpression, aSource.IsEmpty() ?
133  wxString( FMT_CLIPBOARD ) : aSource );
134  PushReader( stringReader );
135  init();
136 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:188
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
static const KEYWORD empty_keywords[1]
Definition: dsnlexer.cpp:121
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
#define FMT_CLIPBOARD
Definition: dsnlexer.cpp:35
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
Is a LINE_READER that reads from a multiline 8 bit wide std::string.
Definition: richio.h:237
void init()
Definition: dsnlexer.cpp:40

References FMT_CLIPBOARD, init(), and PushReader().

◆ DSNLEXER() [4/4]

DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
LINE_READER aLineReader = NULL 
)

Initialize a DSN lexer and prepares to read from aLineReader which is already open, and may be in use by other DSNLEXERs also.

No ownership is taken of aLineReader. This enables it to be used by other DSNLEXERs.

Parameters
aKeywordTableis an array of #KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aLineReaderis any subclassed instance of LINE_READER, such as STRING_LINE_READER or FILE_LINE_READER. No ownership is taken.

Definition at line 105 of file dsnlexer.cpp.

106  :
107  iOwnReaders( false ),
108  start( nullptr ),
109  next( nullptr ),
110  limit( nullptr ),
111  reader( nullptr ),
112  keywords( aKeywordTable ),
113  keywordCount( aKeywordCount )
114 {
115  if( aLineReader )
116  PushReader( aLineReader );
117  init();
118 }
void PushReader(LINE_READER *aLineReader)
Manage a stack of LINE_READERs in order to handle nested file inclusion.
Definition: dsnlexer.cpp:188
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
void init()
Definition: dsnlexer.cpp:40

References init(), and PushReader().

◆ ~DSNLEXER()

DSNLEXER::~DSNLEXER ( )
virtual

Definition at line 139 of file dsnlexer.cpp.

140 {
141  if( iOwnReaders )
142  {
143  // delete the LINE_READERs from the stack, since I own them.
144  for( READER_STACK::iterator it = readerStack.begin(); it!=readerStack.end(); ++it )
145  delete *it;
146  }
147 }
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:494
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:502

References iOwnReaders, and readerStack.

Member Function Documentation

◆ CurLine()

const char* DSNLEXER::CurLine ( ) const
inline

Return the current line of text from which the CurText() would return its token.

Definition at line 424 of file dsnlexer.h.

425  {
426  return (const char*)(*reader);
427  }

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurLineNumber()

int DSNLEXER::CurLineNumber ( ) const
inline

Return the current line number within my LINE_READER.

Definition at line 416 of file dsnlexer.h.

417  {
418  return reader->LineNumber();
419  }
virtual unsigned LineNumber() const
Return the line number of the last line read from this LINE_READER.
Definition: richio.h:135
LINE_READER * reader
Definition: dsnlexer.h:505

References LINE_READER::LineNumber(), and reader.

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurOffset()

int DSNLEXER::CurOffset ( ) const
inline

Return the byte offset within the current line, using a 1 based index.

Returns
a one based index into the current line.

Definition at line 444 of file dsnlexer.h.

445  {
446  return curOffset + 1;
447  }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519

References curOffset.

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurSource()

const wxString& DSNLEXER::CurSource ( ) const
inline

Return the current LINE_READER source.

Returns
source of the lines of text, e.g. a filename or "clipboard".

Definition at line 434 of file dsnlexer.h.

435  {
436  return reader->GetSource();
437  }
virtual const wxString & GetSource() const
Returns the name of the source of the lines in an abstract sense.
Definition: richio.h:109
LINE_READER * reader
Definition: dsnlexer.h:505

References LINE_READER::GetSource(), and reader.

Referenced by Duplicate(), Expecting(), NeedNUMBER(), NextTok(), and Unexpected().

◆ CurStr()

const std::string& DSNLEXER::CurStr ( ) const
inline

Return a reference to current token in std::string form.

Definition at line 399 of file dsnlexer.h.

400  {
401  return curText;
402  }
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText.

◆ CurText()

const char* DSNLEXER::CurText ( ) const
inline

Return a pointer to the current token's text.

Definition at line 391 of file dsnlexer.h.

392  {
393  return curText.c_str();
394  }
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText.

Referenced by CADSTAR_ARCHIVE_PARSER::LoadArchiveFile(), PCAD2KICAD::LoadInputFile(), scanAtom(), and scanList().

◆ CurTok()

int DSNLEXER::CurTok ( ) const
inline

Return whatever NextTok() returned the last time it was called.

Definition at line 230 of file dsnlexer.h.

231  {
232  return curTok;
233  }
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:521

References curTok.

Referenced by Scan(), and scanList().

◆ Duplicate()

void DSNLEXER::Duplicate ( int  aTok)

Throw an IO_ERROR exception with a message saying specifically that aTok is a duplicate of one already seen in current context.

Parameters
aTokis the token/keyword type which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 346 of file dsnlexer.cpp.

347 {
348  wxString errText = wxString::Format(
349  _("%s is a duplicate"), GetTokenString( aTok ).GetData() );
351 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
wxString GetTokenString(int aTok) const
Return a quote wrapped wxString representation of a token value.
Definition: dsnlexer.cpp:303
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
#define _(s)
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), GetTokenString(), and THROW_PARSE_ERROR.

◆ Expecting() [1/2]

void DSNLEXER::Expecting ( int  aTok) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokis the token/keyword type which was expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 322 of file dsnlexer.cpp.

323 {
324  wxString errText = wxString::Format(
325  _( "Expecting %s" ), GetTokenString( aTok ) );
327 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
wxString GetTokenString(int aTok) const
Return a quote wrapped wxString representation of a token value.
Definition: dsnlexer.cpp:303
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
#define _(s)
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), GetTokenString(), and THROW_PARSE_ERROR.

Referenced by NeedLEFT(), NeedRIGHT(), NeedSYMBOL(), and NeedSYMBOLorNUMBER().

◆ Expecting() [2/2]

void DSNLEXER::Expecting ( const char *  aTokenList) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokenListis the token/keyword type which was expected at the current input location, e.g. "pin|graphic|property".
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 330 of file dsnlexer.cpp.

331 {
332  wxString errText = wxString::Format(
333  _( "Expecting '%s'" ), wxString::FromUTF8( text ) );
335 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
#define _(s)
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), text, and THROW_PARSE_ERROR.

◆ findToken()

int DSNLEXER::findToken ( const std::string &  aToken) const
protected

Take aToken string and looks up the string in the keywords table.

Parameters
aTokenis a string to lookup in the keywords table.
Returns
with a value from the enum DSN_T matching the keyword text, or DSN_SYMBOL if aToken is not in the keywords table.

Definition at line 229 of file dsnlexer.cpp.

230 {
231  KEYWORD_MAP::const_iterator it = keyword_hash.find( tok.c_str() );
232 
233  if( it != keyword_hash.end() )
234  return it->second;
235 
236  return DSN_SYMBOL; // not a keyword, some arbitrary symbol.
237 }
KEYWORD_MAP keyword_hash
fast, specialized "C string" hashtable
Definition: dsnlexer.h:526

References DSN_SYMBOL, and keyword_hash.

Referenced by GetCurStrAsToken(), and NextTok().

◆ FromUTF8()

wxString DSNLEXER::FromUTF8 ( ) const
inline

Return the current token text as a wxString, assuming that the input byte stream is UTF8 encoded.

Definition at line 408 of file dsnlexer.h.

409  {
410  return wxString::FromUTF8( curText.c_str() );
411  }
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText.

Referenced by ReadCommentLines().

◆ GetCurStrAsToken()

int DSNLEXER::GetCurStrAsToken ( ) const
inline

Used to support "loose" matches (quoted tokens).

Definition at line 246 of file dsnlexer.h.

247  {
248  return findToken( curText );
249  }
int findToken(const std::string &aToken) const
Take aToken string and looks up the string in the keywords table.
Definition: dsnlexer.cpp:229
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curText, and findToken().

◆ GetTokenString()

wxString DSNLEXER::GetTokenString ( int  aTok) const

Return a quote wrapped wxString representation of a token value.

Definition at line 303 of file dsnlexer.cpp.

304 {
305  wxString ret;
306 
307  ret << wxT("'") << wxString::FromUTF8( GetTokenText(aTok) ) << wxT("'");
308 
309  return ret;
310 }
const char * GetTokenText(int aTok) const
Return the C string representation of a DSN_T value.
Definition: dsnlexer.cpp:284

References GetTokenText().

Referenced by Duplicate(), Expecting(), and Unexpected().

◆ GetTokenText()

const char * DSNLEXER::GetTokenText ( int  aTok) const

Return the C string representation of a DSN_T value.

Definition at line 284 of file dsnlexer.cpp.

285 {
286  const char* ret;
287 
288  if( aTok < 0 )
289  {
290  return Syntax( aTok );
291  }
292  else if( (unsigned) aTok < keywordCount )
293  {
294  ret = keywords[aTok].name;
295  }
296  else
297  ret = "token too big";
298 
299  return ret;
300 }
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * name
unique keyword.
Definition: dsnlexer.h:41
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
static const char * Syntax(int aTok)
Definition: dsnlexer.cpp:240

References keywordCount, keywords, KEYWORD::name, and Syntax().

Referenced by GetTokenString().

◆ init()

void DSNLEXER::init ( )
protected

Definition at line 40 of file dsnlexer.cpp.

41 {
42  curTok = DSN_NONE;
43  prevTok = DSN_NONE;
44 
45  stringDelimiter = '"';
46 
47  specctraMode = false;
48  space_in_quoted_tokens = false;
49  commentsAreTokens = false;
50 
51  curOffset = 0;
52 
53 #if 1
54  if( keywordCount > 11 )
55  {
56  // resize the hashtable bucket count
57  keyword_hash.reserve( keywordCount );
58  }
59 
60  // fill the specialized "C string" hashtable from keywords[]
61  const KEYWORD* it = keywords;
62  const KEYWORD* end = it + keywordCount;
63 
64  for( ; it < end; ++it )
65  {
66  keyword_hash[it->name] = it->token;
67  }
68 #endif
69 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:516
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:524
const char * name
unique keyword.
Definition: dsnlexer.h:41
KEYWORD_MAP keyword_hash
fast, specialized "C string" hashtable
Definition: dsnlexer.h:526
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:518
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:525
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:521
Hold a keyword string and its unique integer token.
Definition: dsnlexer.h:39
char stringDelimiter
Definition: dsnlexer.h:513
int token
a zero based index into an array of KEYWORDs
Definition: dsnlexer.h:42

References commentsAreTokens, curOffset, curTok, DSN_NONE, keyword_hash, keywordCount, keywords, KEYWORD::name, prevTok, space_in_quoted_tokens, specctraMode, stringDelimiter, and KEYWORD::token.

Referenced by DSNLEXER().

◆ isStringTerminator()

bool DSNLEXER::isStringTerminator ( char  cc) const
inlineprotected

Definition at line 483 of file dsnlexer.h.

484  {
485  if( !space_in_quoted_tokens && cc == ' ' )
486  return true;
487 
488  if( cc == stringDelimiter )
489  return true;
490 
491  return false;
492  }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
char stringDelimiter
Definition: dsnlexer.h:513

References space_in_quoted_tokens, and stringDelimiter.

Referenced by NextTok().

◆ IsSymbol()

bool DSNLEXER::IsSymbol ( int  aTok)
static

Test a token to see if it is a symbol.

This means it cannot be a special delimiter character such as DSN_LEFT, DSN_RIGHT, #DSN_QUOTE, etc. It may however, coincidentally match a keyword and still be a symbol.

Definition at line 313 of file dsnlexer.cpp.

314 {
315  // This is static and not inline to reduce code space.
316 
317  // if aTok is >= 0, then it is a coincidental match to a keyword.
318  return aTok == DSN_SYMBOL || aTok == DSN_STRING || aTok >= 0;
319 }

References DSN_STRING, and DSN_SYMBOL.

Referenced by NeedSYMBOL(), and NeedSYMBOLorNUMBER().

◆ NeedLEFT()

void DSNLEXER::NeedLEFT ( )

Call NextTok() and then verifies that the token read in is a DSN_LEFT.

Exceptions
IO_ERRORif the next token is not a DSN_LEFT

Definition at line 362 of file dsnlexer.cpp.

363 {
364  int tok = NextTok();
365  if( tok != DSN_LEFT )
366  Expecting( DSN_LEFT );
367 }
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:509
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:322

References DSN_LEFT, Expecting(), and NextTok().

◆ NeedNUMBER()

int DSNLEXER::NeedNUMBER ( const char *  aExpectation)

Call NextTok() and then verifies that the token read is type DSN_NUMBER.

Returns
the actual token read in.
Exceptions
IO_ERRORif the next token does not satisfy the above test.

Definition at line 396 of file dsnlexer.cpp.

397 {
398  int tok = NextTok();
399  if( tok != DSN_NUMBER )
400  {
401  wxString errText = wxString::Format(
402  _( "need a number for '%s'" ), wxString::FromUTF8( aExpectation ).GetData() );
404  }
405  return tok;
406 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
#define _(s)
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:509
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), DSN_NUMBER, Format(), NextTok(), and THROW_PARSE_ERROR.

◆ NeedRIGHT()

void DSNLEXER::NeedRIGHT ( )

Call NextTok() and then verifies that the token read in is a DSN_RIGHT.

Exceptions
IO_ERRORif the next token is not a DSN_RIGHT

Definition at line 370 of file dsnlexer.cpp.

371 {
372  int tok = NextTok();
373  if( tok != DSN_RIGHT )
374  Expecting( DSN_RIGHT );
375 }
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:509
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:322

References DSN_RIGHT, Expecting(), and NextTok().

◆ NeedSYMBOL()

int DSNLEXER::NeedSYMBOL ( )

Call NextTok() and then verifies that the token read in satisfies IsSymbol().

Returns
the actual token read in.
Exceptions
IO_ERRORif the next token does not satisfy IsSymbol().

Definition at line 378 of file dsnlexer.cpp.

379 {
380  int tok = NextTok();
381  if( !IsSymbol( tok ) )
383  return tok;
384 }
static bool IsSymbol(int aTok)
Test a token to see if it is a symbol.
Definition: dsnlexer.cpp:313
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:509
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:322

References DSN_SYMBOL, Expecting(), IsSymbol(), and NextTok().

◆ NeedSYMBOLorNUMBER()

int DSNLEXER::NeedSYMBOLorNUMBER ( )

Call NextTok() and then verifies that the token read in satisfies bool IsSymbol() or the next token is DSN_NUMBER.

Returns
the actual token read in.
Exceptions
IO_ERRORif the next token does not satisfy the above test.

Definition at line 387 of file dsnlexer.cpp.

388 {
389  int tok = NextTok();
390  if( !IsSymbol( tok ) && tok!=DSN_NUMBER )
391  Expecting( "a symbol or number" );
392  return tok;
393 }
static bool IsSymbol(int aTok)
Test a token to see if it is a symbol.
Definition: dsnlexer.cpp:313
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:509
void Expecting(int aTok) const
Throw an IO_ERROR exception with an input file specific error message.
Definition: dsnlexer.cpp:322

References DSN_NUMBER, Expecting(), IsSymbol(), and NextTok().

◆ NextTok()

int DSNLEXER::NextTok ( )

Return the next token found in the input file or DSN_EOF when reaching the end of file.

Users should wrap this function to return an enum to aid in grammar debugging while running under a debugger, but leave this lower level function returning an int (so the enum does not collide with another usage).

Returns
the type of token found next.
Exceptions
IO_ERRORonly if the LINE_READER throws it.

Definition at line 509 of file dsnlexer.cpp.

510 {
511  const char* cur = next;
512  const char* head = cur;
513 
514  prevTok = curTok;
515 
516  if( curTok == DSN_EOF )
517  goto exit;
518 
519  if( cur >= limit )
520  {
521 L_read:
522  // blank lines are returned as "\n" and will have a len of 1.
523  // EOF will have a len of 0 and so is detectable.
524  int len = readLine();
525  if( len == 0 )
526  {
527  cur = start; // after readLine(), since start can change, set cur offset to start
528  curTok = DSN_EOF;
529  goto exit;
530  }
531 
532  cur = start; // after readLine() since start can change.
533 
534  // skip leading whitespace
535  while( cur<limit && isSpace( *cur ) )
536  ++cur;
537 
538  // If the first non-blank character is #, this line is a comment.
539  // Comments cannot follow any other token on the same line.
540  if( cur<limit && *cur=='#' )
541  {
542  if( commentsAreTokens )
543  {
544  // Grab the entire current line [excluding end of line char(s)] as the
545  // current token. The '#' character may not be at offset zero.
546 
547  while( limit[-1] == '\n' || limit[-1] == '\r' )
548  --limit;
549 
550  curText.clear();
551  curText.append( start, limit );
552 
553  cur = start; // ensure a good curOffset below
555  head = limit; // do a readLine() on next call in here.
556  goto exit;
557  }
558  else
559  goto L_read;
560  }
561  }
562  else
563  {
564  // skip leading whitespace
565  while( cur<limit && isSpace( *cur ) )
566  ++cur;
567  }
568 
569  if( cur >= limit )
570  goto L_read;
571 
572  if( *cur == '(' )
573  {
574  curText = *cur;
575  curTok = DSN_LEFT;
576  head = cur+1;
577  goto exit;
578  }
579 
580  if( *cur == ')' )
581  {
582  curText = *cur;
583  curTok = DSN_RIGHT;
584  head = cur+1;
585  goto exit;
586  }
587 
588  // Non-specctraMode, understands and deciphers escaped \, \r, \n, and \".
589  // Strips off leading and trailing double quotes
590  if( !specctraMode )
591  {
592  // a quoted string, will return DSN_STRING
593  if( *cur == stringDelimiter )
594  {
595  // copy the token, character by character so we can remove doubled up quotes.
596  curText.clear();
597 
598  ++cur; // skip over the leading delimiter, which is always " in non-specctraMode
599 
600  head = cur;
601 
602  while( head<limit )
603  {
604  // ESCAPE SEQUENCES:
605  if( *head =='\\' )
606  {
607  char tbuf[8];
608  char c;
609  int i;
610 
611  if( ++head >= limit )
612  break; // throw exception at L_unterminated
613 
614  switch( *head++ )
615  {
616  case '"':
617  case '\\': c = head[-1]; break;
618  case 'a': c = '\x07'; break;
619  case 'b': c = '\x08'; break;
620  case 'f': c = '\x0c'; break;
621  case 'n': c = '\n'; break;
622  case 'r': c = '\r'; break;
623  case 't': c = '\x09'; break;
624  case 'v': c = '\x0b'; break;
625 
626  case 'x': // 1 or 2 byte hex escape sequence
627  for( i=0; i<2; ++i )
628  {
629  if( !isxdigit( head[i] ) )
630  break;
631  tbuf[i] = head[i];
632  }
633  tbuf[i] = '\0';
634  if( i > 0 )
635  c = (char) strtoul( tbuf, NULL, 16 );
636  else
637  c = 'x'; // a goofed hex escape sequence, interpret as 'x'
638  head += i;
639  break;
640 
641  default: // 1-3 byte octal escape sequence
642  --head;
643  for( i=0; i<3; ++i )
644  {
645  if( head[i] < '0' || head[i] > '7' )
646  break;
647  tbuf[i] = head[i];
648  }
649  tbuf[i] = '\0';
650  if( i > 0 )
651  c = (char) strtoul( tbuf, NULL, 8 );
652  else
653  c = '\\'; // a goofed octal escape sequence, interpret as '\'
654  head += i;
655  break;
656  }
657 
658  curText += c;
659  }
660 
661  else if( *head == '"' ) // end of the non-specctraMode DSN_STRING
662  {
663  curTok = DSN_STRING;
664  ++head; // omit this trailing double quote
665  goto exit;
666  }
667 
668  else
669  curText += *head++;
670 
671  } // while
672 
673  // L_unterminated:
674  wxString errtxt( _( "Un-terminated delimited string" ) );
676  cur - start + curText.length() );
677  }
678  }
679 
680  else // is specctraMode, tests in this block should not occur in KiCad mode.
681  {
682  /* get the dash out of a <pin_reference> which is embedded for example
683  like: U2-14 or "U2"-"14"
684  This is detectable by a non-space immediately preceding the dash.
685  */
686  if( *cur == '-' && cur>start && !isSpace( cur[-1] ) )
687  {
688  curText = '-';
689  curTok = DSN_DASH;
690  head = cur+1;
691  goto exit;
692  }
693 
694  // switching the string_quote character
695  if( prevTok == DSN_STRING_QUOTE )
696  {
697  static const wxString errtxt( _("String delimiter must be a single character of ', \", or $"));
698 
699  char cc = *cur;
700  switch( cc )
701  {
702  case '\'':
703  case '$':
704  case '"':
705  break;
706  default:
708  }
709 
710  curText = cc;
711 
712  head = cur+1;
713 
714  if( head<limit && !isSep( *head ) )
715  {
717  }
718 
720  goto exit;
721  }
722 
723  // specctraMode DSN_STRING
724  if( *cur == stringDelimiter )
725  {
726  ++cur; // skip over the leading delimiter: ",', or $
727 
728  head = cur;
729 
730  while( head<limit && !isStringTerminator( *head ) )
731  ++head;
732 
733  if( head >= limit )
734  {
735  wxString errtxt( _( "Un-terminated delimited string" ) );
737  }
738 
739  curText.clear();
740  curText.append( cur, head );
741 
742  ++head; // skip over the trailing delimiter
743 
744  curTok = DSN_STRING;
745  goto exit;
746  }
747  } // specctraMode
748 
749  // non-quoted token, read it into curText.
750  curText.clear();
751 
752  head = cur;
753  while( head<limit && !isSep( *head ) )
754  curText += *head++;
755 
756  if( isNumber( curText.c_str(), curText.c_str() + curText.size() ) )
757  {
758  curTok = DSN_NUMBER;
759  goto exit;
760  }
761 
762  if( specctraMode && curText == "string_quote" )
763  {
765  goto exit;
766  }
767 
768  curTok = findToken( curText );
769 
770 exit: // single point of exit, no returns elsewhere please.
771 
772  curOffset = cur - start;
773 
774  next = head;
775 
776  return curTok;
777 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:516
bool isSep(char cc)
return true if cc is an s-expression separator character
Definition: dsnlexer.cpp:442
bool isStringTerminator(char cc) const
Definition: dsnlexer.h:483
static bool isNumber(const char *cp, const char *limit)
Function isNumber returns true if the next sequence of text is a number: either an integer,...
Definition: dsnlexer.cpp:459
int findToken(const std::string &aToken) const
Take aToken string and looks up the string in the keywords table.
Definition: dsnlexer.cpp:229
const char * next
Definition: dsnlexer.h:496
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
const char * limit
Definition: dsnlexer.h:497
#define NULL
static bool isSpace(char cc)
Function isSpace tests for whitespace.
Definition: dsnlexer.cpp:415
#define _(s)
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:518
int readLine()
Definition: dsnlexer.h:454
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
const char * start
Definition: dsnlexer.h:495
std::string curText
the text of the current token
Definition: dsnlexer.h:522
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:521
char stringDelimiter
Definition: dsnlexer.h:513

References _, commentsAreTokens, CurLine(), CurLineNumber(), CurOffset(), curOffset, CurSource(), curText, curTok, DSN_COMMENT, DSN_DASH, DSN_EOF, DSN_LEFT, DSN_NUMBER, DSN_QUOTE_DEF, DSN_RIGHT, DSN_STRING, DSN_STRING_QUOTE, exit, findToken(), isNumber(), isSep(), isSpace(), isStringTerminator(), limit, next, NULL, prevTok, readLine(), specctraMode, start, stringDelimiter, and THROW_PARSE_ERROR.

Referenced by CADSTAR_ARCHIVE_PARSER::LoadArchiveFile(), PCAD2KICAD::LoadInputFile(), NeedLEFT(), NeedNUMBER(), NeedRIGHT(), NeedSYMBOL(), NeedSYMBOLorNUMBER(), ReadCommentLines(), Scan(), and scanList().

◆ PopReader()

LINE_READER * DSNLEXER::PopReader ( )

Delete the top most LINE_READER from an internal stack of LINE_READERs and in the case of FILE_LINE_READER this means the associated FILE is closed.

The most recently used former LINE_READER on the stack becomes the current LINE_READER and its previous position in its input stream and the its latest line number should pertain. PopReader always starts reading from a new line upon returning to the previous LINE_READER. A pop is only possible if there are at least 2 #LINE_READERs on the stack, since popping the last one is not supported.

Returns
the LINE_READER that was in use before the pop, or NULL if there was not at least two readers on the stack and therefore the pop failed.

Definition at line 200 of file dsnlexer.cpp.

201 {
202  LINE_READER* ret = nullptr;
203 
204  if( readerStack.size() )
205  {
206  ret = reader;
207  readerStack.pop_back();
208 
209  if( readerStack.size() )
210  {
211  reader = readerStack.back();
212  start = reader->Line();
213 
214  // force a new readLine() as first thing.
215  limit = start;
216  next = start;
217  }
218  else
219  {
220  reader = nullptr;
221  start = dummy;
222  limit = dummy;
223  }
224  }
225  return ret;
226 }
An abstract class from which implementation specific LINE_READERs may be derived to read single lines...
Definition: richio.h:80
char * Line() const
Return a pointer to the last line that was read in.
Definition: richio.h:117
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:502
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
char dummy[1]
when there is no reader.
Definition: dsnlexer.h:498

References dummy, limit, LINE_READER::Line(), next, reader, readerStack, and start.

◆ PrevTok()

int DSNLEXER::PrevTok ( ) const
inline

Return whatever NextTok() returned the 2nd to last time it was called.

Definition at line 238 of file dsnlexer.h.

239  {
240  return prevTok;
241  }
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:518

References prevTok.

◆ PushReader()

void DSNLEXER::PushReader ( LINE_READER aLineReader)

Manage a stack of LINE_READERs in order to handle nested file inclusion.

This function pushes aLineReader onto the top of a stack of LINE_READERs and makes it the current LINE_READER with its own #GetSource(), line number and line text. A grammar must be designed such that the "include" token (whatever its various names), and any of its parameters are not followed by anything on that same line, because PopReader always starts reading from a new line upon returning to the original LINE_READER.

Definition at line 188 of file dsnlexer.cpp.

189 {
190  readerStack.push_back( aLineReader );
191  reader = aLineReader;
192  start = (const char*) (*reader);
193 
194  // force a new readLine() as first thing.
195  limit = start;
196  next = start;
197 }
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:502
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495

References limit, next, reader, readerStack, and start.

Referenced by DSNLEXER().

◆ ReadCommentLines()

wxArrayString * DSNLEXER::ReadCommentLines ( )

Check the next sequence of tokens and reads them into a wxArrayString if they are comments.

Reading continues until a non-comment token is encountered, and such last read token remains as CurTok() and as CurText(). No push back or "un get" mechanism is used for this support. Upon return you simply avoid calling NextTok() for the next token, but rather CurTok().

Returns
Heap allocated block of comments or NULL if none. The caller owns the allocation and must delete if not NULL.

Definition at line 780 of file dsnlexer.cpp.

781 {
782  wxArrayString* ret = nullptr;
783  bool cmt_setting = SetCommentsAreTokens( true );
784  int tok = NextTok();
785 
786  if( tok == DSN_COMMENT )
787  {
788  ret = new wxArrayString();
789 
790  do
791  {
792  ret->Add( FromUTF8() );
793  }
794  while( ( tok = NextTok() ) == DSN_COMMENT );
795  }
796 
797  SetCommentsAreTokens( cmt_setting );
798 
799  return ret;
800 }
bool SetCommentsAreTokens(bool val)
Change the handling of comments.
Definition: dsnlexer.h:289
int NextTok()
Return the next token found in the input file or DSN_EOF when reaching the end of file.
Definition: dsnlexer.cpp:509
wxString FromUTF8() const
Return the current token text as a wxString, assuming that the input byte stream is UTF8 encoded.
Definition: dsnlexer.h:408

References DSN_COMMENT, FromUTF8(), NextTok(), and SetCommentsAreTokens().

◆ readLine()

int DSNLEXER::readLine ( )
inlineprotected

Definition at line 454 of file dsnlexer.h.

455  {
456  if( reader )
457  {
458  reader->ReadLine();
459 
460  unsigned len = reader->Length();
461 
462  // start may have changed in ReadLine(), which can resize and
463  // relocate reader's line buffer.
464  start = reader->Line();
465 
466  next = start;
467  limit = next + len;
468 
469  return len;
470  }
471  return 0;
472  }
char * Line() const
Return a pointer to the last line that was read in.
Definition: richio.h:117
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
LINE_READER * reader
Definition: dsnlexer.h:505
unsigned Length() const
Return the number of bytes in the last line read from this LINE_READER.
Definition: richio.h:143
const char * start
Definition: dsnlexer.h:495
virtual char * ReadLine()=0
Read a line of text into the buffer and increments the line number counter.

References LINE_READER::Length(), limit, LINE_READER::Line(), next, reader, LINE_READER::ReadLine(), and start.

Referenced by NextTok().

◆ SetCommentsAreTokens()

bool DSNLEXER::SetCommentsAreTokens ( bool  val)
inline

Change the handling of comments.

If set true, comments are returned as single line strings with a terminating newline. Otherwise they are consumed by the lexer and not returned.

Definition at line 289 of file dsnlexer.h.

290  {
291  bool old = commentsAreTokens;
292  commentsAreTokens = val;
293  return old;
294  }
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:516

References commentsAreTokens.

Referenced by ReadCommentLines().

◆ SetSpaceInQuotedTokens()

bool DSNLEXER::SetSpaceInQuotedTokens ( bool  val)
inline

Change the setting controlling whether a space in a quoted string isa terminator.

Parameters
valIf true, means

Definition at line 273 of file dsnlexer.h.

274  {
275  bool old = space_in_quoted_tokens;
276 
277  if( specctraMode )
279 
280  return old;
281  }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507

References space_in_quoted_tokens, and specctraMode.

◆ SetSpecctraMode()

void DSNLEXER::SetSpecctraMode ( bool  aMode)

Change the behavior of this lexer into or out of "specctra mode".

If specctra mode, then: -#) stringDelimiter can be changed. -#) KiCad quoting protocol is not in effect. -#) space_in_quoted_tokens is functional else none of the above are true.

The default mode is non-specctra mode, meaning: -#) stringDelimiter cannot be changed. -#) KiCad quoting protocol is in effect. -#) space_in_quoted_tokens is not functional.

Definition at line 149 of file dsnlexer.cpp.

150 {
151  specctraMode = aMode;
152  if( aMode )
153  {
154  // specctra mode defaults, some of which can still be changed in this mode.
155  space_in_quoted_tokens = true;
156  }
157  else
158  {
159  space_in_quoted_tokens = false;
160  stringDelimiter = '"';
161  }
162 }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:514
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
char stringDelimiter
Definition: dsnlexer.h:513

References space_in_quoted_tokens, specctraMode, and stringDelimiter.

◆ SetStringDelimiter()

char DSNLEXER::SetStringDelimiter ( char  aStringDelimiter)
inline

Change the string delimiter from the default " to some other character and return the old value.

Parameters
aStringDelimiterThe character in lowest 8 bits.
Returns
The old delimiter in the lowest 8 bits.

Definition at line 258 of file dsnlexer.h.

259  {
260  int old = stringDelimiter;
261 
262  if( specctraMode )
263  stringDelimiter = aStringDelimiter;
264 
265  return old;
266  }
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:507
char stringDelimiter
Definition: dsnlexer.h:513

References specctraMode, and stringDelimiter.

◆ SyncLineReaderWith()

bool DSNLEXER::SyncLineReaderWith ( DSNLEXER aLexer)

Usable only for DSN lexers which share the same LINE_READER.

Synchronizes the pointers handling the data read by the LINE_READER. Allows 2 #DNSLEXER objects to share the same current line, when switching from a #DNSLEXER to another #DNSLEXER

Parameters
aLexerthe model.
Returns
true if the sync can be made ( at least the same line reader ).

Definition at line 165 of file dsnlexer.cpp.

166 {
167  // Synchronize the pointers handling the data read by the LINE_READER
168  // only if aLexer shares the same LINE_READER, because only in this case
169  // the char buffer is be common
170 
171  if( reader != aLexer.reader )
172  return false;
173 
174  // We can synchronize the pointers which handle the data currently read
175  start = aLexer.start;
176  next = aLexer.next;
177  limit = aLexer.limit;
178 
179  // Sync these parameters is not mandatory, but could help
180  // for instance in debug
181  curText = aLexer.curText;
182  curOffset = aLexer.curOffset;
183 
184  return true;
185 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:519
const char * next
Definition: dsnlexer.h:496
const char * limit
Definition: dsnlexer.h:497
LINE_READER * reader
Definition: dsnlexer.h:505
const char * start
Definition: dsnlexer.h:495
std::string curText
the text of the current token
Definition: dsnlexer.h:522

References curOffset, curText, limit, next, reader, and start.

◆ Syntax()

const char * DSNLEXER::Syntax ( int  aTok)
static

Definition at line 240 of file dsnlexer.cpp.

241 {
242  const char* ret;
243 
244  switch( aTok )
245  {
246  case DSN_NONE:
247  ret = "NONE";
248  break;
249  case DSN_STRING_QUOTE:
250  ret = "string_quote"; // a special DSN syntax token, see specctra spec.
251  break;
252  case DSN_QUOTE_DEF:
253  ret = "quoted text delimiter";
254  break;
255  case DSN_DASH:
256  ret = "-";
257  break;
258  case DSN_SYMBOL:
259  ret = "symbol";
260  break;
261  case DSN_NUMBER:
262  ret = "number";
263  break;
264  case DSN_RIGHT:
265  ret = ")";
266  break;
267  case DSN_LEFT:
268  ret = "(";
269  break;
270  case DSN_STRING:
271  ret = "quoted string";
272  break;
273  case DSN_EOF:
274  ret = "end of input";
275  break;
276  default:
277  ret = "???";
278  }
279 
280  return ret;
281 }

References DSN_DASH, DSN_EOF, DSN_LEFT, DSN_NONE, DSN_NUMBER, DSN_QUOTE_DEF, DSN_RIGHT, DSN_STRING, DSN_STRING_QUOTE, and DSN_SYMBOL.

Referenced by GetTokenText().

◆ Unexpected() [1/2]

void DSNLEXER::Unexpected ( int  aTok) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokis the token/keyword type which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 338 of file dsnlexer.cpp.

339 {
340  wxString errText = wxString::Format(
341  _( "Unexpected %s" ), GetTokenString( aTok ) );
343 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
wxString GetTokenString(int aTok) const
Return a quote wrapped wxString representation of a token value.
Definition: dsnlexer.cpp:303
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
#define _(s)
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), GetTokenString(), and THROW_PARSE_ERROR.

Referenced by Scan(), and scanList().

◆ Unexpected() [2/2]

void DSNLEXER::Unexpected ( const char *  aToken) const

Throw an IO_ERROR exception with an input file specific error message.

Parameters
aTokenis the token which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 354 of file dsnlexer.cpp.

355 {
356  wxString errText = wxString::Format(
357  _( "Unexpected '%s'" ), wxString::FromUTF8( text ) );
359 }
int CurLineNumber() const
Return the current line number within my LINE_READER.
Definition: dsnlexer.h:416
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:164
#define _(s)
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, const CPTREE &aTree)
Output a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:200
int CurOffset() const
Return the byte offset within the current line, using a 1 based index.
Definition: dsnlexer.h:444
const char * CurLine() const
Return the current line of text from which the CurText() would return its token.
Definition: dsnlexer.h:424
const wxString & CurSource() const
Return the current LINE_READER source.
Definition: dsnlexer.h:434

References _, CurLine(), CurLineNumber(), CurOffset(), CurSource(), Format(), text, and THROW_PARSE_ERROR.

Member Data Documentation

◆ commentsAreTokens

bool DSNLEXER::commentsAreTokens
protected

true if should return comments as tokens

Definition at line 516 of file dsnlexer.h.

Referenced by init(), NextTok(), and SetCommentsAreTokens().

◆ curOffset

int DSNLEXER::curOffset
protected

offset within current line of the current token

Definition at line 519 of file dsnlexer.h.

Referenced by CurOffset(), init(), NextTok(), and SyncLineReaderWith().

◆ curText

std::string DSNLEXER::curText
protected

the text of the current token

Definition at line 522 of file dsnlexer.h.

Referenced by CurStr(), CurText(), FromUTF8(), GetCurStrAsToken(), NextTok(), and SyncLineReaderWith().

◆ curTok

int DSNLEXER::curTok
protected

the current token obtained on last NextTok()

Definition at line 521 of file dsnlexer.h.

Referenced by CurTok(), init(), and NextTok().

◆ dummy

char DSNLEXER::dummy[1]
protected

when there is no reader.

Definition at line 498 of file dsnlexer.h.

Referenced by PopReader().

◆ iOwnReaders

bool DSNLEXER::iOwnReaders
protected

on readerStack, should I delete them?

Definition at line 494 of file dsnlexer.h.

Referenced by ~DSNLEXER().

◆ keyword_hash

KEYWORD_MAP DSNLEXER::keyword_hash
protected

fast, specialized "C string" hashtable

Definition at line 526 of file dsnlexer.h.

Referenced by findToken(), and init().

◆ keywordCount

unsigned DSNLEXER::keywordCount
protected

count of keywords table

Definition at line 525 of file dsnlexer.h.

Referenced by GetTokenText(), and init().

◆ keywords

const KEYWORD* DSNLEXER::keywords
protected

table sorted by CMake for bsearch()

Definition at line 524 of file dsnlexer.h.

Referenced by GetTokenText(), and init().

◆ limit

const char* DSNLEXER::limit
protected

Definition at line 497 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), readLine(), and SyncLineReaderWith().

◆ next

const char* DSNLEXER::next
protected

Definition at line 496 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), readLine(), and SyncLineReaderWith().

◆ prevTok

int DSNLEXER::prevTok
protected

curTok from previous NextTok() call.

Definition at line 518 of file dsnlexer.h.

Referenced by init(), NextTok(), and PrevTok().

◆ reader

LINE_READER* DSNLEXER::reader
protected

◆ readerStack

READER_STACK DSNLEXER::readerStack
protected

all the LINE_READERs by pointer.

no ownership. ownership is via readerStack, maybe, if iOwnReaders

Definition at line 502 of file dsnlexer.h.

Referenced by PopReader(), PushReader(), and ~DSNLEXER().

◆ space_in_quoted_tokens

bool DSNLEXER::space_in_quoted_tokens
protected

blank spaces within quoted strings

Definition at line 514 of file dsnlexer.h.

Referenced by init(), isStringTerminator(), SetSpaceInQuotedTokens(), and SetSpecctraMode().

◆ specctraMode

bool DSNLEXER::specctraMode
protected

if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_in_quoted_tokens is functional else not.

Definition at line 507 of file dsnlexer.h.

Referenced by init(), NextTok(), SetSpaceInQuotedTokens(), SetSpecctraMode(), and SetStringDelimiter().

◆ start

const char* DSNLEXER::start
protected

Definition at line 495 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), readLine(), and SyncLineReaderWith().

◆ stringDelimiter

char DSNLEXER::stringDelimiter
protected

Definition at line 513 of file dsnlexer.h.

Referenced by init(), isStringTerminator(), NextTok(), SetSpecctraMode(), and SetStringDelimiter().


The documentation for this class was generated from the following files: