The token column will get wrong when include file has macro

[WoolenWang]

I use snoar-cxx as a tool to analysis the c/cxx code to get all node. and then write the code checklist by using xpath.

when the code file include the headers has some macro the token column will get wrong .
i use lua lang Interpreter src to test this .

i change the toolkit code just add one include dir

[guwirth]

Hi @WoolenWang, think the problem is a string concatenation problem? The preprocessor is creating one string from the two below (result: "line1-line2") and is counting the line number wrong? Can you verify this please.

Sample 1:

char test[] =
   "line1-"
   "line2";
char line_after;

Sample 2:

char test[] =
   "line1-"
   "line2-"
   "line3";
char line_after;

[guwirth]

@WoolenWang just another hint: xpath is no more supported in current versions of SQ.

[WoolenWang]

@guwirth Thanks a lot for the reply.

i try to reduce the scope of the problem code, here is the code::

/*
** $Id: lapi.c,v 2.171.1.1 2013/04/12 18:48:47 roberto Exp $
** Lua API
** See Copyright Notice in lua.h
*/


static TValue *index2addr (lua_State *L, int idx) {
  CallInfo *ci = L->ci;
  if ( idx > 0) {
    TValue *o = ci->func + idx;
    api_check(L, idx <= ci->top - (ci->func + 1), "unacceptable index");
    if (o >= L->top) return NONVALIDVALUE;
    else return o;
  }
  else if (!ispseudo(idx)) {  /* negative index */
    api_check(L, idx != 0 && -idx <= L->top - (ci->func + 1), "invalid index");
    return L->top + idx;
  }
  else if (idx == LUA_REGISTRYINDEX)
    return &G(L)->l_registry;
  else {  /* upvalues */
    idx = LUA_REGISTRYINDEX - idx;
    api_check(L, idx <= MAXUPVAL + 1, "upvalue index too large");
    if (ttislcf(ci->func))  /* light C function? */
      return NONVALIDVALUE;  /* it has no upvalues */
    else {
      CClosure *func = clCvalue(ci->func);
      return (idx <= func->nupvalues) ? &func->upvalue[idx-1] : NONVALIDVALUE;
    }
  }
}

use cxx tookit wil get the column wrong but the line is fine

and i will get the same wrong column in my checklist code.

[WoolenWang]

@guwirth
The problem seem's not a repeatable problem. but with lua src lapi.c it repeat.
when i try the same code the first time, it parse wrong , and i parse the source code again it get right.
when it is wrong i find that the expression ::
c api_check(L, idx <= ci->top - (ci->func + 1), "unacceptable index");
parse wrong as ::

you can try to download the lua src analysis:: http://www.lua.org/ftp/lua-5.3.3.tar.gz
the file i scan is lapi.c

[guwirth]

@WoolenWang thanks for your investigation. Could you provide a simple sample please. (Less lines as needed to reproduce the issue.)

[guwirth]

Simple sample to reproduce:

#define api_check(a, b, c) x(a, b, c)

void test () {
    api_check(Param1, Param2, Param3); // offset wrong: seems to use offset of macro x instead of api_check?
}

char test[] =
   "line1-"
   "line2-"
   "line3"; // end of line and offset wrong
char line_after;

[guwirth]

@WoolenWang it's not a bug - it's a feature.

The point is that you are adding a source file to the Source Code window and are pressing Parse. Parsing is a two step process. First step is that the preprocessor is parsing the code. In case the source code contains macros the resulting output is different to the original source code. In a second step the C++ parser is creating the AST. The SSLR toolkit displays this AST then on the right hand window. The AST is the AST for the preprocessed file. The problem is now that the AST does not match with the original source code file which is still displayed on the left hand side.

So the AST tokenLine and tokenColumn values are correct (for the preprocessed result).