cpd visitor: IllegalArgumentException: 208 is not a valid line offset for pointer #952

benrsx · 2016-09-20T15:56:56Z

Hi,

I run sonar cxx 0.9.7 analysis with sonar 5.6.1 and sonnar-scanner 2.7 and got a problem when parsing multi-lines string.

ERROR: Error during SonarQube Scanner execution
org.sonar.squidbridge.api.AnalysisException: Unable to analyze file: /var/projects/sonar/x/x/x/x/x.cxx
at org.sonar.squidbridge.AstScanner.scanFiles(AstScanner.java:131)
at org.sonar.plugins.cxx.squid.CxxSquidSensor.execute(CxxSquidSensor.java:128)
at org.sonar.batch.sensor.SensorWrapper.analyse(SensorWrapper.java:57)
at org.sonar.batch.phases.SensorsExecutor.executeSensor(SensorsExecutor.java:58)
at org.sonar.batch.phases.SensorsExecutor.execute(SensorsExecutor.java:50)
at org.sonar.batch.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:83)
at org.sonar.batch.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:192)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.batch.scan.ProjectScanContainer.scan(ProjectScanContainer.java:241)
at org.sonar.batch.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:236)
at org.sonar.batch.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:226)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.batch.task.ScanTask.execute(ScanTask.java:47)
at org.sonar.batch.task.TaskContainer.doAfterStart(TaskContainer.java:86)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.batch.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:106)
at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:119)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:244)
at org.sonarsource.scanner.api.EmbeddedScanner.runAnalysis(EmbeddedScanner.java:154)
at org.sonarsource.scanner.cli.Main.runAnalysis(Main.java:110)
at org.sonarsource.scanner.cli.Main.execute(Main.java:72)
at org.sonarsource.scanner.cli.Main.main(Main.java:60)
Caused by: java.lang.IllegalArgumentException: 208 is not a valid line offset for pointer. File [moduleKey=x:master, relative=x/x/x/x.cxx, basedir=/var/projects/sonar/x] has 72 character(s) at line 1089
at org.sonar.api.internal.google.common.base.Preconditions.checkArgument(Preconditions.java:148)
at org.sonar.api.batch.fs.internal.DefaultInputFile.checkValid(DefaultInputFile.java:218)
at org.sonar.api.batch.fs.internal.DefaultInputFile.newPointer(DefaultInputFile.java:209)
at org.sonar.api.batch.fs.internal.DefaultInputFile.newRange(DefaultInputFile.java:240)
at org.sonar.plugins.cxx.cpd.CxxCpdVisitor.visitToken(CxxCpdVisitor.java:78)
at com.sonar.sslr.impl.ast.AstWalker.visitToken(AstWalker.java:107)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:86)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.walkAndVisit(AstWalker.java:69)
at org.sonar.squidbridge.AstScanner.scanFiles(AstScanner.java:110)
... 31 more

The string in th cxx file is
sscanf( ligne, "%s%3s%d%d%lf%d%d%lf%lf%lf%lf%lf%lf%lf%lf%lf%lf"
"%d%d%d/%d/%d%d/%d/%d%d%d%d%d/%d/%d%d/%d/%d%d%d%d%d/%d/%d%d/%d/%d%d"
"%d%d%d/%d/%d%d/%d/%d%d%d%d%d/%d/%d%d/%d/%d%d%d%d%d/%d/%d%d/%d/%d%d%d\n", ...

The problem is that the string token is joined in one line and his length differ from the source line length.

If I remove JoinStringsPreprocessor.java this error disappear

guwirth · 2016-09-20T20:30:24Z

@robenj please try again with latest snapshot. #945 should catch such an exception.

scmbuildguy · 2016-09-20T23:34:57Z

Hi @guwirth,

I get the same error with the latest code.

This was working in v0.9.6 and now it doesn't seem to handle multiple line strings without some addtional syntax. Seems as though commit 3693569 could be an issue, however, that is only an assumption.

Example:

Original test code:
string str = "hello"
"world"

I fixed it by modifying:
string str = "hello"
<< "world"

I have a lot of code with the original format, so I am hoping this can be fixed.

Best Regards,

--Mark

benrsx · 2016-09-21T09:28:37Z

@guwirth I tried with the latest snapshot, I can see the exception for highligthing

Highligthing error in file '/var/projects/sonar/x/x/x/x/x.cxx' at line:1089, column:31

but my problem come just after

guwirth · 2016-09-21T11:22:59Z

Hi @robenj did a short test guwirth@3f3a591. It's working?

benrsx · 2016-09-21T13:05:29Z

I did a sonar scan on your test file after a clone and a build of the branch refactor/highlighting and I get the same error :

DEBUG: Highligthing error in file '/home/webadm/sonar-cxx-refactor-highlighting/sonar-cxx-plugin/src/test/resources/org/sonar/plugins/cxx/highlighter.cc' at line:79, column:21
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 5.722s
INFO: Final Memory: 40M/116M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarQube Scanner execution
org.sonar.squidbridge.api.AnalysisException: Unable to analyze file: /home/webadm/sonar-cxx-refactor-highlighting/sonar-cxx-plugin/src/test/resources/org/sonar/plugins/cxx/highlighter.cc
at org.sonar.squidbridge.AstScanner.scanFiles(AstScanner.java:131)
at org.sonar.plugins.cxx.squid.CxxSquidSensor.execute(CxxSquidSensor.java:128)
at org.sonar.batch.sensor.SensorWrapper.analyse(SensorWrapper.java:57)
at org.sonar.batch.phases.SensorsExecutor.executeSensor(SensorsExecutor.java:58)
at org.sonar.batch.phases.SensorsExecutor.execute(SensorsExecutor.java:50)
at org.sonar.batch.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:83)
at org.sonar.batch.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:192)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.batch.scan.ProjectScanContainer.scan(ProjectScanContainer.java:241)
at org.sonar.batch.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:236)
at org.sonar.batch.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:226)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.batch.task.ScanTask.execute(ScanTask.java:47)
at org.sonar.batch.task.TaskContainer.doAfterStart(TaskContainer.java:86)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.batch.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:106)
at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:119)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:244)
at org.sonarsource.scanner.api.EmbeddedScanner.runAnalysis(EmbeddedScanner.java:154)
at org.sonarsource.scanner.cli.Main.runAnalysis(Main.java:110)
at org.sonarsource.scanner.cli.Main.execute(Main.java:72)
at org.sonarsource.scanner.cli.Main.main(Main.java:60)
Caused by: java.lang.IllegalArgumentException: 35 is not a valid line offset for pointer. File [moduleKey=PLUGIN, relative=resources/org/sonar/plugins/cxx/highlighter.cc, basedir=/home/webadm/sonar-cxx-refactor-highlighting/sonar-cxx-plugin/src/test] has 29 character(s) at line 79
at org.sonar.api.internal.google.common.base.Preconditions.checkArgument(Preconditions.java:148)
at org.sonar.api.batch.fs.internal.DefaultInputFile.checkValid(DefaultInputFile.java:218)
at org.sonar.api.batch.fs.internal.DefaultInputFile.newPointer(DefaultInputFile.java:209)
at org.sonar.api.batch.fs.internal.DefaultInputFile.newRange(DefaultInputFile.java:240)
at org.sonar.plugins.cxx.cpd.CxxCpdVisitor.visitToken(CxxCpdVisitor.java:82)
at com.sonar.sslr.impl.ast.AstWalker.visitToken(AstWalker.java:107)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:86)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.visitChildren(AstWalker.java:99)
at com.sonar.sslr.impl.ast.AstWalker.visit(AstWalker.java:87)
at com.sonar.sslr.impl.ast.AstWalker.walkAndVisit(AstWalker.java:69)
at org.sonar.squidbridge.AstScanner.scanFiles(AstScanner.java:110)
... 31 more

guwirth · 2016-09-21T13:37:30Z

Hi there is

DEBUG: Highligthing error in file '/home/webadm/sonar-cxx-refactor-highlighting/sonar-cxx-plugin/src/test/resources/org/sonar/plugins/cxx/highlighter.cc' at line:79, column:21

which is in code https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/sonar-cxx-plugin/src/main/java/org/sonar/plugins/cxx/highlighter/CxxHighlighter.java#L153. So exception is handled and only a debug log file item. Don't know why scanner is saying failure?

benrsx · 2016-09-21T14:35:59Z

The problem come from the token length in the file CxxCpdVisitor.java

After adding a try/catch the scan run without errors :

  try {
      TextRange range = inputFile.newRange(token.getLine(), token.getColumn(), token.getLine(), token.getColumn() + token.getValue().length());
      cpdTokens.addToken(range, text);
  }catch (Exception e){
      LOG.debug("Token error at line: {}, columns: {}, {} is not a valid line offset for pointer",token.getLine(),token.getColumn(),token.getColumn() + token.getValue().length());
  }

guwirth · 2016-09-21T15:36:52Z

Hi @robenj txh for the hint. Sorry my fault: mixed up the cpd and the highlighting fix - but rout cause is the same. Error is here: https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/sonar-cxx-plugin/src/main/java/org/sonar/plugins/cxx/cpd/CxxCpdVisitor.java#L82. Will add a try/catch to avoid this.

Better solution would be to handle this inside of preprocessor. Point is only that current preprocessor is deprecated and I'm not sure if it is worth to spend the effort.

guwirth added the question label Sep 20, 2016

guwirth added this to the 0.9.7 milestone Sep 20, 2016

guwirth self-assigned this Sep 20, 2016

guwirth added bug and removed question labels Sep 21, 2016

guwirth changed the title ~~IllegalArgumentException: 208 is not a valid line offset for pointer~~ cpd visitor: IllegalArgumentException: 208 is not a valid line offset for pointer Sep 21, 2016

guwirth mentioned this issue Sep 21, 2016

CxxCpdVisitor: ignore range errors #954

Merged

guwirth closed this as completed in #954 Sep 21, 2016

jmecosta mentioned this issue Oct 3, 2016

Vote: default behavior invalid issues and messages #758

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cpd visitor: IllegalArgumentException: 208 is not a valid line offset for pointer #952

cpd visitor: IllegalArgumentException: 208 is not a valid line offset for pointer #952

benrsx commented Sep 20, 2016

guwirth commented Sep 20, 2016

scmbuildguy commented Sep 20, 2016

benrsx commented Sep 21, 2016

guwirth commented Sep 21, 2016

benrsx commented Sep 21, 2016

guwirth commented Sep 21, 2016

benrsx commented Sep 21, 2016

guwirth commented Sep 21, 2016

cpd visitor: IllegalArgumentException: 208 is not a valid line offset for pointer #952

cpd visitor: IllegalArgumentException: 208 is not a valid line offset for pointer #952

Comments

benrsx commented Sep 20, 2016

guwirth commented Sep 20, 2016

scmbuildguy commented Sep 20, 2016

benrsx commented Sep 21, 2016

guwirth commented Sep 21, 2016

benrsx commented Sep 21, 2016

guwirth commented Sep 21, 2016

benrsx commented Sep 21, 2016

guwirth commented Sep 21, 2016