TypeError: Cannot read property 'userTags' of undefined

[tgurr]

Expected behavior

Display the report.

Actual behavior

When trying to access a report, e.g. https://greenbone.local/report/3c7cc9bb-7133-4362-be24-6f6a112d058c

An error occurred on this page.
Please try again.
Hide Error Details

is displayed.

TypeError: Cannot read property 'userTags' of undefined

    in Unknown
    in Unknown
    in n
    in w
    in Unknown
    in n
    in Unknown
    in n
    in n
    in Unknown
    in Tg
    in Unknown
    in w
    in a
    in n
    in Unknown
    in Unknown
    in t
    in t
    in n
    in main
    in t
    in Main
    in Unknown
    in t
    in withLayout(Main)
    in div
    in Unknown
    in t
    in Styled(Layout)
    in n
    in withRouter(n)
    in Unknown
    in n
    in w
    in withRouter(Connect(n))
    in Unknown
    in n
    in w
    in withRouter(Connect(n))
    in Unknown
    in t
    in t
    in Unknown
    in Unknown
    in n
    in w
    in l
    in n
    in n
TypeError: Cannot read property 'userTags' of undefined
    at https://greenbone.local/static/js/main.572708e3.chunk.js:1:1530033
    at Gi (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:684338)
    at bc (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:730750)
    at us (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:723277)
    at cs (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:723202)
    at Zc (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:720211)
    at https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:671596
    at t.unstable_runWithPriority (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:747183)
    at Yo (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:671305)
    at Vo (https://greenbone.local/static/js/2.d9a8af99.chunk.js:2:671541)

Additionally I also run into another similiar issue which is already reported: #2389

Steps to reproduce

1. Upgrade from GVM 11 to 20.08
2. Additionally keep the old /usr/share/gvm/gvmd/report_formats like stated in the release notes
3. Try to access a report, it doesn't matter if it's an old one from GVM 11 or one created by 20.08
4. See the error message

Note that it tries to search for a non-existing

md manage:WARNING:2020-08-24 13h19.27 CEST:9033: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/generate

in the logs.

GVM versions

gsa: (gsad --version)
Greenbone Security Assistant 20.08.0

gvm: (gvmd --version)
Greenbone Vulnerability Manager 20.08.0
Manager DB revision 233

openvas-scanner: (openvassd --version)
OpenVAS 20.8.0

gvm-libs:
20.8.0

Environment

Operating system:
Linux exherbo 5.6.14 #1 SMP Fri May 22 11:12:32 CEST 2020 x86_64 GNU/Linux

Installation method / source: (packages, source installation)
packages (but source based distribution)

Logfiles

md manage:WARNING:2020-08-24 13h19.27 CEST:9033: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/generate

[cfi-gb]

I have seen something similar if the "Feed Import Owner" wasn't set at all:

https://github.com/greenbone/gvmd/blob/v20.8.0/INSTALL.md#set-the-feed-import-owner

or the migration of the feed objects/data has failed (e.g. due to permission issues or similar).

#2367 had improved the logging for me and had shown the correct message but this might be a separate case here.

[tgurr]

I've already set the "Feed Import Owner" as stated in the release/upgrade notes for 20.08 and I did so again now just to be sure, it doesn't change anything for me.

I also tried the latest code of the gsa-20.08 branch (8fad03c) which includes #2367 that you mentioned, but I still get the same error in the web interface and the logs like in my initial report.

Is the already mentioned md manage:WARNING:2020-08-27 11h42.04 CEST:8585: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/generate I have in my logs everytime I try to access a report of any relevance here? Which part of GVM is expected to ship that file?

[bjoernricks]

@tgurr are you able to get the XML output for one of the reports? Either via gvm-cli or via the network tab of your browser?

[stefanost67]

Same happens for me if I try to access a report. I get the error described here: #2389 if I just try to access the reports page. Reports downloaded via gmv-cli are OK.

[bjoernricks]

This issue is about a crash at the report details page (a single report) right? In this case there must be something wrong with the returned XML by gvmd. Therefore we need the XML output of <get_reports report_id="..." filter="details=1"/>.

[stefanost67]

Here is the output:

<get_reports_response status="200" status_text="OK">admin2020-08-27T10:57:19Z<creation_time>2020-08-27T10:57:19Z</creation_time><modification_time>2020-08-27T11:11:54Z</modification_time>0<in_use>0</in_use>Immediate scan of IP 172.17.9.62<report_format id="a994b278-1f62-11e1-96ac-406186ea4fc5">XML</report_format>apply_overrides=0 min_qod=70 first=1 rows=10 sort=nameapply_overrides=0min_qod=70first=1rows=10sort=namenameascending<report_count>211</report_count></get_reports_response>

[bjoernricks]

that's all? nothing else? Is the XML report format available at the report formats list?

Oh I did a mistake in the XML requests. It should be

<get_reports report_id="..." details="1"/>

[stefanost67]

There isn't much more... am I doing something wrong then?

<get_reports_response status="200" status_text="OK">admin2020-08-27T10:57:19Z<creation_time>2020-08-27T10:57:19Z</creation_time><modification_time>2020-08-27T11:11:54Z</modification_time>0<in_use>0</in_use>Immediate scan of IP 172.17.9.62<report_format id="a994b278-1f62-11e1-96ac-406186ea4fc5">XML</report_format>apply_overrides=0 min_qod=70 first=1 rows=10 sort=nameapply_overrides=0min_qod=70first=1rows=10sort=namenameascending<report_count>211</report_count></get_reports_response>

[bjoernricks]

It's missing the whole report. Personally I think it is because the XML report format isn't migrated correctly but I am not sure about that.

[stefanost67]

I see the real results in the database. So I first have to isolate the xml problem I guess

[tgurr]

Guess you're few steps further already, I'm still trying to battle with gvm-cli. In the browser via
https://domain.local/gmp?token=b76c381d-cb82-4460-adda-a761e2da88c1&cmd=get_report&report_id=4229b14c-b771-4695-b3a7-f10e70130694&details=1

I get

<envelope><version>20.08.0</version><vendor_version></vendor_version><token>b76c381d-cb82-4460-adda-a761e2da88c1</token><time>Thu Aug 27 13:45:18 2020 CEST</time><timezone>Europe/Berlin</timezone><login>admin</login><session>1598529461</session><role>Admin</role><severity>nist</severity><i18n>de</i18n><client_address>xxx.xxx.xxx.xxx</client_address><backend_operation>0.07</backend_operation><get_report><get_reports_response status="200" status_text="OK"><report id="4229b14c-b771-4695-b3a7-f10e70130694" format_id="a994b278-1f62-11e1-96ac-406186ea4fc5" extension="xml" content_type="text/xml"><owner><name>admin</name></owner><name>2020-08-24T12:05:33Z</name><comment></comment><creation_time>2020-08-24T12:05:33Z</creation_time><modification_time>2020-08-24T12:17:33Z</modification_time><writable>0</writable><in_use>0</in_use><task id="d651a833-c233-4518-a952-21d69bd0cb56"><name>host.domain.local</name></task><report_format id="a994b278-1f62-11e1-96ac-406186ea4fc5"><name>XML</name></report_format></report><filters id=""><term>apply_overrides=0 min_qod=70 first=1 rows=50 sort=name</term><keywords><keyword><column>apply_overrides</column><relation>=</relation><value>0</value></keyword><keyword><column>min_qod</column><relation>=</relation><value>70</value></keyword><keyword><column>first</column><relation>=</relation><value>1</value></keyword><keyword><column>rows</column><relation>=</relation><value>50</value></keyword><keyword><column>sort</column><relation>=</relation><value>name</value></keyword></keywords></filters><sort><field>name<order>ascending</order></field></sort><reports start="1" max="1000"/><report_count>30<filtered>1</filtered><page>1</page></report_count></get_reports_response></get_report></envelope>

I didn't have success with gvm-cli yet:
su -c "gvm-cli --gmp-username xxx --gmp-password xxx socket --socketpath /run/gvm/gvmd.sock --xml \"<get_reports report_id="4229b14c-b771-4695-b3a7-f10e70130694" details="1"/>\"" -s /bin/bash gvm
which results in
Invalid XML

but since it seems to work for @stefanost67 and in the browser I'm probably just quoting something wrong(?).

I also do have real results in the database and I can also view them via the /results page and setting a filter for the specific host.

[stefanost67]

Yes, I guess the quotes are wrong.
See this:
sudo -u gvm gvm-cli --gmp-username=xxxxx --gmp-password=xxxxxx --protocol GMP socket --socketpath /usr/local/var/run/gvmd.sock --xml "<get_reports report_id="177a3e68-b6fd-484d-ab60-c09fb3a3509e" format_id="c1645568-627a-11e3-a660-406186ea4fc5" details="1"/>"

[stefanost67]

The backslashes are hidden in the comment. I guess you'll see them in the e-mail

[bjoernricks]

Btw. gvm-cli has a --pretty option to format the XML response.

[tgurr]

Still no luck for me with gvm-cli sorry, still getting Invalid XML everytime:
sudo -u gvm gvm-cli --gmp-username=xxx --gmp-password=xxx --protocol GMP socket --socketpath /run/gvm/gvmd.sock --xml "<get_reports report_id="4229b14c-b771-4695-b3a7-f10e70130694" format_id="c1645568-627a-11e3-a660-406186ea4fc5" details="1"/>"

[stefanost67]

In order to show the correct escaping I have to copy the string as a code:
sudo -u gvm gvm-cli --gmp-username=xxxxx --gmp-password=xxxxxx --protocol GMP socket --socketpath /usr/local/var/run/gvmd.sock --xml "<get_reports report_id=\"177a3e68-b6fd-484d-ab60-c09fb3a3509e\" format_id=\"c1645568-627a-11e3-a660-406186ea4fc5\" details=\"1\"/>"

[Virsacer]

My report looks like this:

<get_reports_response status="200" status_text="OK">
  <report id="442e3a6b-048b-4d62-bdfe-e2c3964d0e53" format_id="a994b278-1f62-11e1-96ac-406186ea4fc5" extension="xml" content_type="text/xml">
    <owner>
      <name>admin</name>
    </owner>
    <name>2020-08-26T21:22:53Z</name>
    <comment/>
    <creation_time>2020-08-26T21:22:53Z</creation_time>
    <modification_time/>
    <writable>0</writable>
    <in_use>0</in_use>
    <task id="4c55118c-6340-46e8-85d8-ae5f4f0c6cfb">
      <name>TEST</name>
    </task>
    <report_format id="a994b278-1f62-11e1-96ac-406186ea4fc5">
      <name>XML</name>
    </report_format>
  </report>
  <filters id="">
    <term>apply_overrides=0 min_qod=70 first=1 rows=10 sort=name</term>
    <keywords>
      <keyword>
        <column>apply_overrides</column>
        <relation>=</relation>
        <value>0</value>
      </keyword>
      <keyword>
        <column>min_qod</column>
        <relation>=</relation>
        <value>70</value>
      </keyword>
      <keyword>
        <column>first</column>
        <relation>=</relation>
        <value>1</value>
      </keyword>
      <keyword>
        <column>rows</column>
        <relation>=</relation>
        <value>10</value>
      </keyword>
      <keyword>
        <column>sort</column>
        <relation>=</relation>
        <value>name</value>
      </keyword>
    </keywords>
  </filters>
  <sort>
    <field>name<order>ascending</order></field>
  </sort>
  <reports start="1" max="1000"/>
  <report_count>1<filtered>1</filtered><page>1</page></report_count>
</get_reports_response>

When using details="1", I get "Remote closed the connection"

[tgurr]

@stefanost67 thanks a lot, my bad sorry. Now I get output as well, with the mentioned --pretty flag:

<get_reports_response status="200" status_text="OK">
  <report id="4229b14c-b771-4695-b3a7-f10e70130694" format_id="c1645568-627a-11e3-a660-406186ea4fc5" extension="csv" content_type="text/csv">
    <owner>
      <name>admin</name>
    </owner>
    <name>2020-08-24T12:05:33Z</name>
    <comment/>
    <creation_time>2020-08-24T12:05:33Z</creation_time>
    <modification_time>2020-08-24T12:17:33Z</modification_time>
    <writable>0</writable>
    <in_use>0</in_use>
    <task id="d651a833-c233-4518-a952-21d69bd0cb56">
      <name>host.domain.local</name>
    </task>
    <report_format id="c1645568-627a-11e3-a660-406186ea4fc5">
      <name>CSV Results</name>
    </report_format>
  </report>
  <filters id="">
    <term>apply_overrides=0 min_qod=70 first=1 rows=50 sort=name</term>
    <keywords>
      <keyword>
        <column>apply_overrides</column>
        <relation>=</relation>
        <value>0</value>
      </keyword>
      <keyword>
        <column>min_qod</column>
        <relation>=</relation>
        <value>70</value>
      </keyword>
      <keyword>
        <column>first</column>
        <relation>=</relation>
        <value>1</value>
      </keyword>
      <keyword>
        <column>rows</column>
        <relation>=</relation>
        <value>50</value>
      </keyword>
      <keyword>
        <column>sort</column>
        <relation>=</relation>
        <value>name</value>
      </keyword>
    </keywords>
  </filters>
  <sort>
    <field>name<order>ascending</order></field>
  </sort>
  <reports start="1" max="1000"/>
  <report_count>30<filtered>1</filtered><page>1</page></report_count>
</get_reports_response>

so it's also missing to include the actual report for me and is identical to the one I get via the browser tools.

[stefanost67]

so, is this a bug in gvmd or are we doing something wrong?

[bjoernricks]

so, is this a bug in gvmd or are we doing something wrong?

Personally I can't answer this question at the moment. It will take some time to do further investigation.

[stefanost67]

hello, I do not know what happened, I rebuilt my docker image on a new VM, ran my docker-compose on it and now reports are working.
I'll let you know as soon as/if I understand the root case of the issue we were having.

[timopollmeier]

Are you using a different user as the feed owner and for accessing the report?
If so, is the report user able to access the XML report format on the report formats page or with the <get_report_formats/> GMP command?

[stefanost67]

Hello, I am not sure I understand what happened, but now I can correctly see and download reports on two different, twin systems.
I suspect I had an issue on the docker-compose network setup, but I can not tell for sure.

[stefanost67]

report retrieval works both via gvm-tools and via gsa.

[Virsacer]

Do you use the released versions or the git branches?
I have rebuilt everything (from the released versions), created a new database with only one user but the problem persists...

Did you migrate existing data?

[tgurr]

Are you using a different user as the feed owner and for accessing the report?

For me, no, I currently only have a single user in my setup. That one is admin and this is the one I set as the Feed Import Owner.

[stefanost67]

Do you use the released versions or the git branches?

I use the 20.08 git branch

I have rebuilt everything (from the released versions), created a new database with only one user but the problem persists...

Did you migrate existing data?

no, I build everything from scratch

[tgurr]

So since noone gave my question regarding the actual error that I have in my log
md manage:WARNING:2020-08-27 11h42.04 CEST:8585: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/generate
any attention I now just copied the contents of /var/lib/gvm/gvmd/report_formats/a994b278-1f62-11e1-96ac-406186ea4fc5/ which seems to be the report format for xml into /var/lib/gvm/gvmd/report_formats/ where gvmd complains it's missing a generate script and bingo I get proper output from gvm-cli as well as being able to browse the results via gsad again. Doesn't sound like a solution for me at all, but maybe it can help for further investigating of the root cause.

[bjoernricks]

So since noone gave my question regarding the actual error that I have in my log
md manage:WARNING:2020-08-27 11h42.04 CEST:8585: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/generate
any attention I now just copied the contents of /var/lib/gvm/gvmd/report_formats/a994b278-1f62-11e1-96ac-406186ea4fc5/ which seems to be the report format for xml into /var/lib/gvm/gvmd/report_formats/ where gvmd complains it's missing a generate script and bingo I get proper output from gvm-cli as well as being able to browse the results via gsad again. Doesn't sound like a solution for me at all, but maybe it can help for further investigating of the root cause.

Sounds like

The migration from a previous GVM release to 20.08 requires the report_formats of the previous release kept at gvm/gvmd/report_formats/*. Please make sure to keep the files and folders there when doing the upgrade.

to me

[timopollmeier]

Are you able to see report formats, regardless of whether you are using multiple users or not?

[tgurr]

@bjoernricks I'm pretty sure I followed the upgrade notes and copied over the old ones, however even if not, it should't just search in the root path instead of any uuid subdirectory or am I missing something?

@timopollmeier in gsad it looks good I think

edit: output of sudo -u gvm gvm-cli --gmp-username=xxx --gmp-password=xxx --protocol GMP socket --socketpath /run/gvm/gvmd.sock --xml "<get_report_formats/>" --pretty

<get_report_formats_response status="200" status_text="OK">
  <report_format id="5057e5cc-b825-11e4-9d0e-28d24461215b">
    <owner>
      <name/>
    </owner>
    <name>Anonymous XML</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>xml</extension>
    <content_type>text/xml</content_type>
    <summary>Anonymous version of the raw XML report</summary>
    <description>Complete scan report in GMP XML format.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="910200ca-dc05-11e1-954f-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>ARF</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>xml</extension>
    <content_type>text/xml</content_type>
    <summary>Asset Reporting Format v1.0.0.</summary>
    <description>NIST Asset Reporting Format 1.1 compliant document.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="5ceff8ba-1f62-11e1-ab9f-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>CPE</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>csv</extension>
    <content_type>text/csv</content_type>
    <summary>Common Platform Enumeration CSV table.</summary>
    <description>CPE stands for Common Platform Enumeration.  It is a structured naming scheme for
information technology systems, platforms, and packages.  In other words: CPE
provides a unique identifier for virtually any software product that is known for
a vulnerability.

The CPE dictionary is maintained by MITRE and NIST.  MITRE also maintains CVE
(Common Vulnerability Enumeration) and other relevant security standards.

The report selects all CPE tables from the results and forms a single table
as a comma separated values file.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="9087b18c-626c-11e3-8892-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>CSV Hosts</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>csv</extension>
    <content_type>text/csv</content_type>
    <summary>CSV host summary.</summary>
    <description>Base host information and result counts as comma separated values. Text fields starting with "=", "@", "+", "-" will have a single quote "'" added so spreadsheet software does not interpret them as formulas.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="c1645568-627a-11e3-a660-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>CSV Results</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>csv</extension>
    <content_type>text/csv</content_type>
    <summary>CSV result list.</summary>
    <description>List of results as comma separated values. Text fields starting with "=", "@", "+", "-" will have a single quote "'" added so spreadsheet software does not interpret them as formulas.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="77bd6c4a-1f62-11e1-abf0-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>ITG</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>csv</extension>
    <content_type>text/csv</content_type>
    <summary>German "IT-Grundschutz-Kataloge" report.</summary>
    <description>Tabular report on the German "IT-Grundschutz-Kataloge",
as published and maintained by the German Federal Agency for IT-Security.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="a684c02c-b531-11e1-bdc2-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>LaTeX</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>tex</extension>
    <content_type>text/plain</content_type>
    <summary>LaTeX source file.</summary>
    <description>Report as LaTeX source file for further processing.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="9ca6fe72-1f62-11e1-9e7c-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>NBE</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>nbe</extension>
    <content_type>text/plain</content_type>
    <summary>Legacy OpenVAS report.</summary>
    <description>The traditional OpenVAS Scanner text based format.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="c402cc3e-b531-11e1-9163-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>PDF</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>pdf</extension>
    <content_type>application/pdf</content_type>
    <summary>Portable Document Format report.</summary>
    <description>Scan results in Portable Document Format (PDF).</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="9e5e5deb-879e-4ecc-8be6-a71cd0875cdd">
    <owner>
      <name/>
    </owner>
    <name>Topology SVG</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>svg</extension>
    <content_type>image/svg+xml</content_type>
    <summary>Network topology SVG image.</summary>
    <description>Scan results in topologic structure as scalable vector graphics.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="a3810a62-1f62-11e1-9219-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>TXT</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>txt</extension>
    <content_type>text/plain</content_type>
    <summary>Plain text report.</summary>
    <description>Plain text report, best viewed with fixed font size.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="c15ad349-bd8d-457a-880a-c7056532ee15">
    <owner>
      <name/>
    </owner>
    <name>Verinice ISM</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2020-05-19T10:14:19Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>vna</extension>
    <content_type>application/zip</content_type>
    <summary>Greenbone Verinice ISM Report, v3.2.0.</summary>
    <description>Information Security Management Report for Verinice import, version 3.2.0.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="50c9950a-f326-11e4-800c-28d24461215b">
    <owner>
      <name/>
    </owner>
    <name>Verinice ITG</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>vna</extension>
    <content_type>application/zip</content_type>
    <summary>Greenbone Verinice ITG Report, v1.0.1.</summary>
    <description>IT-Grundschutz Report for Verinice import, version 1.0.1.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <report_format id="a994b278-1f62-11e1-96ac-406186ea4fc5">
    <owner>
      <name/>
    </owner>
    <name>XML</name>
    <comment/>
    <creation_time>2019-10-21T12:12:47Z</creation_time>
    <modification_time>2019-10-21T12:12:47Z</modification_time>
    <writable>1</writable>
    <in_use>0</in_use>
    <permissions>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
      <permission>
        <name>get_report_formats</name>
      </permission>
    </permissions>
    <extension>xml</extension>
    <content_type>text/xml</content_type>
    <summary>Raw XML report.</summary>
    <description>Anonymized scan report in GMP XML format.</description>
    <predefined>1</predefined>
    <trust>yes<time>2020-07-30T18:43:23+02:00</time></trust>
    <active>1</active>
  </report_format>
  <filters id="">
    <term>first=1 rows=50 sort=name</term>
    <keywords>
      <keyword>
        <column>first</column>
        <relation>=</relation>
        <value>1</value>
      </keyword>
      <keyword>
        <column>rows</column>
        <relation>=</relation>
        <value>50</value>
      </keyword>
      <keyword>
        <column>sort</column>
        <relation>=</relation>
        <value>name</value>
      </keyword>
    </keywords>
  </filters>
  <sort>
    <field>name<order>ascending</order></field>
  </sort>
  <report_formats start="1" max="1000"/>
  <report_format_count>14<filtered>14</filtered><page>14</page></report_format_count>
</get_report_formats_response>

[Virsacer]

Sounds like

The migration from a previous GVM release to 20.08 requires the report_formats of the previous release kept at gvm/gvmd/report_formats/*. Please make sure to keep the files and folders there when doing the upgrade.

to me

On a new installation it should definitely not be neccessary to "keep" files from another version...

Now I rebuilt everything using the git branches, new DB as always and I see this in the log:

event report_format:MESSAGE:2020-08-31 18h57.19 utc:135: Report format XML (a994b278-1f62-11e1-96ac-406186ea4fc5) has been created by admin

Also copied "generate" and "report_format.xml" from

/gvm/var/lib/gvm/gvmd/report_formats/43baba63-37de-4d54-9bb1-1c7074e4845a/a994b278-1f62-11e1-96ac-406186ea4fc5

to

/gvm/var/lib/gvm/gvmd/report_formats

but still not working :-/

[mattmundell]

Is the already mentioned md manage:WARNING:2020-08-27 11h42.04 CEST:8585: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/generate I have in my logs everytime I try to access a report of any relevance here?

This path is not correct. gvmd should always be looking for "generate" in a dir with a UUID. I'm not sure why the UUID is missing, will look closer.

I now just copied the contents of /var/lib/gvm/gvmd/report_formats/a994b278-1f62-11e1-96ac-406186ea4fc5/ which seems to be the report format for xml into /var/lib/gvm/gvmd/report_formats/ where gvmd complains it's missing a generate script and bingo

Sounds like you've just masked the error, and the other report formats won't work.

[mattmundell]

I'm not sure why the UUID is missing, will look closer.

Oh, this happens when the report format has no owner, which means your report formats have not been migrated.

You can see in the GSA image the little sunglass icon, which means that the user does not own the report format.

I guess something is going wrong when you set the Feed Import Owner. Any logs when you do that? Are you using the right UUID?

[tgurr]

I guess something is going wrong when you set the Feed Import Owner. Any logs when you do that? Are you using the right UUID?

su -c "gvmd --get-users --verbose" -s /bin/bash gvm
admin 3794f4cf-c16b-45b2-a47f-7e0e6a385250

md   main:MESSAGE:2020-09-01 13h49.06 utc:2414308:    Greenbone Vulnerability Manager version 20.08.0 (DB revision 233)
md manage:   INFO:2020-09-01 13h49.06 utc:2414308:    Getting users.

su -c "gvmd --modify-setting 78eceaec-3385-11ea-b237-28d24461215b --value 3794f4cf-c16b-45b2-a47f-7e0e6a385250" -s /bin/bash gvm

runs without any error (would be great if this could be fixed to fail if there's an error or at least show some error). The logs however show the actual problem:

md   main:MESSAGE:2020-09-01 13h49.43 utc:2415553:    Greenbone Vulnerability Manager version 20.08.0 (DB revision 233)
md manage:   INFO:2020-09-01 13h49.43 utc:2415553:    Modifying setting.
md manage:WARNING:2020-09-01 13h49.44 utc:2415553: move_report_format_dir: rename /usr/share/gvm/gvmd/report_formats/c1645568-627a-11e3-a660-406186ea4fc5 to /var/lib/gvm/gvmd/report_formats/3794f4cf-c16b-45b2-a47f-7e0e6a385250/c1645568-627a-11e3-a660-406186ea4fc5
md manage:WARNING:2020-09-01 13h49.44 utc:2415553: move_report_format_dir: rename /usr/share/gvm/gvmd/report_formats/c1645568-627a-11e3-a660-406186ea4fc5 to /var/lib/gvm/gvmd/report_formats/3794f4cf-c16b-45b2-a47f-7e0e6a385250/c1645568-627a-11e3-a660-406186ea4fc5: Permission denied
md manage:WARNING:2020-09-01 13h49.44 utc:2415553: migrate_predefined_report_formats: failed at report format c1645568-627a-11e3-a660-406186ea4fc5

That makes perfect sense, files in /usr/share are usually not owned by any user, of course also not the the gvm user. Shouldn't the migration rather try to copy them to /var/lib/gvm/gvmd/report_formats instead of trying to rename/move them?

[mattmundell]

@Virsacer I'm not sure you're seeing the same thing.

On a new installation it should definitely not be neccessary to "keep" files from another version...

For a fresh install it is not required.

But be sure to remove any old files if you are trying a fresh install.

Now I rebuilt everything using the git branches, new DB as always and I see this in the log:

event report_format:MESSAGE:2020-08-31 18h57.19 utc:135: Report format XML (a994b278-1f62-11e1-96ac-406186ea4fc5) has been created by admin

Sounds good.

Also copied "generate" and "report_format.xml" from

/gvm/var/lib/gvm/gvmd/report_formats/43baba63-37de-4d54-9bb1-1c7074e4845a/a994b278-1f62-11e1-96ac-406186ea4fc5

to

/gvm/var/lib/gvm/gvmd/report_formats

Please don't do this, it's just going to cause confusion.

but still not working :-/

You don't give details. I think you should even create a new issue.

[Virsacer]

@mattmundell Actually I did: #2389
It was mentioned in the first post but conversation got pace here and the root-cause seems to be the same:
Incomplete report returned by "get_reports" -> #2418 (comment)

[mattmundell]

Shouldn't the migration rather try to copy them to /var/lib/gvm/gvmd/report_formats instead of trying to rename/move them?

Yes, good idea. Done in greenbone/gvmd#1286.

EDIT: corrected PR

[mattmundell]

It was mentioned in the first post but conversation got pace here and the root-cause seems to be the same:

OK, I see. But it's a different issue, because you get a successful Report format XML (a994b278-1f62-11e1-96ac-406186ea4fc5) has been created by admin message, and you get the report XML back, whereas tgurr was not getting the report XML.

When using details="1", I get "Remote closed the connection"

This looks like a problem. Check your log when you give details="1". Something is probably failing there.

[Virsacer]

When using details="1", I get "Remote closed the connection"

This looks like a problem. Check your log when you give details="1". Something is probably failing there.

Well this is strange:

md manage:WARNING:2020-09-02 00h30.53 CEST:720: sql_exec_internal: PQexec failed: ERROR:  could not load library "/gvm/lib/libgvm-pg-server.so": libgvm_base.so.20: cannot open shared object file: No such file or directory
 (7)
md manage:WARNING:2020-09-02 00h30.53 CEST:720: sql_exec_internal: SQL: WITH permissions_subject     AS (SELECT * FROM permissions         WHERE subject_location               = 0         AND ((subject_type = 'user'               AND subject                   = (SELECT id FROM users WHERE users.uuid = '9285fecd-e048-4655-aeb0-4415d05da5fc'))              OR (subject_type = 'group'                  AND subject                      IN (SELECT DISTINCT "group"                          FROM group_users                          WHERE "user"                                = (SELECT id FROM users WHERE users.uuid = '9285fecd-e048-4655-aeb0-4415d05da5fc')))              OR (subject_type = 'role'                  AND subject                      IN (SELECT DISTINCT role                          FROM role_users                          WHERE "user"                                = (SELECT id FROM users WHERE users.uuid = '9285fecd-e048-4655-aeb0-4415d05da5fc'))))),     super_on_users     AS (SELECT DISTINCT *         FROM (SELECT resource FROM permissions_subject               WHERE name = 'Super'               AND resource_type = 'user'               UNION               SELECT "user" FROM role_users               WHERE role                     IN (SELECT resource                         FROM permissions_subject                         WHERE name = 'Super'                         AND resource_type = 'role')               UNION               SELECT "user" FROM group_users               WHERE "group"                     IN (SELECT resource                         FROM permissions_subject                         WHERE name = 'Super'                         AND resource_type = 'group'))              AS all_users)SELECT results.id, results.uuid, nvts.name AS name, '' AS comment,  iso_time (date, opts.user_zone) AS creation_time,  iso_time (date, opts.user_zone) AS modification_time, date AS created, date AS modified, (SELECT name FROM users WHERE users.id = results.owner) AS _owner, results.owner, host, port AS location, nvt, severity_to_type (severity) AS original_type, severity_to_type ((SELECT new_severity FROM result_new_severities                  WHERE result_new_severities.result = results.id                  AND result_new_severities.user = opts.user_id                  AND result_new_severities.override = opts.override                  AND result_new_severities.dynamic = opts.dynamic                  LIMIT 1)) AS type, (SELECT autofp FROM results_autofp WHERE (result = results.id) AND (autofp_selection = opts.autofp)) AS auto_type, description, task, report AS report_rowid, (SELECT cvss_base FROM nvts WHERE nvts.oid =  nvt) AS cvss_base, nvt_version, severity AS original_severity, (SELECT new_severity FROM result_new_severities WHERE result_new_severities.result = results.id AND result_new_severities.user = opts.user_id AND result_new_severities.override = opts.override AND result_new_severities.dynamic = opts.dynamic LIMIT 1) AS severity, nvts.name AS vulnerability, date, (SELECT uuid FROM reports WHERE id = report) AS report_id, nvts.solution_type AS solution_type, results.qod AS qod, results.qod_type, (CASE WHEN (hostname IS NULL) OR (hostname = '') THEN (SELECT value FROM report_host_details       WHERE name = 'hostname'         AND report_host = (SELECT id FROM report_hosts                            WHERE report_hosts.host=results.host                            AND report_hosts.report = results.report)       LIMIT 1) ELSE hostname END) AS hostname, (SELECT uuid FROM tasks WHERE id = task) AS task_id, nvts.cve AS cve, path, (SELECT CASE WHEN host IS NULL             THEN NULL             ELSE (SELECT uuid FROM hosts                   WHERE id = (SELECT host FROM host_identifiers                               WHERE source_type = 'Report Host'                               AND name = 'ip'                               AND source_id                                   = (SELECT uuid                                      FROM reports                                      WHERE id = results.report)                               AND value = results.host                               LIMIT 1))             END), (SELECT CASE        WHEN EXISTS (SELECT * FROM notes                     WHERE (result = results.id                            OR (result = 0 AND nvt = results.nvt))                     AND (task = 0 OR task = results.task))        THEN 1        ELSE 0        END), (SELECT CASE        WHEN EXISTS (SELECT * FROM overrides                     WHERE (result = results.id                            OR (result = 0 AND nvt = results.nvt))                     AND (task = 0 OR task = results.task))        THEN 1        ELSE 0        END), (SELECT EXISTS (SELECT * FROM tickets                WHERE id IN (SELECT ticket FROM ticket_results                             WHERE result = results.id                             AND result_location                                 = 0))), nvts.summary, nvts.insight, nvts.affected, nvts.impact, nvts.solution, nvts.detection, nvts.family, nvts.tag, (ARRAY (SELECT name::text        FROM cert_bund_advs        WHERE id IN (SELECT adv_id FROM cert_bund_cves                     WHERE cve_name IN (SELECT ref_id                                        FROM vt_refs                                        WHERE vt_oid = results.nvt                                        AND type = 'cve'))        ORDER BY name DESC)), (ARRAY (SELECT name::text        FROM dfn_cert_advs        WHERE id IN (SELECT adv_id FROM dfn_cert_cves                     WHERE cve_name IN (SELECT ref_id                                        FROM vt_refs                                        WHERE vt_oid = results.nvt                                        AND type = 'cve'))        ORDER BY name DESC)) FROM results  LEFT OUTER JOIN nvts ON results.nvt = nvts.oid , (SELECT   'user_zone'::text AS user_zone,   1 AS user_id,   0 AS autofp,   0 AS override,   0 AS dynamic) AS opts WHERE  ((results.owner   = (SELECT id FROM users WHERE users.uuid = '9285fecd-e048-4655-aeb0-4415d05da5fc'))  OR EXISTS (SELECT * FROM permissions_subject             WHERE name = 'Super'             AND (resource = 0))  OR results.owner IN (SELECT *                     FROM super_on_users)  OR EXISTS (SELECT id FROM permissions_subject  WHERE resource = results.id  AND resource_type = 'result'  AND resource_location = 0  AND (t ())) OR EXISTS (SELECT id FROM permissions_subject  WHERE resource = results.task  AND resource_type = 'task'  AND (t ()))) AND (report = 1)  AND severity != -3.0 AND (results.qod >= CAST (70 AS INTEGER)) ORDER BY lower (nvts.name) ASC LIMIT 10 OFFSET 0;
md manage:WARNING:2020-09-02 00h30.53 CEST:720: next: sql_exec_internal failed

[tgurr]

Shouldn't the migration rather try to copy them to /var/lib/gvm/gvmd/report_formats instead of trying to rename/move them?

Yes, good idea. Done in greenbone/gvmd#1285.

Correct PR seems to be greenbone/gvmd#1286. After applying the changes I was able to successfully run the migration this time and at least for me the initial report seems to be fixed with that. Thanks to everyone involved!

[mattmundell]

md manage:WARNING:2020-09-02 00h30.53 CEST:720: sql_exec_internal: PQexec failed: ERROR: could not load library "/gvm/lib/libgvm-pg-server.so": libgvm_base.so.20: cannot open shared object file: No such file or directory (7)

The means Postgres can't find the gvmd server-side library.

Best solution is probably to add /gvm/lib/ to /etc/ld.so.conf and then run ldconfig (as root). See number 6 here.

This serve-side library is a pain but there's not an easy way for us to get rid of it.

[mattmundell]

Closing because @tgurr confirmed it working.

[BasvanH]

@mattmundell, is there a way to re-generate the report_formats folder? I had removed my /opt/gvm folder and retained my database, and then did a new build of 20.08. Now I get the TypeError: Cannot read property 'userTags' of undefined error when opening a report.

[BasvanH]

@mattmundell, is there a way to re-generate the report_formats folder? I had removed my /opt/gvm folder and retained my database, and then did a new build of 20.08. Now I get the TypeError: Cannot read property 'userTags' of undefined error when opening a report.

I got them back from backup and run the Feed owner ship. Now it's working again.