Some new logs cannot be dumped with archive logs

[Gomeow]

Hello everyone,

I have some problem with archive evtx logs on windows server 2016(1607).

Some new logs cannot be shown when using evtxdump.

How it happens: Install one 2016 server, set 'Security Log' to Archieve when is full. Then dump the 'Security.evtx' without open it in eventviewer.
I found out the logs canbe correctly dumped when double click on the evtx file (open it with windows eventviewer).
And also the file was changed only by open it with windows eventviewer.(Some times the evtx file sizes changed at all.)

The different dump only with opend the file:

Here is the evtx file.
Security.zip

Thanks.

[qjerome]

Hi @Gomeow,

Did you think about comparing the number of events shown in Event Viewer and the number of event dumped with evtxdump ? If that number is the same for a given file it is likely the issue is not on evtxdump side.

Please let me know what is the result of this comparison.

Cheers,

[Gomeow]

Hi @qjerome ,

They do have difference and here are some snap. Just using the file uploaded before.(with 1.2.2 release)

There should be 460 recoreds in the evtx file.

Thanks.

[qjerome]

Thank you for your feedback. This is indeed very strange.
Let me investigate this and I will revert back to you.

[qjerome]

It seems the Security.evtx file is not consistent.
There is a dump of the file header:

File Header: Security.evtx

Magic: ElfFile
FirstChunkNum: 0
LastChunkNum: 3
NumNextRecord: 320
HeaderSpace: 128
MinVersion: 0x0001
MaxVersion: 0x0003
SizeHeader: 4096
ChunkCount: 4
Flags: 0x00000001
CheckSum: 0x62778716

You can see that there are 4 chunks, a chunk being a structure where several events are encoded.
On the other hand, if we use the carving feature of evtxdump that is looking for chunk header magic number and parse everything looking like a chunk, we see that 6 chunks are actually parsed, and you actually get the expected number of events.

./evtxdump-386 -c Security.evtx | wc -l
2020/07/09 14:01:21 INFO - Parsing Chunk @ Offset: 4096 (0x00001000)
2020/07/09 14:01:22 INFO - Parsing Chunk @ Offset: 69632 (0x00011000)
2020/07/09 14:01:22 INFO - Parsing Chunk @ Offset: 135168 (0x00021000)
2020/07/09 14:01:22 INFO - Parsing Chunk @ Offset: 200704 (0x00031000)
2020/07/09 14:01:22 INFO - Parsing Chunk @ Offset: 266240 (0x00041000)
2020/07/09 14:01:22 INFO - Parsing Chunk @ Offset: 331776 (0x00051000)
460 --> that is the result of wc, so the count of events

So my theory is the following, your file has a wrong header for a reason I ignore. When you open it in Event Viewer, it sees that there is an error and fixes the header. That is why after you have opened the file, it gets modified on disk and then the number of event is correct with evtxdump. So I would not qualify this issue as a bug in evtxdump because the tool parses the data correctly with the data it has, it is just that the header is incorrect. I will not provide any fix to address that issue since there is already the carving option to force chunk parsing when that is needed. Though I am considering implementing a verify option to diagnose this kind of issue more easily.

Thanks for reporting that issue.

[qjerome]

Hey @Gomeow,

After a deeper analysis I noticed that your file was in a "dirty" state.
This information is available in the Flags member of the File Header structure.
So I fixed evtxdump so that when the file is in "dirty" state the Header gets repaired before further processing.
You can use the last release of evtxdump and you will get the expected number of events.

Thanks again for reporting this issue that allowed me to improve the tool.