Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lack of documentation #42

Open
JayFoxRox opened this issue Nov 1, 2017 · 8 comments
Open

Lack of documentation #42

JayFoxRox opened this issue Nov 1, 2017 · 8 comments

Comments

@JayFoxRox
Copy link

This is hardly documented. I could not make it work as I'm not sure which version of construct I need. I also don't know wether this is for python2 or python3.

Some examples also depend on the pefile module which is not listed.

The README should be updated and mention installation instructions including all dependencies and their respective versions.

@moyix
Copy link
Owner

moyix commented Nov 1, 2017

Excellent suggestions. Maybe you could make a pull request?

@JayFoxRox
Copy link
Author

I'd do it, but I couldn't make it work at all. So I also can't tell others how to use it.

I only have debug symbols of Xbox executables in PDB2 format (stored in *.idb files). I'm not even sure if pdbparse would support them if pdbparse was working for me.

My experience so far has been:

  • Install pdbparse through pip: pdbparse wouldn't detect the PDB2 files as the file magic was a binary string to standard string comparison which failed.
  • Pull git version and install contruct: construct 2.8+ does not work (See Construct 2.8+ unsupported #41 ), so I followed the 2.8 transition guide for about 30 minutes and then gave up.
  • Now pull construct 2.5.5 but pdbparse would still have problems with dbihdr = DBIHeader.parse_stream(stream) in dbi.py. This caused me to give up finally. I already had construct uninstalled so I couldn't even make an issue about this.
  • I don't really know what each example does either, so I didn't know what to expect.
  • Also because pefile did not load .xbe (.exe for Xbox) I couldn't try at least one of the examples, even after transitioning some code to construct 2.8+ (hackily, as I never worked with construct). This issue is specific to me, but I wouldn't know what to expect when making a workaround for pefile. So lots of upfront work I was not willing to invest.

I'm on Linux and don't have any other way to look at these files.
Basically I'm just desperate to know what's in those debug files.
However, I'm not desperate enough to spend more than 2 hours on this without being guaranteed that it will eventually do exactly what it has to do. I need pdbparse to work on another project, so spending too much time with pdbparse is not an option.

I just wanted to leave my feedback here, in the hopes someone who knows more about pdbparse and construct would fix it.
If I had a quick way to at least have a stable pdbparse setup[1] I'd probably contribute back a patch or two. But the initial trouble I already had has exhausted me for now.

It's really a shame because pdbparse seems to be the only good documentation and tool to work with these files. A lot of work has gone into it and it's basically hindered by a lack of documentation and dependency on a legacy lib.

[1] stable setup; that is: base functionality working or at least knowledge that bugs I'm having are not caused by dependencies or local issues

@moyix
Copy link
Owner

moyix commented Nov 1, 2017

It used to be true that pdbparse was the only tool to work with PDBs, but that's not true any more. Microsoft released the source code to their PDB library, and LLVM now has an implementation of a PDB parser as well: https://llvm.org/docs/CommandGuide/llvm-pdbutil.html

I agree that it would be great to upgrade to a newer version of Construct, but realistically I'm not likely to have time to do this any time soon. So patches are appreciated, but otherwise it is probably simpler to just install an older version of Construct.

I also have never encountered the Xbox idb files – are you sure these are really PDB files?

@JayFoxRox
Copy link
Author

I can't get llvm-pdbutil working either. It says llvm-pdbutil: MSF Error: The data is in an unexpected format. MSF magic header doesn't match, so I assume this only works for PDB7, not for the older PDB format. I'd assume the same for the MS library.
A python solution would also be best for me as the rest of my tools are also written in python.

Also, yes: I'm fairly certain these files are pdb files. The magic matches and these files are called "vc70.idb". The Original Xbox Development Kit (XDK) was based on Microsoft Visual C++ 6.0 (and later .NET 2003 and 2005. Not sure about .NET 2002). The Original Xbox is running a modified Windows 2000 kernel with staticly linked executables (PE files, but with slightly different header, therefore XBE instead of EXE).


It would already be immensely usefull if pdbparse contained example pdb files and / or example output + a listing of what each sample does.

Also I'm still not sure wether it even works with Construct 2.5.5 or if I have to go back even further.

@moyix
Copy link
Owner

moyix commented Nov 2, 2017

Construct 2.5.2 and lower should work (this is the version that's specified in setup.py, FWIW: https://github.com/moyix/pdbparse/blob/master/setup.py#L13). Getting it through git and doing python setup.py install should have pulled in the right version automatically.

If possible can you share a sample of one of these idb files? The PDB2 support is pretty old (they stopped using in the rest of Windows it after Windows 2000; I'm really surprised that the xbox used it), so there might be some easily-fixed bugs in there.

@mrolle45
Copy link

You can look at the magic signature of your vc70.idb file by saying type vc70.idb on Windows, or using the strings command on unix/linux. A PDB V2 file should come out as Microsoft C/C++ program database 2.00. If you look at the file in a text editor, binary dump, etc., the file will begin with Microsoft C/C++ program database 2.00\r\n\032JG\0\0\0.
I've found that pdbparse with construct 2.5.5 works for me with normal V7 PDB files.

@rohaaan
Copy link

rohaaan commented Apr 23, 2018

FYI
use command pip install construct==2.5.2
the setup.py says construct <= 2.5.2

@rohaaan
Copy link

rohaaan commented Apr 23, 2018

FYI
use command pip install pefile==2016.3.28
setup.py does not mention it but this version works flawless

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants