Merge pull request #425 from Washi1337/development

5.2.0

Directory.Build.props

@@ -1,13 +1,13 @@
 <Project>
 
     <PropertyGroup>
-        <Copyright>Copyright © Washi 2016-2022</Copyright>
+        <Copyright>Copyright © Washi 2016-2023</Copyright>
         <PackageProjectUrl>https://github.com/Washi1337/AsmResolver</PackageProjectUrl>
         <PackageLicense>https://github.com/Washi1337/AsmResolver/LICENSE.md</PackageLicense>
         <RepositoryUrl>https://github.com/Washi1337/AsmResolver</RepositoryUrl>
         <RepositoryType>git</RepositoryType>
         <LangVersion>10</LangVersion>
-        <Version>5.1.0</Version>
+        <Version>5.2.0</Version>
     </PropertyGroup>
 
 </Project>

appveyor.yml

@@ -4,7 +4,7 @@
       - master
 
   image: Visual Studio 2022
-  version: 5.1.0-master-build.{build}
+  version: 5.2.0-master-build.{build}
   configuration: Release
 
   skip_commits:
@@ -33,7 +33,7 @@
       - development
 
   image: Visual Studio 2022
-  version: 5.1.0-dev-build.{build}
+  version: 5.2.0-dev-build.{build}
   configuration: Release
 
   skip_commits:

docs/dotnet/bundles.rst

@@ -3,7 +3,7 @@ AppHost / SingleFileHost Bundles
 
 Since the release of .NET Core 3.1, it is possible to deploy .NET assemblies as a single binary. These files are executables that do not contain a traditional .NET metadata header, and run natively on the underlying operating system via a platform-specific application host bootstrapper.
 
-AsmResolver supports extracting the embedded files from these types of binaries. Additionally, given an application host template provided by the .NET SDK, AsmResolver also supports constructing new bundles as well. All relevant code is found in the following namespace:
+AsmResolver supports extracting the embedded files from these types of binaries. Additionally, given the original file or an application host template provided by the .NET SDK, AsmResolver also supports constructing new bundles as well. All relevant code is found in the following namespace:
 
 .. code-block:: csharp
 
@@ -96,14 +96,14 @@ Constructing new bundled executable files requires a template file that AsmResol
 - ``<DOTNET-INSTALLATION-PATH>/sdk/<version>/AppHostTemplate``
 - ``<DOTNET-INSTALLATION-PATH>/packs/Microsoft.NETCore.App.Host.<runtime-identifier>/<version>/runtimes/<runtime-identifier>/native``
 
-Using this template file, it is then possible to write a new bundled executable file using ``WriteUsingTemplate``:
+Using this template file, it is then possible to write a new bundled executable file using ``WriteUsingTemplate`` and the ``BundlerParameters::FromTemplate`` method:
 
 .. code-block:: csharp
 
     BundleManifest manifest = ...
     manifest.WriteUsingTemplate(
         @"C:\Path\To\Output\File.exe",
-        new BundlerParameters(
+        BundlerParameters.FromTemplate(
             appHostTemplatePath: @"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Host.win-x64\6.0.0\runtimes\win-x64\native\apphost.exe",
             appBinaryPath: @"HelloWorld.dll"));
 
@@ -117,12 +117,38 @@ For bundle executable files targeting Windows, it may be required to copy over s
     BundleManifest manifest = ...
     manifest.WriteUsingTemplate(
         @"C:\Path\To\Output\File.exe",
-        new BundlerParameters(
+        BundlerParameters.FromTemplate(
             appHostTemplatePath: @"C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Host.win-x64\6.0.0\runtimes\win-x64\native\apphost.exe",
             appBinaryPath: @"HelloWorld.dll",
             imagePathToCopyHeadersFrom: @"C:\Path\To\Original\HelloWorld.exe"));
 
-``BundleManifest`` also defines other ```WriteUsingTemplate`` overloads taking ``byte[]``, ``IDataSource`` or ``IPEImage`` instances instead of paths.
+
+If you do not have access to a template file (e.g., if the SDK is not installed) but have another existing PE file that was packaged in a similar fashion, it is then possible to use this file as a template instead by extracting the bundler parameters using the ``BundlerParameters::FromExistingBundle`` method. This is in particularly useful when trying to patch existing AppHost bundles:
+
+.. code-block:: csharp
+
+    string inputPath = @"C:\Path\To\Bundled\HelloWorld.exe";
+    string outputPath = Path.ChangeExtension(inputPath, ".patched.exe");
+
+    // Read manifest.
+    var manifest = BundleManifest.FromFile(inputPath);
+
+    /* ... Make changes to manifest and its files ... */ 
+    
+    // Repackage bundle using existing bundle as template.
+    manifest.WriteUsingTemplate(
+        outputPath, 
+        BundlerParameters.FromExistingBundle(
+            originalFile: inputPath, 
+            appBinaryPath: mainFile.RelativePath));
+
+
+.. warning::
+
+    The ``BundlerParameters.FromExistingBundle`` method applies heuristics on the input file to determine the parameters for patching the input file. As heuristics are not perfect, this is not guaranteed to always work.
+
+
+``BundleManifest`` and ``BundlerParameters`` also define overloads of the ``WriteUsingTemplate`` and ``FromTemplate`` / ``FromExistingBundle`` respectively, taking ``byte[]``, ``IDataSource`` or ``IPEImage`` instances instead of file paths.
 
 
 Managing Files

docs/dotnet/type-signatures.rst

@@ -11,8 +11,8 @@ All relevant classes in this document can be found in the following namespaces:
     using AsmResolver.DotNet.Signatures.Types;
 
 
-Overview of all Type Signatures 
--------------------------------
+Overview
+--------
 
 Basic leaf type signatures: 
 
@@ -78,8 +78,8 @@ Corlib type signatures can also be looked up by their element type, by their ful
 If an invalid element type, name or type descriptor is passed on, these methods return ``null``.
 
 
-TypeDefOrRefSignature
----------------------
+Class and Struct Types
+----------------------
 
 The ``TypeDefOrRefSignature`` class is used to reference types in either the ``TypeDef`` or ``TypeRef`` (and sometimes ``TypeSpec``) metadata table. 
 
@@ -101,8 +101,8 @@ Alternatively, ``CreateTypeReference`` can be used on any ``IResolutionScope``:
     While it is technically possible to reference a basic type such as ``System.Int32`` as a ``TypeDefOrRefSignature``, it renders the .NET module invalid by most implementations of the CLR. Always use the ``CorLibTypeSignature`` to reference basic types within your blob signatures.
 
 
-GenericInstanceTypeSignature
-----------------------------
+Generic Instance Types
+----------------------
 
 The ``GenericInstanceTypeSignature`` class is used to instantiate generic types with type arguments:
 
@@ -128,8 +128,8 @@ Alternatively, a generic instance can also be generated via the ``MakeGenericTyp
     // listOfString now contains a reference to List<string>.
 
 
-FunctionPointerTypeSignature
-----------------------------
+Function Pointer Types
+----------------------
 
 Function pointer signatures are strongly-typed pointer types used to describe addresses to functions or methods. In AsmResolver, they are represented using a ``MethodSignature``:
 
@@ -175,8 +175,8 @@ To quickly transform any ``ITypeDescriptor`` into a ``TypeSignature``, it is pos
 Likewise, a ``TypeSignature`` can also be converted back to a ``ITypeDefOrRef``, which can be referenced using a metadata token, using the ``TypeSignature.ToTypeDefOrRef()`` method.
 
 
-Decorating type signatures
---------------------------
+Decorating Types
+----------------
 
 Type signatures can be annotated with extra properties, such as an array or pointer specifier.
 
@@ -205,7 +205,7 @@ Below an overview of all factory shortcut methods:
 +===================================================================+==================================================================================================================+
 | ``MakeArrayType(int dimensionCount)``                             | Wraps the type in a new ``ArrayTypeSignature`` with ``dimensionCount`` zero based dimensions with no upperbound. |
 +-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
-| ``MakeArrayType(ArrayDimension[] dimensinos)``                    | Wraps the type in a new ``ArrayTypeSignature`` with ``dimensions`` set as dimensions                             |
+| ``MakeArrayType(ArrayDimension[] dimensions)``                    | Wraps the type in a new ``ArrayTypeSignature`` with ``dimensions`` set as dimensions                             |
 +-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
 | ``MakeByReferenceType()``                                         | Wraps the type in a new ``ByReferenceTypeSignature``                                                             |
 +-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
@@ -219,3 +219,70 @@ Below an overview of all factory shortcut methods:
 +-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
 | ``MakeGenericInstanceType(TypeSignature[] typeArguments)``        | Wraps the type in a new ``GenericInstanceTypeSignature`` with the provided type arguments.                       |
 +-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
+
+
+
+Comparing Types
+---------------
+
+Type signatures can be tested for semantic equivalence using the ``SignatureComparer`` class. 
+Most use-cases of this class will not require any customization. 
+In these cases, the default ``SignatureComparer`` can be used:
+
+.. code-block:: csharp
+
+    var comparer = SignatureComparer.Default;
+
+
+However, if you wish to configure the comparer (e.g., for relaxing some of the declaring assembly version comparison rules), it is possible to create a new instance instead:
+
+.. code-block:: csharp
+
+    var comparer = new SignatureComparer(SignatureComparisonFlags.AllowNewerVersions);
+
+
+Once a comparer is obtained, we can test for type equality using any of the overloaded ``Equals`` methods:
+
+.. code-block:: csharp
+
+    TypeSignature type1 = ...;
+    TypeSignature type2 = ...;
+
+    if (comparer.Equals(type1, type2)) 
+    {
+        // type1 and type2 are semantically equivalent.
+    }
+
+
+The ``SignatureComparer`` class implements various instances of the ``IEqualityComparer<T>`` interface, and as such, it can be used as a comparer for dictionaries and related types:
+
+.. code-block:: csharp
+
+    var dictionary = new Dictionary<TypeSignature, TValue>(comparer);
+
+
+.. note:: 
+
+    The ``SignatureComparer`` class also implements equality comparers for other kinds of metadata, such as field and method descriptors and their signatures.
+
+
+In some cases, however, exact type equivalence is too strict of a test.
+Since .NET facilitates an object oriented environment, many types will inherit or derive from each other, making it difficult to pinpoint exactly which types we would need to compare to test whether two types are compatible with each other.  
+
+Section I.8.7 of the ECMA-335 specification defines a set of rules that dictate whether values of a certain type are compatible with or assignable to variables of another type. 
+These rules are implemented in AsmResolver using the ``IsCompatibleWith`` and ``IsAssignableTo`` methods:
+
+.. code-block:: csharp
+
+    if (type1.IsCompatibleWith(type2)) 
+    {
+        // type1 can be converted to type2.
+    }
+
+
+.. code-block:: csharp
+
+    if (type1.IsAssignableTo(type2)) 
+    {
+        // Values of type1 can be assigned to variables of type2.
+    }

docs/index.rst

@@ -72,3 +72,14 @@ Table of Contents:
    dotnet/type-memory-layout
    dotnet/bundles
    dotnet/advanced-pe-image-building.rst
+
+
+
+.. toctree::
+   :maxdepth: 1
+   :caption: PDB Symbols
+   :name: sec-pdb
+
+   pdb/index
+   pdb/basics
+   pdb/symbols

docs/pdb/basics.rst

@@ -0,0 +1,113 @@
+Basic I/O
+=========
+
+Every PDB image interaction is done through classes defined by the ``AsmResolver.Symbols.Pdb`` namespace:
+
+.. code-block:: csharp
+
+    using AsmResolver.Symbols.Pdb;
+
+
+Creating a new PDB Image
+------------------------
+
+Creating a new image can be done by instantiating a ``PdbImage`` class:
+
+.. code-block:: csharp
+
+    var image = new PdbImage();
+
+
+Opening a PDB Image
+-------------------
+
+Opening a PDB Image can be done through one of the ``FromXXX`` methods from the ``PdbImage`` class:
+
+.. code-block:: csharp
+
+    byte[] raw = ...
+    var image = PdbImage.FromBytes(raw);
+    
+.. code-block:: csharp
+
+    var image = PdbImage.FromFile(@"C:\myfile.pdb");
+
+.. code-block:: csharp
+
+    MsfFile msfFile = ...
+    var image = PdbImage.FromFile(msfFile);
+
+.. code-block:: csharp
+
+    BinaryStreamReader reader = ...
+    var image = PdbImage.FromReader(reader);
+
+
+If you want to read large files (+100MB), consider using memory mapped I/O instead:
+
+.. code-block:: csharp
+
+    using var service = new MemoryMappedFileService();
+    var image = PdbImage.FromFile(service.OpenFile(@"C:\myfile.pdb"));
+
+
+Writing a PDB Image
+-------------------
+
+Writing PDB images directly is currently not supported yet, however there are plans to making this format fully serializable.
+
+
+Creating a new MSF File
+-----------------------
+
+Multi-Stream Format (MSF) files are files that form the backbone structure of all PDB images. 
+AsmResolver fully supports this lower level type of access to MSF files using the ``MsfFile`` class.
+
+To create a new MSF file, use one of its constructors:
+
+.. code-block:: csharp
+
+    var msfFile = new MsfFile();
+
+
+.. code-block:: csharp
+
+    var msfFile = new MsfFile(blockSize: 4096);
+
+
+Opening an MSF File
+-------------------
+
+Opening existing MSF files can be done in a very similar fashion as reading a PDB Image:
+
+.. code-block:: csharp
+
+    byte[] raw = ...
+    var msfFile = MsfFile.FromBytes(raw);
+
+.. code-block:: csharp
+
+    var msfFile = MsfFile.FromFile(@"C:\myfile.pdb");
+
+.. code-block:: csharp
+
+    BinaryStreamReader reader = ...
+    var msfFile = MsfFile.FromReader(reader);
+
+    
+Similar to reading PDB images, if you want to read large files (+100MB), consider using memory mapped I/O instead:
+
+.. code-block:: csharp
+
+    using var service = new MemoryMappedFileService();
+    var msfFile = MsfFile.FromFile(service.OpenFile(@"C:\myfile.pdb"));
+
+
+Writing an MSF File
+-------------------
+
+Writing an MSF file can be done through one of the ``Write`` method overloads.
+
+.. code-block:: csharp
+
+    msfFile.Write(@"C:\myfile.patched.pdb");

docs/pdb/index.rst

@@ -0,0 +1,14 @@
+Overview
+========
+
+The Program Database (PDB) file format is a format developed by Microsoft for storing debugging information about binary files. 
+PDBs are typically constructed based on the original source code the binary was compiled with, and lists various symbols that the source code defines and/or references.
+
+Since version 5.0, AsmResolver provides a work-in-progress implementation for reading (and sometimes writing) PDB files to allow for better analysis of compiled binaries. 
+This implementation is fully managed, and thus does not depend on libraries such as the Debug Interface Access (DIA) that only work on the Windows platform.
+Furthermore, this project also aims to provide additional documentation on the file format, to make it more accessible to other developers.
+
+.. warning:: 
+
+    As the PDB file format is not very well documented, and mostly is reverse engineered from the official implementation provided by Microsoft, not everything in this API is finalized or stable yet. 
+    As such, this part of AsmResolver's API is still likely to undergo some breaking changes as development continues.