[Bug]: core dump with BitArchiveWriter

[gbook]

bit7z version

4.0.x

Compilation options

BIT7Z_AUTO_FORMAT

7-zip version

v23.01

7-zip shared library used

7z.dll / 7z.so

Compilers

GCC

Compiler versions

No response

Architecture

x86_64

Operating system

Linux

Operating system versions

RHEL 8

Bug description

The following code, specifically the BitArchiveWriter.compressTo() function,

    try {
        using namespace bit7z;
#ifdef Q_OS_WINDOWS
        Bit7zLibrary lib("C:/Program Files/7-Zip/7z.dll");
#else
        Bit7zLibrary lib("/usr/libexec/p7zip/7z.so");
#endif
        if (archivePath.endsWith(".zip", Qt::CaseInsensitive)) {
            BitArchiveWriter archive(lib, BitFormat::Zip);
            archive.setUpdateMode(UpdateMode::Update);
            archive.addDirectory(dir.toStdString());
            archive.compressTo(archivePath.toStdString());
        }
        else {
            BitArchiveWriter archive(lib, BitFormat::SevenZip);
            archive.setUpdateMode(UpdateMode::Update);
            archive.addDirectory(dir.toStdString());
            archive.compressTo(archivePath.toStdString()); /* <----- core dump ----- */
        }
        m = "Successfully compressed directory [" + dir + "] to archive [" + archivePath + "]";
        return true;
    }
    catch ( const bit7z::BitException& ex ) {
        /* Do something with ex.what()...*/
        m = "Unable to compress directory into archive using bit7z library [" + QString(ex.what()) + "]";
        return false;
    }

produces the following error at runtime

free(): double free detected in tcache 2
Aborted (core dumped)

Steps to reproduce

No response

Expected behavior

No response

Relevant compilation output

No response

Code of Conduct

• By submitting this issue, I agree to follow bit7z's Code of Conduct

[rikyoz]

Hi!
I see from the code that you're using p7zip's 7z.so on Linux.
By default, bit7z uses the latest 7-Zip implementation of the Windows COM interfaces, which is incompatible with p7zip's (since 7-Zip v23.01).
This leads to problems like yours, where the 7z.so uses one implementation, and bit7z another, resulting in a crash.
You can build bit7z to be compatible with the p7zip's 7z.so by enabling the CMake option BIT7Z_USE_LEGACY_IUNKNOWN; this should fix your problem.

[gbook]

Thank you for the reply. Unfortunately that did not fix it. I rebuilt bit7z with the BIT7Z_USE_LEGACY_IUNKNOWN flag set to true, but I still get the same error.

…

On Fri, May 3, 2024 at 3:28 PM Riccardo ***@***.***> wrote: Hi! I see from the code that you're using p7zip's 7z.so on Linux. By default, bit7z uses the latest 7-Zip implementation of the Windows COM interfaces, which is incompatible with p7zip's (at least since 7-Zip v23.01). This leads to problems like yours, where the 7z.so uses one implementation, and bit7z another, resulting in a crash. You can build bit7z to be compatible with the p7zip's 7z.so by enabling the CMake option BIT7Z_USE_LEGACY_IUNKNOWN; this should fix your problem. — Reply to this email directly, view it on GitHub <#206 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7K5B5QXMSQNLJOU2PVK3DZAPQNDAVCNFSM6AAAAABHF7LSAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJTGY2DANJWGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rikyoz]

I see.
I'll try to replicate the issue

[gbook]

I'm on RHEL8 if that helps at all.

…

On Fri, May 3, 2024, 4:45 PM Riccardo ***@***.***> wrote: I see. I'll try to replicate the issue — Reply to this email directly, view it on GitHub <#206 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7K5B7SBFR7Y7IAFG4R2IDZAPZPRAVCNFSM6AAAAABHF7LSAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJTG4ZTQMZWGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rikyoz]

Unfortunately, I wasn't able to replicate the issue.

Specifically:

• As I don't have access to RHEL 8, I've set up a VM running Fedora 28 (which is the base for RHEL 8).

• For the tests, I've used the default GCC version (8.3.1), the default p7zip's 7z.so (package p7zip-plugins), and the default Qt 5 development packages provided on Fedora 28.

• Inside the VM, I've built bit7z as follows:

  git clone https://github.com/rikyoz/bit7z && cd bit7z
  mkdir build && cd build
  cmake .. -DBIT7Z_AUTO_FORMAT=ON -DBIT7Z_USE_LEGACY_IUNKNOWN=ON
  cmake --build . --config Release
  

• Always inside the VM, I created a barebone Qt 5 Widgets application project in Qt Creator that uses the bit7z library built as in the previous step.

  • I've done this to try replicate your issue, as it seems that your code is part of a Qt application.

• My app GUI looks like this:
  

  • The first QLineEdit and QPushButton couple allows to select the directory to be compressed.
  • The second one allows to select the destination archive.
  • The last QPushButton executes your code, which I wrapped inside a method of a dummy Compressor class:

  // compressor.h
  class Compressor {
      QString m;
    public:
      const QString& resultMessage() {
          return m;
      }
  
      bool compressDirectory(const QString& dir, const QString& archivePath) {
          try {
              using namespace bit7z;
      #ifdef Q_OS_WINDOWS
              Bit7zLibrary lib("C:/Program Files/7-Zip/7z.dll");
      #else
              Bit7zLibrary lib("/usr/libexec/p7zip/7z.so");
      #endif
              if (archivePath.endsWith(".zip", Qt::CaseInsensitive)) {
                  BitArchiveWriter archive(lib, BitFormat::Zip);
                  archive.setUpdateMode(UpdateMode::Update);
                  archive.addDirectory(dir.toStdString());
                  archive.compressTo(archivePath.toStdString());
              }
              else {
                  BitArchiveWriter archive(lib, BitFormat::SevenZip);
                  archive.setUpdateMode(UpdateMode::Update);
                  archive.addDirectory(dir.toStdString());
                  archive.compressTo(archivePath.toStdString());
              }
              m = "Successfully compressed directory [" + dir + "] to archive [" + archivePath + "]";
              return true;
          }
          catch ( const bit7z::BitException& ex ) {
              /* Do something with ex.what()...*/
              m = "Unable to compress directory into archive using bit7z library [" + QString(ex.what()) + "]";
              return false;
          }
      }
  };
  
  // mainwindow.cpp
  void MainWindow::on_pushButton_clicked() {
      Compressor compressor;
      bool result = compressor.compressDirectory(ui->directoryLineEdit->text(), ui->archiveLineEdit->text());
      if (result) {
          QMessageBox::information(this, tr("Compression successful"), compressor.resultMessage(), QMessageBox::Ok);
      } else {
          QMessageBox::critical(this, tr("Compression error"), compressor.resultMessage(), QMessageBox::Ok);
      }
  }

• I've tested the code both by creating a Zip and a 7z archive, and it worked in both cases, without any crash:
  
  

[gbook]

That is very helpful! At least I know it works. I'll keep trying on my end.

…

On Sat, May 4, 2024, 1:12 PM Riccardo ***@***.***> wrote: Unfortunately, I wasn't able to replicate the issue. Specifically: - As I don't have access to RHEL 8, I've set up a VM running Fedora 28 (which is the base for RHEL 8). - For the tests, I've used the default GCC version (8.3.1), the default p7zip's 7z.so (package p7zip-plugins), and the default Qt 5 development packages provided on Fedora 28. - Inside the VM, I've built bit7z as follows: git clone https://github.com/rikyoz/bit7z && cd bit7z mkdir build && cd build cmake .. -DBIT7Z_AUTO_FORMAT=ON -DBIT7Z_USE_LEGACY_IUNKNOWN=ON cmake --build . --config Release - Always inside the VM, I created a barebone Qt 5 Widgets application project in Qt Creator that uses the bit7z library built as in the previous step. - I've done this to try replicate your issue, as it seems that your code is part of a Qt application. - My app GUI looks like this: image.png (view on web) <https://github.com/rikyoz/bit7z/assets/1334291/6b71ea1d-596f-429f-bccc-06bcd39b4e88> - The first QLineEdit and QPushButton couple allows to select the directory to be compressed. - The second one allows to select the destination archive. - The last QPushButton executes your code, which I wrapped inside a method of a dummy Compressor class: // compressor.hclass Compressor { QString m; public: const QString& resultMessage() { return m; } bool compressDirectory(const QString& dir, const QString& archivePath) { try { using namespace bit7z; #ifdef Q_OS_WINDOWS Bit7zLibrary lib("C:/Program Files/7-Zip/7z.dll"); #else Bit7zLibrary lib("/usr/libexec/p7zip/7z.so"); #endif if (archivePath.endsWith(".zip", Qt::CaseInsensitive)) { BitArchiveWriter archive(lib, BitFormat::Zip); archive.setUpdateMode(UpdateMode::Update); archive.addDirectory(dir.toStdString()); archive.compressTo(archivePath.toStdString()); } else { BitArchiveWriter archive(lib, BitFormat::SevenZip); archive.setUpdateMode(UpdateMode::Update); archive.addDirectory(dir.toStdString()); archive.compressTo(archivePath.toStdString()); } m = "Successfully compressed directory [" + dir + "] to archive [" + archivePath + "]"; return true; } catch ( const bit7z::BitException& ex ) { /* Do something with ex.what()...*/ m = "Unable to compress directory into archive using bit7z library [" + QString(ex.what()) + "]"; return false; } } }; // mainwindow.cppvoid MainWindow::on_pushButton_clicked() { Compressor compressor; bool result = compressor.compressDirectory(ui->directoryLineEdit->text(), ui->archiveLineEdit->text()); if (result) { QMessageBox::information(this, tr("Compression successful"), compressor.resultMessage(), QMessageBox::Ok); } else { QMessageBox::critical(this, tr("Compression error"), compressor.resultMessage(), QMessageBox::Ok); } } - I've tested the code both by creating a Zip and a 7z archive, and it worked in both cases, without any crash: image.png (view on web) <https://github.com/rikyoz/bit7z/assets/1334291/d8bbf14e-2fde-4620-840d-17af8b7bd01e> image.png (view on web) <https://github.com/rikyoz/bit7z/assets/1334291/83e5bef4-3125-434f-800a-aa1535ed454f> — Reply to this email directly, view it on GitHub <#206 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7K5BZCCWSOTXNJNBZ6AX3ZAUJINAVCNFSM6AAAAABHF7LSAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUGMYDMMZUGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[gbook]

It worked! (once) I was using the 4.0.4 code and I updated to the 4.0.6, rebuilt it, and it worked. My program built, it ran, and created a .7z file successfully.

However, after that one time working, now my program doesn't build correctly :-( I don't know what changed, but now every time I build my program against the bit7z library I get this error

linking nidb
../../bin/squirrel//libsquirrel.so: undefined reference to `bit7z::BitInputArchive::ConstIterator::operator*()'
collect2: error: ld returned 1 exit status
make: *** [Makefile:353: nidb] Error 1

In my case nidb depends on libsquirrel which depends on libbit7z64. I'm able to build the libsquirrel library, which depends on bit7z. But when trying to build any executable that depends on bit7z, I get link errors like these:

linking squirrel
squirrel.o: In function `squirrel::RemoveDirectoryFromArchive(QString, QString, QString&)':
squirrel.cpp:(.text+0x18171): undefined reference to `bit7z::BitInputArchive::ConstIterator::operator*()'
squirrel.cpp:(.text+0x18399): undefined reference to `bit7z::BitInputArchive::ConstIterator::operator*()'
squirrel.o: In function `bit7z::BitExtractor<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>::extractMatching(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned char, std::allocator<unsigned char> >&, bit7z::FilterPolicy) const':
squirrel.cpp:(.text._ZNK5bit7z12BitExtractorIRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEE15extractMatchingES8_S8_RSt6vectorIhSaIhEENS_12FilterPolicyE[_ZNK5bit7z12BitExtractorIRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEE15extractMatchingES8_S8_RSt6vectorIhSaIhEENS_12FilterPolicyE]+0xaf): undefined reference to `bit7z::BitInputArchive::ConstIterator::operator*()'
collect2: error: ld returned 1 exit status
make: *** [Makefile:313: squirrel] Error 1

[gbook]

Now it works, consistently. Building bit7z from scratch solved my problem. Thank you!

[rikyoz]

Building bit7z from scratch solved my problem.

The recent v4.0.6 fixed the operator* of the BitInputArchive's iterator, which couldn't be called on const iterator objects.

There was probably a mismatch between your bit7z's headers and binary, which was fixed by rebuilding it.

Thank you!

You're welcome!
Thank you for using bit7z! 🙏