Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bps patching bug #806

Closed
Mihara opened this issue Nov 5, 2023 · 7 comments · Fixed by #816
Closed

bps patching bug #806

Mihara opened this issue Nov 5, 2023 · 7 comments · Fixed by #816
Assignees
Labels
bug A confirmed issues that needs fixing

Comments

@Mihara
Copy link

Mihara commented Nov 5, 2023

Paste the command

That really shouldn't matter in this case, but if you insist.

#!/bin/bash

igir copy zip \
    --input "testcase" \
    --patch "testcase-patches" \
    --output "TestRoms" \
    -vvv

Describe the bug

I have attempted to automatically apply a large number of translation patches, so far only to SNES roms, and immediately ran into a problem: IPS patch files acquired from RomHacking.net, while they do list a specific rom they apply to, do not necessarily work on the rom listed as is: sometimes, they expect a header, even though versions listed in the No-Intro database never have them.

After I realized that, I spent most of a day converting 200+ patches acquired from RomHacking to BPS, by creating a headered or unheadered rom as required, applying the IPS, verifying that the result runs, and creating a BPS from that, using flips for all the stages of the process except adding a rom header. (That, I just did with a python script that launched flips.)

That took most of a day, but now I have a large number of BPS patches that definitely work with the rom files I actually have.

When attempting to run igir on the lot, however, I get garbled and broken patches. When applying patches manually with flips, they work -- but most importantly, SHA1 hashes of the files patched with igir an patched with flips are different, and that means there's definitely a bug somewhere.

Here's an example .bps, I have a lot more of those if you need any.

sha1sum says:

a56a550c460285d06fbbcf2d8e41690bd9b9ba13  46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) flips patched.sfc
0cc6d8ac6a80a9892467f3d03e4493772885c9aa  46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) igir patched.sfc
6a9c8fb28eb524fc26561f9e193380edab3a8a58  46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc

The hash of the file produced by igir seems to be consistent, at least, but invariably, it's the flips patched copy that runs, while the igir patched copy doesn't.

Expected behavior

Patching should be consistent with other tools making BPS files. Failing that, flips could be in the wrong, in which case I need to file a bug report elsewhere, but I find this somewhat less likely.

Debug logs

Nothing of interest in there:

 ______   ______   ______  _______  
|      \ /      \ |      \|       \ 
 \$$$$$$|  $$$$$$\ \$$$$$$| $$$$$$$\
  | $$  | $$|    \  | $$  | $$    $$   ROM collection manager
  | $$  | $$|    \  | $$  | $$    $$   https://igir.io/
  | $$  | $$ \$$$$  | $$  | $$$$$$$\
 _| $$_ | $$__| $$ _| $$_ | $$  | $$   v2.1.0
|   $$ \ \$$    $$|   $$ \| $$  | $$
 \$$$$$$  \$$$$$$  \$$$$$$ \$$   \$$


[14:21:12.195] WARN:  No DAT files provided, consider using some for the best results!
[14:21:12.196] INFO:  ROMScanner: scanning ROM files
[14:21:12.214] DEBUG: ROMScanner: found 1 ROM file
[14:21:12.238] INFO:  ROMScanner: done scanning ROM files
[14:21:12.238] INFO:  ROMHeaderProcessor: processing file headers
[14:21:12.251] TRACE: ROMHeaderProcessor: testcase/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: reading potentially headered file by filename: .smc
[14:21:12.256] TRACE: ROMHeaderProcessor: testcase/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: found non-applicable header by filename: .smc
[14:21:12.256] DEBUG: ROMHeaderProcessor: found headers in 0 ROMs
[14:21:12.256] INFO:  ROMHeaderProcessor: done processing file headers
[14:21:12.259] INFO:  FileIndexer: indexing 1 file
[14:21:12.266] DEBUG: FileIndexer: found 1 unique file
[14:21:12.266] INFO:  FileIndexer: done indexing files
Scanning for ROMs ... 1 file found
[14:21:12.267] INFO:  PatchScanner: scanning patch files
[14:21:12.270] DEBUG: PatchScanner: found 1 patch file
[14:21:12.303] TRACE: PatchScanner: testcase-patches/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) B090235A.bps: found patch by extension: object
[14:21:12.303] INFO:  PatchScanner: done scanning patch files
Scanning for patches ... 1 unique patch found
[14:21:12.304] INFO:  DATGameInferrer: inferring DATs for 1 ROM
[14:21:12.304] DEBUG: DATGameInferrer: inferred 1 DAT
[14:21:12.305] INFO:  DATGameInferrer: done inferring DATs
[14:21:12.305] INFO:  processing 1 DAT
[14:21:12.305] INFO:  DATParentInferrer: inferring parents for 1 game
[14:21:12.308] DEBUG: DATParentInferrer: testcase: grouped to 1 parent
[14:21:12.308] INFO:  DATParentInferrer: done inferring parents
[14:21:12.308] INFO:  DATMergerSplitter: testcase: merging & splitting
[14:21:12.308] DEBUG: DATMergerSplitter: testcase: DAT doesn't have parent/clone info, doing nothing
[14:21:12.308] INFO:  DATFilter: testcase: filtering DAT
[14:21:12.309] DEBUG: DATFilter: testcase: filtered to 1/1 game (1.5MiB)
[14:21:12.309] INFO:  DATFilter: testcase: done filtering DAT
[14:21:12.310] INFO:  CandidateGenerator: testcase: generating candidates
[14:21:12.318] TRACE: CandidateGenerator: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): found 1 candidate
[14:21:12.318] DEBUG: CandidateGenerator: testcase: generated 1.5MiB of 1 candidate for 1 parent
[14:21:12.318] INFO:  CandidateGenerator: testcase: done generating candidates
[14:21:12.318] INFO:  CandidatePatchGenerator: testcase: generating patched candidates
[14:21:12.319] DEBUG: CandidatePatchGenerator: testcase: 1 unique patches found
[14:21:12.320] INFO:  CandidatePatchGenerator: testcase: done generating patched candidates
[14:21:12.320] TRACE: CandidatePatchGenerator: testcase: testcase/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: patch candidate generated: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc
[14:21:12.321] INFO:  CandidatePreferer: testcase: filtering candidates
[14:21:12.322] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): 1 candidate before filtering
[14:21:12.322] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): 1 candidate after filtering
[14:21:12.322] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): 1 candidate before filtering
[14:21:12.322] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): 1 candidate after filtering
[14:21:12.322] DEBUG: CandidatePreferer: testcase: filtered to 3MiB of 2 candidates for 2 parents
[14:21:12.322] INFO:  CandidatePreferer: testcase: done filtering candidates
[14:21:12.322] INFO:  CandidatePostProcessor: testcase: processing candidates
[14:21:12.325] INFO:  CandidatePostProcessor: testcase: done processing candidates
[14:21:12.326] INFO:  CandidateMergeSplitValidator: testcase: validating merged & split ROM sets
[14:21:12.326] INFO:  CandidateMergeSplitValidator: testcase: done validating merged & split ROM sets
[14:21:12.327] INFO:  CandidateWriter: testcase: writing 2 candidates
[14:21:12.327] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): writing 1 candidate
[14:21:12.328] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): writing 1 candidate
[14:21:12.328] TRACE: CandidateWriter: testcase: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip: not overwriting existing zip
[14:21:12.328] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): no raw files to write
[14:21:12.328] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): done writing 1 candidate
[14:21:12.328] TRACE: CandidateWriter: testcase: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: not overwriting existing zip
[14:21:12.328] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): no raw files to write
[14:21:12.328] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): done writing 1 candidate
[14:21:12.328] INFO:  CandidateWriter: testcase: done writing 2 candidates
[14:21:12.329] INFO:  StatusGenerator: testcase: generating ROM statuses
[14:21:12.331] INFO:  StatusGenerator: testcase: done generating ROM statuses
testcase ... 1 games, 1 retail releases, 1 patched games written
[14:21:12.331] INFO:  done processing 1 DAT

DAT(s) used

No response

igir version

v2.1.0

Node.js version

v20.8.1

Operating system

Ubuntu 22.04.3 LTS

Additional context

In addition, some of the zipped patched roms produced are missing file extensions -- dots in the middle of the filename are apparently involved -- and the rom name G.O.D - Mezame yo to Yobu Koe ga Kikoe (Japan).zip confuses the patcher enough that there's a stray O.zip in the result.

@Mihara Mihara added the potential-bug A potential issue that needs confirmation and/or triage label Nov 5, 2023
@emmercm
Copy link
Owner

emmercm commented Nov 5, 2023

I think I've been seeing some BPS issues with my collection as well, I'll try to investigate.

As an aside, BPS patches have the source file's CRC32 in them, you don't need to put it in your patch filename 🙂. https://igir.io/roms/patching/#rom-checksums has the docs on this.

@emmercm
Copy link
Owner

emmercm commented Nov 5, 2023

@Mihara this log indicates there is already a file in your output directory, and that you did not provide the --overwrite or --overwrite-invalid option:

[14:21:12.328] TRACE: CandidateWriter: testcase: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: not overwriting existing zip

If that file is not runnable then it's the fault of some other process.


Given the patch file you linked, I was able to successfully patch the ROM:

$ npm start -- copy zip clean --input "Downloads/46 Okunen Monogatari - Harukanaru Eden e (Japan).7z" --patch "Downloads/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) B090235A.bps" --output temp

 ______   ______   ______  _______  
|      \ /      \ |      \|       \ 
 \$$$$$$|  $$$$$$\ \$$$$$$| $$$$$$$\
  | $$  | $$|    \  | $$  | $$    $$   ROM collection manager
  | $$  | $$|    \  | $$  | $$    $$   https://igir.io/
  | $$  | $$ \$$$$  | $$  | $$$$$$$\
 _| $$_ | $$__| $$ _| $$_ | $$  | $$   v2.1.0
|   $$ \ \$$    $$|   $$ \| $$  | $$
 \$$$$$$  \$$$$$$  \$$$$$$ \$$   \$$


WARN:  No DAT files provided, consider using some for the best results!
✓ Scanning for ROMs ·········· | 1 file found
✓ Scanning for patches ······· | 1 unique patch found
✓ Downloads ·················· | 1 games, 1 retail releases, 1 patched games written
✓ Cleaning output directory ·· | 1 file recycled

$ crc32 temp/*
1bab99c8        temp/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip
b1946ac7        temp/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip

@Mihara
Copy link
Author

Mihara commented Nov 6, 2023

As an aside, BPS patches have the source file's CRC32 in them, you don't need to put it in your patch filename 🙂. https://igir.io/roms/patching/#rom-checksums has the docs on this.

If I have that, my script that applies them without igir doesn't have to know BPS to apply them.

@Mihara this log indicates there is already a file in your output directory, and that you did not provide the --overwrite or --overwrite-invalid option:

This one doesn't, because I replicated the test case again in a clean environment and made sure to only run it once:

 ______   ______   ______  _______  
|      \ /      \ |      \|       \ 
 \$$$$$$|  $$$$$$\ \$$$$$$| $$$$$$$\
  | $$  | $$|    \  | $$  | $$    $$   ROM collection manager
  | $$  | $$|    \  | $$  | $$    $$   https://igir.io/
  | $$  | $$ \$$$$  | $$  | $$$$$$$\
 _| $$_ | $$__| $$ _| $$_ | $$  | $$   v2.1.0
|   $$ \ \$$    $$|   $$ \| $$  | $$
 \$$$$$$  \$$$$$$  \$$$$$$ \$$   \$$


[06:12:20.429] WARN:  No DAT files provided, consider using some for the best results!
[06:12:20.431] INFO:  ROMScanner: scanning ROM files
[06:12:20.456] DEBUG: ROMScanner: found 1 ROM file
[06:12:20.496] INFO:  ROMScanner: done scanning ROM files
[06:12:20.497] INFO:  ROMHeaderProcessor: processing file headers
[06:12:20.515] TRACE: ROMHeaderProcessor: testcase/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: reading potentially headered file by filename: .smc
[06:12:20.531] TRACE: ROMHeaderProcessor: testcase/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: found non-applicable header by filename: .smc
[06:12:20.531] DEBUG: ROMHeaderProcessor: found headers in 0 ROMs
[06:12:20.531] INFO:  ROMHeaderProcessor: done processing file headers
[06:12:20.539] INFO:  FileIndexer: indexing 1 file
[06:12:20.548] DEBUG: FileIndexer: found 1 unique file
[06:12:20.548] INFO:  FileIndexer: done indexing files
Scanning for ROMs ... 1 file found
[06:12:20.551] INFO:  PatchScanner: scanning patch files
[06:12:20.561] DEBUG: PatchScanner: found 1 patch file
[06:12:20.604] TRACE: PatchScanner: testcase-patches/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) B090235A.bps: found patch by extension: object
[06:12:20.604] INFO:  PatchScanner: done scanning patch files
Scanning for patches ... 1 unique patch found
[06:12:20.606] INFO:  DATGameInferrer: inferring DATs for 1 ROM
[06:12:20.606] DEBUG: DATGameInferrer: inferred 1 DAT
[06:12:20.607] INFO:  DATGameInferrer: done inferring DATs
[06:12:20.607] INFO:  processing 1 DAT
[06:12:20.608] INFO:  DATParentInferrer: inferring parents for 1 game
[06:12:20.615] DEBUG: DATParentInferrer: testcase: grouped to 1 parent
[06:12:20.615] INFO:  DATParentInferrer: done inferring parents
[06:12:20.615] INFO:  DATMergerSplitter: testcase: merging & splitting
[06:12:20.615] DEBUG: DATMergerSplitter: testcase: DAT doesn't have parent/clone info, doing nothing
[06:12:20.616] INFO:  DATFilter: testcase: filtering DAT
[06:12:20.617] DEBUG: DATFilter: testcase: filtered to 1/1 game (1.5MiB)
[06:12:20.617] INFO:  DATFilter: testcase: done filtering DAT
[06:12:20.618] INFO:  CandidateGenerator: testcase: generating candidates
[06:12:20.631] TRACE: CandidateGenerator: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): found 1 candidate
[06:12:20.631] DEBUG: CandidateGenerator: testcase: generated 1.5MiB of 1 candidate for 1 parent
[06:12:20.631] INFO:  CandidateGenerator: testcase: done generating candidates
[06:12:20.631] INFO:  CandidatePatchGenerator: testcase: generating patched candidates
[06:12:20.632] DEBUG: CandidatePatchGenerator: testcase: 1 unique patches found
[06:12:20.634] INFO:  CandidatePatchGenerator: testcase: done generating patched candidates
[06:12:20.635] TRACE: CandidatePatchGenerator: testcase: testcase/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: patch candidate generated: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc
[06:12:20.635] INFO:  CandidatePreferer: testcase: filtering candidates
[06:12:20.637] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): 1 candidate before filtering
[06:12:20.637] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): 1 candidate after filtering
[06:12:20.638] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): 1 candidate before filtering
[06:12:20.638] TRACE: CandidatePreferer: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): 1 candidate after filtering
[06:12:20.638] DEBUG: CandidatePreferer: testcase: filtered to 3MiB of 2 candidates for 2 parents
[06:12:20.638] INFO:  CandidatePreferer: testcase: done filtering candidates
[06:12:20.638] INFO:  CandidatePostProcessor: testcase: processing candidates
[06:12:20.645] INFO:  CandidatePostProcessor: testcase: done processing candidates
[06:12:20.646] INFO:  CandidateMergeSplitValidator: testcase: validating merged & split ROM sets
[06:12:20.647] INFO:  CandidateMergeSplitValidator: testcase: done validating merged & split ROM sets
[06:12:20.648] INFO:  CandidateWriter: testcase: writing 2 candidates
[06:12:20.649] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): writing 1 candidate
[06:12:20.650] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): writing 1 candidate
[06:12:20.651] TRACE: CandidateWriter: testcase: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip: writing 1 archive entry ...
[06:12:20.651] TRACE: CandidateWriter: testcase: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: writing 1 archive entry ...
[06:12:20.885] TRACE: CandidateWriter: testcase: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip: wrote 1 archive entry
[06:12:20.886] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): no raw files to write
[06:12:20.886] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan): done writing 1 candidate
[06:12:21.501] TRACE: CandidateWriter: testcase: TestRoms/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: wrote 1 archive entry
[06:12:21.502] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): no raw files to write
[06:12:21.502] TRACE: CandidateWriter: testcase: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): done writing 1 candidate
[06:12:21.505] INFO:  CandidateWriter: testcase: done writing 2 candidates
[06:12:21.506] INFO:  StatusGenerator: testcase: generating ROM statuses
[06:12:21.509] INFO:  StatusGenerator: testcase: done generating ROM statuses
testcase ... 1 games, 1 retail releases, 1 patched games written
[06:12:21.510] INFO:  done processing 1 DAT

The resulting file still has a hash different from the one patched by flips, and it's still the same:

a56a550c460285d06fbbcf2d8e41690bd9b9ba13  46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) - flips.sfc
0cc6d8ac6a80a9892467f3d03e4493772885c9aa  46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) - igir.sfc
6a9c8fb28eb524fc26561f9e193380edab3a8a58  46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc

What next?

@emmercm
Copy link
Owner

emmercm commented Nov 10, 2023

@Mihara could you add test as an extra command to that? I'm curious what it says, it should say failure.

So that I can try to reproduce, could you paste the CRC32 of those three files instead? BPS patches include an expected output CRC32 in them.

@Mihara
Copy link
Author

Mihara commented Nov 11, 2023

Here. Initial setup:

$ ls -1R
.:
known-good
patch
rom
runtest

./known-good:
'46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc'

./patch:
'46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) B090235A.bps'

./rom:
'46 Okunen Monogatari - Harukanaru Eden e (Japan).zip'

known-good/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc is the patched rom my script produces through running flips.

Contents of runtest:

#!/bin/bash

rm -rf rom-output igir.log

echo ==== IGIR OUTPUT ==== > igir.log
igir copy zip test -vvv --input rom --output rom-output --patch patch >> igir.log

echo ==== CRC32 ==== >> igir.log
crc32 known-good/* rom-output/* >> igir.log
find rom-output -name '*.zip' -exec unzip -v {} \; >> igir.log

Results in igir.log:

==== IGIR OUTPUT ====
 ______   ______   ______  _______  
|      \ /      \ |      \|       \ 
 \$$$$$$|  $$$$$$\ \$$$$$$| $$$$$$$\
  | $$  | $$|    \  | $$  | $$    $$   ROM collection manager
  | $$  | $$|    \  | $$  | $$    $$   https://igir.io/
  | $$  | $$ \$$$$  | $$  | $$$$$$$\
 _| $$_ | $$__| $$ _| $$_ | $$  | $$   v2.1.0
|   $$ \ \$$    $$|   $$ \| $$  | $$
 \$$$$$$  \$$$$$$  \$$$$$$ \$$   \$$


[08:43:19.843] WARN:  No DAT files provided, consider using some for the best results!
[08:43:19.845] INFO:  ROMScanner: scanning ROM files
[08:43:19.861] DEBUG: ROMScanner: found 1 ROM file
[08:43:19.884] INFO:  ROMScanner: done scanning ROM files
[08:43:19.884] INFO:  ROMHeaderProcessor: processing file headers
[08:43:19.895] TRACE: ROMHeaderProcessor: rom/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: reading potentially headered file by filename: .smc
[08:43:19.901] TRACE: ROMHeaderProcessor: rom/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: found non-applicable header by filename: .smc
[08:43:19.901] DEBUG: ROMHeaderProcessor: found headers in 0 ROMs
[08:43:19.901] INFO:  ROMHeaderProcessor: done processing file headers
[08:43:19.904] INFO:  FileIndexer: indexing 1 file
[08:43:19.910] DEBUG: FileIndexer: found 1 unique file
[08:43:19.910] INFO:  FileIndexer: done indexing files
Scanning for ROMs ... 1 file found
[08:43:19.912] INFO:  PatchScanner: scanning patch files
[08:43:19.914] DEBUG: PatchScanner: found 1 patch file
[08:43:19.943] TRACE: PatchScanner: patch/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486) B090235A.bps: found patch by extension: object
[08:43:19.943] INFO:  PatchScanner: done scanning patch files
Scanning for patches ... 1 unique patch found
[08:43:19.943] INFO:  DATGameInferrer: inferring DATs for 1 ROM
[08:43:19.944] DEBUG: DATGameInferrer: inferred 1 DAT
[08:43:19.944] INFO:  DATGameInferrer: done inferring DATs
[08:43:19.944] INFO:  processing 1 DAT
[08:43:19.945] INFO:  DATParentInferrer: inferring parents for 1 game
[08:43:19.948] DEBUG: DATParentInferrer: rom: grouped to 1 parent
[08:43:19.948] INFO:  DATParentInferrer: done inferring parents
[08:43:19.948] INFO:  DATMergerSplitter: rom: merging & splitting
[08:43:19.948] DEBUG: DATMergerSplitter: rom: DAT doesn't have parent/clone info, doing nothing
[08:43:19.948] INFO:  DATFilter: rom: filtering DAT
[08:43:19.949] DEBUG: DATFilter: rom: filtered to 1/1 game (1.5MiB)
[08:43:19.949] INFO:  DATFilter: rom: done filtering DAT
[08:43:19.949] INFO:  CandidateGenerator: rom: generating candidates
[08:43:19.955] TRACE: CandidateGenerator: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan): found 1 candidate
[08:43:19.955] DEBUG: CandidateGenerator: rom: generated 1.5MiB of 1 candidate for 1 parent
[08:43:19.955] INFO:  CandidateGenerator: rom: done generating candidates
[08:43:19.955] INFO:  CandidatePatchGenerator: rom: generating patched candidates
[08:43:19.956] DEBUG: CandidatePatchGenerator: rom: 1 unique patches found
[08:43:19.956] INFO:  CandidatePatchGenerator: rom: done generating patched candidates
[08:43:19.957] TRACE: CandidatePatchGenerator: rom: rom/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc: patch candidate generated: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip|46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc
[08:43:19.957] INFO:  CandidatePreferer: rom: filtering candidates
[08:43:19.958] TRACE: CandidatePreferer: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan): 1 candidate before filtering
[08:43:19.958] TRACE: CandidatePreferer: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan): 1 candidate after filtering
[08:43:19.958] TRACE: CandidatePreferer: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): 1 candidate before filtering
[08:43:19.958] TRACE: CandidatePreferer: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): 1 candidate after filtering
[08:43:19.958] DEBUG: CandidatePreferer: rom: filtered to 3MiB of 2 candidates for 2 parents
[08:43:19.958] INFO:  CandidatePreferer: rom: done filtering candidates
[08:43:19.958] INFO:  CandidatePostProcessor: rom: processing candidates
[08:43:19.961] INFO:  CandidatePostProcessor: rom: done processing candidates
[08:43:19.962] INFO:  CandidateMergeSplitValidator: rom: validating merged & split ROM sets
[08:43:19.962] INFO:  CandidateMergeSplitValidator: rom: done validating merged & split ROM sets
[08:43:19.963] INFO:  CandidateWriter: rom: writing 2 candidates
[08:43:19.963] TRACE: CandidateWriter: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan): writing 1 candidate
[08:43:19.964] TRACE: CandidateWriter: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): writing 1 candidate
[08:43:19.964] TRACE: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip: writing 1 archive entry ...
[08:43:19.964] TRACE: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: writing 1 archive entry ...
[08:43:20.111] TRACE: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip: wrote 1 archive entry
[08:43:20.112] TRACE: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip: testing zip
[08:43:20.114] TRACE: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip: test passed
[08:43:20.114] TRACE: CandidateWriter: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan): no raw files to write
[08:43:20.114] TRACE: CandidateWriter: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan): done writing 1 candidate
[08:43:20.379] TRACE: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: wrote 1 archive entry
[08:43:20.379] TRACE: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: testing zip
[08:43:20.380] ERROR: CandidateWriter: rom: rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip: written zip has the file 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc with the CRC 9c58e328, expected 5173188d
[08:43:20.380] TRACE: CandidateWriter: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): no raw files to write
[08:43:20.380] TRACE: CandidateWriter: rom: 46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486): done writing 1 candidate
[08:43:20.382] INFO:  CandidateWriter: rom: done writing 2 candidates
[08:43:20.383] INFO:  StatusGenerator: rom: generating ROM statuses
[08:43:20.384] INFO:  StatusGenerator: rom: done generating ROM statuses
rom ... 1 games, 1 retail releases, 1 patched games written
[08:43:20.384] INFO:  done processing 1 DAT
 
==== CRC32 ====
5173188d	known-good/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc
b0c5f493	rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip
0a538234	rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip
Archive:  rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan).zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
 1572864  Defl:N  1208472  23% 2023-11-11 05:43 b090235a  46 Okunen Monogatari - Harukanaru Eden e (Japan).sfc
--------          -------  ---                            -------
 1572864          1208472  23%                            1 file
Archive:  rom-output/46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
 1572864  Defl:N  1206329  23% 2023-11-11 05:43 9c58e328  46 Okunen Monogatari - Harukanaru Eden e (Japan) (En)(Retranslation, EVO - Search for Eden)(RH5486).sfc
--------          -------  ---                            -------
 1572864          1206329  23%                            1 file

Indeed, adding test results in an error and the broken patched file not being written when there's no zip. When zip is on the list of commands, we get a zipped broken patched file.

@emmercm
Copy link
Owner

emmercm commented Nov 16, 2023

I'm able to reproduce it on my end.

  • Source ROM CRC: b090235a
  • Patch CRC before: b090235a
  • Patch CRC after/expected: 5173188d
  • Output file CRC (bad): 9c58e328

@emmercm emmercm self-assigned this Nov 16, 2023
@emmercm emmercm added bug A confirmed issues that needs fixing and removed potential-bug A potential issue that needs confirmation and/or triage labels Nov 16, 2023
Copy link

🔒 Inactive issue lock

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Comment generated by the GitHub Lock Issues workflow.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug A confirmed issues that needs fixing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants