[Issue] Only iOS - user could upload the duplicate photo

[sync-by-unito]

User story

As a user of the asset upload feature on the capture app, I want the app to prevent me from uploading duplicate photos, so that I can avoid creating unnecessary duplicates and efficiently manage my assets.

Reproduce step

1. Take a photo on native camera
2. upload it & complete the asset register process
3. choose the same photo and upload
4. It will have the two assets but they are from the same photo.

Example:

﹍https://nftsearch.site/asset-profile?cid=bafybeiafqrzdvtf44rmt2zngqvqr6mommtpxvyctvvzdmkqp3gc6tccq5a﹍

﹍https://nftsearch.site/asset-profile?cid=bafybeiclmlojwqypwpuhz2akzd3pmfi3pl3poql24qdvxjzausanti4pie﹍

Expectation

1. The user should not be able to upload the duplicate photo on the backend.
2. When user wants to upload the duplicate photo it should show the right error message let user know this asset exists on the blockchain

Additional information:

1. Only found in iOS, android will just show the wrong error message.
2. Currently found on iOS & can’t reproduce every time, the ratio is about 20%.

┆Issue is synchronized with this Asana task by Unito
┆Created By: Kenny Hung

[sync-by-unito]

➤ Sam commented:

Kenny Hung, just to double check

Android show wrong error message you mean this message?

https://app.asana.com/app/asana/-/get_asset?asset_id=1204361548656097 ( https://app.asana.com/app/asana/-/get_asset?asset_id=1204361548656097 )

[sync-by-unito]

➤ Kenny Hung commented:

Sam Yes, like the similar error messge, but I didn't see it before.

I suggest the error message could be more simple, like (Need Tammy Yang's confirmation.)

"The asset you are trying to upload already exists on the blockchain. Please upload a different asset."

About Chinese, need Sofia Yan's help.

[sync-by-unito]

➤ Kenny Hung commented:

But this task is about avoiding user could upload the same photo file more 1 time. (Error message content is another task. [Issue] Error handling for uploading duplicate photo ( https://app.asana.com/0/1201016280880500/1204361832181061/f ))

[sync-by-unito]

➤ Sam commented:

Kenny Hung thank you for your feedback.

[sync-by-unito]

➤ Sam commented:

Kenny Hung, can you try again on iOS same version.

• Take photo with iOS native camera
• Upload photo
• Wait until it's registered
• Upload same photo

I think you will get Internet connection ... error this time.

[sync-by-unito]

➤ Kenny Hung commented:

Sam

Yes, but this issue can’t reproduce every time, the ratio is about 20%.

when you try more times, almost 5 times, user could upload 2 times successfully.

[sync-by-unito]

➤ Tammy Yang commented:

James ChienOlgaBofu Chen need your help

We encounter the problem that on iOS duplicate photos are allowed to upload.

Steps to reproduce:

1. Take a photo with the native camera on iPhone 8
2. Upload and wait until the registration completed
3. Use Capture App to choose the same photo and upload again
4. Repeat Bump @types/node from 12.12.50 to 14.0.23 #3 for 30 times, there are some chances (around 1-2 times) that users can still upload

Example for the issue:

1. The original uploaded one: https://nftsearch.site/asset-profile?cid=bafybeiafqrzdvtf44rmt2zngqvqr6mommtpxvyctvvzdmkqp3gc6tccq5a ( https://nftsearch.site/asset-profile?cid=bafybeiafqrzdvtf44rmt2zngqvqr6mommtpxvyctvvzdmkqp3gc6tccq5a )
2. The duplicated one (the one that should not be uploaded successfully): https://nftsearch.site/asset-profile?cid=bafybeiclmlojwqypwpuhz2akzd3pmfi3pl3poql24qdvxjzausanti4pie ( https://nftsearch.site/asset-profile?cid=bafybeiclmlojwqypwpuhz2akzd3pmfi3pl3poql24qdvxjzausanti4pie )

Debugging:

I do notice that XMP injection of the two files are different:

• XMP 1 ( https://jimpl.com/results/cruuiVbjPBH1N6jgzLcFZTZm?target=exif )
• XMP 2 ( https://jimpl.com/results/1x4Wu8PqWJw6iRbAoXAx4qpb?target=exif )

As you can see, the image filename is different and the extension is wrong.

My question:

Do you know if XMP injection may affect the IPFS cid?

I am trying to figure out is it because of any injection done by the mobile OS which causes the issue.

Any other suggestions are also welcome.

[sync-by-unito]

➤ Tammy Yang commented:

Additional debugging, I downloaded the two images from Kenny's example and does the pixel-by-pixel comparison (source code see https://pastebin.com/a6Zre8Zs ( https://pastebin.com/a6Zre8Zs )), they are identical (i.e. the difference is must from header)

[sync-by-unito]

➤ Tammy Yang commented:

Exif are all the same, too (source code to check see https://pastebin.com/8aShmRN2 ( https://pastebin.com/8aShmRN2 ))

TileWidth                : 512

TileLength               : 512

GPSInfo                  : 2058

ResolutionUnit           : 2

ExifOffset               : 240

Make                     : Apple

Model                    : iPhone 8

Software                 : 14.8.1

Orientation              : 1

DateTime                 : 2023:04:10 17:54:49

XResolution              : 72.0

YResolution              : 72.0

HostComputer             : iPhone 8

[sync-by-unito]

➤ Tammy Yang commented:

However, the interesting part is, if I upload the two photos again to our IPFS gateway using this API ( https://docs.numbersprotocol.io/developers/more-tools/ipfs-pin-file-to-ipfs ) (which look identical from every aspect except the filename in XMP) again, the Nid are indeed different 😅.

[sync-by-unito]

➤ Sam commented:

Tammy Yang, regarding CAI injection I found this in changelog ( https://github.com/numbersprotocol/capture-lite/blob/master/CHANGELOG.md#changed-32 ).

And this is related commit ( f5803fd ) in 0.39.0.

I think there is no CAI injection logic from the capture app.

[sync-by-unito]

➤ Tammy Yang commented:

Well... their header is the IDENTICAL XD

Source code: https://pastebin.com/JzNGVXj3 ( https://pastebin.com/JzNGVXj3 )

Result printed:

File type: jpg

Header: b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x00\x00H\x00H\x00\x00\xff\xe1\t\x0cExif\x00\x00'

[sync-by-unito]

➤ Bofu Chen commented:

Extract and save the headers of the two photos

$ exiv2 -p a pr bafybeiafqrzdvtf44rmt2zngqvqr6mommtpxvyctvvzdmkqp3gc6tccq5a.jpeg | tee bafybeiafqrzdvtf44rmt2zngqvqr6mommtpxvyctvvzdmkqp3gc6tccq5a.exiv2.txt

$ exiv2 -p a pr bafybeiclmlojwqypwpuhz2akzd3pmfi3pl3poql24qdvxjzausanti4pie.jpeg | tee bafybeiclmlojwqypwpuhz2akzd3pmfi3pl3poql24qdvxjzausanti4pie.exiv2.txt

Comparing the difference between the headers, the only difference is Exif.Photo.MakerNote

$ diff -Naru bafybeiafqrzdvtf44rmt2zngqvqr6mommtpxvyctvvzdmkqp3gc6tccq5a.exiv2.txt bafybeiclmlojwqypwpuhz2akzd3pmfi3pl3poql24qdvxjzausanti4pie.exiv2.txt

--- bafybeiafqrzdvtf44rmt2zngqvqr6mommtpxvyctvvzdmkqp3gc6tccq5a.exiv2.txt 2023-04-17 20:21:23

+++ bafybeiclmlojwqypwpuhz2akzd3pmfi3pl3poql24qdvxjzausanti4pie.exiv2.txt 2023-04-17 20:21:50

@@ -29,7 +29,7 @@

Exif.Photo.Flash                             Short       1  No, auto

Exif.Photo.FocalLength                       Rational    1  4.0 mm

Exif.Photo.SubjectArea                       Short       4  2015 1511 2217 1330

-Exif.Photo.MakerNote                         Undefined 1192  65 112 112 108 101 32 105 79 83 0 0 1 77 77 0 31 0 1 0 9 0 0 0 1 0 0 0 12 0 2 0 7 0 0 2 46 0 0 1 136 0 3 0 7 0 0 0 104 0 0 3 182 0 4 0 9 0 0 0 1 0 0 0 1 0 5 0 9 0 0 0 1 0 0 0 197 0 6 0 9 0 0 0 1 0 0 0 192 0 7 0 9 0 0 0 1 0 0 0 1 0 8 0 10 0 0 0 3 0 0 4 30 0 12 0 10 0 0 0 2 0 0 4 54 0 13 0 9 0 0 0 1 0 0 0 13 0 14 0 9 0 0 0 1 0 0 0 0 0 15 0 9 0 0 0 1 0 0 0 2 0 16 0 9 0 0 0 1 0 0 0 1 0 20 0 9 0 0 0 1 0 0 0 5 0 23 0 9 0 0 0 1 0 0 32 0 0 25 0 9 0 0 0 1 0 0 0 0 0 26 0 2 0 0 0 6 0 0 4 70 0 31 0 9 0 0 0 1 0 0 0 0 0 32 0 2 0 0 0 37 0 0 4 76 0 33 0 10 0 0 0 1 0 0 4 114 0 37 0 9 0 0 0 1 0 0 0 0 0 38 0 9 0 0 0 1 0 0 0 0 0 39 0 10 0 0 0 1 0 0 4 122 0 43 0 2 0 0 0 37 0 0 4 130 0 45 0 9 0 0 0 1 0 0 20 134 0 46 0 9 0 0 0 1 0 0 0 0 0 47 0 9 0 0 0 1 0 0 0 133 0 54 0 9 0 0 0 1 0 0 6 26 0 55 0 9 0 0 0 1 0 0 0 1 0 59 0 9 0 0 0 1 0 0 0 0 0 60 0 9 0 0 0 1 0 0 0 0 0 0 0 0 98 112 108 105 115 116 48 48 79 17 2 0 174 0 65 0 128 0 175 0 166 0 245 0 254 0 222 0 200 0 208 0 220 0 226 0 232 0 218 0 188 0 177 0 96 0 178 0 175 0 156 0 230 0 236 0 227 0 180 0 174 0 175 0 177 0 177 0 173 0 170 0 171 0 175 0 228 0 188 0 153 0 229 0 224 0 220 0 203 0 163 0 160 0 162 0 162 0 163 0 163 0 165 0 167 0 174 0 193 0 153 0 231 0 234 0 217 0 217 0 201 0 161 0 158 0 159 0 160 0 160 0 161 0 162 0 164 0 173 0 162 0 232 0 239 0 236 0 217 0 217 0 201 0 162 0 159 0 159 0 159 0 160 0 160 0 162 0 162 0 172 0 239 0 240 0 239 0 236 0 217 0 217 0 202 0 162 0 160 0 159 0 159 0 159 0 160 0 161 0 160 0 172 0 241 0 239 0 239 0 237 0 219 0 219 0 203 0 163 0 161 0 160 0 160 0 161 0 160 0 161 0 159 0 171 0 243 0 242 0 240 0 239 0 220 0 220 0 205 0 164 0 162 0 162 0 162 0 161 0 161 0 162 0 158 0 171 0 245 0 244 0 243 0 242 0 222 0 222 0 207 0 161 0 160 0 160 0 163 0 163 0 163 0 163 0 158 0 171 0 247 0 247 0 246 0 245 0 225 0 224 0 209 0 164 0 165 0 160 0 165 0 165 0 164 0 164 0 159 0 172 0 250 0 249 0 249 0 248 0 229 0 227 0 212 0 171 0 168 0 168 0 167 0 167 0 166 0 166 0 160 0 173 0 0 1 252 0 253 0 251 0 233 0 231 0 216 0 174 0 172 0 170 0 168 0 168 0 168 0 168 0 160 0 174 0 15 1 0 1 1 1 0 1 238 0 235 0 226 0 184 0 175 0 173 0 171 0 170 0 170 0 170 0 157 0 159 0 34 1 11 1 5 1 5 1 253 0 253 0 240 0 212 0 180 0 176 0 174 0 174 0 175 0 171 0 132 0 130 0 44 1 38 1 17 1 9 1 253 0 11 1 255 0 234 0 197 0 181 0 178 0 185 0 203 0 189 0 122 0 128 0 53 1 54 1 46 1 21 1 252 0 252 0 5 1 243 0 224 0 193 0 174 0 170 0 184 0 186 0 129 0 128 0 0 8 0 0 0 0 0 0 2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 12 98 112 108 105 115 116 48 48 212 1 2 3 4 5 6 7 8 85 102 108 97 103 115 85 118 97 108 117 101 85 101 112 111 99 104 89 116 105 109 101 115 99 97 108 101 16 1 19 0 0 19 229 185 116 129 187 16 0 18 59 154 202 0 8 17 23 29 35 45 47 56 58 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 0 0 32 59 0 2 104 255 255 255 204 243 0 1 123 7 255 255 177 161 0 0 78 210 0 0 0 159 0 0 1 0 0 0 0 189 0 0 1 0 113 56 50 53 115 0 52 54 53 67 56 49 49 55 45 48 57 55 69 45 52 57 49 51 45 65 57 57 69 45 70 65 56 50 68 55 54 65 55 53 49 49 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 66 49 69 65 57 55 50 52 45 51 50 54 56 45 52 52 49 66 45 56 53 66 56 45 67 48 55 50 48 65 69 50 48 66 69 69 0 0

+Exif.Photo.MakerNote                         Undefined 1192  65 112 112 108 101 32 105 79 83 0 0 1 77 77 0 31 0 1 0 9 0 0 0 1 0 0 0 12 0 2 0 7 0 0 2 46 0 0 1 136 0 3 0 7 0 0 0 104 0 0 3 182 0 4 0 9 0 0 0 1 0 0 0 1 0 5 0 9 0 0 0 1 0 0 0 197 0 6 0 9 0 0 0 1 0 0 0 192 0 7 0 9 0 0 0 1 0 0 0 1 0 8 0 10 0 0 0 3 0 0 4 30 0 12 0 10 0 0 0 2 0 0 4 54 0 13 0 9 0 0 0 1 0 0 0 13 0 14 0 9 0 0 0 1 0 0 0 0 0 15 0 9 0 0 0 1 0 0 0 2 0 16 0 9 0 0 0 1 0 0 0 1 0 20 0 9 0 0 0 1 0 0 0 5 0 23 0 9 0 0 0 1 0 0 32 0 0 25 0 9 0 0 0 1 0 0 0 0 0 26 0 2 0 0 0 6 0 0 4 70 0 31 0 9 0 0 0 1 0 0 0 0 0 32 0 2 0 0 0 37 0 0 4 76 0 33 0 10 0 0 0 1 0 0 4 114 0 37 0 9 0 0 0 1 0 0 0 0 0 38 0 9 0 0 0 1 0 0 0 0 0 39 0 10 0 0 0 1 0 0 4 122 0 43 0 2 0 0 0 37 0 0 4 130 0 45 0 9 0 0 0 1 0 0 20 134 0 46 0 9 0 0 0 1 0 0 0 0 0 47 0 9 0 0 0 1 0 0 0 133 0 54 0 9 0 0 0 1 0 0 6 26 0 55 0 9 0 0 0 1 0 0 0 1 0 59 0 9 0 0 0 1 0 0 0 0 0 60 0 9 0 0 0 1 0 0 0 0 0 0 0 0 98 112 108 105 115 116 48 48 79 17 2 0 174 0 65 0 128 0 175 0 166 0 245 0 254 0 222 0 200 0 208 0 220 0 226 0 232 0 218 0 188 0 177 0 96 0 178 0 175 0 156 0 230 0 236 0 227 0 180 0 174 0 175 0 177 0 177 0 173 0 170 0 171 0 175 0 228 0 188 0 153 0 229 0 224 0 220 0 203 0 163 0 160 0 162 0 162 0 163 0 163 0 165 0 167 0 174 0 193 0 153 0 231 0 234 0 217 0 217 0 201 0 161 0 158 0 159 0 160 0 160 0 161 0 162 0 164 0 173 0 162 0 232 0 239 0 236 0 217 0 217 0 201 0 162 0 159 0 159 0 159 0 160 0 160 0 162 0 162 0 172 0 239 0 240 0 239 0 236 0 217 0 217 0 202 0 162 0 160 0 159 0 159 0 159 0 160 0 161 0 160 0 172 0 241 0 239 0 239 0 237 0 219 0 219 0 203 0 163 0 161 0 160 0 160 0 161 0 160 0 161 0 159 0 171 0 243 0 242 0 240 0 239 0 220 0 220 0 205 0 164 0 162 0 162 0 162 0 161 0 161 0 162 0 158 0 171 0 245 0 244 0 243 0 242 0 222 0 222 0 207 0 161 0 160 0 160 0 163 0 163 0 163 0 163 0 158 0 171 0 247 0 247 0 246 0 245 0 225 0 224 0 209 0 164 0 165 0 160 0 165 0 165 0 164 0 164 0 159 0 172 0 250 0 249 0 249 0 248 0 229 0 227 0 212 0 171 0 168 0 168 0 167 0 167 0 166 0 166 0 160 0 173 0 0 1 252 0 253 0 251 0 233 0 231 0 216 0 174 0 172 0 170 0 168 0 168 0 168 0 168 0 160 0 174 0 15 1 0 1 1 1 0 1 238 0 235 0 226 0 184 0 175 0 173 0 171 0 170 0 170 0 170 0 157 0 159 0 34 1 11 1 5 1 5 1 253 0 253 0 240 0 212 0 180 0 176 0 174 0 174 0 175 0 171 0 132 0 130 0 44 1 38 1 17 1 9 1 253 0 11 1 255 0 234 0 197 0 181 0 178 0 185 0 203 0 189 0 122 0 128 0 53 1 54 1 46 1 21 1 252 0 252 0 5 1 243 0 224 0 193 0 174 0 170 0 184 0 186 0 129 0 128 0 0 8 0 0 0 0 0 0 2 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 12 98 112 108 105 115 116 48 48 212 1 2 3 4 5 6 7 8 85 102 108 97 103 115 85 118 97 108 117 101 89 116 105 109 101 115 99 97 108 101 85 101 112 111 99 104 16 1 19 0 0 19 229 185 116 129 187 18 59 154 202 0 16 0 8 17 23 29 39 45 47 56 61 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 0 0 32 59 0 2 104 255 255 255 204 243 0 1 123 7 255 255 177 161 0 0 78 210 0 0 0 159 0 0 1 0 0 0 0 189 0 0 1 0 113 56 50 53 115 0 52 54 53 67 56 49 49 55 45 48 57 55 69 45 52 57 49 51 45 65 57 57 69 45 70 65 56 50 68 55 54 65 55 53 49 49 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 66 49 69 65 57 55 50 52 45 51 50 54 56 45 52 52 49 66 45 56 53 66 56 45 67 48 55 50 48 65 69 50 48 66 69 69 0 0

Exif.Photo.SubSecTimeOriginal                Ascii       4  028

Exif.Photo.SubSecTimeDigitized               Ascii       4  028

Exif.Photo.FlashpixVersion                   Undefined   4  1.00

[sync-by-unito]

➤ Sam commented:

I think I narrow down bug surface.

In capture app there is function called setAsset (

capture-lite/src/app/shared/repositories/proof/proof.ts

Line 131 in d9f8764

async setAssets(assets: Assets) {

) which

• generates ( https://github.com/numbersprotocol/capture-lite/blob/master/src/app/shared/repositories/proof/proof.ts#L134 ) sha256 from base64 of the asset (in capture app it's referred as index)

Then in capture app we check (

capture-lite/src/app/shared/database/table/capacitor-filesystem-table/capacitor-filesystem-table.ts

Line 106 in d9f8764

this.assertNoConflictWithExistedTuples(tuples, comparator);

) if there is no 2 proofs with the same index (sha256 of base64 file)

• if yes we upload to backend
• if no we show Duplicate error

Here is what I did iOS 0.77.2

• I pick image from gallery 1, 2, 3, 4, 5 (I also pause upload so everything happens inside capture app without asset registration etc)
• then I try to pick image 5 (expected to show Duplicate error but it didn't)

And it happened because for the same base64 (image 5) capture app generated (

capture-lite/src/app/utils/crypto/crypto.ts

Line 13 in d9f8764

export async function sha256WithBase64(base64: string) {

) different sha256 🤯 (I think this is a place to start debugging from).

Here are the logs from capture app iOS logs:

[sync-by-unito]

➤ Bofu Chen commented:

Kenny HungSamTammy Yang If the failure rate is medium (1/30) and only happens on iOS, I recommend that we release first and ensure to fix this issue in this sprint.

[sync-by-unito]

➤ Tammy Yang commented:

SamKenny Hung Let's do this. Which ever with lowest fail rate and highest quality, let's go with it and set conditional pass. This issue may need a bit more efforts and I will ask James Chien's support to check together in the next sprint.

[sync-by-unito]

➤ Sam commented:

Kenny Hung, so 0.77.2 (1) firebase release is ok?

If yes I will start prepearing code review

We choose 0.77.2(1) firebase release as candidate right?

[sync-by-unito]

➤ Kenny Hung commented:

Sam Yes!

Please submit code reivew v230321-ionic based on 0.77.2

You could refer Comment by @kenny Hung on v230310-capture-app-ionic(merge v230223) ( https://app.asana.com/0/0/1204010561520024/1204419595745090/f ).

[sync-by-unito]

➤ Kenny Hung commented:

Sam

Another thing, after you prepare code, and will start v230413-capture-app-ionic ( https://app.asana.com/0/0/1204223578973623 ) please also make sure the firebase release includes the 0.77.2 implemented.

[sync-by-unito]

➤ Tammy Yang commented:

If [CFM] Update upload review mechanism ( https://app.asana.com/0/0/1204451854984158 ) is fixed, I believe this issue will no longer exist. Though, still good to find out what happens in the App side to change the image.