Bug: 画像添付ファイルを含むページのアーカイブをインポートすると画像がリンクが切れになる #6313

n79s · 2022-07-20T13:27:50Z

Environment

Host

item	version
OS	AmazonLinux2 Kernel 5.10
GROWI	5.0.11
node.js	16.15.1
npm	8.11.0
Using Docker	yes
Using [growi-docker-compose][growi-docker-compose]	yes

Client

item	version
OS	Windows11
browser	Chrome 103.0.5060.114

How to reproduce? (再現手順)

Growi 5.0.11をgrowi-docker-compose利用で2環境（別サーバ）準備。A環境 / B環境と呼称
両環境とも同一バージョンで、ファイルアップロードの設定はMongoDB(GridFS)
A環境で添付ファイル（画像）を含むページを作成
A環境で添付ファイル含むページのアーカイブ（条件は後述）
B環境でアーカイブしたA環境のZIPファイルをインポート（条件は後述）

アーカイブの条件

インポートの条件

What happens? (症状)

添付ファイルが復元されない。（リンク切れとなる）

What is the expected result? (期待される動作)

添付ファイルも含めてページが復元される。

Note

同一環境でのアーカイブ→インポートの場合は表示されます。

docker-composeで利用させていただいています。アーカイブ→インポートで添付ファイルも含めて復元出来ると大変ありがたく、起票させていただきました。ドキュメントで何か見逃していたり、アーカイブの手順やインポートの手順が誤っているようでしたらすみません。

The text was updated successfully, but these errors were encountered:

yuki-takei · 2022-07-21T16:13:34Z

@n79s ご報告ありがとうございます。実装しておきながらなんですが export はかなり力業でやっているので、gridfs の chunks を正常に復元できるかどうかはかなり怪しいなあと思っています。

過去にこういう issue があったのですが、最近再現実験をしてみても再現せずクローズしました。
#2103

インポート時にサーバーサイドでエラー等出ていないでしょうか？もし可能でしたら出力を共有していただけると助かります。よろしくお願いします。

n79s · 2022-07-22T12:46:21Z

回答ありがとうございます。追加のログになります。

インポート時に画面では下記Failedがでます。

新規の添付ファイルはinsertされるのですが、一度インポートした内容はupsertでインポートしてもFailedになるようでした。

Failedの文字リンククリックで表示されるダイアログの内容は以下でした。

AttachmentFiles.FilesのFailedの内容

{"code":11000,"index":0,"errmsg":"E11000 duplicate key error collection: growi.attachmentFiles.files index: _id_ dup key: { _id: ObjectId('62d7e2f675a5f4a0d663110a') }","op":{"_id":"62d7e2f675a5f4a0d663110a","length":88797,"chunkSize":261120,"uploadDate":"2022-07-20T11:11:50.571Z","filename":"1d40d20d682b2f19cc1610ddf11ba83f.png","contentType":"image/png"}}
{"code":11000,"index":1,"errmsg":"E11000 duplicate key error collection: growi.attachmentFiles.files index: _id_ dup key: { _id: ObjectId('62d7f1db75a5f4a0d66319df') }","op":{"_id":"62d7f1db75a5f4a0d66319df","length":26944,"chunkSize":261120,"uploadDate":"2022-07-20T12:15:23.636Z","filename":"dd6b59662853b5e945d73077aa394a29.png","contentType":"image/png"}}
{"code":11000,"index":2,"errmsg":"E11000 duplicate key error collection: growi.attachmentFiles.files index: _id_ dup key: { _id: ObjectId('62d7f1e775a5f4a0d6631a28') }","op":{"_id":"62d7f1e775a5f4a0d6631a28","length":5288,"chunkSize":261120,"uploadDate":"2022-07-20T12:15:35.736Z","filename":"7dac85c72e37ba4f615c92fbbd23d864.png","contentType":"image/png"}}

AttachmentFiles.ChunksのFailedの内容

{"code":11000,"index":0,"errmsg":"E11000 duplicate key error collection: growi.attachmentFiles.chunks index: _id_ dup key: { _id: ObjectId('62d7e2f675a5f4a0d663110d') }","op":{"_id":"62d7e2f675a5f4a0d663110d","files_id":"62d7e2f675a5f4a0d663110a","n":0,"data":"aVZ・・・・Ua1N1UW1DQw=="}}
{"code":11000,"index":1,"errmsg":"E11000 duplicate key error collection: growi.attachmentFiles.chunks index: _id_ dup key: { _id: ObjectId('62d7f1db75a5f4a0d66319e2') }","op":{"_id":"62d7f1db75a5f4a0d66319e2","files_id":"62d7f1db75a5f4a0d66319df","n":0,"data":"aVZC・・・・RXJrSmdnZz09"}}
{"code":11000,"index":2,"errmsg":"E11000 duplicate key error collection: growi.attachmentFiles.chunks index: _id_ dup key: { _id: ObjectId('62d7f1e775a5f4a0d6631a2b') }","op":{"_id":"62d7f1e775a5f4a0d6631a2b","files_id":"62d7f1e775a5f4a0d6631a28","n":0,"data":"aVZC・・・・PUks1Q1lJST0="}}

上記インポート実行時のdocker-compose logs（特にエラーのような内容はありませんでした）

app_1            | {"name":"express","hostname":"82351dce94ae","pid":14,"req_id":"5085b27b-d405-44dd-8872-70d30ac4cbd5","level":50,"msg":"::ffff:10.8.0.66 <-- GET /_api/v3/in-app-notification/list?limit=6 HTTP/1.1 503 39 http://172.31.20.136/admin/importer Chrome 103.0 Windows 10.0.0 1.685284 ms","time":"2022-07-22T06:00:51.165Z","v":0}
app_1            | {"name":"express","hostname":"82351dce94ae","pid":14,"req_id":"e7002c4f-9832-41dd-8c73-bcd8004f7052","level":50,"msg":"::ffff:10.8.0.66 <-- GET /_api/v3/in-app-notification/status HTTP/1.1 503 39 http://172.31.20.136/admin/importer Chrome 103.0 Windows 10.0.0 1.576375 ms","time":"2022-07-22T06:00:51.180Z","v":0}
app_1            | {"name":"express","hostname":"82351dce94ae","pid":14,"req_id":"e3d3244a-7784-41fc-a8e6-d511afc2f9b0","level":30,"msg":"::ffff:10.8.0.66 <-- POST /_api/v3/import/ HTTP/1.1 200 2 http://172.31.20.136/admin/importer Chrome 103.0 Windows 10.0.0 2.549268 ms","time":"2022-07-22T06:00:51.312Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"global.gc() invoked.","time":"2022-07-22T06:00:51.372Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Importing pages has completed.","time":"2022-07-22T06:00:51.431Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"global.gc() invoked.","time":"2022-07-22T06:00:51.437Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Importing attachmentFiles.files has completed.","time":"2022-07-22T06:00:51.499Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"global.gc() invoked.","time":"2022-07-22T06:00:51.501Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Importing revisions has completed.","time":"2022-07-22T06:00:51.565Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"global.gc() invoked.","time":"2022-07-22T06:00:51.570Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Importing attachments has completed.","time":"2022-07-22T06:00:51.639Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"global.gc() invoked.","time":"2022-07-22T06:00:51.647Z","v":0}
app_1            | {"name":"growi:services:ImportService","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Importing attachmentFiles.chunks has completed.","time":"2022-07-22T06:00:51.702Z","v":0}
app_1            | {"name":"growi:services:page","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Page migration processing: (migratedPages=4)","time":"2022-07-22T06:00:51.726Z","v":0}
app_1            | {"name":"growi:services:page","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Successfully updated all descendantCount of public pages.","time":"2022-07-22T06:00:51.761Z","v":0}
app_1            | {"name":"growi:services:page","hostname":"82351dce94ae","pid":14,"level":30,"msg":"Successfully migrated all public pages.","time":"2022-07-22T06:00:51.769Z","v":0}
app_1            | {"name":"express","hostname":"82351dce94ae","pid":14,"req_id":"e5c3064b-d6aa-4533-8b0e-3eabd8b7d1bc","level":50,"msg":"::ffff:10.8.0.66 <-- GET /_api/v3/in-app-notification/list?limit=6 HTTP/1.1 503 39 http://172.31.20.136/admin/importer Chrome 103.0 Windows 10.0.0 1.911181 ms","time":"2022-07-22T06:00:54.342Z","v":0}
app_1            | {"name":"express","hostname":"82351dce94ae","pid":14,"req_id":"2e3a9d99-dd70-4365-9da2-e7723710cd6b","level":50,"msg":"::ffff:10.8.0.66 <-- GET /_api/v3/in-app-notification/status HTTP/1.1 503 39 http://172.31.20.136/admin/importer Chrome 103.0 Windows 10.0.0 1.565051 ms","time":"2022-07-22T06:00:54.380Z","v":0}
app_1            | {"name":"express","hostname":"82351dce94ae","pid":14,"req_id":"93200fa9-ed64-4036-863e-7ee763b15279","level":50,"msg":"::ffff:10.8.0.66 <-- GET /_api/v3/in-app-notification/status HTTP/1.1 503 39 http://172.31.20.136/admin/importer Chrome 103.0 Windows 10.0.0 1.59451 ms","time":"2022-07-22T06:00:56.318Z","v":0}
app_1            | {"name":"express","hostname":"82351dce94ae","pid":14,"req_id":"f2eb5e73-d83f-4324-8906-292ead93cfa8","level":50,"msg":"::ffff:10.8.0.66 <-- GET /_api/v3/in-app-notification/status HTTP/1.1 503 39 http://172.31.20.136/admin/importer Chrome 103.0 Windows 10.0.0 1.704283 ms","time":"2022-07-22T06:00:56.892Z","v":0}

ただ、インポート時にはサーバサイドにはエラー出ていないようでしたが、インポート後に添付画像を含むページを参照した時に下記のエラーがサーバサイドに出ておりました。

app_1            | {"name":"growi","hostname":"82351dce94ae","pid":14,"level":50,"msg":"Uncaught Exception:  MongoGridFSChunkError: ChunkIsWrongSize: Got unexpected length: 12748, expected: 9560
    at /opt/growi/node_modules/mongodb/lib/gridfs/download.js:176:62
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at /opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:477:68
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at completeEndSession (/opt/growi/node_modules/mongodb/lib/sessions.js:147:17)
    at /opt/growi/node_modules/mongodb/lib/sessions.js:157:13
    at maybePromise (/opt/growi/node_modules/mongodb/lib/utils.js:494:5)
    at ClientSession.endSession (/opt/growi/node_modules/mongodb/lib/sessions.js:133:41)
    at cleanupCursor (/opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:530:32)
    at /opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:477:24
    at /opt/growi/node_modules/mongodb/lib/cursor/find_cursor.js:66:13
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at /opt/growi/node_modules/mongodb/lib/operations/execute_operation.js:50:17
    at callbackWithRetry (/opt/growi/node_modules/mongodb/lib/operations/execute_operation.js:81:20)
    at /opt/growi/node_modules/mongodb/lib/cmap/connection_pool.js:272:25
    at handleOperationResult (/opt/growi/node_modules/mongodb/lib/sdam/server.js:363:9)","time":"2022-07-22T06:01:37.705Z","v":0}
app_1            | {"name":"growi","hostname":"82351dce94ae","pid":14,"level":50,"msg":"Uncaught Exception:  MongoGridFSChunkError: ChunkIsWrongSize: Got unexpected length: 24208, expected: 18156
    at /opt/growi/node_modules/mongodb/lib/gridfs/download.js:176:62
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at /opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:477:68
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at completeEndSession (/opt/growi/node_modules/mongodb/lib/sessions.js:147:17)
    at /opt/growi/node_modules/mongodb/lib/sessions.js:157:13
    at maybePromise (/opt/growi/node_modules/mongodb/lib/utils.js:494:5)
    at ClientSession.endSession (/opt/growi/node_modules/mongodb/lib/sessions.js:133:41)
    at cleanupCursor (/opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:530:32)
    at /opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:477:24
    at /opt/growi/node_modules/mongodb/lib/cursor/find_cursor.js:66:13
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at /opt/growi/node_modules/mongodb/lib/operations/execute_operation.js:50:17
    at callbackWithRetry (/opt/growi/node_modules/mongodb/lib/operations/execute_operation.js:81:20)
    at /opt/growi/node_modules/mongodb/lib/cmap/connection_pool.js:272:25
    at handleOperationResult (/opt/growi/node_modules/mongodb/lib/sdam/server.js:363:9)","time":"2022-07-22T06:01:37.712Z","v":0}
app_1            | {"name":"growi","hostname":"82351dce94ae","pid":14,"level":50,"msg":"Uncaught Exception:  MongoGridFSChunkError: ChunkIsWrongSize: Got unexpected length: 8632, expected: 6472
    at /opt/growi/node_modules/mongodb/lib/gridfs/download.js:176:62
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at /opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:477:68
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at completeEndSession (/opt/growi/node_modules/mongodb/lib/sessions.js:147:17)
    at /opt/growi/node_modules/mongodb/lib/sessions.js:157:13
    at maybePromise (/opt/growi/node_modules/mongodb/lib/utils.js:494:5)
    at ClientSession.endSession (/opt/growi/node_modules/mongodb/lib/sessions.js:133:41)
    at cleanupCursor (/opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:530:32)
    at /opt/growi/node_modules/mongodb/lib/cursor/abstract_cursor.js:477:24
    at /opt/growi/node_modules/mongodb/lib/cursor/find_cursor.js:66:13
    at /opt/growi/node_modules/mongodb/lib/utils.js:508:9
    at /opt/growi/node_modules/mongodb/lib/operations/execute_operation.js:50:17
    at callbackWithRetry (/opt/growi/node_modules/mongodb/lib/operations/execute_operation.js:81:20)
    at /opt/growi/node_modules/mongodb/lib/cmap/connection_pool.js:272:25
    at handleOperationResult (/opt/growi/node_modules/mongodb/lib/sdam/server.js:363:9)","time":"2022-07-22T06:01:37.714Z","v":0}

yuki-takei · 2022-07-30T22:24:56Z

#2103 と同じですね。

ChunkIsWrongSize: Got unexpected length: 8632, expected: 6472

おそらくアーカイブの作り方から問題がある気がします。

報告ありがとうございました。ちょっと直近では着手できないのですが、折を見て調査してみます。

n79s added the type/bug label Jul 20, 2022

yuki-takei added the 1️⃣ phase/under-investigation label Jul 30, 2022

yuki-takei removed the type/bug label May 25, 2023

yuki-takei added 5️⃣ phase/resolved and removed 1️⃣ phase/under-investigation labels Aug 21, 2024

yuki-takei mentioned this issue Aug 21, 2024

fix: Import data #8994

Merged

yuki-takei closed this as completed in #8994 Aug 22, 2024

yuki-takei closed this as completed in 9fbf716 Aug 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: 画像添付ファイルを含むページのアーカイブをインポートすると画像がリンクが切れになる #6313

Bug: 画像添付ファイルを含むページのアーカイブをインポートすると画像がリンクが切れになる #6313

n79s commented Jul 20, 2022

yuki-takei commented Jul 21, 2022

n79s commented Jul 22, 2022

yuki-takei commented Jul 30, 2022

Bug: 画像添付ファイルを含むページのアーカイブをインポートすると画像がリンクが切れになる #6313

Bug: 画像添付ファイルを含むページのアーカイブをインポートすると画像がリンクが切れになる #6313

Comments

n79s commented Jul 20, 2022

Environment

Host

Client

How to reproduce? (再現手順)

What happens? (症状)

What is the expected result? (期待される動作)

Note

yuki-takei commented Jul 21, 2022

n79s commented Jul 22, 2022

インポート時に画面では下記Failedがでます。

上記インポート実行時のdocker-compose logs（特にエラーのような内容はありませんでした）

yuki-takei commented Jul 30, 2022