Caused by: java.lang.OutOfMemoryError: Java heap space #417

kazurayam · 2023-04-30T07:09:17Z

I created src/test/java/issues/Issue416Test.java

It failed due to OutOfMemoryError

> Task :test
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] jobName=NISA_Chronos, store=src/test/fixtures/issue#416/store
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] currentMaterialList.getJobTimestamp()=20230430_085419
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] currentMaterialList.size()=44
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] priorTo=20230430_085419
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] func=com.kazurayam.materialstore.base.reduce.Reducer$$Lambda$366/0x000000080020d440@69796bd0
[Test worker] INFO com.kazurayam.materialstore.core.StoreImpl - [reflect] priorTo=20230430_085419
[Test worker] INFO com.kazurayam.materialstore.core.StoreImpl - [reflect] jt=20230430_081455
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] previousMaterialList.getJobTimestamp()=_
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] previousMaterialList.size()=0
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.zipper.Zipper - #zipMaterials foundLeftCount=0 is unusual
[Test worker] INFO com.kazurayam.materialstore.base.reduce.Reducer - [chronos] reducedMPG.size()=44
[Test worker] DEBUG com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #process mpg.getCountTotal()=44
[Test worker] INFO com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #process {"checked":false,"isBachelor":true,"diffRatio":0.0,"fileTypeExtension":"csv","queryOnMetadata":{"URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/23.xlsx", "URL.port":"80", "URL.protocol":"https", "seq":"1", "sheet_index":"0", "sheet_name":"指定インデックス投資信託"},"identification":"{\"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\", \"seq\":\"1\", \"sheet_index\":\"0\", \"sheet_name\":\"指定インデックス投資信託\"}","left":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"},"right":{"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"0f58b2d68b0e043644b8b1436f4814f87cf8e459","fileType":"csv","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "sheet_index":{"key":"sheet_index","value":"0","matchedByAster":true}, "sheet_name":{"key":"sheet_name","value":"指定インデックス投資信託","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"sheet_index\":\"0\", \"sheet_name\":\"指定インデックス投資信託\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/0f58b2d68b0e043644b8b1436f4814f87cf8e459.csv","diffability":"AS_TEXT"},"diff":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"}}
[Test worker] WARN com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #stuffDiffByDiffer left Material was NULL_OBJECT. right={"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"0f58b2d68b0e043644b8b1436f4814f87cf8e459","fileType":"csv","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "sheet_index":{"key":"sheet_index","value":"0","matchedByAster":true}, "sheet_name":{"key":"sheet_name","value":"指定インデックス投資信託","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"sheet_index\":\"0\", \"sheet_name\":\"指定インデックス投資信託\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/0f58b2d68b0e043644b8b1436f4814f87cf8e459.csv","diffability":"AS_TEXT"}
[Test worker] INFO com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #process {"checked":false,"isBachelor":true,"diffRatio":0.0,"fileTypeExtension":"csv","queryOnMetadata":{"URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/23.xlsx", "URL.port":"80", "URL.protocol":"https", "seq":"1", "sheet_index":"1", "sheet_name":"指定インデックス投資信託以外の投資信託（アクティブ運用投信等）"},"identification":"{\"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\", \"seq\":\"1\", \"sheet_index\":\"1\", \"sheet_name\":\"指定インデックス投資信託以外の投資信託（アクティブ運用投信等）\"}","left":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"},"right":{"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"a057919e32886466213e5393068d27f39bd48f5f","fileType":"csv","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "sheet_index":{"key":"sheet_index","value":"1","matchedByAster":true}, "sheet_name":{"key":"sheet_name","value":"指定インデックス投資信託以外の投資信託（アクティブ運用投信等）","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"sheet_index\":\"1\", \"sheet_name\":\"指定インデックス投資信託以外の投資信託（アクティブ運用投信等）\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/a057919e32886466213e5393068d27f39bd48f5f.csv","diffability":"AS_TEXT"},"diff":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"}}
[Test worker] WARN com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #stuffDiffByDiffer left Material was NULL_OBJECT. right={"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"a057919e32886466213e5393068d27f39bd48f5f","fileType":"csv","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "sheet_index":{"key":"sheet_index","value":"1","matchedByAster":true}, "sheet_name":{"key":"sheet_name","value":"指定インデックス投資信託以外の投資信託（アクティブ運用投信等）","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"sheet_index\":\"1\", \"sheet_name\":\"指定インデックス投資信託以外の投資信託（アクティブ運用投信等）\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/a057919e32886466213e5393068d27f39bd48f5f.csv","diffability":"AS_TEXT"}
[Test worker] INFO com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #process {"checked":false,"isBachelor":true,"diffRatio":0.0,"fileTypeExtension":"csv","queryOnMetadata":{"URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/23.xlsx", "URL.port":"80", "URL.protocol":"https", "seq":"1", "sheet_index":"2", "sheet_name":"上場株式投資信託（ETF）"},"identification":"{\"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\", \"seq\":\"1\", \"sheet_index\":\"2\", \"sheet_name\":\"上場株式投資信託（ETF）\"}","left":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"},"right":{"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"c97fb9280a513fd9bf90904ed36e756bae5873c8","fileType":"csv","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "sheet_index":{"key":"sheet_index","value":"2","matchedByAster":true}, "sheet_name":{"key":"sheet_name","value":"上場株式投資信託（ETF）","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"sheet_index\":\"2\", \"sheet_name\":\"上場株式投資信託（ETF）\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/c97fb9280a513fd9bf90904ed36e756bae5873c8.csv","diffability":"AS_TEXT"},"diff":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"}}
[Test worker] WARN com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #stuffDiffByDiffer left Material was NULL_OBJECT. right={"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"c97fb9280a513fd9bf90904ed36e756bae5873c8","fileType":"csv","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "sheet_index":{"key":"sheet_index","value":"2","matchedByAster":true}, "sheet_name":{"key":"sheet_name","value":"上場株式投資信託（ETF）","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"sheet_index\":\"2\", \"sheet_name\":\"上場株式投資信託（ETF）\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/c97fb9280a513fd9bf90904ed36e756bae5873c8.csv","diffability":"AS_TEXT"}
[Test worker] INFO com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #process {"checked":false,"isBachelor":true,"diffRatio":0.0,"fileTypeExtension":"xlsx","queryOnMetadata":{"URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/23.xlsx", "URL.port":"80", "URL.protocol":"https", "seq":"1"},"identification":"{\"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\", \"seq\":\"1\"}","left":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"},"right":{"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"cd1befebe8336e3837aeffd2856220a4e7b7e4ab","fileType":"xlsx","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/cd1befebe8336e3837aeffd2856220a4e7b7e4ab.xlsx","diffability":"UNABLE"},"diff":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"}}
[Test worker] WARN com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #stuffDiffByDiffer left Material was NULL_OBJECT. right={"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"cd1befebe8336e3837aeffd2856220a4e7b7e4ab","fileType":"xlsx","metadata":{"seq":{"key":"seq","value":"1","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/23.xlsx","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"seq\":\"1\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/23.xlsx\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/cd1befebe8336e3837aeffd2856220a4e7b7e4ab.xlsx","diffability":"UNABLE"}
[Test worker] INFO com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #process {"checked":false,"isBachelor":true,"diffRatio":0.0,"fileTypeExtension":"png","queryOnMetadata":{"URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/24.pdf", "URL.port":"80", "URL.protocol":"https", "page":"1", "seq":"5"},"identification":"{\"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/24.pdf\", \"URL.port\":\"80\", \"URL.protocol\":\"https\", \"page\":\"1\", \"seq\":\"5\"}","left":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"},"right":{"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"3a98c4ba471f11462d06a4c94ef4daa4010a466a","fileType":"png","metadata":{"page":{"key":"page","value":"1","matchedByAster":true}, "seq":{"key":"seq","value":"5","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/24.pdf","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"page\":\"1\", \"seq\":\"5\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/24.pdf\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/3a98c4ba471f11462d06a4c94ef4daa4010a466a.png","diffability":"AS_IMAGE"},"diff":{"jobName":"_","jobTimestamp":"_","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"_/_/objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"}}
[Test worker] WARN com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #stuffDiffByDiffer left Material was NULL_OBJECT. right={"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"3a98c4ba471f11462d06a4c94ef4daa4010a466a","fileType":"png","metadata":{"page":{"key":"page","value":"1","matchedByAster":true}, "seq":{"key":"seq","value":"5","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/24.pdf","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{\"page\":\"1\", \"seq\":\"5\", \"URL.host\":\"www.fsa.go.jp\", \"URL.path\":\"/policy/nisa/20170614-2/24.pdf\", \"URL.port\":\"80\", \"URL.protocol\":\"https\"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/3a98c4ba471f11462d06a4c94ef4daa4010a466a.png","diffability":"AS_IMAGE"}

org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 1.
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:63)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy5.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.OutOfMemoryError: Java heap space
	at ru.yandex.qatools.ashot.comparison.PointsMarkupPolicy.addDiffPoint(PointsMarkupPolicy.java:41)
	at ru.yandex.qatools.ashot.comparison.ImageDiff.addDiffPoint(ImageDiff.java:51)
	at ru.yandex.qatools.ashot.comparison.ImageDiffer.markDiffPoints(ImageDiffer.java:81)
	at ru.yandex.qatools.ashot.comparison.ImageDiffer.makeDiff(ImageDiffer.java:57)
	at ru.yandex.qatools.ashot.comparison.ImageDiffer.makeDiff(ImageDiffer.java:96)
	at com.kazurayam.materialstore.base.reduce.differ.ImageDifferToPNG.stuffDiff(ImageDifferToPNG.java:41)
	at com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor.stuffDiffByDiffer(DiffingMPGProcessor.java:68)
	at com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor.process(DiffingMPGProcessor.java:44)
	at com.kazurayam.materialstore.base.inspector.InspectorImpl.reduceAndSort(InspectorImpl.java:45)
	at issues.Issue416Test.test_reproduce(Issue416Test.java:49)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$166/0x00000008001b7c40.apply(Unknown Source)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall$$Lambda$167/0x00000008001b7040.apply(Unknown Source)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$$Lambda$327/0x00000008001f8c40.apply(Unknown Source)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)

The text was updated successfully, but these errors were encountered:

kazurayam · 2023-04-30T07:19:16Z

https://www.jetbrains.com/help/idea/increasing-memory-heap.html

I allocated 4096MB, 8192MB, but was not enough.

kazurayam · 2023-04-30T07:28:23Z

OutOfMemoryError was raised at

[Test worker] INFO com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #process {"checked":false,"isBachelor":true,"diffRatio":0.0,"fileTypeExtension":"png","queryOnMetadata":{"URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/24.pdf", "URL.port":"80", "URL.protocol":"https", "page":"1", "seq":"5"},"identification":"{"URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/24.pdf", "URL.port":"80", "URL.protocol":"https", "page":"1", "seq":"5"}","left":{"jobName":"","jobTimestamp":"","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"//objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"},"right":{"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"3a98c4ba471f11462d06a4c94ef4daa4010a466a","fileType":"png","metadata":{"page":{"key":"page","value":"1","matchedByAster":true}, "seq":{"key":"seq","value":"5","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/24.pdf","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{"page":"1", "seq":"5", "URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/24.pdf", "URL.port":"80", "URL.protocol":"https"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/3a98c4ba471f11462d06a4c94ef4daa4010a466a.png","diffability":"AS_IMAGE"},"diff":{"jobName":"","jobTimestamp":"","id":"0000000000000000000000000000000000000000","fileType":"","metadata":{},"identification":"{}","relativeUrl":"//objects/0000000000000000000000000000000000000000.","diffability":"UNABLE"}}

[Test worker] WARN com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor - #stuffDiffByDiffer left Material was NULL_OBJECT. right={"jobName":"NISA_Chronos","jobTimestamp":"20230430_085419","id":"3a98c4ba471f11462d06a4c94ef4daa4010a466a","fileType":"png","metadata":{"page":{"key":"page","value":"1","matchedByAster":true}, "seq":{"key":"seq","value":"5","matchedByAster":true}, "URL.host":{"key":"URL.host","value":"www.fsa.go.jp","matchedByAster":true}, "URL.path":{"key":"URL.path","value":"/policy/nisa/20170614-2/24.pdf","matchedByAster":true}, "URL.port":{"key":"URL.port","value":"80","matchedByAster":true}, "URL.protocol":{"key":"URL.protocol","value":"https","matchedByAster":true}},"identification":"{"page":"1", "seq":"5", "URL.host":"www.fsa.go.jp", "URL.path":"/policy/nisa/20170614-2/24.pdf", "URL.port":"80", "URL.protocol":"https"}","relativeUrl":"NISA_Chronos/20230430_085419/objects/3a98c4ba471f11462d06a4c94ef4daa4010a466a.png","diffability":"AS_IMAGE"}

This message tells that DiffMPGProcessor is trying to stuff a diff of PNG while the left Material was NULL_OBJECT. Is it worth? --- No.

kazurayam · 2023-04-30T07:42:29Z

com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor has the following method:

private MaterialProduct stuffDiffByDiffer(final MaterialProduct materialProduct)
            throws MaterialstoreException {
        IFileType fileType;
        if (materialProduct.getLeft().equals(Material.NULL_OBJECT)) {
            logger.warn(String.format("#stuffDiffByDiffer left Material was NULL_OBJECT. right=%s",
                    materialProduct.getRight()));
            fileType = materialProduct.getRight().getIndexEntry().getFileType();
        } else if (materialProduct.getRight().equals(Material.NULL_OBJECT)) {
            logger.warn(String.format("#stuffDiffByDiffer right Material was NULL_OBJECT. left=%s",
                    materialProduct.getLeft()));
            fileType = materialProduct.getLeft().getIndexEntry().getFileType();
        } else {
            fileType = materialProduct.getRight().getIndexEntry().getFileType();
        }
        Differ differ = differs.get(fileType);
        return differ.stuffDiff(materialProduct);
    }

kazurayam · 2023-04-30T13:15:40Z

I added src/test/java/issues/Issue416Test.java
I could reproduce the OutOfMemoryError with this test.

So I am able to study how to fix this problem.

kazurayam · 2023-05-01T01:28:16Z

I added a test case in
https://github.com/kazurayam/materialstore/blob/develop/src/test/java/issues/Issue417Test.java

@Test
    public void test_AShot_imageDiff_how_much_memory_it_requires() throws MaterialstoreException, IOException {
        Path noMaterialFoundPNG = projectDir
                .resolve("src/main/resources/com/kazurayam/materialstore/core/NoMaterialFound.png");
        BufferedImage leftImage = ImageIO.read(noMaterialFoundPNG.toFile());
        Path fixturePNG = projectDir
                .resolve("src/test/fixtures/issue#417/3a98c4ba471f11462d06a4c94ef4daa4010a466a.png");
        BufferedImage rightImage = ImageIO.read(fixturePNG.toFile());
        ImageDiffer imgDiff = new ImageDiffer();
        ImageDiff imageDiff = imgDiff.makeDiff(leftImage, rightImage);
        // this causes OutOfMemoryError
        assertNotNull(imageDiff);
    }

The left image has the size of 1440 × 576, it looks like this:

The right image has the size of 3445 × 4872, it looks like this:

These 2 PNG images are very different.

Caused by: java.lang.OutOfMemoryError: Java heap space
	at ru.yandex.qatools.ashot.comparison.PointsMarkupPolicy.addDiffPoint(PointsMarkupPolicy.java:41)
	at ru.yandex.qatools.ashot.comparison.ImageDiff.addDiffPoint(ImageDiff.java:51)
	at ru.yandex.qatools.ashot.comparison.ImageDiffer.markDiffPoints(ImageDiffer.java:81)
        ...

OutOfMemoryError was raised at ImageDiffer.java:81 becase there found too much different pixels between the 2 PNG images.

kazurayam · 2023-05-01T01:32:07Z

com.kazurayam.materialstore.base.reduce.DiffingMPGProcessor#stuffDiffByDiffer() tries to make a diff of 2 PNG images

It is a mistake.

The left Material was NULL_OBJECT, therefore the stuffDiffByDiffer() should abandon making a diff. I should regard

https://user-images.githubusercontent.com/4467417/235387855-945d8c6e-a173-418a-8c73-4b9d0edcd11f.png

as a diff result.

kazurayam · 2023-05-01T12:26:14Z

done at v0.16.2

kazurayam · 2023-05-10T03:10:24Z

pazone/ashot#359

use ImageMarkupPolicy instead of PointsMarkupPolicy if you expect high number of differences

kazurayam · 2023-05-11T14:21:16Z

done at 0.16.6-SNAPSHOT

kazurayam added a commit that referenced this issue Apr 30, 2023

#417 OutOfMemoryError reproduced

af1c7cc

kazurayam added a commit that referenced this issue May 1, 2023

#417

b461359

kazurayam added a commit that referenced this issue May 1, 2023

#417 fixed OutOfMemoryError by ImageDifferToPng#stuffDiff()

18cb5c5

kazurayam added a commit that referenced this issue May 1, 2023

#417

90dc0ed

kazurayam closed this as completed May 1, 2023

kazurayam reopened this May 10, 2023

kazurayam added a commit that referenced this issue May 11, 2023

#417 use ImageMarkupPolicy instead of PointsMarkupPolicy

f870163

kazurayam closed this as completed May 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Caused by: java.lang.OutOfMemoryError: Java heap space #417

Caused by: java.lang.OutOfMemoryError: Java heap space #417

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented May 1, 2023 •

edited

Loading

kazurayam commented May 1, 2023 •

edited

Loading

kazurayam commented May 1, 2023

kazurayam commented May 10, 2023

kazurayam commented May 11, 2023

Caused by: java.lang.OutOfMemoryError: Java heap space #417

Caused by: java.lang.OutOfMemoryError: Java heap space #417

Comments

kazurayam commented Apr 30, 2023 • edited Loading

kazurayam commented Apr 30, 2023 • edited Loading

kazurayam commented Apr 30, 2023 • edited Loading

kazurayam commented Apr 30, 2023 • edited Loading

kazurayam commented Apr 30, 2023 • edited Loading

kazurayam commented May 1, 2023 • edited Loading

kazurayam commented May 1, 2023 • edited Loading

kazurayam commented May 1, 2023

kazurayam commented May 10, 2023

kazurayam commented May 11, 2023

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented Apr 30, 2023 •

edited

Loading

kazurayam commented May 1, 2023 •

edited

Loading

kazurayam commented May 1, 2023 •

edited

Loading