[ETH][ETC] Dag creation error - System.OutOfMemoryException: ethash_full_new IO or memory error

[eakova]

Note

Since there is a closed issue and #186 , It could be better to have open a new one to review this problem.
Current dev and master branch code fails while ETH and ETC dag creation.
Attaching C++ native projects to the solution could help other developers( like me) to fix the issue. it's quite hard to debug with preloaded dlls.

Description

ETH and ETC dag creation stops the node with this error:
System.OutOfMemoryException: ethash_full_new IO or memory error

Steps to Reproduce

OS: Windows 10 or Windows 2012 Datacenter
Ram: 12 GB Ram
CPU: 6 Core AMD
Parity v1.10.0-20180320 beta
Parity , Postgres and VS 2017 are installed in the same VMs.

Already tried these options:

"dagDir": "c:\eth\dag\"
"dagDir": "c:/eth/dag/",
"dagDir": "/eth/dag/",

config.js is below:

{
"logging": {
"level": "info",
"enableConsoleLog": true,
"enableConsoleColors": true,
"logFile": "logtxt.log",
"logBaseDirectory": "",
"perPoolLogFile": true
},
"banning": {
"manager": "integrated",
"banOnJunkReceive": true,
"banOnInvalidShares": false
},
"notifications": {
"enabled": true,
"email": {
"host": "smtp.example.com",
"port": 587,
"user": "user",
"password": "password",
"fromAddress": "[email protected]",
"fromName": "support"
},
"admin": {
"enabled": false,
"emailAddress": "",
"notifyBlockFound": true
}
},
"persistence": {
"postgres": {
"host": "127.0.0.1",
"port": 5432,
"user": "miningcore",
"password": "miningcore",
"database": "miningcore"
}
},
"paymentProcessing": {
"enabled": true,
"interval": 600,
"shareRecoveryFile": "recovered-shares.txt"
},
"pools": [
{
"id": "eth1",
"enabled": true,
"dagDir": "c:\eth\dag\",
"coin": {
"type": "ETH"
},
"address": "0x*******************",
"rewardRecipients": [
{
"type": "op",
"address": "0x*******************",
"percentage": 1
}
],
"blockRefreshInterval": 500,
"clientConnectionTimeout": 600,
"banning": {
"enabled": true,
"time": 600,
"invalidPercent": 50,
"checkThreshold": 50
},
"ports": {
"3072": {
"difficulty": 0.1,
"varDiff": {
"minDiff": 0.05,
"maxDiff": null,
"targetTime": 15,
"retargetTime": 90,
"variancePercent": 30
}
}
},
"daemons": [
{
"host": "127.0.0.1",
"port": 8545,
"user": "user",
"password": "password"
}
],
"paymentProcessing": {
"enabled": true,
"minimumPayment": 0.01,
"minimumPaymentToPaymentId": 5.0,
"payoutScheme": "PPLNS",
"payoutSchemeConfig": {
"factor": 2.0
},
"coinbasePassword": " ",
"keepUncles": false,
"keepTransactionFees": false
}
}
]
}

Expected behavior

Proper DAG creation without error.

Actual behavior

0 sized DAG created with memory error.

[Konstantin35]

Fix the settings
"dagDir": "C:/eth/dag"
automatic selection of the swap file

[eakova]

@Konstantin35 I've already tried these options:

"dagDir": "c:\eth\dag"
"dagDir": "c:/eth/dag/",
"dagDir": "/eth/dag/",

nothing changed.

By the way , what do you mean with "automatic selection of the swap file" ?

[eakova]

The bug is at this line in io.c

if (fseek(f, (long int)(file_size + ETHASH_DAG_MAGIC_NUM_SIZE - 1), SEEK_SET) != 0)

Regarding
https://github.com/ethereum/cpp-ethereum/blob/develop/libethash/io.c
https://github.com/ethereum/cpp-ethereum/blob/7d8a1b9b4a878367a79d4641085d517567c72e6f/libethash/io_posix.c

The 64 bit needs to use fseeko for linux and _fseeki64 for windows. Soon I will be sending PR but I am sure repo owner can fix this quickly.

[oliverw]

@eakova Nice analysis.

[bahelit]

I've applied #329 but miningcore still crashes when generating the dag.

Also added a swap file to the server and set the dagDir in the config file but the problem still exists.

[dhagell]

changing it to:
_fseeki64(f, (__int64)(file_size + ETHASH_DAG_MAGIC_NUM_SIZE - 1), SEEK_SET);

for windows 10 worked for me. additionally, changing the cast type.
Thanks.
https://stackoverflow.com/questions/4034227/on-windows-fseeki64-does-not-seek-to-seek-end-correctly-for-large-files

[peysab]

Hey guys,
I'm facing the same issue, I ran the parity on an ubuntu server and trying to connect to it using a windows, any idea what to do?