ECIP-? Limit DAG growth

[arvicco]

No description provided.

[arvicco]

In order to make Ethereum mining memory-hard, its PoW algo employs DAG. It started at 1GB size and is growing linearly with the blockchain size. There are several problems with it, if we want PoW to be viable long-term for ETC.

1. Over time the DAG outpaces most GPU memory size, and GPU mining may become impossible.
2. Even the DAG size above 2GB is pushing the limits of memory bandwidth of current commercial GPUs, so the hashrate decreases dramatically.
3. ... Anything else?

So far I've seen the following suggestions to deal with this issue:

1. Tweak with the DAG expansion parameters to slow or stop its growth (ECIP-1010 Delay Difficulty Bomb Explosion #4 (comment))
2. Extend the epoch (DAG regeneration interval) length from current 30000 blocks to something else, without doing any other changes
3. Make DAG oscillate cyclically between 1 and 2 GB sizes (ECIP-1010 Delay Difficulty Bomb Explosion #4 (comment))

[arvicco]

@mikeyb Was going to take a stab on ECIP for DAG issue, but other suggestions welcome as well.

[marcusrbrown]

Hey - the replay attack ECIP is in progress, currently named 1011 in my branch. I suggest that we leave ECIPs open as PRs while they are discussed instead of duplicating efforts by creating new issues. That way, if someone other than the original author wants to contribute edits, they can pull in that branch and push to the PR directly.

Let's leave the ECIP number empty (simply refer to the PR by the GitHub id, e.g. #6) until the ECIP is accepted as a draft and merged. When the ECIP is ready to move into another phase, then the author (or anyone else) can open a new PR for that ECIP, which will have a number. This makes it easier to work simultaneously without worrying about what particular number a ECIP has.

[trustfarm-dev]

copy from 1010 thread,

I think remove difficulties bomb , we must consider DAG size.

Easily , real examples I describe,
AMD 280x real hash rate case::
2015.08 , DAG 1GB , Hash 26Mh/s
2016.09 , DAG 1.6GB , Hash 15Mh/s

So, simple rough idea suggestion is ,

method 1.

1. DAG Size and DAG-initialvector Initialzation , if DAG Size over 2GB.
   that's to reset 1GB.

method 2.
2. DAG Size over 2GB , after another 30000Blocks, DAG Size will decrease , and 1GB then increase.
make swing in 1GB ~ 2GB Boundary.

[trustfarm-dev]

important information. it's already discussed in forum.ethereum.org DAG Simulations.

@mikeyb commented 14 days ago • edited
Are we also considering a DAG file size slowdown? If DAG size generation keeps up at the constant pace PoW mining will become obsolete long before the frozen bomb goes off.

I suggest we either extend epoch times or slow down DAG size

Unless I am missing something? If we plan to keep PoW in any form, this will also have to be considered.

https://github.com/Genoil/dagSimCL

[trustfarm-dev]

also copy from previous notation. it's important materials.

@realcodywburns commented 13 days ago • edited
It looks like the dag is more or less a secondary bomb. It was designed to keep Asics out of mining, but it was never limited because the thought was PoS would come. If it is allowed to keep growing it will essentially make mining only available to those who can afford high dollar cards with high ram. Essentially having the same effect as Asics, centralizing the mining power to a few. @mikeyb is working on an ecip and doing further testing. My recommendation would be to stop the dag growth and keep it at a number high enough to prevent Asics but low enough to keep everyone equal. Also aim to have a 'dag steering committee' or something that reviews the gpu market and makes recommendations everytime a system upgrade is announced.

[trustfarm-dev]

http://forum.ethereum.org/search?Page=p7&Search=DAG

then, there's real miner's pain-points.
Finally, GPU manufaturer and seller will win. if DAG has increase always.

[trustfarm-dev]

copy from previous threads related on DAG size.

@realcodywburns commented 13 days ago
@elaineo it depends on the purpose of the dag in the grand scheme of ethereum classic. At it's current rate it will reach 2gb in February 2017 and 2.5gb ~August 2017. The limiting factor in the mining cards seems to be the bus speed and cards with 3gb or less become useless around 2.2gb dag size. If the intent is to stop Asics from coming, the current dag would work for a few years. If we want 'normal users' to be able to mine with current rigs, then 2gb should work just fine. The serious miners do not buy off the shelf components. Asics or not, they can dump the cash on custom gear that will centralize the hash power.

[trustfarm-dev]

copy from previous threads related on DAG Size

@arvicco commented 13 hours ago
@trustfarm-dev Hmmm, cyclic DAG size oscillations, instead of constant bloat... interesting, I like this. Since the only function of DAG was to make mining memory-hard and hamper ASIC development, this approach serves such purpose just as well.

[arvicco]

@igetgames Good points on leaving the ECIP numbers empty. The problem is that absent the dedicated Github issue (like, for DAG bloat here), people start discussing it in unrelated issues - like DAG discussion regularly resurfacing in ECIP-1010 issue. So, it's better to have a dedicated issue to discuss each topic, and once ECIPs are ready that address it (possibly, competing proposals, as I mentioned there are different approaches to fixing DAG issue, for example) they will be linked from this more general topic discussion issue.

[marcusrbrown]

@arvicco That makes sense. The way I mentioned works better when the ECIP comes first. Linking PRs and issues is trivial so either approach should be fine. As far as the subject, you can have ECIP-? as a placeholder as you do now, or just leave it empty and keep the topic (assuming it will be linked to a ECIP PR in the PRs message).

[trustfarm-dev]

I also agree on @arvicco @igetgames comments.
In view of developer and disccussion , I think it's more efficient , related material's spread in one threads.
So I've copied , I hope this DAG issue will be treated more seriously. In the miner view , DAG size increaments are big issues.

[arvicco]

@trustfarm-dev Yes, it definitely needs to be resolved at the same time as diff bomb per ECIP-1010

[mikeyb]

DAG size is not an issue. As HBM becomes standard ( http://wccftech.com/amd-rx-490-4k-gaming-card/ ) the point at which dag size is affected is much much greater than we have to worry about for quite a few years, and by then the hardware will support it most likely as well. rx480s are already capable of using HBM gpu memory, but currently aren't (AMD makes more profit selling the cheaper GDDR5 ram).

Miners should be encouraged to upgrade their mining hardware as time goes on, as with any PoW coin when technology improves, your profit margins are best with the latest tech.

rx480 is the best bang for the buck these days, and I cant imagine 280x being very profitable for much longer anyways.

If we would like to setup a discussion about this, let me know. This is coming from someone who was very pro dag size modifications, but after exploring every angle it is just not necessary. Yes it will knock out some home miners with 2GB cards, and eventually anyone using GDDR5 gpu cards.

A final note, the DAG size is directly related to memory bandwidth. This should continue to increase, and I think AMD might smarten up and start making mining specific GPU cards.

rx490 chart from url

[trustfarm-dev]

@mikeyb I slight agree your saying.
But, In the view of CryptoCurrency miner , they should have a chances and opportunities , if someone have a Latest GPU , even though old-fashioned GPU or CPU, or Anyother Brands of GPU, participate easily, prevent ASIC , it's needs.

We can't enforce or drive miner to change their devices.

ETC developer's are not a sales man of AMD.

So, Pro-Miner will be choose their device. but, most of entering ETC mining. expensive GPU is big barrier for participate in ETC community.

Thus, I suggest that DAG size discussion should be treated in this time with Difficulties Bomb issues.

[mikeyb]

1. I am a miner. I have mined bitcoin, scrypt coins and now ethash coins.
2. Bitcoin miners upgrade their hardware when their hardware becomes obsolete. When CPU mining was outdated, miners moved to GPU mining. When GPU mining was outdated, miners moved to ASIC mining. When the original ASIC mining was outdated, miners moved to more powerful ASICs (S3 -> S5 -> S7 -> S9).
3. We are not forcing anyone to change their devices. If they want to stay current as with any mining operation (even mining gold in the real world), they will upgrade or fall behind.
4. I am not a sales man of AMD. I am a salesman of not changing the protocol when unnecessary.

So no, I highly advise against changing protocol to support a very small handful of miners who refuse to upgrade their mining operation to not fall behind on the times. The DAG size is not an issue, the refusal of miners to upgrade is however. The latest GPU cards are the same price as the 280x when it was released. Price will not keep people from mining.

Also, to keep the DAG size under 2GB WILL cause ASICs to become popular for ethash mining. The only thing keeping ethash ASIC resistance is the fact that making a ASIC for it would cost a bit of money as each ASIC chip needs enough memory to fit the DAG. If the DAG stays below 2GB, then it is cheaper for me to make an ASIC device as I only need 2GB per ASIC instead of 3GB.

[trustfarm-dev]

In case of pro-miner will spend their money to get higher hashrates.
I'm also higher experienced on mining.

And, then who will get earn many money? when you upgrade devices? also, you get , but most of miner has earned funds to spend for upgrade.

In case of memory hard DAG mining , miner should spend their funds every year.
It's not a good , I thinks.

In case of ASIC device, Dagger hasimoto algorithm is memory hard algorithm, it's also different than script which has similar memory hard algorithm.
I think, ASIC miner will not come out easily. Investment will not return compared GPU mining.

In case of Bitcoin mining , they have sha digest algorithm itself. so, asic miner will come out easily.
But, ETHash isn't.

My deep meaning of this comments are, make ETC to more friendly beginner and try to mining.
Lower the entrance - barrier. It'll makes ETC will grown and popular.

Even though, easy entering ETC world, get a higher funds, they upgrade their devices, surely.
And if DAG size will be grown over 3~4GB, It'll not easy to pumpup hash.

refer simulation of genoil's dagger,
http://forum.ethereum.org/discussion/3947/dagger-simulator

It's very critical issues on miner, making blocks.

We needs to discuss on this things now.

[mikeyb]

If you are experienced in mining then you will understand that Heliox's dag simulations in that thread are old as dirt (using old as dirt gpu cards). The more memory bandwidth you have, the more DAG you can process. A rx490 will be able to handle a 5+GB dag file.

As you can see from https://docs.google.com/spreadsheets/d/1ZXNrSCNV0HGWU7zOTUyIIRUGv5M44P6wiAZclpY4Y2Q/edit#gid=127344520 the DAG file wont even hit 3GB for another 1+ years.

There is nothing to discuss as far as I am concerned. Just because you dont want to upgrade your mining hardware, doesn't mean we should decrease network security.

We are not changing protocol to make it easier or cheaper for people to mine, this is how ASICs gain popularity and then your 280x is even more worthless.

Tell me, can you explain WHY hash rates drop when the DAG hits a certain file size for a specific card? I think I might be having a discussion with someone who does not even understand why their hashrates would drop, therefore asking for changes to something they don't even understand as the underlying issue.

Also, when I continue to build my mining farm with newer and faster GPU cards, your 2GB 280x will become more and more obsolete as the network hashing power grows, so regardless of if we stop the DAG growth, your 280x will become obsolete anyways.

[trustfarm-dev]

@mikeyb
Firstly, I'm ETC supporter. don't mis-undertanding of me.
You can get information about on http://security7218.org/en/sub_2.html
I know geth and ethereum weakness and support. when and eth , dao hack.
And never stick to fasten to 280x supporter, it's just examples.
Please check, What I've hope and considerations.
DAG Size is small thing. but very important. most of miner thinks that who have worrying on that.

And Network block chain security is more strengthen by participation of miner and community users.
It's not wholly depends on Hardware.

[mikeyb]

I think it is a great idea to get people mining, and I feel we can implement ways for the masses to utilize share based ownership so anyone even with a some change can get into mining. No need to change the DAG protocol at this point. If for some reason memory bandwidth stops growing, we can work out a simplified solution many years down the road.

The idea is everyone works together to make the most efficient use of our hashing and buying power.

[evieira55]

Just a thought here but what if we keep the DAG at a constant size when we also diffuse the Difficulty Bomb?. It would incentivize modern day cards but at the same time it would remain minable by everyone to include 2 gb cards for a LONG time. and in 4 years we can see where we are at. We all know that for a World computer to exist there needs to be alot of distributed HPC (I.E GPU's mining) so why not keep as many people in business by mining as possible? it will not hurt anyone it will just increase hashrate which we all know is good.

[mikeyb]

This can probably be closed now as limiting DAG is highly unnecessary with the introduction of high bandwidth gpu memory designs. While I understand those with 2GB cards want to still mine, upgrading to 4GB (selling 2GB cards to offset cost) isn't a dramatic cost increase. Not to mention cards running 2GB are likely to be so energy inefficient soon that it wont be profitable, especially as the network difficulty continues to rise. Also the rate of DAG size growth really slows down around 2.5GB and by the time we reach that size HBM based gpu chips will be standard and can easily chew through DAG files of this size.

[elaineo]

Is it worth revisiting the issue in the future?

[mikeyb]

According to the DAG size growth tab on the ETC Bomb-Sheet we wont hit 2.5 GB DAG size until 10/27/2022 and 2.61 GB DAG size around 1/13/2093. We can probably revisit sometime around 12/6/4047 when it hits 2.75 GB 😂

[elaineo]

@mikeyb ok i'll put it on my calendar 😆

[trustfarm-dev]

@elaineo @mikeyb @arvicco We' also consider it with blocktime.
from above ETC Bomb-sheet , in delayed beginning of block time increasing is around 04/2018. next year.
Block time is also important factor for blockchain, it also influenced by DAG Size.
I think close this ECIP now, in some time, we need to consider again.
Happy New Year. All

[evieira55]

another side of this that we need to consider is. are there any benefits at all to a faster growing DAG? as in. increase the DAG size growth in order to incentive better cards mining on the chain I.E more expensive machines as in bitcoin which leads to a more valuable token. just a thought

…

On Wed, Jan 18, 2017 at 5:53 PM, Trustfarm-heart ***@***.***> wrote: @elaineo <https://github.com/elaineo> @mikeyb <https://github.com/mikeyb> @arvicco <https://github.com/arvicco> We' also consider it with blocktime. from above ETC Bomb-sheet , in delayed beginning of block time increasing is around 04/2018. next year. Block time is also important factor for blockchain, it also influenced by DAG Size. I think close this ECIP now, in some time, we need to consider again. Happy New Year. All — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#6 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABcPA9Mir7FBAVVHEReKJGI-rgdxzu42ks5rTtAogaJpZM4KI7-q> .

-- Eliezer Fernandes Vieira

[evieira55]

we can even consider a total DAG size decrease in order to get MORE cards on the chain. its our token we can do whatever pleases us :) On Thu, Jan 19, 2017 at 10:11 AM, Eliezer Vieira <[email protected]> wrote:

…

another side of this that we need to consider is. are there any benefits at all to a faster growing DAG? as in. increase the DAG size growth in order to incentive better cards mining on the chain I.E more expensive machines as in bitcoin which leads to a more valuable token. just a thought On Wed, Jan 18, 2017 at 5:53 PM, Trustfarm-heart ***@***.*** > wrote: > @elaineo <https://github.com/elaineo> @mikeyb <https://github.com/mikeyb> > @arvicco <https://github.com/arvicco> We' also consider it with > blocktime. > from above ETC Bomb-sheet , in delayed beginning of block time increasing > is around 04/2018. next year. > Block time is also important factor for blockchain, it also influenced by > DAG Size. > I think close this ECIP now, in some time, we need to consider again. > Happy New Year. All > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#6 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABcPA9Mir7FBAVVHEReKJGI-rgdxzu42ks5rTtAogaJpZM4KI7-q> > . > -- Eliezer Fernandes Vieira

-- Eliezer Fernandes Vieira

[mikeyb]

Increasing size on a faster scale would lead to centralization as entry requires more upfront costs.
Decreasing has no real benefit. 2G cards are a very small sector of the network hashrate and security. No need to change protocol for that. Not to mention GPU asics are getting better/available

[realcodywburns]

We would reach 2.5 gb dag by block 5 million. That could be an issue.

[mikeyb]

The performance hit should be quite minimal at 2.5GB unless using very old 4GB cards, the newer currently used cards can run dags in the 2.6GB+ range without performance hits. By the time this becomes and issue, the correction would be to upgrade your hardware to 2017 standards as HBM should really be going mainstream this year in most video cards.

[trustfarm-dev]

@mikeyb @arvicco @igetgames @elaineo @evieira55
Now, 2.2GB , and End of Year 2.7GB then 3GB GPU can't mining.

https://www.ddengle.com/miningbitcoin/1842874
Google translate help you.

[mikeyb]

GPU can mine just fine. Upgrade older cards just as you would upgrade a car, or computer or whatever else loses potency over time. ASICs are here anyways, GPU obsolete soon.

[trustfarm-dev]

Now GPU for mining is global shortages.
So, Can not mine other's participations.
Only Big-Size of miners will do mining. like , Most of Bitcoin is mined by china mainland, Several mining company.

[trustfarm-dev]

It means that, Etherbased coin mining is also centralized, like bitcoin.
Is it for goal of De-Centralized Blockchain Coin?
I'm NOT! Strongly.
Now, Final Winner is AMD and NVidia. GPU Manufacturer.

[mikeyb]

Then buy more hash power...not sure what you want a team of developers to do about a something one person feels is an issue. If you feel some of the code/protocol needs changed, write it and submit a pull request for review. If the changes proposed are deemed worthy, I am sure your changes will be approved.

Sorry you gotta upgrade your electronics every few years, it is this way with all things, not just mining.

[mikeyb]

P.S. I remember back in 2013 when there was global GPU shortages and gpu cards were 2.5x their normal price. Things worked out just fine. You should be buying HBM vega cards going forward, the dag size is NOT an issue.

[mikeyb]

See my previous post on this. #6 (comment)

Consider this my final response on the issue. I will gladly review any submitted proposals you submit for review in the future.

[dollebrekel]

Sorry to hop in so lately. Yes hmb2 can hold more dag file size. I have seen that Samsung is busy working on gddr6. In my point of few with the specs they already give it is better then hmb2 for the dag file. When the gddr6 will be released I am not sure. I will test soon the vega + epoch. And I will hive you an update how much the hmb2 can handle.

[RorschachRev]

Ty elaineo. New to thread, I posted in the wrong spot.
I don't think that planning to force GPU into an obsolete state by an artificial constraint will increase the security hardening of ETC. There are many negatives and no positives to having an extremely large DAG and the only reason to not make the change to a smaller DAG is not wanting to change anything. ETH has a DAG reset every 30k blocks. If we did the same thing, it would be a reset at 5 mil blocks for the era and the next era reset would be 10 mil blocks.

Remember that the DAG grows with block rate increases, and if we have more miners, more users, more support from community, then the DAG will grow faster. A large DAG prevents the adoption of new users and miners (although there are much more pressing reasons.) We should have a large influx of new users when the next ETH hard fork for Parity is announced, we should support those miners with a reasonably sized DAG.

@mikeyb Even if the performance hit is minimal on a class of cards in the market it is an artificial and unnecessary performance hit. We should be looking at scalability solutions, not excusing making the system artificially slower for loyal supporters.

[mikeyb]

Except you miss the fact that the hardware you want to hold on to for so long will be a negative profit generating hardware device, thus negating any reason to hold on to it.

[RorschachRev]

@mikeyb The economic reality is affected by multiple things, including price / difficulty. I think your assumptions for the secondary graphics card market and upgrades are likely ignoring the labor costs involved and overestimating the secondary market price for used graphics cards. The key point here is user adoption and migration from other networks. If our price goes up, our difficulty grows slowly, and we support the same cards as other networks then we have a lower transition cost and the barriers to user adoption are lower.

What benefit is there from a high DAG size? None proposed.
What benefit is there from maintaining a consistent DAG size? User adoption, not needing to generate more than 1-2GB of garbage data, network improvements for miners with some configurations, lower startup times for a new miner or a node reboot, consistent behavior within expected results and long term viability of PoW as a network. No increased vulnerability.
What benefits and tradeoffs from having a smaller DAG size? (300mb-500mb) Same benefits as before but with diminishing returns and an increased vulnerability to theoretical attacks.

I will submit some patches in the next 1-2 weeks (work for hire takes priority) but I'm already dealing with this type of code in Go.

[realcodywburns]

rebooting this in a new ECIP soon-ish