Generating DAG consumes all the CPU quota

[wi1dcard]

Other tasks and HTTP requests to shadow when generating DAG will be slowed down as generating DAG consumes all CPU quotas.

[xiaoch05]

The CPU usage has been limited by https://github.com/darwinia-network/shadow/blob/master/ffi/pkg/shadow/ffi/mod.go#L34 , the test passed locally. But I don't know why it can't be limited on production env. Also for the issue #160, it has different performance. So is there any way to deploy a test shadow to simulate the production env?

[wi1dcard]

It is too high of the hard-coded limit. We have only 2 cores in total for shadow on production.

Besides, it isn't a good option to limit the GOMAXPROCS directly. I think it might be better if we can separate it into 2 components (such as API and workers) or using some mechanisms like coroutine to schedule the tasks and use the CPU resources more efficiently.

[xiaoch05]

You're right, we need to deliver all these configurations to users. Currently, shadow has no configure file and these configurations are in go part. So it has a little difficult to do this.

We have used coroutine to process the DAG. The GOMAXPROCS setting is not only the CPU usage limit but also the number of coroutines. The file is too big, so we need to write it continuously, and should use the full CPU resource given to it. In fact, the goroutine will not use other's CPU resources. Maybe 1 core is suitable for this situation.

[wi1dcard]

In fact, the goroutine will not use other's CPU resources.

What do you mean by "others' CPU resources"?

[xiaoch05]

In fact, the goroutine will not use other's CPU resources.

What do you mean by "others' CPU resources"?

The GOMAXPROCS will not limit the CPU usage in rust. Others services such as HTTP are not affected.