Should we have a certification procedure

[chaals]

The Specification has included a section on conformance and "EthTrust Certification", which basically sets some conditions for certification, notes that it is not a guarantee of anything more than certain tests having been performed, and that it is moderated by usage in the community and there is no formal oversight body.

As editor I have repeatedly heard private feedback that this is concerning, because there is a tendency on the part of a number of developers to request the lowest and simplest possible level of certification and then act as if they have a guarantee for the security of their work.

Likewise, I have repeatedly heard feedback in private that it is important to customers, and to the overall success of EthTrust in raising the quality level of security reviews and thus improving the overall security in the ecosystem, that there is a well-known and reliable certification scheme.

I doubt that this issue has a clear "correct" resolution, but I think it is important to consider it and get broad input, hence filing it first as a public issue here.

Some possible outcomes:

• Some public recorded discussions (e.g. conference panels, or webinars) on the issue.
• Written documents outlining the pros and cons of a certification scheme
• Removing the certification section from the specification
• Working with several organisations to develop certification schemes for those who claim competence in reviewing the security of Smart Contracts according to the EthTrust specification
• EEA nominating a specific authorised provider, or several such, of certification either for Smart Contracts or for bodies that certify the producers of such certification.

There are other possibilities, and as can be seen, some of those outlined above exclude others. That is deliberate: I believe it is important that the industry gets broad input on the issue before trying to reach a proposed answer.

[cylon56]

Some sort of clear certification procedure is critical to getting real adoption of EthTrust. There are currently no strong incentives or calls to action with the current spec that leads to auditors or project adopting EthTrust explicitly, even though the spec is clearly written with official certification in mind. It effectively functions as best practice guidelines at the moment.

I'm open to ideas on how to achieve either within the EEA or by partnering with other organizations more suited to support a certification program.

[dmuhs]

I agree with @cylon56 here and the point that there is no 100% correct solution to this problem. That being said, here are some thoughts to make an argument for splitting the certification off into a separate specification.

The EthTrust specification primarily focuses on security level requirements, which are only a subset of work required in a wider certification program. It would be beneficial to put certification details into a separate document. This approach allows the EthTrust spec to remain focused on its core purpose — outlining security levels and properties — without the added complexity of certification details.

A separate document dedicated to certification could provide an overview of the process, including who conducts it, the required skills, and how reviewers are certified. This separation would also allow for a more in-depth exploration of concrete certification aspects, such as incentivizing projects to aim for higher certification levels instead of sticking with the lowest one and communicating the meaning of these levels to end users.

Maintaining a distinct certification document allows the EthTrust specification to become more focused on the level requirements. Readers looking for information on security levels would not have to sift through certification details, making the spec more user-friendly and focused.

The current spec outlines security issues but lacks detailed guidelines on detecting and addressing these issues, especially complex vulnerabilities like front-running and upgradeability concerns. A certification document could include a chapter on review guidelines, offering examples, tools, and configurations to standardize the review process.

Lastly, the certification document could provide greater details about the relationship between the EEA, editors, and ecosystem partners offering certification. This structure would make it easier to rapidly iterate on the certification process without affecting the core EthTrust spec itself.

[the-kscar]

Is there a specific team that wants an audit for which you could do a test run? I'd also be curious to hear thoughts on monetizing a certification like this, given the significant amount of work it would require.

[gkkobakhidze]

I also agree with @dmuhs about separating out the certification process from the security spec itself. If and when we want the certification to be a popularly used process, it should ideally evolve parallel and even separately from the security spec itself. In other words, the process for certification could be updated to a different version several times within the release cycle of just one version of the security spec. Or vice-versa for that matter. At the moment the way they are coupled together, it isn't quite possible.

There are a few reasons to do that.

First, it is clearly going to be something that needs a lot of input from the community, which would only come after people try it. Adjusting the certification process within a fairly concrete and set-in-stone security spec (at least between version releases) would be difficult.

Second, it would allow the EEA editors and contributors to switch gears and specialize between the certification and the security spec. If someone wants to evangelize the certification process and communicates and promotes this to the ecosystem, they would be able to do so by only focusing on the certification section without having to stay 100% on top of what the latest proxy deployment strategy is (just an example). Service standards, conflict of interest guidelines, transparency guidelines etc. - there is just a lot to be mentioned for the certification, so it may warrant a separate document of its own.

Third, the security spec should ideally not worry about how it conformance is certified, just what would qualify for conformance. I would draw an analogy with common ERC/EIP standards - they provide what it would mean for a piece of code to be considered compliant, but they don't bother explaining how one should ensure that, what could be misrepresented, that it is ultimately up to the reviewer etc. etc.