Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于MSE准则迭代求解最优scale的疑问 #25

Open
weihChen opened this issue Oct 12, 2023 · 2 comments
Open

关于MSE准则迭代求解最优scale的疑问 #25

weihChen opened this issue Oct 12, 2023 · 2 comments

Comments

@weihChen
Copy link

你好,在函数forward_net_octav中有如下mse准则下迭代求解最优scale的代码:

abs_x = np.abs(ort_inputs[i])
s_n = abs_x.sum() / abs_x[abs_x > 0].size
for _ in range(20):
    s_n_plus_1 = abs_x[abs_x > s_n].sum() / \
               (1 / (4 ** 8) / 3 / unsigned * abs_x[abs_x <= s_n].size + abs_x[abs_x > s_n].size)
    if np.abs(s_n_plus_1 - s_n) < 1e-6:
        break
    s_n = s_n_plus_1

想请问下这里

 s_n_plus_1 = abs_x[abs_x > s_n].sum() / \
               (1 / (4 ** 8) / 3 / unsigned * abs_x[abs_x <= s_n].size + abs_x[abs_x > s_n].size)

迭代更新scale公式的物理含义是什么呢?是如何推导得到的呢?

@gushiqiao
Copy link
Contributor

您好,函数forward_net_octav以及mse校准是参考Optimally Clipped Tensors And Vectors(OCTAV)方法的实现,论文链接: https://arxiv.org/pdf/2206.06501.pdf, 上述代码对应的公式对应文中的公式6
企业微信截图_16974230578073

@weihChen
Copy link
Author

好的,多谢啦🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants