-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RELAY]Add rsqrt, exp(a,x) and log(b,x) operators #2769
Comments
It might be worth considering following Halide's distinction between fast_inv_sqrt (which maps to the approximate inverse sqrt functions available in vectorized formats on most ISAs) and inv_sqrt (the more numerically precise but expensive variant). |
Well, there may be some ambiguity in my description. I am not referring to the trade-off between high speed and high precision. My main point is that Relay/NNVM needs to keep as much semantics as possible as the highest level of TVM. In addition to When Translate From: 我的本意不是高速和高精度之间的取舍 我在从 mxnet 转化的时候发现了 rsqrt 的缺失, 我已经打了个补丁, tf 也有人补了, 但是 caffe, onnx 还没有. 我认为从其他框架转化的时候, Relay 应该最大程度的保留各种语义, 只加一个 rsqrt 也很丑, rcbrt 呢还有... 除了幂函数 x^-a 是这种情况, 广义指数 exp(a,x) 和广义对数 log(b,x) 也没有这样的 OP. 我暂时使用了换底公式补了一下, 量化的话损失也不小, log2,log10 很多模型也要用. |
|
log, exp and rsqrt are in relay now, I will send a pr to add them all in tf frontend |
I request to add
tvm.relay.op.tensor.rsqrt
(and
tvm.relay.op.tensor.exponent
,tvm.relay.op.tensor.logarithm
)rsqrt(x) = 1/sqrt(x), but only on math.
considering numerical errors, it is a completely different situation
consider quantification the problem may be more serious
I don't know whether
power
has considered thisif so, just add
rsqrt
intvm.relay.frontend
The text was updated successfully, but these errors were encountered: