Skip to content

Commit

Permalink
Merge branch 'sabercrombie/superfluous_clamp' into 'master'
Browse files Browse the repository at this point in the history
Remove misleading/unnecessary clamp in metal path

See merge request machine-learning/dorado!776
  • Loading branch information
tijyojwad committed Dec 18, 2023
2 parents 5fa4de7 + 4fa5db1 commit 7109c1c
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions dorado/basecall/metal/nn.metal
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,9 @@ inline float conv_activation(float x) {
// tanh or SiLU / swish activation.
const float y = kConvTanhActivation ? tanh_fast(x) : x * sigmoid(x);
if (kConvOutputClamp) {
// Note: the lower bound is inoperative, since SiLU(x) has a min. of ~-0.28.
// Needs to updated anyway, once clamp range is nailed down.
return clamp(y, -0.5f, 3.5f);
// We only clamp in the case of SiLU/swish, which has a minimum of ~0.28.
// Only an upper bound need be imposed.
return min(y, 3.5f);
}
return y;
}
Expand Down

0 comments on commit 7109c1c

Please sign in to comment.