google flax Ideas · Discussions · GitHub

Transformers with Flax - T5, BERT, RoBERTa, MT3 and more
Show and tell 8bitmp3
Announcement: new signature for `MultiHeadDotProductAttention`'s `__call__` method
General chiamp

Sort by: Latest activity

Categories

Ideas Discussions

Share ideas for new features

You must be logged in to vote

NNX MultiHeadDotProductAttention

PVirie asked Oct 11, 2024 in Ideas · Closed · Answered

2
You must be logged in to vote

Why dont we use a single Dense layer in LSTM as e.g. dm-Haiku does

leonl42 asked Dec 17, 2023 in Ideas · Answered

3
You must be logged in to vote

Feature request: optionally sow attention weight in dot_product_attention

JyChang012 asked Feb 13, 2023 in Ideas · Answered

3
You must be logged in to vote

Missing trending activations like GEGLU & Mish

HMUNACHI asked Nov 14, 2023 in Ideas · Answered

6
You must be logged in to vote

mish activation function.

Nightbringers asked Oct 4, 2023 in Ideas · Unanswered

1
You must be logged in to vote

Feature request: spectral normalization

rdilip asked Feb 23, 2023 in Ideas · Closed · Answered

3
You must be logged in to vote

Reservoir Computing with Flax

Wotaker asked Jul 26, 2023 in Ideas · Unanswered

1
You must be logged in to vote

Avoid passing deterministic to complicated nested modules by using variable dict

JyChang012 asked Mar 3, 2023 in Ideas · Answered

2
You must be logged in to vote

Context manager for mixed precision training

jatentaki asked Apr 5, 2022 in Ideas · Unanswered

2
You must be logged in to vote

Addition of Pretrained models: ResNet

MrinalTyagi asked Jan 15, 2023 in Ideas · Unanswered

0
You must be logged in to vote

Feature Request: Example Documentation

shenoynikhil asked Jan 8, 2023 in Ideas · Answered

1
You must be logged in to vote

A Global Average Pooling Function?

codymlewis asked Nov 15, 2022 in Ideas · Answered

6
You must be logged in to vote

Default stride for pooling functions

dogeplusplus asked Apr 3, 2022 in Ideas · Answered

2
You must be logged in to vote

Feature Request: Native dynamic data loaders/generators

mayureshagashe2105 asked Jul 21, 2022 in Ideas · Unanswered

1
You must be logged in to vote

A Moving Averge layer?

lkhphuc asked May 25, 2022 in Ideas · Unanswered

3
You must be logged in to vote

Feature request: functionality similar to torch.nn.utils.vector_to_parameters

timdhondt1 asked Apr 14, 2022 in Ideas · Answered

2
You must be logged in to vote

Improving CLU's Metric API

cgarciae asked Mar 21, 2022 in Ideas · Unanswered

4
You must be logged in to vote

Implementation of the asymmetric attention

manifest asked Nov 5, 2021 in Ideas · Unanswered

4
You must be logged in to vote

Training and evaluation API

manifest asked Aug 23, 2021 in Ideas · Unanswered

0
You must be logged in to vote

A self-organizing map example

automataIA asked Aug 19, 2021 in Ideas · Unanswered

0
You must be logged in to vote

Add mean_init and var_init parameters to the BatchNorm module

matthias-wright asked May 29, 2021 in Ideas · Unanswered

0
You must be logged in to vote

Add model zoo

machineko asked May 11, 2021 in Ideas · Answered

3
You must be logged in to vote

Allow dot_product_attention to support normalization functions other than softmax

CoderPat asked May 4, 2021 in Ideas · Unanswered

0
You must be logged in to vote

[REQ] Allow overriding __init__ of struct.dataclass

PhilipVinc asked Apr 30, 2021 in Ideas · Unanswered

8
You must be logged in to vote

split params and variables in .init()

GJBoth asked Mar 2, 2021 in Ideas · Answered

7