Fix BERT fp16 bugs, add test

[MoisesHer]

Description

Fixing fp16 bug in a previous PR: #1264
A bug was introduced in the last commit (merging with numpy upstream) of that PR when solving a conflict. Apologies
Bug only affects fp16 case, returning NaNs
Model was also not hybridized after being cast.
[x] Both issues are solved in this PR.
[x] A test was added to compare FP32 vs. FP16 results in BERT inference

cc @dmlc/gluon-nlp-team @sxjscience

[codecov]

Codecov Report

Merging #1270 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1270   +/-   ##
=======================================
  Coverage   81.75%   81.75%           
=======================================
  Files          52       52           
  Lines        6862     6862           
=======================================
  Hits         5610     5610           
  Misses       1252     1252           

Impacted Files	Coverage Δ
src/gluonnlp/data/tokenizers/sentencepiece.py	75.44% <0.00%> (-0.60%)	⬇️
src/gluonnlp/data/tokenizers/yttm.py	82.75% <0.00%> (+0.86%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e0b293e...9ce2552. Read the comment docs.

[szha]

Could you add a test for the expected data type?

[sxjscience]

@MoisesHer Would you help add a test in https://github.com/dmlc/gluon-nlp/blob/numpy/tests/test_models_bert.py? You can do a similar forward test as in

gluon-nlp/tests/test_models_roberta.py

Lines 16 to 48 in e78a24e

	@pytest.mark.remote_required
	@pytest.mark.parametrize('model_name', list_pretrained_roberta())
	def test_roberta(model_name):
	# test from pretrained
	assert len(list_pretrained_roberta()) > 0
	with tempfile.TemporaryDirectory() as root:
	cfg, tokenizer, params_path =\
	get_pretrained_roberta(model_name, root=root)
	assert cfg.MODEL.vocab_size == len(tokenizer.vocab)
	roberta_model = RobertaModel.from_cfg(cfg)
	roberta_model.load_parameters(params_path)
	# test forward
	batch_size = 3
	seq_length = 32
	vocab_size = len(tokenizer.vocab)
	input_ids = mx.np.array(
	np.random.randint(
	2,
	vocab_size,
	(batch_size, seq_length)
	),
	dtype=np.int32
	)
	valid_length = mx.np.array(
	np.random.randint(
	seq_length // 2,
	seq_length,
	(batch_size,)
	),
	dtype=np.int32
	)
	x = roberta_model(input_ids, valid_length)

and compare the final result.

[sxjscience]

Need to wait for the GPU CI.

[sxjscience]

@MoisesHer The GPU CI should be functional now. Would you try to create a new PR to add FP16 functionality?
Sorry for the confusion, we may still need to fix the GPU CI test.

[sxjscience]

You can refer to

gluon-nlp/tests/test_models_bert.py

Line 15 in a842bf3

def test_bert_small_cfg(compute_layout, ctx):

. Here, we just add a ctx to the arguments so it becomes a fixture.

[sxjscience]

LGTM

[sxjscience]

CC @zheyuye @szha @hymzoque I'll merge this in first.