Regarding embedding returned by the model

[divyag11]

I am loading bert-base-nli-mean-tokens model to get the embedding of the sentence. I had a question, from which layer of the bert base model you are taking the embedding that is being returned by the bert-base-nli-mean-token model. Please let me know.

[nreimers]

Hi @divyag11
The last layer (the output layer) is used. These output are then usually mean-pooled.

Best
Nils Reimers

[divyag11]

okay, the last layer of pretrained bert base model you are taking OR the last layer of the bert model trained on nli task is taken?

[nreimers]

The last layer of BERT, fine tuned on NLI data, is used. The classifier for NLI is discarded / not used.

[divyag11]

okay,thanks a lot @nreimers