Skip to content

TinkerMob/keras_albert_model

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ALBERT

Unofficial implementation of ALBERT.

Install

python setup.py install

or

# Install latest version:
pip install git+https://github.com/TinkerMob/keras_albert_model.git

# Install specific version:
pip install git+https://github.com/TinkerMob/[email protected]

Current versions of dependencies:

  • keras==2.3.0
  • tensorflow==2.0.0

Build model

from keras_albert_model import build_albert

model = build_albert(token_num=30000, training=True)
model.summary()

Load checkpoint

You can load pretrained model provided by brightmart/albert_zh:

from keras_albert_model import load_brightmart_albert_zh_checkpoint

model = load_brightmart_albert_zh_checkpoint('path_to_checkpoint_folder')
model.summary()

Select output layers

from keras_albert_model import build_albert

model = build_albert(token_num=30000, training=False, output_layers=[-1, -2, -3, -4])
model.summary()

About

A Lite BERT

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages