carefree-learn 0.1.15

Release Notes

carefree-learn 0.1.15 improved overall performances.

DDP

Since PyTorch is introducing ZeRO optimizer, we decided to remove deepspeed dependency and use native DDP from PyTorch.

results = cflearn.ddp(tr_file, world_size=2)
predictions = results.m.predict(te_file)

JitLSTM

Since native RNNs of PyTorch do not support dropouts on w_ih and w_hh, we followed the official implementation of jit version LSTM and implemented these dropouts.

m = cflearn.make(
    "rnn",
    model_config={
        "pipe_configs": {
            "rnn": {
                "extractor": {
                    "cell": "JitLSTM"
                }
            }
        }
    }
)

Misc

• Fixed NNB when std is 0 (177363e).
• Fixed summary in some edge cases (945ca15, f95f667, 2768153).
• Introduced ONNXWrapper for more general ONNX exports (226de5b).

---

• Optimized Transformer (b09916b).
• Upgraded PyTorch dependency (a596031).
• Supported reuse_extractor in PipeInfo (149aa49).
• Implemented HighwayBlock (3dad99e, 436ebab) and Introduced *FCNNConfig (e0670f7).
• Implemented Initializer.orthogonal (1019114) and Optimized initializations of RNN (2193706).