Deep and Wide算法是将Embedding的结果直接输入DNN进一步提取高阶特特交叉, 最后将一阶特征与高阶特征组合起来进行预测, 其构架如下:
- SimpleInputLayer: 稀疏数据输入层, 对稀疏高维数据做了特别优化, 本质上是一个FCLayer
- Embedding: 隐式嵌入层, 如果特征非one-hot, 则乘以特征值
- FCLayer: DNN中最常见的层, 线性变换后接传递函数
- SumPooling: 将多个输入的数据做element-wise的加和, 要求输入具本相同的shape
- SimpleLossLayer: 损失层, 可以指定不同的损失函数
override def buildNetwork(): Unit = {
val wide = new SimpleInputLayer("input", 1, new Identity(),
JsonUtils.getOptimizerByLayerType(jsonAst, "SparseInputLayer"))
val embeddingParams = JsonUtils.getLayerParamsByLayerType(jsonAst, "Embedding")
.asInstanceOf[EmbeddingParams]
val embedding = new Embedding("embedding", embeddingParams.outputDim, embeddingParams.numFactors,
embeddingParams.optimizer.build()
)
val hiddenLayer = JsonUtils.getFCLayer(jsonAst, embedding)
val join = new SumPooling("sumPooling", 1, Array[Layer](wide, hiddenLayer))
new SimpleLossLayer("simpleLossLayer", join, lossFunc)
}
Deep and wide的参数较多, 需要用Json配置文件的方式指定(关于Json配置文件的完整说明请参考Json说明), 一个典型的例子如下:
{
"data": {
"format": "dummy",
"indexrange": 148,
"numfield": 13,
"validateratio": 0.1
},
"model": {
"modeltype": "T_DOUBLE_SPARSE_LONGKEY",
"modelsize": 148
},
"train": {
"epoch": 10,
"numupdateperepoch": 10,
"lr": 0.1,
"decay": 0.8
},
"default_optimizer": {
"type": "momentum",
"momentum": 0.9,
"reg2": 0.01
},
"layers": [
{
"name": "wide",
"type": "simpleinputlayer",
"outputdim": 1,
"transfunc": "identity"
},
{
"name": "embedding",
"type": "embedding",
"numfactors": 8,
"outputdim": 104
},
{
"name": "fclayer",
"type": "FCLayer",
"inputlayer": "embedding",
"outputdims": [
100,
100,
1
],
"transfuncs": [
"relu",
"relu",
"identity"
]
},
{
"name": "sumPooling",
"type": "SumPooling",
"outputdim": 1,
"inputlayers": [
"wide",
"fclayer"
]
},
{
"name": "simplelosslayer",
"type": "simplelosslayer",
"lossfunc": "logloss",
"inputlayer": "sumPooling"
}
]
}
runner="com.tencent.angel.ml.core.graphsubmit.GraphRunner"
modelClass="com.tencent.angel.ml.classification.WideAndDeep"
$ANGEL_HOME/bin/angel-submit \
--angel.job.name DeepFM \
--action.type train \
--angel.app.submit.class $runner \
--ml.model.class.name $modelClass \
--angel.train.data.path $input_path \
--angel.workergroup.number $workerNumber \
--angel.worker.memory.gb $workerMemory \
--angel.ps.number $PSNumber \
--angel.ps.memory.gb $PSMemory \
--angel.task.data.storage.level $storageLevel \
--angel.task.memorystorage.max.gb $taskMemory
对深度学习模型, 其数据, 训练和网络的配置请优先使用Json文件指定.