API Refactorization

[wangkuiyi]

Layers: @jacquesqiao should be done in 2017-2-23

• [调研]implement more layers in v2 API
  • issue implement more layers in v2 API #1402
  • [100%] add cost function add cost function in v2.layer #1416
  • [100%] add ParamAttr add ParamAttr in V2 #1431
  • should be done in 2017-2-23
• [100%]specify data types of data layers
  • Merged add type to layer.data #1386
  • Done
• [90%] add topology. to be discussed
  • add Topology to handle actions on network #1400
  • should be done in 2017-2-23
• [80%]调通 RNN @luotao1 + @jacquesqiao
  • should be done in 2017-2-23
    • 2017-2-23 单层已经调通，正在优化
    • 等待projection+mixlayer来实现多层嵌套的rnn。
• [80%] 实现projection+mixlayer @qingqing01 + @jacquesqiao
  • 2017-2-23 方案讨论确定。

Data: should be done in 2017-2-27

• [100%]data feeder (from data reader to Argument) @qingqing01
  • Data Feeder #1374
• [99%]ReaderCreator/ReaderDecorator @helinwang
  • create buffered data reader decorator and tests #1364
  • create reader creator decorators: shuffle, compose, chain #1411
  • use terminology *data reader creator* instead of *data reader*, and r… #1412
  • create numpy array reader creator and text reader creator #1424
  • add decorator: map_readers, change from google python style to pep 287 #1425

• Use sklearn/NLTK to get dataset, it should be complete before 2-25.
  - [20%]Data Package @wen-bo-yang @wangkuiyi
  • Data Package job in implement Dataset Reader in Paddle Book #1419
  • SRL data should be done in 2017-2-23

• [70%] paddle.data.caching @reyoung
  • Data Fetcher
    • design Design of data.fetcher in Paddle Data Package #1394
    • implementation Implement of paddle.v2.data.fetcher #1405
  • It seems that fetcher is not needed. Maybe we should fetch data independently.
  • Should be done in 2017-2-23

Activations should be done in 2017-2-23

• [100%]C++ getLayerOutput() and Expose To SWIG and update python getLayerOutput @gangliao LayerOutput for single machine multiple devices #1353
  - [100%] Paddle Docker CPU and GPU
  Update docker image's context #1443

Event: @reyoung should be done in 2017-2-23

• [70%]Get Strong Typed Metric From paddle::Evaluator:

  • also, there are several tiny fixes merged to develop. Unify Evaluator::create method #1371 Unify print layer, ValuePrinter evaluator. #1372 Remove sequence information in GradientPrinter. #1373
  • issue Get Strong Typed Metric From paddle::Evaluator #1389
  • PR Feature/evaluator value #1375
  • Should be done in 2017-2-23

• [没开始]补全Event？

• [调研]IPython 画图？

  • Document a trick for dynamic plotting in Jupyter/iPython #1415

• [没开始]其他的Demo？

[qingqing01]

从semantic_role_labeling出发，写API训练，一些想到的问题：

网络配置

ParameterAttribute、xx_projection等其他不在python/paddle/v2中的如何引入包。这些目前在： python/paddle/trainer_config_helpers

确实要包装成Python代码。 在 #1402 列了出具体要做的工作。除了上述工作，还有RNN layers相关的一些工作。 请 @jacquesqiao @luotao1 @gangliao @qingiqng 来分工。

训练

parameters 初始化问题：C++还是Python？如果是python需要提供常用初始化接口吧？

目前通过ParameterAttribute 来指定parameter的初始化方法。目前支持两种方法：Gaussian RNG 和 uniform distributed RNG。实际上有很多方法来初始化参数： http://stats.stackexchange.com/questions/47590/what-are-good-initial-weights-in-a-neural-network @reyoung @qingqing01

optimizer: 其他demo需要完善python/paddle/v2/optimizer.py内容，目前只有Adam，Adamax。

请 @qingqing01

L2-Regularization 如何设置？ 5 / 8 demo有正则

v2 API已经支持了——在创建trainer的时候指定。未来可能要支持per-layer的regularization。

Testing during training event_handler需要支持

@reyoung 会来写 paddle.test 和 paddle.infer

其他

疑问：网络连接如下：

          B_cost_layer：是cost层, paddle.parameters.create(B_layer)即可训练
        / 
A_layer
        \ 
         C_layer  和 B_layer共享参数，testing用。
 

按照训练topology=B_cost_layer即可，topology=[B_cost_layer, C_layer]吗? 否则的话testing如何拿到C_layer的输出？

    trainer.train(train_data_reader=train_reader,
                  topology=[B_cost_layer, C_layer],
                  ...)

[wangkuiyi]

Testing during training：event_handler需要支持

在我脑海里，我理解你给出的例子可以这么描述：

a_layer = paddle.layer.SomeLayer()
b_cost = paddle.layer.SomeCost(a_layer)
c_cost = paddle.layer.SomeOtherCost(a_layer)

parameters = paddle.parameters.create([b_cost, c_cost])

def event_handler(event):
    if isinstande(event, paddle.event.EndIteration):
        print paddle.test(c_cost,
                   parameters,
                   paddle.dataset.mnist.testing_data())
        
paddle.train(b_cost,
             parameters,
             paddle.dataset.mnist.training_data(),
             event_handler)

请看这样可以吗？

[reyoung]

This issue has been done.