use terminology data reader creator instead of data reader, and r… #1412

helinwang · 2017-02-21T22:50:07Z

…emove lambda usage from design doc

wangkuiyi · 2017-02-21T23:05:27Z

doc/design/reader/README.md

-def data_reader_bool(t):
-	while True:
-		yield t
+def data_reader_creator_bool(t):


data_reader_bool ==> reader_creator_bool

wangkuiyi · 2017-02-21T23:05:33Z

doc/design/reader/README.md

-	while True:
-		yield t
+def data_reader_creator_bool(t):
+	def creator:


creator ==> reader

wangkuiyi · 2017-02-21T23:06:10Z

doc/design/reader/README.md


 For example, we want to use a source of real images (reusing mnist dataset), and a source of fake images as input for [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661).

 We can do:

 ```python
-def data_reader_fake_image():
+def data_reader_creator_fake_image():


data_reader_creator_fake_image ==> reader_creator_random_image

wangkuiyi · 2017-02-21T23:07:53Z

doc/design/reader/README.md


 For example, we want to use a source of real images (reusing mnist dataset), and a source of fake images as input for [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661).

 We can do:

 ```python
-def data_reader_fake_image():
+def data_reader_creator_fake_image():
 	while True:


reader_creator_random_image 应该有参数表示图像大小

def reader_creator_random_image(width, height): def reader(): while True: yield numpy.random.uniform(-1, 1, size=width*height)

wangkuiyi · 2017-02-21T23:13:33Z

doc/design/reader/README.md


-Paddle reads data from data reader during training. It will be passed into `paddle.train` as a parameter.
+Paddle reads data from *data reader* during training. *data reader creator* (or *reader creator*) creates a *data reader* when invoked. *reader creator* will be passed into `paddle.train` as a parameter.


At training and testing time, PaddlePaddle programs need to read data. To ease the users' work to write data readign code, we define that

A reader is a function that reads data (from file, network, random number generator, etc) and yields data items.

A reader creator is a function that returns a reader function.

A reader decorator is a function, which accepts one or more readers, and returns a reader.

and provide frequently used reader creators and reader decorators.

wangkuiyi · 2017-02-21T23:15:06Z

doc/design/reader/README.md


-Data reader is a function with no parameter that creates a iterable (anything can be used in `for x in iterable`):
+Data reader creator is a function with no parameter that creates a iterable (anything can be used in `for x in iterable`):


I think a reader creator can have parameters. For example:

def reader_creator_text(filename): def reader(): ... read from filename ... return reader

Here the creator does have one paraemter.

wangkuiyi · 2017-02-21T23:15:40Z

doc/design/reader/README.md

@@ -41,11 +41,11 @@ label_layer = paddle.layer.data("label", ...)
 paddle.train(paddle.dataset.mnist, {"image":0, "label":1}, 128, 10, ...)
 ```

-## Data Reader Decorators
+## Data Reader Creator Decorator


Reader Creator Decorator ==> Reader Decorator

helinwang requested a review from wangkuiyi February 21, 2017 22:50

use terminology *data reader creator* instead of *data reader*, and r…

9b3cdb1

…emove lambda usage from design doc

wangkuiyi requested changes Feb 21, 2017

View reviewed changes

correct data reader and data reader creator usage

478b0c8

wangkuiyi requested changes Feb 21, 2017

View reviewed changes

fix according to comments

5688c38

helinwang force-pushed the design branch from 98625cf to 5688c38 Compare February 21, 2017 23:26

more fixes

794c220

wangkuiyi approved these changes Feb 22, 2017

View reviewed changes

wangkuiyi mentioned this pull request Feb 22, 2017

API Refactorization #1392

Closed

helinwang merged commit 5a1d926 into PaddlePaddle:develop Feb 22, 2017

lizexu123 pushed a commit to lizexu123/Paddle that referenced this pull request Feb 23, 2024

Add demo for evaling on paddle-inference. (PaddlePaddle#1412)

2f7b54d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use terminology data reader creator instead of data reader, and r… #1412

use terminology data reader creator instead of data reader, and r… #1412

helinwang commented Feb 21, 2017

wangkuiyi Feb 21, 2017

helinwang Feb 21, 2017

wangkuiyi Feb 21, 2017

helinwang Feb 21, 2017

wangkuiyi Feb 21, 2017

helinwang Feb 21, 2017

wangkuiyi Feb 21, 2017

helinwang Feb 21, 2017

wangkuiyi Feb 21, 2017

helinwang Feb 21, 2017

wangkuiyi Feb 21, 2017

helinwang Feb 21, 2017

wangkuiyi Feb 21, 2017

helinwang Feb 21, 2017


		Paddle reads data from data reader during training. It will be passed into `paddle.train` as a parameter.
		Paddle reads data from data reader during training. data reader creator (or reader creator) creates a data reader when invoked. reader creator will be passed into `paddle.train` as a parameter.


		Data reader is a function with no parameter that creates a iterable (anything can be used in `for x in iterable`):
		Data reader creator is a function with no parameter that creates a iterable (anything can be used in `for x in iterable`):

use terminology *data reader creator* instead of *data reader*, and r… #1412

use terminology *data reader creator* instead of *data reader*, and r… #1412

Conversation

helinwang commented Feb 21, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

use terminology data reader creator instead of data reader, and r… #1412

use terminology data reader creator instead of data reader, and r… #1412