Some possible optimization

[vedmathai]

I may be wrong but doesn't
mini_batch.append(self.memory.popleft())
do better job than
mini_batch.append(self.memory[i])
in

def expReplay(self, batch_size):
	mini_batch = []
	l = len(self.memory)
	for i in xrange(l - batch_size + 1, l):
                 mini_batch.append(self.memory[i])

It is much faster too.

[rockmany2000]

Vedmathai , Your code is very different from the original code.
popleft() will remove data from queue . then len(agent.memory) will be 0 . then 1 train / step will become 1 train / 32 steps.
The episode loop will become quick , but I don't know what the result will happen.