*** buffer overflow detected ***: terminated

[ZiyiLiubird]

When I was running textworld via Python, the programmer threw "*** buffer overflow detected ***: terminated" error after finishing several env tasks. Do you have any suggestions for how to fix this error? Thanks!

[MarcCote]

Hi. thanks for reporting this. Do you have an code example to reproduce this? Is there any stacktrace information other than the error message?

[ZiyiLiubird]

Hi. thanks for reporting this. Do you have an code example to reproduce this? Is there any stacktrace information other than the error message?

Thanks for your reply. Specifically, I created about 100 textworld envs, and then I utilized following codes to manage these envs:

import os
import gym
import textworld.gym

with open("logo.txt", "r") as f:
    logo = f.read()

class EnvWrapper(object):
    def __init__(self, rank:int, comm_size:int, st_task:int, all_task_nums:int, max_episode_steps:int=52):
        self.rank = rank
        self.min_task_id = all_task_nums // comm_size * rank + st_task
        self.max_task_id = all_task_nums // comm_size * (rank + 1) + st_task
        print(f"min task id: {self.min_task_id}")
        print(f"max task id: {self.max_task_id}")
        # if self.max_task_id > all_task_nums:
        #     self.max_task_id = all_task_nums
        self.current_task_id = self.min_task_id
        self.envs_dir = "/mnt/vepfs/devel/ziyiliu/tw_games"
        self.max_episode_steps = max_episode_steps

    def generate_new_task(self):
        env_address = os.path.join(self.envs_dir, f"game{self.current_task_id}", f"custom_game{self.current_task_id}.z8")
        env_id = textworld.gym.register_game(env_address, max_episode_steps=self.max_episode_steps)
        self.env = gym.make(env_id)

    def step(self, action:str):
        obs, reward, done, info = self.env.step(action)
        obs = obs.split('>')[0]
        return obs, reward, done, info

    def reset(self):
        self.generate_new_task()
        obs, infos = self.env.reset()
        obs = obs.split(logo)[-1].split('>')[0]
        return obs
    
    def render(self):
        self.env.render()

    def check_update(self,):
        self.current_task_id += 1
        self.env.close()
        if self.current_task_id == self.max_task_id:
            return True
        return False

If one task is done or exceeds the maximum steps, i.e., 50 (not 52 in the envwrapper), then I will call check_update() function and reset() function to generate a new task env.

Usually, “*** buffer overflow detected ***: terminated” error occurs between running after 4-90 envs.

[MarcCote]

Do you know if it's always the same game that fails? Does it depends on the actions sent to the game or is the error happening when loading the game?

Can you share those 100 games (all .ni, .json and .z8 files)?

Also what OS system are you running TextWorld on?

[MarcCote]

I think I might have found the issue. See microsoft/jericho#74 and the fix (microsoft/jericho#75).