Error running Hadrrec application

[abhyshr]

From Himanshu Sharma [email protected]

The python HADREC env. code with the RL library we are developing is giving error after running (~60 episodes).Below is the error. 
 

########################################################################

Traceback (most recent call last):
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/driver/main.py", line 38, in
    exa_learner.run()
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/base/learner_base.py", line 113, in run
    self.workflow.run(self)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/utils/profile.py", line 135, in wrapper_profile
    return func(*args, **kwargs)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/workflows/workflow_vault/sync_learner_ARS.py", line 779, in run
    self.ARS_actorWrapper(exalearner,nepisodes)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/workflows/workflow_vault/sync_learner_ARS.py", line 745, in ARS_actorWrapper
    return_bool = self.actor(exalearner, nepisodes,comm_flag=False)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/workflows/workflow_vault/sync_learner_ARS.py", line 629, in actor
    self.reset_env(exalearner)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/utils/introspect.py", line 311, in wrapper
    result = func(*args, **kwargs)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/workflows/workflow_vault/sync_learner_ARS.py", line 396, in reset_env
    self.current_state = exalearner.env.reset()
  File "/people/shar306/.conda/envs/Py_GridPackV3/lib/python3.10/site-packages/gym/core.py", line 232, in reset
    return self.env.reset(**kwargs)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/envs/env_vault/HadrecWrapper_V1.py", line 82, in reset
    return self.env.reset(flag=1)
  File "/qfs/projects/ecp_exalearn/hsharma/Exrl_pars/EXARL/exarl/envs/env_vault/Hadrec_dir/exarl_env/Hadrec.py", line 408, in reset
    self.hadapp.solvePowerFlowBeforeDynSimu(self.simu_input_file, 0)
RuntimeError: PETSc error (0): Error detected in C PETSc
 
############################################################################################################

[wperkins]

The code being used here is the feature/init-comm (#119) branch.

How is an "episode" run? Is a gridpack.Environment created for each, or are all "episodes" run with the same gridpack.Environment?

If the latter, this may be the infamous PETScPrefix problem. In the input, before each <PETScOptions> node add a node like <PETScPrefix>pfx#</PETScPrefix>
where # a character unique to each entry, e.g., pfx1, pfx2, etc. So, in input_tamu500_step005.xml (if that's what you are using), the <Powerflow> <LinearSolver> entry would look like

    <LinearSolver>
      <SolutionTolerance>1.0E-11 </SolutionTolerance> 
      <PETScPrefix>pfx1</PETScPrefix>
      <PETScOptions>
        <!-- -ksp_view -->
        -ksp_type richardson
        -pc_type lu
        -pc_factor_mat_solver_type petsc
        -ksp_max_it 1
      </PETScOptions>
    </LinearSolver>

Subsequent <PETScPrefix> entries need to be unique.

[abhyshr]

Thanks @wperkins. Adding <PETScPrefix>pfx1</PETScPrefix worked. One more reason for removing the random prefix set to PETSc options. I don't think it adds value for what we are trying to do and it is hard to debug.