Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alfoa/generic codeinterface fix #844

Merged
merged 20 commits into from
Nov 16, 2018
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
1996c8e
dai
alfoa Aug 1, 2018
8925566
Merge brant checkout workshop_:wq
alfoa Aug 1, 2018
13bcea1
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Aug 24, 2018
95b615e
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Sep 11, 2018
6c6661d
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Sep 25, 2018
7f21d87
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Oct 17, 2018
2c71fc3
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Oct 19, 2018
393ac20
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Oct 29, 2018
afb259b
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Nov 7, 2018
2f8f657
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Nov 8, 2018
7087437
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Nov 9, 2018
493fa02
oky
alfoa Nov 9, 2018
c7130e1
Merge branch 'devel' of https://github.com/idaholab/raven into devel
alfoa Nov 12, 2018
b990006
fix generic code interface (parsing)
alfoa Nov 13, 2018
5ffb6ca
ok
alfoa Nov 14, 2018
b838818
removed trailingspaces
alfoa Nov 15, 2018
5c86d1b
Update test_BasicStatistics.xml
alfoa Nov 15, 2018
35e806c
addrressed comments
alfoa Nov 15, 2018
66ae608
Merge branch 'alfoa/genericCodeinterfaceFix' of https://github.com/id…
alfoa Nov 15, 2018
fe72486
fixed test
alfoa Nov 16, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion framework/CodeInterfaces/Generic/GenericCodeInterface.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,5 +182,5 @@ def createNewInput(self,currentInputFiles,origInputFiles,samplerType,**Kwargs):
origfiles.append(inputFile)
parser = GenericParser.GenericParser(infiles)
parser.modifyInternalDictionary(**Kwargs)
parser.writeNewInput(currentInputFiles,origfiles)
parser.writeNewInput(infiles,origfiles)
return currentInputFiles
4 changes: 2 additions & 2 deletions framework/Models/Code.py
Original file line number Diff line number Diff line change
Expand Up @@ -848,8 +848,8 @@ def submit(self, myInput, samplerType, jobHandler, **kwargs):
a mandatory key is the sampledVars'that contains a dictionary {'name variable':value}
@ Out, None
"""
prefix = kwargs['prefix'] if 'prefix' in kwargs else None
uniqueHandler = kwargs['uniqueHandler'] if 'uniqueHandler' in kwargs.keys() else 'any'
prefix = kwargs.get("prefix")
uniqueHandler = kwargs.get("uniqueHandler",'any')

## These two are part of the current metadata, so they will be added before
## the job is started, so that they will be captured in the metadata and match
Expand Down
62 changes: 32 additions & 30 deletions framework/Models/Model.py
Original file line number Diff line number Diff line change
Expand Up @@ -369,8 +369,9 @@ def submit(self, myInput, samplerType, jobHandler, **kwargs):
a mandatory key is the sampledVars'that contains a dictionary {'name variable':value}
@ Out, None
"""
prefix = kwargs['prefix'] if 'prefix' in kwargs else None
uniqueHandler = kwargs['uniqueHandler'] if 'uniqueHandler' in kwargs.keys() else 'any'
prefix = kwargs.get("prefix")
uniqueHandler = kwargs.get("uniqueHandler",'any')
forceThreads = kwargs.get("forceThreads",False)

## These kwargs are updated by createNewInput, so the job either should not
## have access to the metadata, or it needs to be updated from within the
Expand All @@ -382,37 +383,38 @@ def submit(self, myInput, samplerType, jobHandler, **kwargs):
## works, we are unable to pass a member function as a job because the
## pp library loses track of what self is, so instead we call it from the
## class and pass self in as the first parameter
jobHandler.addJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, prefix, metadata=metadata, modulesToImport=self.mods, uniqueHandler=uniqueHandler)
jobHandler.addJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, prefix, metadata=metadata, modulesToImport=self.mods, uniqueHandler=uniqueHandler, forceUseThreads=forceThreads)

def submitAsClient(self, myInput, samplerType, jobHandler, **kwargs):
"""
This will submit an individual sample to be evaluated by this model to a
specified jobHandler as a client job. Note, some parameters are needed
by createNewInput and thus descriptions are copied from there.
@ In, myInput, list, the inputs (list) to start from to generate the new
one
@ In, samplerType, string, is the type of sampler that is calling to
generate a new input
@ In, jobHandler, JobHandler instance, the global job handler instance
@ In, **kwargs, dict, is a dictionary that contains the information
coming from the sampler, a mandatory key is the sampledVars' that
contains a dictionary {'name variable':value}
@ Out, None
"""
prefix = kwargs['prefix'] if 'prefix' in kwargs else None
uniqueHandler = kwargs['uniqueHandler'] if 'uniqueHandler' in kwargs.keys() else 'any'
#def submitAsClient(self, myInput, samplerType, jobHandler, **kwargs):
#"""
#This will submit an individual sample to be evaluated by this model to a
#specified jobHandler as a client job. Note, some parameters are needed
#by createNewInput and thus descriptions are copied from there.
#@ In, myInput, list, the inputs (list) to start from to generate the new
#one
#@ In, samplerType, string, is the type of sampler that is calling to
#generate a new input
#@ In, jobHandler, JobHandler instance, the global job handler instance
#@ In, **kwargs, dict, is a dictionary that contains the information
#coming from the sampler, a mandatory key is the sampledVars' that
#contains a dictionary {'name variable':value}
#@ Out, None
#"""
#prefix = kwargs.get("prefix")
#uniqueHandler = kwargs.get("uniqueHandler",'any')
#forceThreads = kwargs.get("forceThreads",False)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment explaining why we are saving this material, and on what conditions it can be removed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed


## These kwargs are updated by createNewInput, so the job either should not
## have access to the metadata, or it needs to be updated from within the
## evaluateSample function, which currently is not possible since that
## function does not know about the job instance.
metadata = kwargs
### These kwargs are updated by createNewInput, so the job either should not
### have access to the metadata, or it needs to be updated from within the
### evaluateSample function, which currently is not possible since that
### function does not know about the job instance.
#metadata = kwargs

## This may look a little weird, but due to how the parallel python library
## works, we are unable to pass a member function as a job because the
## pp library loses track of what self is, so instead we call it from the
## class and pass self in as the first parameter
jobHandler.addClientJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, prefix, metadata=metadata, modulesToImport=self.mods, uniqueHandler=uniqueHandler)
### This may look a little weird, but due to how the parallel python library
### works, we are unable to pass a member function as a job because the
### pp library loses track of what self is, so instead we call it from the
### class and pass self in as the first parameter
#jobHandler.addClientJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, prefix, metadata=metadata, modulesToImport=self.mods, uniqueHandler=uniqueHandler, forceUseThreads=forceThreads)

def addOutputFromExportDictionary(self,exportDict,output,options,jobIdentifier):
"""
Expand Down
5 changes: 3 additions & 2 deletions framework/Models/PostProcessor.py
Original file line number Diff line number Diff line change
Expand Up @@ -203,12 +203,13 @@ def submit(self,myInput,samplerType,jobHandler,**kwargs):
a mandatory key is the sampledVars'that contains a dictionary {'name variable':value}
@ Out, None
"""

kwargs['forceThreads'] = True
Model.submit(self,myInput, samplerType, jobHandler,**kwargs)
## This may look a little weird, but due to how the parallel python library
## works, we are unable to pass a member function as a job because the
## pp library loses track of what self is, so instead we call it from the
## class and pass self in as the first parameter
jobHandler.addJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, str(0), modulesToImport=self.mods, forceUseThreads=True)
#jobHandler.addJob((self, myInput, samplerType, kwargs), self.__class__.evaluateSample, str(0), modulesToImport=self.mods, forceUseThreads=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, justification for keeping commented code.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed


def evaluateSample(self, myInput, samplerType, kwargs):
"""
Expand Down
5 changes: 5 additions & 0 deletions tests/framework/CodeInterfaceTests/GenericInterface/inp.three
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<?xml version="1.0" ?>
<properties>
<x>10</x>
<z>1</z>
</properties>
2 changes: 2 additions & 0 deletions tests/framework/CodeInterfaceTests/test_generic_interface.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
<Files>
<Input name="one.xml" type="">one.xml</Input>
<Input name="inp.two" type="">inp.two</Input>
<Input name="inp.three" type="">inp.three</Input>
<Input name="mesh" type="">dummy.e</Input>
<Input name="a_dummy_file_for_subdirectory" type="" subDirectory="testSubDirectory">dummy_file_for_subdirectory.dummy</Input>
</Files>
Expand All @@ -36,6 +37,7 @@
<clargs arg="python" type="prepend"/>
<clargs arg="-i" extension=".xml" type="input"/>
<clargs arg="-a" extension=".two" type="input"/>
<clargs arg="-a" extension=".three" type="input"/>
<clargs arg="-o" type="output"/>
</Code>
</Models>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -313,9 +313,6 @@
<PointSet name="checkIfMatrixCanBeComputed_basicStatPP">
<Output>checkIfMatrixCanBeComputed_vars</Output>
</PointSet>
<PointSet name="autoCorrelationTest_basicStatPP">
<Output>autoCorrelationTest_vars</Output>
</PointSet>
<PointSet name="CreateInfinity_basicStatPP">
<Output>CreateInfinity_vars</Output>
</PointSet>
Expand Down