model.identify_effect when the data doesn't match the graph.

[matanma]

Hi.
First - I'm excited and grateful about dowhy - kudos for your work.
I've been reading the documentation here, stating that: "Identification can be achieved without access to the data, acccesing only the graph. ". I've then constructed a digraph and a CausalModel with this graph as an input, and an arbitrary (e.g. empty) data. model.identify_effect() then identified no effect. When adding data appropriately corresponding to the graph - the effects were properly identified.
If this is indeed required, I'd clarify the documentation, and also issue some message whenever the data doesn't correspond to the graph.
Thanks.

[emrekiciman]

Hi @matanma,

Thanks for flagging this. We should improve the documentation for this.

While the identify_effect() function doesn't use the data values, it does look at the columns in the data frame to determine which features are observed vs. not observed.

I.e., from that notebook you linked to, if we create a sample dataset, and then

data = dowhy.datasets.linear_dataset(beta=10,
        num_common_causes=5,
        num_instruments = 2,
        num_effect_modifiers=1,
        num_samples=5000,
        treatment_is_binary=True,
        stddev_treatment_noise=10,
        num_discrete_common_causes=1)
df = data["df"]  # this dataframe has all the data

# create an empty dataframe that only has the columns [X0,Z0,Z1,W0,W1,W2,W3,W4,v0]
empty_df_with_columns = pd.DataFrame(columns=['X0','Z0','Z1','W0','W1','W2','W3','W4','v0'])

# create a completely empty dataframe, no columns
empty_df = pd.DataFrame()

Then create a causal model object and call identify effect:

model=CausalModel(
        #data = empty_df,  --> fails to identify an estimand because all features are believed to be unobserved
        data = empty_df_with_columns,  # --> succeeds because the columns indicate features will be observed
        treatment=data["treatment_name"],
        outcome=data["outcome_name"],
        graph=data["gml_graph"]
        )

identified_estimand = model.identify_effect()
print(identified_estimand)

We will see a valid estimand if we pass in empty_df_with_columns but will not find a valid estimand if we pass in empty_df as DoWhy will think that all features are unobserved.

Does this address the issue you were seeing?

And to reiterate, you are completely correct that this should be updated in the docs. I'll mark this discussion as an issue.

Emre

[matanma]

Thanks very much for the prompt reply @emrekiciman.
If the documentation is modified, I suggest referring also to the case where the columns of the dataframe may not perfectly match the graph. For example, if there are additional columns to the ones specified in the graph, or if there's a typo in one of the columns names - what happens then? In general, I suggest issuing at least some warning in such cases, otherwise one may not be aware that the model doesn't properly identify a causal effect due to e.g. a typo in one of the columns names.
Thanks.

[emrekiciman]

Thanks @matanma That is a good point about additional warnings.