SimplE model definition not correct?

[dschaehi]

Hi, as far as I know, SimplE defines for each entity two different embeddings: one for head and one for tail. So for a triple (i, j, k) you need to average h_i * r_j * t_k and h_j * r^{-1}_j * t_i. It seem that this is not reflected in your code. The model in your code seems to be symmetric, where as SimplE is not symmetric.

OpenKE/models/SimplE.py

Lines 23 to 24 in 2380e9d

	def _calc_avg(self, h, t, r, r_inv):
	return (- torch.sum(h * r * t, -1) - torch.sum(h * r_inv * t, -1))/2

[Joker-Song]

Hi, as far as I know, SimplE defines for each entity two different embeddings: one for head and one for tail. So for a triple (i, j, k) you need to average h_i * r_j * t_k and h_j * r^{-1}_j * t_i. It seem that this is not reflected in your code. The model in your code seems to be symmetric, where as SimplE is not symmetric.

OpenKE/models/SimplE.py

Lines 23 to 24 in 2380e9d

	def _calc_avg(self, h, t, r, r_inv):
	return (- torch.sum(h * r * t, -1) - torch.sum(h * r_inv * t, -1))/2

the implementation reference https://github.com/Mehran-k/SimplE/blob/master/simplE_avg.py. In simpleE_avg.py h_i = t_j , t_k = h_j

[dschaehi]

As you can see in the following line of original source code there are h1_emb and h2_emb for the head entities and t1_emb and t2_emb for tail entities. So SimplE has head and tail representations for each entity.

self.init_scores = (tf.reduce_sum(tf.multiply(tf.multiply(self.h1_emb, self.r1_emb), self.t1_emb), 1) + tf.reduce_sum(tf.multiply(tf.multiply(self.h2_emb, self.r2_emb), self.t2_emb), 1)) / 2.0

https://github.com/Mehran-k/SimplE/blob/7c9edc7d2c299fffd2164e976e312aa8996a0e28/simplE_avg.py#L33

[dfdazac]

@dschaehi was there a reason for closing? I think this issue is still present in the OpenKE implementation. In the original implementation the embedding for an entity depends on the position in the triple:

https://github.com/Mehran-k/SimplE/blob/29108230b63920afa38067b1aff8b8d53d07ed01/simplE_avg.py#L21-L35

[dschaehi]

@dfdazac: No, I closed it because there didn't seemed to be a response. I can reopen it.

[Mehran-k]

Hi all, I'm the main author of the SimplE paper. I have received emails asking me if the OpenKE implementation of SimplE is correct or not so I thought I post a public response here. I can confirm that the OpenKE implementation is indeed incorrect and there are two issues (one major, one minor) in it:
Major issue: As pointed out by @dschaehi there's a major issue in the model definition. SimplE requires two embedding vectors per entity, one to be used when the entity is the head and one to be used when the entity is the tail. In the OpenKE implementation, there is only one embedding vector per entity which hurts the model by making it almost identical to DistMult.
Minor issue: This implementation corresponds to a variant of SimplE which we called SimplE-ignr in the paper. It takes the average of the two predictions during training but only uses one of the predictions during testing (see https://github.com/thunlp/OpenKE/blob/OpenKE-PyTorch/openke/module/model/SimplE.py#L54). The standard SimplE model takes the average of the two predictions for both training and testing.

For a correct pytorch implementation of SimplE, I recommend this repo: https://github.com/baharefatemi/SimplE/blob/master/SimplE.py