Multi-outputs & class_weight

[bityangke]

Assume our model have two outputs :
output 1 'class' for classification
output 2 'location' for regression

Now we have the imbalance dataset(eg. binary classification, class '0': 98 percent, class '1': 2 percent), so we need set the class_weight params in model.fit() function, but for output 2 'location' regression task, we do not need class_weight.
How can we set the class_weight in this situation?
Does "class_weight = {0: ratio 1 , 1: ratio 2} is applied the classification task and ignore the regression task by default？

[bityangke]

And a second situation is that:
output 1 'class_1' for classification
output 2 'class_2' for classification

Both outputs do the classification, but we want to give each output different class_weight params.
How can we do this? (I am not sure whether this situation makes sense or not. )
Maybe Like this?:
class_weight = { 'class_1': {0: ratio 1 , 1: ratio 2} , 'class_2': {0: ratio 3 , 1: ratio 4} }

[fchollet]

You should pass a dict with string keys corresponding to the names of your outputs. Like you describe.

…

On 15 December 2016 at 15:14, Ke Young ***@***.***> wrote: And a second situation is that: output 1 'class_1' for classification output 2 'class_2' for classification Both outputs do the classification, but we want to give each output different class_weight params. How can we do this? (I am not sure whether this situation makes sense or not. ) Maybe Like this?: class_weight = { 'class_1': {0: ratio 1 , 1: ratio 2} , 'class_2': {0: ratio 3 , 1: ratio 4} } — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4735 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AArWb6wWFGIUv7e82crHb9ggQ4M2oyAnks5rIcnJgaJpZM4LOrKk> .

[bityangke]

Thanks for your quick response.
If we use the one-hot code.
For the two classes classification, should the class_weight like this? :
class_weight = {10: weight 1, 11: weight 2}
10 and 11 repesent the one hot codes of the two classes

[fchollet]

No, use integers. 0 and 1.

…

On 15 December 2016 at 15:31, Ke Young ***@***.***> wrote: Thanks for your quick response. If we use the one-hot code. For the two classes classification, should the class_weight like this? : class_weight = {10: weight 1, 11: weight 2} 10 and 11 repesent the one hot codes of the two classes — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4735 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AArWb98elmzifptTjsU3SK1SYX-_wIT3ks5rIc23gaJpZM4LOrKk> .

[olix20]

@fchollet what is the right way to set weights given multi-class, one-hot encoded outputs? e.g. y = [0, 1, 1, 1, 0]

[shamoons]

Yup. Same issue

[pratiklodha95]

@olix20 did you find a solution ?

[QuickLearner171998]

For a multi output model how can I change class_weights during training ie after each step according to a batch.

[adriangb]

In 2020 / TF 2.3.0 this does not work: class_weights = {'output1': {0: 1, 1: 10}, 'output2': {0: 5, 1: 1, 2: 10}}

[tingjhenjiang]

this discussion should be reopened as of Dec 11, 2020 as this issue is still not resolved.
Here is what it comes to me:
ValueError: Expected ``class_weight`` to be a dict with keys from 0 to one less than the number of classes, found {'THEORETICAL': {0: 0.9251916468411314, 1: 1.0879701585327946}, 'ENGINEERING': {0: 0.9703354588300527, 1: 1.031535514294135}, 'EMPIRICAL': {0: 0.720016457519029, 1: 1.6362786348761102}, 'OTHERS': {0: 0.5192107995846313, 1: 13.513513513513514}}

and this discussion is related to discussion 1 and discussion 2.

[kevinashaw]

Agreed. This is really needed.

[kevinashaw]

@fchollet Can we get this re-opened? We are dealing an inherently asymmetric target set with the need for multiple outputs. class_weight is the right way for us to address this. Is there a work around? (Topic is also addressed in tensorflow/tensorflow#41448 (comment))

[MareSeestern]

Any progress?

[adriangb]

I think class weights can always be implemented via sample weights (see sklearn.utils.class_weight.compute_sample_weight). It seems like Keras does support sample weights for each output, so maybe using compute_sample_weight for each output can work? If you already have sample weights, I think you should be able to do weight = weight * new_weight (but someone with more of a statistics background should confirm this)

[kevinashaw]

@adriangb Ostensibly, Keras does support class weights with multiple outputs, but the implementation is broken, as noted by the error message shown in this thread. Its unclear to me if sklearn can be used in a tensorflow/keras training pipeline. I find adding outside frameworks can be troublesome when exporting models to TF Serving. But I may not have enough knowledge here.

[adriangb]

I'm not sure either. I'm just offering it as an alternative, but obviously it would still be nice if it got fixed.

[niki-j]

Did anyone have any luck with this issue here? The nested dictionary approach as suggested by @fchollet for class_weight earlier seems to be supported for TF <= 2.1.0 only.
A lot of people with such multi-label/multi-output problems seem to have raised this as a big (and unresolved) issue across many recent forums.

[thusinh1969]

Keras is going somewhere :( ... Not even a response from https://github.com/fchollet :(

[ThomasHenckel]

I think class weights can always be implemented via sample weights (see sklearn.utils.class_weight.compute_sample_weight). It seems like Keras does support sample weights for each output, so maybe using compute_sample_weight for each output can work? If you already have sample weights, I think you should be able to do weight = weight * new_weight (but someone with more of a statistics background should confirm this)

Thanks @adriangb
This worked for me, I just wanted to adjust the weight on for the last class

from sklearn.utils.class_weight import compute_sample_weight
class_weight = [{0: 1, 1: 1}, {0: 1, 1: 1}, {0: 1, 1: 1}, {0: 0.8, 1: 1.1}]
sample_weight = compute_sample_weight(class_weight,labels)

Would be nice if Keras would accept the class_weight directly, but until then...

[philosofool]

sample_weight is not supported when using tf.data.Dataset instances, so the need for class_weight with multilabel models remains.