Issue to set metrics with multiple outputs Model

[RPlumey-asulab]

Hi,

I’m using the following setup:

import numpy as np  
import tensorflow as tf  
print(f"NumPy version: {np.__version__}")  # 1.26.4  
print(f"TensorFlow version: {tf.__version__}")  # 2.18.0  
print(f"Keras version: {tf.keras.__version__}")  # 3.9.2  

from tensorflow.keras import Input, Model  
from tensorflow.keras.layers import Dense  

a = Input(shape=(3,), name='a')
b = Dense(3, activation='relu')(a)  
c = Dense(3, activation='relu')(b)  
model = Model(inputs={'a': a}, outputs={'b': b, 'c': c})  
model.summary()  
model.compile(optimizer='adam', loss={'b': 'mse', 'c': 'mae'}, metrics={'b': 'accuracy', 'c': 'accuracy'})  
model.fit({'a': np.random.rand(100, 3)}, {'b': np.random.rand(100, 3), 'c': np.random.rand(100, 3)}, epochs=10)

But I’m getting this error:

ValueError: In the dict argument 'metrics', key 'b' does not correspond to any model output.

Do you know why the model doesn’t recognize 'b' and 'c' as valid output names even though I used them in the outputs dictionary?

Also, when I try using this instead:

model.compile(optimizer='adam', loss={'b': 'mse', 'c': 'mae'}, metrics=[['accuracy'], ['accuracy']])

I get another error:
ValueError: For a model with multiple outputs, when providing the 'metrics' argument as a list, it should have as many entries as the model has outputs. Received: metrics=[['accuracy'], ['accuracy']] of length 2 whereas the model has 0 outputs.

Any idea what’s going on here?

[dhantule]

Hi @RPlumey-asulab, thanks for reporting this.

If you try naming the outputs layers with the name argument, your code should work fine. Also, your model's outputs are dictionaries so you must use dictionaries for metrics and losses, with keys matching layer names.
Attaching a gist for reference.

[RPlumey-asulab]

Hi,

Just to make sure I understand — is it normal that the model throws an error unless the output layers are explicitly named to match the keys used in the outputs dict?
I assumed defining the outputs like {'b': b, 'c': c} would be enough like for the loss, so that part was a bit unclear to me.
Knowing that in my real case it's easier to explain if they don't.

a = Input(shape=(3,5), name='a')

x,y,z = Split(axis=-2, num_or_size_splits=3)(a)

stuff= Stuff(...)
x = stuff(x)
y = stuff(y)
z = stuff(z)

model = Model(inputs={'a': a}, outputs={'x': x, 'y': y, 'z': z})

Thanks for the quick reply.

[dhantule]

Hi @RPlumey-asulab,

If you don't name the layers, Keras would implicitly assign names to them, and you’d need to use those auto-generated names (e.g., 'dense', 'dense_1', etc.) in your loss or metrics dictionaries.

[RPlumey-asulab]

Got it, but.

Just to be sure — in the second case, let’s say I have a Dense layer named "Dense" and it's the output layer (e.g., stuff = Dense(..., name="Dense").
How should I compile the model with metrics in that case? Should I write metrics={"Dense": ['accuracy']} or metrics={"Dense_0": ['accuracy'], "Dense_1": ['accuracy'], "Dense_2": ['accuracy']}?
If it is one of these, it is not consistent with the loss where loss={'x' = 'mse', 'y' = 'mse', 'z' = 'mse'} works

Thanks again!

[dhantule]

Hi @RPlumey-asulab, for multiple named outputs

model = keras.Model(inputs=input, outputs={'x': x, 'y': y, 'z': z})
model.compile(
    optimizer='adam',
    loss={'x': 'mse', 'y': 'mse', 'z': 'mse'},
    metrics={'x': ['mae'], 'y': ['mae'], 'z': ['mae']}
)

for single output layer

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics={"Dense": ['accuracy']} # or just metrics=['accuracy']
)

[RPlumey-asulab]

Hi @dhantule ,

So, with the following code:

import keras
from keras import Input, Model
from keras.layers import Dense, Layer
import tensorflow as tf
import numpy as np

input_ = Input(shape=(3,5), name='input')

class SplitLayer(Layer):
    def call(self, inputs):
        return [ tf.squeeze(t, axis=1)
            for t in tf.split(inputs, num_or_size_splits=inputs.shape[1], axis=1)
        ]

x, y, z = SplitLayer(name='split')(input_)
dense = Dense(3, activation='relu', name='dense')
x = dense(x)
y = dense(y)
z = dense(z)

model = keras.Model(inputs=input_, outputs={'x': x, 'y': y, 'z': z})
model.compile(
    optimizer='adam',
    loss={'x': 'mse', 'y': 'mse', 'z': 'mse'},
    metrics={'x': ['mae'], 'y': ['mae'], 'z': ['mae']}
)
model.summary()

model.fit(
    np.random.rand(100, 3, 5),
    {'x': np.random.rand(100, 3), 'y': np.random.rand(100, 3), 'z': np.random.rand(100, 3)},
    epochs=10
)

I'm getting this error:

ValueError: In the dict argument `metrics`, key 'x' does not correspond to any model output. Received:
metrics={'x': ['mae'], 'y': ['mae'], 'z': ['mae']}

Is this behavior unexpected, or am I doing something wrong here?

Thanks a lot!

Best,
Robin