Turning a multi-class classification into ordinal regression

[haoyang27]

Hi,
I'm a beginner on deep learning and have been trying to turn a multi-class classification task into the ordinal regression. Here's the code regarding the CNN architecture:

`class TrainingModel:
def init(self, hyperparameters, model_type, target_height, target_width, class_names):
""" Creates and prepares a model for training.

    Args:
        hyperparameters (dict): The configuration's hyperparameters.
        model_type (str): Type of model to create.
        target_height (int): Height of input.
        target_width (int): Width of input.
        class_names (list of str): A list of classes.
        
    Returns:
        model (keras Model): The prepared keras model.
        model_type (str): The name of the model type.
    """

    # Catch if the model is not in the model list
    if model_type not in model_list:
        print(colored(f"Warning: Model '{model_type}' not found in the list of possible models: {list(model_list.keys())}"))
        
        # TODO: Create a custom model

    self.model_type = model_type
        
    # Get the model base
    base_model = model_list[model_type](
        include_top=False,
        weights=None,
        input_tensor=None,
        input_shape=(target_height, target_width, hyperparameters['channels']),
        pooling=None
    )

    # Return the prepared model
    avg = keras.layers.GlobalAveragePooling2D()(base_model.output)
    #out = coral.CoralOrdinal(len(class_names))(avg)
    out = keras.layers.Dense(len(class_names), activation="softmax")(avg)
    out = coral.CoralOrdinal(len(class_names))(out)
    self.model = keras.models.Model(inputs=base_model.input, outputs=out)
    
    # Create optimizer and add to model
    optimizer = keras.optimizers.SGD(
        lr=hyperparameters['learning_rate'], 
        momentum=hyperparameters['momentum'], 
        nesterov=True, 
        decay=hyperparameters['decay']
    )
    self.model.compile(
        #loss="sparse_categorical_crossentropy", 
        loss = coral.OrdinalCrossEntropy(),
        optimizer=optimizer,
        #metrics=["accuracy"]
        metrics = ["accuracy",coral.MeanAbsoluteErrorLabels()] 
    )

`

Am I doing this correctly by adding an ordinal layer, changing the loss function and the metrics? Also, does the accuracy actually make sense since I'm getting extremely low value on accuracy (around 0.05, it's around 0.95 when it's a classification). Any help would be greatly appreciated, thanks!!!

Edit: my classes are in the form of ["class1": 0, "class2": 1, "class3": 2, "class4": 3, "class5": 4]

[gmgeorg]

@haoyang27 W/o the actual data it will be hard to pinpoint exactly whats going on here.

Looking at your code, my hunch is though that you have not actually replaced the softmax with ordinal softmax, but added it. I suggest to get rid of the first softmax and just directly pass your last layer to the CoralOrdinal layer.

# out = keras.layers.Dense(len(class_names), activation="softmax")(avg)  # remove this; CoralOrdinal _is_ the ordinal softmax
out = coral.CoralOrdinal(len(class_names))(avg)

Also what do your classes represent? are they actual ordinal? (if not, then you should just stick to softmax as ordinal will be based on incorrect assumptions of ordered labels)

Also suggest to not use "accuracy" as an evaluation metric here, since that is measuring the point prediction, not the probabilistic prediction quality. You can have great probabilistic prediction model, yet the decision of picking the class will give bad metrics [poor decision making]. See also #22

[haoyang27]

Thanks for the detailed explanation! I've also found out that I should get rid of the softmax layer. The classes are the scoring system for evaluating certain subjects, 1 means score 1 so on and so forth, higher the score the better, it's sort of in the way of ordinal but can also be treated as traditional classification/regression I guess.

For the metrics, indeed using accuracy does not fit with regression. Again, appreciated for your response!