Custom Loss Function in Keras

Keras is a library for creating neural networks. It is open source and written in Python. Keras does not support low-level computation but it runs on top of libraries like Theano or Tensorflow.

Keras is developed by Google and is fast, modular, easy to use.

Loss function has a critical role to play in machine learning. Loss is a way of calculating how well an algorithm fits the given data. If predicted values deviate too much from actual values, loss function will produce a very large number.

So our aim is to reduce the value produced by loss function with the help of optimization function.

Different loss functions in Keras:

Keras provides a bunch of loss functions. For example below is the few commonly used loss function for Keras:

Mean Squared Error

The mean squared error loss function measures the average of the squares of the errors. This is calculated effectively as the average squared difference between the predicted values and the actual value.

Mean Absolute Error

The mean absolute error is effectively the measure of difference between two continuous variables. The mean absolute error is an average of the absolute errors e = Y-X and Y is prediction and X is the true value.

What is custom loss function

A custom loss function in Keras will improve the machine learning model performance in the ways we want. Hence this is very useful for solving specific problems efficiently.

Sometimes we need to use a loss function that is not provided by default in Keras. In that case we can construct our own custom loss function and pass to the function model.compile as a parameter.

We can create a custom loss function in Keras by writing a function that returns a scalar and takes the two arguments namely true value and predicted value.

Creating custom loss function

Now let us start creating the custom loss function. For this we need to import below libraries. First one is numpy which stands for numerical python and second one is keras.backend.

Backend is a library used in Keras which is for performing most of the computation like tensor products, convolutions and other similar activities with the help of libraries like Tensorflow. Hence backend engine can perform the computation and develop the various models. As we might guess Tensorflow is the default backend engine which can be changed as well.

import numpy

import keras.backend as kerback

Then we need to generate the sample data for calculating the loss value using the custom loss function that we will create. For that we will take the help of numpy library. Numpy has a class called random and the method called rand, we are going to use the rand method with two parameters 79 and 89 as shown in below code.

Similarly to generate predicted values we are going to use rand method of random class for numpy. We will be passing the two values 79 and 89 as parameter to this function as well.

actual = numpy.random.rand(79,89)

predicted = numpy.random.rand(79,89)

After we are ready with actual and predicted values we are going to create a custom loss function called my_loss as shown below. The my_loss function will take two parameters namely actual and predicted. After that inside the my_loss function we are going to use keras backend as kerback and call the mean function on this class. The mean function will take kerback.sum as argument.

The kerback.sum method will take the kerback.square as argument. The square function is taking the difference of actual and predicted values and it is dividing it by actual multiplied by predicted. This is our custom implementation.

The my_loss function is returning the my_loss value as shown the code below.

def my_loss(actual, predicted):

my_loss_value=kerback.mean(kerback.sum(kerback.square((actual-predicted)/(actual*predicted))))

return my_loss_value

After the above my_loss function returns the value, we are going to print it. We are passing keras backend eval function as argument of the print function.

The eval function is taking the my_loss function as parameter and the my_loss function , in turn taking the actual and predicted values as its parameter.

print(kerback.eval(my_loss(actual,predicted)))

You can finally use this loss function in model.compile function as below.

model.compile(loss=my_loss,optimizer=optimizer)

Compiling keras model utilises some efficient numerical libraries implicitly also called backend which is actually like TensorFlow and Theano. The backend part of keras has the ability to choose best way to represent the network for training automatically.

While we start compiling the model we need to specify few additional properties compared to when training the model.

One important thing to specify is the loss function to be used to calculate a bunch of weights.

Note that the compile function has another parameter called optimizer: This object denotes the training method. The common used optimizers are named as rmsprop, Adam, sgd etc.

Custom loss function with custom signature:

Up till now, though we have created a custom loss function and used it to compile our model we have not been able to change the number of parameters of our loss function. In some cases you may like to use a third parameter, other than actual and predicted to be used for loss calculation.

In this case one of the ways to implement this is to wrap our standard loss function my_loss (actual, predicted) inside another function with any number of parameters, with whatever you want as third parameter of calculation and return the loss function my_loss (actual, predicted) which has the required signature. The example code is shown below with a wrapper function returning the custom loss function called my_loss. Note that, in this code we are using a third parameter param3 while calculating the loss.

def my_loss_wrapper(param3):

def my_loss(actual, predicted):

my_loss_value=kerback.mean(kerback.sum(kerback.square((actual-predicted)/(param3))))

return my_loss_value

return my_loss

Conclusion:

It all depends upon your particular use case, what kind of loss function you may like to write. I found it is really a bit cleaner to utilise Keras backend rather than TensorFlow directly for simple custom loss functions like this one. I hope you liked this implementation and would like to try a custom loss function on your own for keras.

Recent Posts

Menu