[Machine Learning] Introduction of Softmax function

Softmax

Softmax function, mapping the vector between (0, 1), also represents the probability distribution of each element (classification class) in the vector.

Of course, sine it is a probability distribution, the sum of the vectors should be 1.

Let’s take a look for Softmax formula:

This formula is hard to understand, you can look directly at the code:

# -*- coding: utf-8 -*-
import numpy as np

inputs = np.array([1, 4, 9, 7, 5])

def softmax(inputs):
     return np.exp(inputs)/sum(np.exp(inputs))

outputs = softmax(inputs)
for n in range(len(outputs)):
     print('{} -> {}'.format(inputs[n], outputs[n]))

Output:

1 -> 0.00028901145493871657
4 -> 0.005804950249395781
9 -> 0.8615310049461178
7 -> 0.11659554257150641
5 -> 0.015779490778041354

And the sum is 1.

print(sum(outputs))

Output:

1.0

Application

• You can call this function from Keras and PyTorch
• Softmax function often used in multi-class prediction