The tanh function is similar to the sigmoid function, but it has the ability to give a negative signal as well. The following diagram illustrates this:
The y function is as follows:
It can be implemented by the following Python code:
def tanh(x):
numerator = 1-np.exp(-2*x)
denominator = 1+np.exp(-2*x)
return numerator/denominator
Now let's look at the softmax function.