[Machine Learning] Tanh 函式介紹與程式實作

Last Updated on 2021-05-08 by Clay

Tanh，又稱為雙曲正切函數，基本的定義如下：

基本上便是 sinh(x) / cosh(x)，繪製出的圖案其實便是將我們輸入的 x 值映射在 [-1, 1] 之間。

以下我寫了一個簡單的範例程式碼來顯示 tanh(x) 的圖形：

# -*- coding: utf-8 -*-
import matplotlib.pyplot as plt
import math

value = -10
x = [value+n*0.01 for n in range(2001)]
y = [math.tanh(v) for v in x]


plt.plot(x, y)
ax = plt.gca()
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')
ax.xaxis.set_ticks_position('bottom')
ax.spines['bottom'].set_position(('data',0))
ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data',0))
plt.show()ㄙㄟ

# -*- coding: utf-8 -*-
import matplotlib.pyplot as plt
import math

value = -10
x = [value+n*0.01 for n in range(2001)]
y = [math.tanh(v) for v in x]


plt.plot(x, y)
ax = plt.gca()
ax.spines[‘right’].set_color(‘none’)
ax.spines[‘top’].set_color(‘none’)
ax.xaxis.set_ticks_position(‘bottom’)
ax.spines[‘bottom’].set_position((‘data’,0))
ax.yaxis.set_ticks_position(‘left’)
ax.spines[‘left’].set_position((‘data’,0))
plt.show()ㄙㄟ

Output:

若是不用 python 內建的 math.tanh 模組，而是改用定義的公式來計算：

def sinh(x):
    return (math.exp(x)-math.exp(-1*x))/2


def cosh(x):
    return (math.exp(x)+math.exp(-1*x))/2


def tanh(x):
    return sinh(x)/cosh(x)

value = -10
x = [value+n*0.01 for n in range(2001)]
y = [tanh(v) for v in x]


plt.plot(x, y)
ax = plt.gca()
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')
ax.xaxis.set_ticks_position('bottom')
ax.spines['bottom'].set_position(('data',0))
ax.yaxis.set_ticks_position('left')
ax.spines['left'].set_position(('data',0))
plt.show()

def sinh(x):
    return (math.exp(x)-math.exp(-1*x))/2


def cosh(x):
    return (math.exp(x)+math.exp(-1*x))/2


def tanh(x):
    return sinh(x)/cosh(x)

value = -10
x = [value+n*0.01 for n in range(2001)]
y = [tanh(v) for v in x]


plt.plot(x, y)
ax = plt.gca()
ax.spines[‘right’].set_color(‘none’)
ax.spines[‘top’].set_color(‘none’)
ax.xaxis.set_ticks_position(‘bottom’)
ax.spines[‘bottom’].set_position((‘data’,0))
ax.yaxis.set_ticks_position(‘left’)
ax.spines[‘left’].set_position((‘data’,0))
plt.show()

Output:

我們也可以得到一樣的結果。

應用

tanh 函數也存在著梯度飽和的問題，跟 sigmoid 函數一樣
收斂比 ReLU 函數慢

References

https://www.sciencedirect.com/science/article/abs/pii/S0375960100007258

Activation function
Loss function
- Cross Entropy
- OtherTransforms

[Machine Learning] Tanh 函式介紹與程式實作

應用

References

Read More

Leave a Reply取消回覆

[Machine Learning] Tanh 函式介紹與程式實作

應用

References

Read More

分享此文：

Leave a Reply取消回覆