Last Updated on 2024-02-20 by Clay
HuggingFace 之前曾經發表過文章表示現在的 LLM最好是依照 ChatML 格式去訓練，在一般情況下，會按照 system、user、assistant 的三種不同角色來進行生成，格式如下：
Read More »[已解決] Mistral 經過 SFTTrainer 微調後不會輸出 eos_token `<|im_end|>`
Last Updated on 2023-12-30 by Clay
/bin/bash: warning: shell level (1000) too high, resetting to 1
Last Updated on 2023-12-27 by Clay
在大型語言模型（Large Language Model, LLM）的微調任務中，監督式微調（Supervised Fine-tuning, SFT）、基於人類反饋強化學習（Reinforcement Learning from Human Feedback, RLHF）和直接偏好優化（DPO）... 等等都是不錯的方法，不過他們之間存在一些差異。
Read More »LLM 微調筆記 - SFT 和 DPO 的差異
Last Updated on 2024-02-29 by Clay
DPO（Direct Preference Optimization, 直接偏好優化）是一種取代 RLHF（Reinforcement Learning from Human Feedback, 基於人類反饋的強化學習）的微調方式。眾所皆知，大型語言模型在經過非監督式學習後能夠學習到大量的知識與理解能力（有些研究者認為是『壓縮並保存』了知識在神經網路權重中）；在監督式學習後學會了流暢地回應我們的問題，或者說是學會了『對話』的能力。
Read More »Direct Preference Optimization (DPO) 訓練方法筆記
Last Updated on 2023-12-24 by Clay
今天當我在一台新的 Linux 筆電上想要安裝 pyaudio（Python 中經常用於錄音的套件）時，我遇到了之前沒有遇過的錯誤：
Read More »[已解決] fatal error: portaudio.h: No such file or directory 9 | #include "portaudio.h" | ^~~~~~~~~~~~~ compilation terminated
Last Updated on 2023-12-21 by Clay
Given n points on a 2D plane where points[i] = [xi, yi], Return the widest vertical area between two points such that no points are inside the area.
Last Updated on 2023-12-20 by Clay
You are given an integer array prices representing the prices of various chocolates in a store. You are also given a single integer money, which represents your initial amount of money.
Last Updated on 2023-12-19 by Clay
An image smoother is a filter of the size 3 x 3 that can be applied to each cell of an image by rounding down the average of the cell and the eight surrounding cells (i.e., the average of the nine cells in the blue smoother). If one or more of the surrounding cells of a cell is not present, we do not consider it in the average (i.e., the average of the four cells in the red smoother).
Last Updated on 2023-12-18 by Clay
IPython 是一個提供互動式運算的系統，可以在各種 shell 跟視覺化界面整合在一起；比方說我們可以透過終端機使用 ipython（前題是這個模組有裝）、或是使用如 VS Code 和 PyCharm 這類的圖形化界面編輯器/IDE。
