110 lines
2.3 KiB
Plaintext
110 lines
2.3 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "129d4b8c",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import numpy\n",
|
|
"import matplotlib.pyplot as plt"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d35579ac",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Study of algorithm\n",
|
|
"\n",
|
|
"* Improve Performance, $P$\n",
|
|
"* At some Task, $T$\n",
|
|
"* With Experience, $E$\n",
|
|
"\n",
|
|
"Learning: **Improving $P$ at $T$ with $E$**\n",
|
|
"\n",
|
|
"* $T$: Output\n",
|
|
"* $E$: Training Data\n",
|
|
"* $P$: Target (or Loss) function\n",
|
|
" * If you cannot measure it, you cannot improve it - Kelvin\n",
|
|
" * e.g.: Error rate, Euclidean distance, Logproba, Information theoretical measures(Mutual information, KL)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "0d742636",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "255ea9b1",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Derivation w.r.t Matrix\n",
|
|
"\n",
|
|
"In the typical $f: \\R \\to \\R$\n",
|
|
"$$f(x) = ax, \\quad df/dx = a$$\n",
|
|
"and\n",
|
|
"\n",
|
|
"$$f(x) = x^2, \\quad df/dx = 2x$$\n",
|
|
"\n",
|
|
"\n",
|
|
"By the way, $f = w^Tx,\\, (w \\in \\R^D, x\\in \\R^D)$\n",
|
|
"\n",
|
|
"$$\\frac{df}{dx} = \\frac{\\partial f}{\\partial x_1}$$\n",
|
|
"\n",
|
|
"\n",
|
|
"In the case $f(x) = x^2 = x^Tx$\n",
|
|
"\n",
|
|
"$$\\frac{df}{dx} = 2x$$\n",
|
|
"\n",
|
|
"for example: $f(x) = w^Tx$\n",
|
|
"\n",
|
|
"$$f(x) = \\sum w_i x_i$$\n",
|
|
"$$\\frac{\\partial f}{\\partial x_i} = w_i$$\n",
|
|
"$$\\therefore \\frac{d f}{d x} = w$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "84468a76",
|
|
"metadata": {},
|
|
"source": [
|
|
"## trace \n",
|
|
"$tr[M] = \\sum^{D}_{i=1}{M_{ii}}$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e8671c90",
|
|
"metadata": {},
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "2025-02-AI (3.12.11)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.12.11"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|