Home|About|Gaming cabinet|Archives|How it works|Starter sites

GeezerVison is showing diffprog.github.io — Notation – The Elements of Differentiable Programming

Notation

Table 1: Naming conventions

Notation	Description
$X \subseteq R^{D}$ X⊆RD	Input space (e.g., features)
$Y \subseteq R^{M}$ Y⊆RM	Output space (e.g., classes)
$S_{k} \subseteq R^{D_{k}}$ Sk⊆RDk	Output space on layer or state $k$ k
$W \subseteq R^{P}$ W⊆RP	Weight space
$Λ \subseteq R^{Q}$ Λ⊆RQ	Hyperparameter space
$Θ \subseteq R^{R}$ Θ⊆RR	Distribution parameter space, logit space
$N$ N	Number of training samples
$T$ T	Number of optimization iterations
$x \in X$ x∈X	Input vector
$y \in Y$ y∈Y	Target vector
$s_{k} \in S_{k}$ sk∈Sk	State vector $k$ k
$w \in W$ w∈W	Network (model) weights
$λ \in Λ$ λ∈Λ	Hyperparameters
$θ \in Θ$ θ∈Θ	Distribution parameters, logits
$π \in [0, 1]$ π∈[0,1]	Probability value
$π \in △^{M}$ π∈△M	Probability vector

Table 2: Naming conventions (continued)

Notation	Description
$f$ f	Network function
$f (\cdot; x)$ f(⋅;x)	Network function with $x$ x fixed
$L$ L	Objective function
$ℓ$ ℓ	Loss function
$κ$ κ	Kernel function
$ϕ$ ϕ	Output embedding, sufficient statistic
$s t e p$ step	Heaviside step function
${l o g i s t i c}_{σ}$ logisticσ	Logistic function with temperature $σ$ σ
$l o g i s t i c$ logistic	Shorthand for ${l o g i s t i c}_{1}$ logistic1
$p_{θ}$ pθ	Model distribution with parameters $θ$ θ
$ρ$ ρ	Data distribution over $X \times Y$ X×Y
$ρ_{X}$ ρX	Data distribution over $X$ X
$μ, σ^{2}$ μ,σ2	Mean and variance
$Z$ Z	Random noise variable