Welcome to the SingularitySingularity

What a language model is, probability over sequences, n-grams, bag-of-words, and why counting fails.

Term frequency, inverse document frequency, and why sparse vectors fail to capture meaning — leading us toward dense, learned representations.

From sparse counts to dense vectors: perceptrons, multi-layer networks, and activation functions.

Loss functions, cross-entropy, softmax, and gradient descent — how a network knows it is wrong and how to improve.

The chain rule applied backward through a network, turning one loss value into gradients for every weight and bias.

Recurrent neural networks, sequence tasks, unrolling through time, and the vanishing gradient problem that motivates gated architectures.

Long Short-Term Memory gates, sequence-to-sequence models, and why attention replaced recurrence for long contexts.

Dense vectors, Word2Vec, analogies, subword tokenization, static and contextual embeddings, positional encoding, and an interactive embedding-space playground.

Self-attention, query-key-value, multi-head attention, and the math behind it — built on top of sequence models.

Layer norm, residual connections, feed-forward networks, and encoder/decoder variants.

Tokenization at scale, next-token prediction, optimizers, and training dynamics.

Greedy, beam, temperature, top-k, top-p, and interactive text generation.

Scaling laws, GPT/LLaMA/Mistral families, MoE, and multimodal extensions.

SFT, RLHF, LoRA, instruction tuning, and parameter-efficient methods.

Prompt engineering, RAG, benchmarks, and failure modes.