These notes capture more than a paper’s abstract. Each summary covers the research question, experimental evidence, limitations, and implications for building real systems.
WikiSkill’s strongest idea is the separation of raw traces, persistent operational memory, and executable skills, with validation-gated rollback only for the executable layer.
The paper correctly prioritizes out-of-distribution rank transfer, but its proposed score conflates predictive validity with risk-adjusted utility.
The paper offers a useful template/realized-graph/trace distinction and reporting protocol, but lacks a reproducible survey methodology.
DataComp-LM establishes a controlled benchmark for dataset research and finds that aggressive model-based quality filtering is more effective than conventional source mixing.
The paper adapts byte pair encoding to learn variable-length subword units, enabling open-vocabulary neural translation without an external dictionary fallback.
The Chinchilla paper shows that model parameters and training tokens should scale in approximately equal proportions, enabling smaller, better-trained models.