Essays, experiments, and field notes.
Archive
Long-form notes on machine learning, experiments, and the systems around them.
ES can learn in Prime-RL-style LLM environments, but it appears to pick up different signals than RL: stronger format and token-control behavior in some tasks, weaker exact solving in others.
Read essayThis essay was originally envisioned to show the end of AI writing detection. I’ve since realized I was wrong. I’ve been writing this essay over the course of the last month. The other night, I extended this further after a discussion about model bias. The proliferation of AI writing will make detection more valuable even if the detection is routinely beaten.
Read essayI wanted to revisit my HeAR model distillation because I was under a time constraint for the hackathon. I could not fully address all aspects of my training with the attention they deserved. Specifically, after the first full run, I noted many questions and concerns in my write-up. I extend the pareto frontier in audio encoders.
Read essayHow I trained HeAR-s, a distilled model of Google's HeAR. This audio model has better audio length extrapolation than the original and is competitive with larger audio embedding models.
Read essayHere is a full financial report produced entirely by AI using a recent github release: https://github.com/Matthew-agi/AI-Streetview-Financial-/tree/main.
Read essayHere I discuss LLMs and ICL potential. Specifically I address my experience and the literature of time-series forecasting using LLMs. I then release a new github for financial analysis using LLMs.
Read essayHere I directly compare ICL, steering vectors and trigram logit bias with the objective to mimic writing style.
Read essayPlay with some classifiers and apple leaves.
Read essayCritical batch size has been readily explored. Here I explore the covariant case and how it relates to RLVR. Critical rollout size implies that we can use gradient noise to determine the optimal number of rollouts per prompt for RLVR datasets.
Read essayThe continual learning debate is a lot of people talking past each other. Here I flesh out my thoughts. For the most part, CL is a prediction on whether the LLM is the system or a sub-system.
Read essayI've been sitting on this for months! LLMs are somewhere better than the average human but (maybe?) lesser than the average hedge fund manager. All that was missing was a proper harness. Here is an Un-edited report from July where Kimi K2 determines whether GOOG is a good investment or not.
Read essayFirst post — a quick sanity check that the blog is wired up.
Read essay