Writing, notes, and projects.
This essay was originally envisioned to show the end of AI writing detection. I’ve since realized I was wrong. I’ve been writing this essay over the course of the last month. The other night, I extended this further after a discussion about model bias. The proliferation of AI writing will make detection more valuable even if the detection is routinely beaten.
I wanted to revisit my HeAR model distillation because I was under a time constraint for the hackathon. I could not fully address all aspects of my training with the attention they deserved. Specifically, after the first full run, I noted many questions and concerns in my write-up. I extend the pareto frontier in audio encoders.
How I trained HeAR-s, a distilled model of Google's HeAR. This audio model has better audio length extrapolation than the original and is competitive with larger audio embedding models.
Here is a full financial report produced entirely by AI using a recent github release: https://github.com/Matthew-agi/AI-Streetview-Financial-/tree/main.
Here I discuss LLMs and ICL potential. Specifically I address my experience and the literature of time-series forecasting using LLMs. I then release a new github for financial analysis using LLMs.
Here I directly compare ICL, steering vectors and trigram logit bias with the objective to mimic writing style.
Play with some classifiers and apple leaves.
Critical batch size has been readily explored. Here I explore the covariant case and how it relates to RLVR. Critical rollout size implies that we can use gradient noise to determine the optimal number of rollouts per prompt for RLVR datasets.
The continual learning debate is a lot of people talking past each other. Here I flesh out my thoughts. For the most part, CL is a prediction on whether the LLM is the system or a sub-system.
I've been sitting on this for months! LLMs are somewhere better than the average human but (maybe?) lesser than the average hedge fund manager. All that was missing was a proper harness. Here is an Un-edited report from July where Kimi K2 determines whether GOOG is a good investment or not.
First post — a quick sanity check that the blog is wired up.