How I trained HeAR-s, a distilled model of Google's HeAR. This audio model has better audio length extrapolation than the original and is competitive with larger audio embedding models.
Writing, notes, and projects.
How I trained HeAR-s, a distilled model of Google's HeAR. This audio model has better audio length extrapolation than the original and is competitive with larger audio embedding models.
Here is a full financial report produced entirely by AI using a recent github release: https://github.com/Matthew-agi/AI-Streetview-Financial-/tree/main.
Here I discuss LLMs and ICL potential. Specifically I address my experience and the literature of time-series forecasting using LLMs. I then release a new github for financial analysis using LLMs.
Here I directly compare ICL, steering vectors and trigram logit bias with the objective to mimic writing style.
Play with some classifiers and apple leaves.
Critical batch size has been readily explored. Here I explore the covariant case and how it relates to RLVR. Critical rollout size implies that we can use gradient noise to determine the optimal number of rollouts per prompt for RLVR datasets.
The continual learning debate is a lot of people talking past each other. Here I flesh out my thoughts. For the most part, CL is a prediction on whether the LLM is the system or a sub-system.
I've been sitting on this for months! LLMs are somewhere better than the average human but (maybe?) lesser than the average hedge fund manager. All that was missing was a proper harness. Here is an Un-edited report from July where Kimi K2 determines whether GOOG is a good investment or not.
First post — a quick sanity check that the blog is wired up.