obsession is a crazy fucking movie

guess i just need to spend a couple more weeks prepping for interviews

yc startup culture is insane

gained like 700 followers in the last 2 weeks idk what i even did tbh, maybe im falling off now? idk feels cool that what i say doesnt just get ignored instatntly

fuck im addicted to coding

watched the interview with the corgi guy hes actually kinda based except for the working 7 days i week and working hella hours i think theres def like a scientifically optmial way to balance work and rest.

why does sf have to be so fuckig expensive why cant we just have cool people wanting to make cool things in a cheaper place man wtf

had an "intro chat" today that turned into a surprise live leetcode medium in the first 2 minutes. lesson: preparation beats talent, and getting caught off guard says nothing about you. funniest part is their own careers page lists "intro conversation" as step 1 and the technical as step 2, so yeah, that one's on him lol. anyway, back to building. honestly the hardest lessons i've ever learned came from shipping to real users, not from interviews.

software is never 100% bug free. and it was never supposed to be.

even the best engineers can't test every possible input. there are too many. and the math actually proves you can't fully prove a program is correct.

so the goal was never zero bugs. it's managing risk down to a level that fits what's at stake.

a todo app and a pacemaker have very different bars.

good engineering isn't perfectionism. it's knowing which bugs matter and letting the rest go.

testing the post pipeline. if you're seeing this, it works. deleting shortly.

had ~10 founders and recruiters in my dms today.

a year ago i couldn't get a single reply. now the question has flipped: which startup do i actually want to join?

i'm not looking for safe or comfortable. i want the team that pushes me way past what i think i'm capable of, surrounded by the smartest, hardest-working people i can find. that's the whole bar.

evals are literally just like the reverse of training ML models, which is pretty cool because you can tune your model based on how good it is. But your eval data better be really good, which is why running it on real data is so valuable. Theres gotta be a company who is selling eval data? like a scale.ai type company that sells high quality eval data? or even hires people to write real world commands to test ability.

man i really want a herman miller chair

prepping for more interviews today, kinda starting to miss the grind of building? ecspecially peeky, i havent had anyt time this week to work on it because of all these interviews which is fine. Just wanna get back to coding tbh, good to take a break maybe, hopefully i can at least finish peeky to where i want it to be before i start working full-time or at least find a collaborator to help maintain it, i do beleive it can help people still.

I got 3 interviews this week along with technical projects, crazy how quick things can ramp up, building peeky was my realization that building something deep end to end is not only great for your resume, but the experience i got from it and handling all the bugs and issues with ai agents along the way actually make me such a stronger engineer and way better positioned for a role at a startup than anything i did before. I should def add an eval to peeky, after talking to a few founders thats the biggest things i think they were looking ofor

its my birthday btw, just turned 24 today. starting to feel like unc ngl lol but whatever, still got alot of years left i hope. another year of building i guess.

gotta retrain the routelet classifier for peeky. its been acting kinda weird lately, a bunch of the tools/integrations changed so the intents are all mismatched now and its misfiring on commands it used to nail. gonna have to rebuild the dataset and retrain the whole thing, will take some time but i think its worth it. the whole app falls apart if the router cant route.

also just got accepted into upenn lmao, pretty hype ngl. now i just gotta figure out how to manage the workload without letting work slip. keeping work at 100% no matter what, school fits around it.

been working on the ui, looks pretty good idk. realised i havent had a break in like 120 days now ummmm

when i get a job just gonna chill for a bit idk

spending some time on Peeky, hindsight and timemachine tdoay. hitting all 3 hard, but mainly just cleanup on peeky, gotta retrain the ML model cuz the data isnt great. then need to work on the mvp for timemachine so i can actually test the backtester even works? also everyone goging crazy over mythos

haiku model broke this morning, right when farza was probbaly testing it. Just my luck bro

no matter how good the ai models get, theres always problems with the projects that i just cant seem to fix, bugs suck.

fallibilism is such a beautiful word and something i never knew i believed in so much.

every time a new model comes out (this time mythos/fable) i give it the same prompt. it's my own little AGI test, to see how good these models actually are on real projects, not just benchmarks and hype.

it's a prompt for a pretty simple flight sim game, and so far no model has come close to pulling it off. better yet, no model has even managed to make the plane fly for longer than 5 seconds.

gonna test it on mythos and see how it handles it.

trying to think of a way to reduce the cost of backtesting since currently i need like 1000 calls just to test one model on like 10 tickers for 3 years. and its not even obvious if the backtesting is gonna be statistically significant on just one run if models are inherently random :/

Aigth this is my first post, been working on a way to backtest LLms on stocks and polymarket to see if i can extract any alpha/ see if theres actually a model that can trade/ predict real world events.

Gonna need to make my own research agent without peek-forward bias so i can accuratley evaluate these llms against each other and see if any can actually predict the future.

here's some quick findings from my recent backtesting project.

Can an LLM trade. Grade it only on data after its training cutoff, where it can't be remembering.

Setup. Prices $p_t$, position $x_t \in \{-1, 0, +1\}$ (short, flat, long). The position is held one bar before it earns, so returns are:

That one-bar shift blocks look-ahead. Period returns compound into total return:

Sharpe is mean return divided by volatility:

Leakage test. Compare the candidate's Sharpe before and after the training cutoff. A model with edge should maintain it; a model that was memorizing should collapse. Subtract out regime drift (using momentum as control):

If $L > 0$ and the 95% bootstrap CI excludes zero, the drop is real leakage, not market regime.

Significance. Permutation test: flip sign of each return at random $P = 2000$ times, count how many have Sharpe as high as the real one:

Bonferroni-adjust for multiple candidates: $p_{\text{adj}} = \min(1, p \cdot N)$.

Results

gpt-4o-mini (real model)

Weekly direction, five assets (AAPL, MSFT, NVDA, SPY, TSLA), split at Oct 2023 cutoff.

No edge post-cutoff (0.101 barely beats control at 0.036). No leakage (CI contains zero).

Synthetic controls (method check)

The memorizer collapses after cutoff. The method catches it. Momentum and noise don't. The leakage test works.

One sentence. The method detects memory, a real model shows no trade edge once it can't cheat.