this is what a pi based LLM is great for; give it it's own email address, then say something like "I'm intrigued, please talk to my security expert, CC'd" and cc the pi.
thing is, we have both local models and local hardware and a true evaluation would do a calculation before openai inflated thw market, before nvidia made circular deals and the other distortions.
i think youd find the ROI is nowhere near the API rates are the "price support" is entirely a figment of billionaires and their parasites trying to corner the market by horde logistics
from the march changes, it looked like they increased cache eviction rates on the VRAM at claude causing everyone to start burning tokens as they had to regen token state.
they still have to cache the tokens. its not completely stateless.
in theory, every conversation is replayed from the beginning. in practice, its only going to be economical to heavily cache the stable portions of the text as tokens inside the GPU
one of the reasons the Cloud providers have such heavy prompts is because that can be cached for all users, but its essentially poisonong the state before you even start. alot of the variability appears related to changing the context rather than the model.
models are expensive and the bean counters know fine tuning and context changes are cheaper. id guess the IPOs are essentially the SOTA EOL.
whats important is the political influece. Just like Trump backtracking on his AI guidance, this is about moving poltical power over models. Whatever excuse chills and 'retrains' them to properly hate women, minotiries and the like. Give how hard Elon has tried and failed to turn models conservative, they need to get the larger models to lick the boot.
reply