Open source does not apply to AI and we should discourage anyone from using that term. All the models are opaque and proprietary. You cannot go into any source code and fix bugs, or add features, or study it to learn more. It's not the same thing at all as open source software.
Yes. The fact is we have no idea what's in these models. People are afraid of mkultra-ed humans that turn into killing machines with a given trigger word, but here we've built machines that are exactly that, not even intentionally! Yes, guard rails have gotten better and reports of LLMs talking people into killing themselves have gone down, but sorry if my trust has been broken!
the difference is, that a model is literally just numbers.
proprietary software is data and code, and while it's true that code is data, the difference is that some data is intended to be for control flow, while in a model it is all the same stuff.
You can point to any single part of an LLM's model and say "this here is a weight" but if you point to any single part of a binary file you will have no idea what you're looking at.
And yes, you could decompile the binary, but that still doesn't give you the entire picture, and all proprietary software does the shady shit on servers these days anyway so you're not even going to find anything interesting.
what if i want to change out the middle part of the training, and then still run the rest as before? or if i want to cull a bunch of the initial training set?
a binary is also just numbers, but we know there's other parts to it
