A framing from that podcast that helped me better understand the role of Bonsai is that it's really a framework for building incremental distributed state machines.
This framing is also in the post. From the readme:
> Bonsai itself -- this library -- is actually more generic than the above makes it sound. It allows you to build general-purpose incremental, composable state machines. Bonsai_web builds on top of that core library, specializing it for interactive browser-based UIs, but we also have Bonsai_term for building interactive terminal-based UIs.
Senior software engineer, ~11 years' experience. Most recently at Lambda (AI inference infra). Before that, ~7 years at Solvuu building a data science platform for genomics research and data analysis. Looking for senior backend / devops roles — full-time.
To me it sounds like a tooling problem. OP seems to be trying to use probabilistic text systems as if they enforce rules, but rule enforcement should really live outside the model. My sense is that there was a failure to verify the agent's intent.
The tooling that invokes the model should really define some kind of guardrails. I feel like there's an analogy to be had here with the difference between an untyped program and a typed program. The typed program has external guardrails that get checked by an external system (the compiler's type checker).
What tooling? It's a probabilistic text generator that runs in a black box on the provider's server. What tooling will have which guardrails to make sure that these scattered markdown files are properly injected and used in the text generation?
That's the million dollar question. Maybe have systems of agents that all validate each other's work? Maybe something needs to be done at the harness level? I don't suppose that we could realistically expect 100% accuracy, but if we take 100% to be the upper limit, we could build systems that get us closer to that ideal.
No no, that’s not what I’m saying. The fact that the data is stored in files is incidental. It could be in a database, in a knowledge graph, derived from so other data Regardless of where it is, something should know to include it in the context, but only when it’s relevant.
So for instance you could start by trying to classify the prompt in some way. If you use an LLM for this, you might need to get it to return a machine parsable data format. Then your harness can pattern match on the classification and use it to enrich the prompt with additional context. The challenge would be in determining how exactly you want to go about this, balancing tradeoffs such as accuracy, cost, time, etc..
For the classification step you might begin with something like "Determine whether the following prompt is a QUESTION or a STATEMENT. Respond using only one of the two words. Prompt: $PROMPT"
You could have multiple back-and-forths like this and at each round you gain more information about the prompt, and you can use that information to determine further classifications and/or context to include.
> Regardless of where it is, something should know to include it in the context,
Magic. You're talking about magic. You keep re-iterating the same faith that "There's some magic way to make probabilistic text generator running in the cloud to never miss local files", where "files" is "files, knowledge graphs, databases etc.".
It doesn't matter how data is stored. You can't know when to include something relevant in the context because the whole thing including context is running in the cloud. You are not in the driver's seat. Literally anything you include locally in the prompt can and will be ignored.
I’m not following. If I run an agent on ollama locally, it’s not in the cloud. I don’t see what cloud has anything to do with the argument.
As to your other point about anything you include in the prompt can and will be ignored. Yes, I agree. You could draw an analogy to how a teacher assigns an in-class reading assignment and follows it up with a reading comprehension quiz. If your mind wanders during the reading you may come to find that you will fail the quiz because “anything you include in the prompt can and will be ignored”. Therefore, the quiz result serves the purpose of an evaluation.
Why would anyone settle for underpaid positions from an agency taking a 7% contract cut, and purging CVs from any external firm also contracting with their services.
Most people figure out this scam very early in life, but some cling to terrible jobs for unfathomable reasons. =3
If folks expect someone to solve problems for them, than 100% people end up unhappy. The old idea of loyalty buying a 30 year career with vertical movement died sometime in the 1990s.
Ikigai chart will help narrow down why people are unhappy:
> If folks expect someone to solve problems for them
In this type of situation, the fundamental issue is that making progress depends on many people acting in unison to increase their bargaining power, which is (a) hard to arrange even if everyone who acted this way would benefit, and (b) actually may be detrimental to some people (usually the high performers).
I agree it is nearly impossible to alter the inertia of existing firms. Most have entrenched process people that defend how things are done right up until a company enters insolvency. Fine if you sell soda or rubber tires, but a death knell for technology or media firms.
In my observations it is usually conditioned fear, personal debt-driven risk aversion, and or failure to even ask if the department above you is really necessary. These days, it is almost always easier to go to another firm if you want a promotion. =3
You know you could just define the verified specs in lean and if performance is a problem, use the lean spec to extract an interface and tests for a more performant language like rust. You could at least in theory use Lean as an orchestrator of verified interfaces.
Strong agree. OCaml's compiler is sofa king good at catching and preventing real bugs that the agents accidentally introduce here and there. It's the same as with humans, except the agents don't complain about the bullshit reasons humans don't like OCaml. They just crank through and produce high quality output.
