A Series of Unfortunate Decisions

Venture
Booking.com

When a person asks a question of an LLM, the LLM responds. But there’s a good chance of an some error in the answer. Depending on the model or the question, it could be a 10% chance or 20% or much higher.
The inaccuracy could be a hallucination (a fabricated answer) or a wrong answer or a partially correct answer.

So a person can enter in many different types of questions & receive many different types of answers, some of which are correct & some of which are not.
In this chart, the arrow out of the LLM represents a correct answer. Askew arrows represent errors.
Today, when we use LLMs, most of the time a human checks the output after every step. But startups are pushing the limits of these models by asking them to chain work.
Imagine I ask an LLM-chain to make a presentation about the best cars to buy for a family of 5 people. First, I ask for a list of those cars, then I ask for a slide on the cost, another on fuel economy, yet another on color selection.
The AI must plan what to do at each step. It starts with finding the car names. Then it searches the web, or its memory, for the data necessary, then it creates each slide.

As AI chains these calls together the universe of potential outcomes explodes.
If at the first step, the LLM errs : it finds 4 cars that exist, 1 car that is hallucinated, & a boat, then the remaining effort is wasted. The error compounds from the first step & the deck is useless.
As we build more complex workloads, managing errors will become a critical part of building products.
Design patterns for this are early. I imagine it this way :

At the end of every step, another model validates the output of the AI. Perhaps this is a classical ML classifier that checks the output of the LLM. It could also be an adversarial network (a GAN) that tries to find errors in the output.
The effectiveness of the overall chained AI system will be dependent on minimizing the error rate at each step. Otherwise, AI systems will make a series of unfortunate decisions & its work won’t be very useful.
—————
Boost Internet Speed
Free Business Hosting
Free Email Account
Dropcatch
Free Secure Email
Secure Email
Cheap VOIP Calls
Free Hosting
Boost Inflight Wifi
Premium Domains
Free Domains

Related Posts

Venture

Theory Two

Today, we’re announcing our second fund of $450m to support our mission of partnering with early stage software companies that leverage technology discontinuities into go-to-market

Boost Inflight Internet- Business Hosting- Secure Email Account- Dropcatch Domain Names- Antisnoop Email- Free Secure Email- Cheap VOIP Calls- Free Hosting and Email- Aero Connectivity- Premium Domains for Sale- Transcom Telecom- Satphone Airtime- Mobile Plans- Free Domain Names- Organic Products- Double Check- Aviation News
Transcom ISP - Transcom VOIP - Free Secure Email - Dropcatch Software - FastApn Inflight - Aero Connect - Premium Domains