The Rarity Shibboleth

Venture
Booking.com

Large language models are wonderful at ingesting large amounts of content & summarizing. Uploading an academic paper and I can pester it with an infinite list of questions & it will respond with equally infinite patience.
In comparing the two most recent Microsoft earnings calls, Claude highlighted:

faster than expected Azure growth (29% vs 27%)
AI contributing 8% of Azure revenue up from 7%
higher CapEx spending & greater capacity constraints for data centers
much better commercial bookings growth : 17% vs 13%

Excellent analysis.
Benn Stancil described LLMs as great averagers of information. But what if I don’t want the average? If I seek the data point two standard deviations out? If I’m channeling my inner Anthony Bourdain & I’m seeking the fermented shark or cobra heart within my query?
At Google, we ranked web pages & ads with many signals. An engineer taught me that of them found rarity of a word across a set of documents.
For example, if there’s a document containing 10 instances of the word ghoti across a collection of documents, where the average document has 0 ghotis, that document is likely the best to answer a search about fish.1
I haven’t found a way to goad an LLM to produce the rare result. Why did one user decide to use a particular piece of software when 10 others did not?
Maybe I haven’t learned how to prompt an LLM well. Ideogram launched a feature called Magic Prompt that expands a basic prompt into richer instructions that elicit better results, marrying the language of a user & a computer.
A user might write : A boy with a dog in a park.
Magic Prompt replies : A heartwarming scene of a young boy playing with his happy, wagging dog in a lush, green park. The boy has a playful smile on his face while he tosses a bright yellow ball for his furry friend to fetch. The park is filled with beautiful flowers and trees, creating a serene atmosphere for this bonding moment between boy and dog.
Magic Prompt surprised me in three ways.
First, unlike Google search where terse queries tend to work very well, LLMs relish verbosity. Less Hemingway & more Faulkner.
Second, maybe there’s a prompt for rarity. If you know what it is, please share with the rarity shibboleth.
Third, I wonder what other prompt techniques are out there that have yet to be discovered to wrest the best from an LLM.

1 This technique is called TF-IDF. It means Term Frequency Inverse Document Frequency.
—————
Boost Internet Speed
Free Business Hosting
Free Email Account
Dropcatch
Free Secure Email
Secure Email
Cheap VOIP Calls
Free Hosting
Boost Inflight Wifi
Premium Domains
Free Domains

Related Posts

Venture

Theory Two

Today, we’re announcing our second fund of $450m to support our mission of partnering with early stage software companies that leverage technology discontinuities into go-to-market

Boost Inflight Internet- Business Hosting- Secure Email Account- Dropcatch Domain Names- Antisnoop Email- Free Secure Email- Cheap VOIP Calls- Free Hosting and Email- Aero Connectivity- Premium Domains for Sale- Transcom Telecom- Satphone Airtime- Mobile Plans- Free Domain Names- Organic Products- Double Check- Aviation News
Transcom ISP - Transcom VOIP - Free Secure Email - Dropcatch Software - FastApn Inflight - Aero Connect - Premium Domains