Hot off the Wires 🔥

We're eavesdropping on the smartest minds in research. 🤫 Don't miss out on what they're cooking up! In this section, we dissect some of the juiciest tech research that holds the key to what's next in tech.⚡

From chatbots that seem to know all the latest tech specs to recommendation engines that understand your preferences better than you do, LLMs are constantly evolving to handle more specific, high-stakes applications. A crucial piece of this puzzle lies in how these models access and use external data, whether to boost accuracy, handle domain-specific queries, or adapt to rapidly changing information.

Several of our previous editions have showcased applications of LLMs using various techniques, such as Retrieval-Augmented Generation (RAG), fine-tuning, and dynamic knowledge integration. And we’re sure many of you might be curious to know about:

• How can LLMs get even better at using external data?

• What exactly are the differences between these techniques?

• When to use which technique?

That’s why when Spark & Trouble came across this recent survey paper from Microsoft Research, they couldn’t resist breaking it down for you!

So, what’s it all about?

External data is crucial for LLMs for several reasons, especially when applied in various fields:

• Enhances Domain-Specific Knowledge: External data allows LLMs to access current and specialized information that may not be included in their initial training.

• Reduces Hallucinations: By integrating external data, LLMs can provide more reliable and accurate outputs.

• Supports Real-World Applications: LLMs need external data to tackle real-world tasks like analyzing financial trends or medical records, helping them make sense of all that complex info.

• Improves Interpretability: Using external data helps LLMs explain their answers better, making it easier for users to understand where the information is coming from and trust the model’s responses more.

However, all external data is not the same! Neither are the requirements of different data-augmented LLM applications. The researchers understood this and came up with a clever way to categorize various applications into 4 major buckets (based on the type of query).

These are fairly intuitive & understanding them will enable you to choose the right set of techniques for your use cases.

Forging the fundamentals

Let’s get up to speed with some of the terminology that you should be aware of to understand the essence of this research:

Under the hood…

Let’s now break down each of the 4 categories of queries for data-augmented LLM applications:

Level 1 - Explicit Fact

Level 1 queries are straightforward, fact-based questions like “Where will the 2024 Summer Olympics be held?” They don’t require complex reasoning—just the right info from external data.

The challenge for LLMs is making sure they retrieve accurate and up-to-date information quickly. If the data isn’t well-organized or easy to access, the model might stumble.

Solutions? Keep external data well-structured, and regularly updated, and use RAG techniques to ensure your AI always finds the right facts fast! This can include a plethora of modalities (documents, images, videos, web pages, etc.)

Level 2 - Implicit Fact

Level 2 queries are like the "brain teasers" for AI—they go beyond simple questions and need a bit of reasoning and linking facts together. Imagine asking, "Who's the majority party in Canberra’s country?" The model first has to know Canberra is in Australia, then find out the leading party.

These questions challenge AI by requiring it to piece together info from different sources, make logical jumps, and avoid misunderstandings.

To tackle this, advanced retrieval techniques like iterative RAG can be used. Here, the model first generates an initial answer based on what it knows. Then, it retrieves additional information to fill in any gaps or correct mistakes in that answer. This process can happen several times, with the model refining its response each time.

Another really cool way to work with such applications is to maintain the external data in a knowledge graph (KG) and use techniques like Graph-RAG to retrieve relevant information optimally from the KG before answering the question.

Level 3 - Interpretable Rationale

Level 3 Queries are the kind of questions where LLMs have to go beyond just giving facts—they need to explain why something is true. For example, asking “Why is it important for a drug to follow FDA guidelines?” means the model has to not only know about the guidelines but also explain their significance.

The challenge here is understanding complex contexts, connecting the dots from multiple sources, and giving a reasoned answer. Moreover, since the context here could be fairly long, optimizing the prompts for LLMs to answer such questions is of great importance.

This is exactly where techniques like prompt tuning & chain of thought (CoT) prompting shine.

Level 4 - Hidden Rationales

Level 4 queries are the trickiest questions, and they need super deeper understanding and reasoning skills to answer—think of asking, "Why do some medications work better for certain patients?"

To tackle these, LLMs face challenges like making connections from hidden data and needing complex reasoning skills, along with a lack of precise documentation to infer the results correctly.

This is where techniques like in-context learning & fine-tuning come in handy. Offline learning is another interesting paradigm to address level 4 queries. It includes techniques that are used to improve the performance of models by learning from past data without needing real-time input. Here are some key methods:

Why does this matter?

Although this may not be a typical ‘breakthrough’ in AI that Spark & Trouble discussed today, this understanding is essential for advancing the field of natural language processing and improving the utility of LLMs in various domains.

Internalizing this classification of queries & the methods to address them can enable AI enthusiasts like you to quickly & correctly choose the right frameworks to build your own AI applications that rely on external data.

Our resident data enthusiast (aka Trouble) would be curious to learn more about your endeavors in this space to assist you in any way possible.

Write to us about what’s on your mind and how might you use these techniques in your projects…

10x Your Workflow with AI 📈

Work smarter, not harder! In this section, you’ll find prompt templates 📜 & bleeding-edge AI tools ⚙️ to free up your time.

Fresh Prompt Alert!🚨

Ready to share your genius with the world but stuck wondering if anyone will actually stay awake through your course? Been there!

This week's Fresh Prompt Alert is your personal instructional design sidekick! It helps you turn your brain dump into a binge-worthy learning experience. It's like having a veteran teacher whispering the perfect lesson plan in your ear.

Ready to be the instructor students rave about? Take the prompt for a spin 👇

Need inspiration?
Check out how we obtained a stellar course outline based on an international bestseller:

Insights from “Thinking, Fast and Slow” by Daniel Kahneman

3 AI Tools You JUST Can't Miss 🤩

• 📒 timeOS Magic Notepad: Organizes your meeting notes and action items while you focus on the conversation

• 🗣️ Aptitude: Let AI conduct and analyze conversations with hundreds of customers

• 🐞 Langtail: The low-code platform for testing AI apps to catch bugs before users see them

• 🖼️ OmniGen: Consistent AI Image Generator Online

• ⚡ Bolt: Prompt, run, edit, and deploy full-stack web apps

Spark 'n' Trouble Shenanigans 😜

Our dynamic duo is here with a meme that truly captures the sentiments of some users after playing around with OpenAI’s much-coveted ‘king-of-reasoning’ o1 model.

(Check out this Reddit thread for more context)

Well, looks like there is still a long way to go for AI models to be hallucination-free!