|

Docs

Lorebooks

Unfortunately there is a limit to how much information can be included in a Character's permanent definition. However, you may have additional information you want the AI to remember.

This is where Lorebooks come in. Each entry in the Lorebook is a keyword with an associated value. When that keyword appears in conversation, the corresponding value is automatically added to the prompt, allowing the AI to better remember what you're talking about.

Lorebook

Lorebooks can be especially helpful for characters from a world with a lot of lore. By leveraging keywords and associated values, the system injects relevant context directly into the system prompt, allowing the AI to maintain accuracy, consistency, and depth throughout an interaction. This mechanism enables the AI to recall specific information when needed rather than processing excessive stored data at all times, thereby optimizing tokens usage and improving response coherence.

Creating a Lorebook Entry

To add a Lorebook Entry, the user must define two components:

  • Key(s): The trigger words that prompt the system to recall specific information. You can configure multiple keys by passing a comma-separated list.
  • Value: The corresponding information injected into the AI’s System Prompt when any of the keys appear in the chat.

Example Lorebook Entry

As an example, you may have:

  • Keys(s): 
sword, blade, weapon
  • Value: 
A well-crafted longsword with an intricate hilt forged by master blacksmiths of the kingdom.

How are Lorebook keys detected

During a conversation, we will look back at the last two user messages, and the last two character messages. If any of the Keys are detected, we will insert the Value in the prompt. A few details:

  • Lorebook Keys are not case sensitive, so "Bath" would match with both "BATH" and "baTH".
  • The Key search will match partial text. So the Key "fire" will match with "I saw a firefighter".
  • If the Keys are detected multiple times, the Value is only added once.
  • Lore entries cannot trigger other lore keywords. Lore entries are not searched within when finding Lore keywords.

In the example above, if any of the last four messages contain "sword," "blade," or "weapon", we will add "A well-crafted longsword with an intricate hilt forged by master blacksmiths of the kingdom." to the prompt. This will help the character understand that the sword is a longsword.

Tips

1. Keep values short and relevant

  • Each value should be concise, ideally a sentence or two. Overly long values may overload the system prompt, leading to confusing or irrelevant responses. Multiple Lorebook Entries can be triggered at once, and they can add up quickly.

2. Be specific with your Keys

  • Vague keywords can trigger unintended responses.
  • Example: A keyword entry for "ape" will trigger for "shape" and "grape", potentially causing unintended responses.

3. Ensure Proper Formatting

  • Separate multiple keywords with commas. Avoid adding extra commas at the beginning or end of the keyword list. "dragon, wyrm, fire-breathing beast" is correct. ", dragon, wyrm, fire-breathing beast," is incorrect (The extra commas at the beginning and/or the end are wrong).

Common Issues and Troubleshooting

Problem: A Lorebook Entry Does Not Trigger
  • Verify that the exact keyword or a close variation appears in recent chat exchanges.
  • Ensure the entry is saved correctly and formatted without unnecessary spaces or punctuation.
Problem: Too Many Lorebooks Triggering at Once
  • Reduce the number of keywords per Lorebook entry.
  • Keep values short to prevent the system prompt from becoming too dense.
Problem: AI Becomes Repetitive or Overly Reliant on the Lorebook

Refine the wording in the values to ensure the AI integrates the information naturally.

If necessary, reduce the number of lorebook entries that rely on frequently used words.

Why Use Lorebooks Instead of Personality Fields?

While it may seem easier to store all relevant information in the Personality field, this approach has several drawbacks:

  • The Personality field is always loaded in every AI response, which can cause memory bloat and reduce efficiency.
  • Lorebooks load dynamically, ensuring only relevant information is applied at the right time.
  • Using Lorebooks prevents critical details from being lost in long conversations while maintaining flexibility.