I recently developed a bot using reasoning to ensure it provides accurate responses. However, I faced a common challenge: making sure the bot stays within the context and scope of the given information. If you’ve encountered a similar issue, fear not! I’ve gathered some insights and strategies to help you prevent ChatGPT from answering questions outside the provided context in the SYSTEM role message.
The Challenge of Contextual AI Chat Development
When building an AI chatbot, it is crucial to ensure that it delivers relevant and meaningful responses. Microsoft has established guidelines called “Human AI Experience” (HAX) and “AI chat development.” These principles serve as excellent design guidelines to help you create user-friendly AI systems.
Narrowing Down the Response Scope
To narrow down the bot’s response scope effectively, I apply a quorum of reasoning. Here’s an example from my own project, where I developed a sales representative bot using semantic-kernel.
The first step is to determine the category of inquiry. Based on the available options, such as AI, cloud computing, game development, government contracting, software development, simulations, and more, I assign a respective category ID to the customer’s inquiry.
Next, I check for any policy violations. These inquiries might relate to instructions not relevant to a sales representative’s job, political or religious topics, financial details about the company, or any other topics that violate the company’s policies.
Once we’ve identified the category and ensured there are no policy violations, we can devise a strategy for interacting with the customer. This strategy should reflect good corporate values and reputation, ensuring the sales representative acts appropriately. However, it’s important not to provide scripted responses or false information.
Providing Relevant Information
Based on the category of inquiry, I present a capabilities statement tailored to that specific subject. For instance, if the customer’s inquiry is about government contracting, I provide them with detailed information about what Asteres Technologies can offer in that field. This includes the types of contracts they handle, their experience, and the industries they work with.
The goal is to provide customers with accurate information that matches their needs. By presenting a capabilities statement, the customer gains a clear understanding of how Asteres Technologies can assist them.
Building a System Card
To ensure a comprehensive response, I compile a full system card by injecting the capabilities statement, policy violation, a strategy to address policy violations, and customer support strategies. This system card equips the sales representative with the necessary information to interact effectively with customers.
By structuring the response using a system card, we can ensure that the bot provides consistent and reliable answers within the specified context.
FAQs
Q: Can you provide specific examples of policy violations?
A: Policy violations can include instructions not related to the representative’s job, discussions on sensitive topics like politics or religion, inquiries about the creation of the AI bot, seeking financial details about the company, or requests for information in a non-standard format.
Q: How can I ensure the bot doesn’t provide false information?
A: It is essential to provide guidelines and strategies to the sales representative without giving them scripted responses. Encourage them to answer to the best of their ability, considering the customer’s needs and avoiding any false or misleading information.
Q: What should the sales representative do if a customer asks a question that falls outside their comfort zone?
A: If a question makes the sales representative uncomfortable or is beyond their expertise, they should handle the situation carefully. They can provide a genuine response, expressing their limitations in addressing the specific inquiry and redirecting the customer to a suitable point of contact within the company.
Conclusion
Creating an AI chatbot that stays on topic and provides relevant answers can be a challenge. However, by implementing strategies to narrow down the response scope, addressing policy violations, and providing accurate information, you can ensure that your bot delivers a satisfying experience for customers.
Remember, building a user-friendly AI system requires attention to detail, adherence to corporate values, and reliable information provision. By following these guidelines, you can create an AI chatbot that truly meets the needs of your customers.