Summary

Building safer large language model (LLM) applications is crucial as generative AI continues to evolve. This article explores how developers can use LangChain Templates and NVIDIA NeMo Guardrails to create secure and accurate LLM applications. By integrating NeMo Guardrails into LangChain Templates, developers can ensure content moderation, enhanced security, and evaluation of LLM responses.

Building Safer LLM Apps with LangChain Templates and NVIDIA NeMo Guardrails

Introduction

As generative AI continues to advance, ensuring the safety and security of large language model (LLM) applications becomes increasingly important. LangChain Templates and NVIDIA NeMo Guardrails offer a powerful solution for developers to build safer LLM apps. This article will guide you through the process of integrating NeMo Guardrails with LangChain Templates to create secure and accurate LLM applications.

What are LangChain Templates?

LangChain Templates are a new way to create, share, maintain, download, and customize LLM-based agents and chains. These templates enable developers to add newer chains and agents that others can use to create custom applications. They integrate seamlessly with FastAPI for building APIs with Python, adding speed and ease of use.

What are NVIDIA NeMo Guardrails?

NVIDIA NeMo Guardrails is a scalable AI guardrail orchestration platform for safeguarding generative AI applications. It provides programmable rules and runtime integration to control user inputs before engaging with the LLM and to validate the final LLM output. NeMo Guardrails ensures safe, reliable, and customizable AI applications, including RAG-enabled AI agents, copilots, and chatbots.

Integrating NeMo Guardrails with LangChain Templates

To integrate NeMo Guardrails with LangChain Templates, developers need to create a directory named guardrails and configure the necessary files such as config.yml, disallowed.co, general.co, and prompts.yml. These configurations define the guardrail flows that control the chatbot’s behavior and ensure it adheres to predefined rules.

Here’s an example of how to integrate NeMo Guardrails with LangChain Templates:

  1. Install LangChain CLI and NVIDIA AI Foundation Endpoints package

    pip install -U langchain-cli
pip install -U langchain_nvidia_aiplay

  2. Download and customize the LangChain Template

    langchain app nvidia_rag_guardrails --package nvidia-rag-canonical

  3. Create the guardrails directory and configure the necessary files

    • Create a directory named guardrails.
    • Configure config.yml, disallowed.co, general.co, and prompts.yml files.

  4. Activate the guardrails flow

    • Include the rails in the config.yml file.
    • Define general configurations for the LLM models, sample conversations, and rails.

Benefits of Integrating NeMo Guardrails with LangChain Templates

  • Content Moderation: NeMo Guardrails provide content moderation to ensure that LLM responses are accurate and contextually relevant.
  • Enhanced Security: NeMo Guardrails offer enhanced security by controlling user inputs and validating the final LLM output.
  • Evaluation of LLM Responses: NeMo Guardrails evaluate LLM responses to prevent cybersecurity attacks like prompt injection.

Example Use Case

Let’s consider an example where we download an existing LangChain template with a RAG use case and integrate NeMo Guardrails.

  • Download the LangChain Template

    langchain app nvidia_rag_guardrails --package nvidia-rag-canonical

  • Set up the ingestion pipeline The downloaded template sets up an ingestion pipeline into a Milvus vector database. The existing ingestion pipeline includes a PDF with information regarding Social Security Benefits. As this dataset contains sensitive information, adding guardrails can help secure the LLM responses and make the existing LangChain Template trustworthy.

  • Integrate NeMo Guardrails Create a directory named guardrails and configure the necessary files such as config.yml, disallowed.co, general.co, and prompts.yml.

Table: Comparison of LangChain Templates with and without NeMo Guardrails

Feature LangChain Templates without NeMo Guardrails LangChain Templates with NeMo Guardrails
Content Moderation No content moderation Provides content moderation to ensure accurate and contextually relevant LLM responses
Security No enhanced security Offers enhanced security by controlling user inputs and validating the final LLM output
Evaluation of LLM Responses No evaluation of LLM responses Evaluates LLM responses to prevent cybersecurity attacks like prompt injection
Customization Limited customization options Provides programmable rules and runtime integration for customizable AI applications

By integrating NeMo Guardrails with LangChain Templates, developers can create safer and more secure LLM applications that meet the evolving needs of generative AI.

Conclusion

In this article, we explored how developers can use LangChain Templates and NVIDIA NeMo Guardrails to build safer LLM applications. By integrating NeMo Guardrails into LangChain Templates, developers can ensure content moderation, enhanced security, and evaluation of LLM responses. This solution provides a powerful tool for creating secure and accurate LLM applications, making it an essential component in the development of generative AI applications.