NAVI Console: Policy-driven Safeguards Comparison

Introduction

Nace encountered challenges verifying machine-generated outputs against task-specific guidelines, particularly affecting enterprise partners needing automated, policy-compliant text generation.

The company developed Nace Automated Verification Intelligence (NAVI), a verification system for reviewing policy-aware outputs and identifying violating text. This blog post argues that current solutions focusing on factual groundedness are insufficient for detecting policy violations, and prompting off-the-shelf LLMs proves inadequate for this task.

Nace created a comprehensive evaluation procedure featuring the Policy Alignment Verification (PAV) Dataset with human-curated policy-aware generation examples. The Safety Model surpasses existing solutions while maintaining "enterprise-grade latency and reliability." A subset of the PAV Dataset is open-sourced.

Topics covered:

1. Current solutions and NAVI overview
2. Methodology for creating custom validation datasets
3. Experimental setup description
4. Performance comparison demonstrating NAVI Safety Model's effectiveness

Existing Solutions

AWS Bedrock Guardrail Amazon's tool evaluates user inputs and model responses against configured policies, including content filters, denied topics, sensitive information filters, and word filters. Primarily focused on content safety and factual verification, it can be adapted for policy alignment tasks.

Azure Groundedness Detection Microsoft's service identifies whether AI-generated content is grounded in input data by comparing output content to provided context, flagging inconsistencies or hallucinations. Although designed for factual verification, its approach can apply to policy alignment verification.

NeMo Guardrails with GPT-4o NVIDIA's open-source toolkit for guardrailing inputs and outputs in AI applications features moderation applied to input, dialog, retrieval, tool use, and outputs. Several modes can be evaluated for policy alignment verification.

Prompt-based Adaptation of Off-the-shelf LLMs This approach uses carefully crafted prompts with existing large language models for policy alignment verification, offering flexibility and potentially competitive performance.

The evaluation finds most solutions focus specifically on factual grounding, with NAVI providing broader policy alignment assessment.

How NAVI Works

NAVI's verification process comprises four stages:

1. Retrieval Pipeline: Processes submitted policy documents into a representation suitable for effective decision-making.
2. Safety Model: Responsible for primary decision-making, outputs verification results indicating "Compliant" or "Noncompliant" with policies.
3. Explanation Module: Provides reasoning for "Noncompliant" labels.
4. Reference Module: Supplies specific references to violated policy sections.

The evaluation focuses on the Safety Model's performance as the core component producing verification results.

Policy Alignment Verification Dataset

The dataset was constructed with specific criteria:

1. Diversity and Realism: Wide range of cases reflecting real-world scenarios applicable to various contexts and challenges.
2. Complexity: Sufficiently complex to present significant challenges for large language models.
3. Specificity: Tailored to policy verification rather than general fact-checking.

Dataset Fields:

Field	Description
query	User's question or prompt triggering the generated response
response	Candidate text requiring policy alignment review
context	Selection of policies relevant to the response
label	Ground truth indicating compliance with specified policies
subset	Thematic grouping of policies and documents
files	Paths to files from which policy knowledge should be extracted
target file	Specific policy file the response targets

Data Collection Process:

1. Applications span diverse industries including Aviation, Communications, Finance, Legal, and Retail, with capability-specific subsets evaluating model behavior.
2. Industry-specific cases focus on representative companies with publicly available policy documents.
3. Multiple violating and policy-following texts were manually created for each subset.
4. Annotation criteria were iteratively refined achieving over 95% inter-annotator agreement.
5. Response text was processed through embedding-based approach to retrieve context.
6. Open-source version includes 6 high-quality industry-specific subsets.

The open-source dataset consists of 125 examples across 6 industry subsets: 70 with positive labels and 55 with negative labels (Noncompliant is positive).

Subset Statistics:

Subset	Number of Files	Number of Examples
AT&T	2	20
Airbnb	2	21
Cadence Bank	2	20
Delta Airlines	2	21
Verisk	40	21
Walgreens	2	22

Dataset accessible on HuggingFace: https://huggingface.co/datasets/nace-ai/policy-alignment-verification-dataset/

Experimental Setup

Relevant Policies

NAVI's superior performance stems partly from its Retrieval Pipeline, which comprehensively presents related policy information across multiple documents. Since vendors have varying context length requirements and lack native document processing support, all competing solutions received processed policy outputs from NAVI's pipeline, ensuring evaluation focused solely on verifiers' ability to identify policy violations.

AWS Bedrock Guardrails Method:

1. Create guardrail using AWS platform with Relevance and Groundness flags following documentation.
2. Run passages against extracted relevant policies, computing Relevance and Groundness scores.
3. Find optimal thresholds for both scores to achieve highest F1 score on target dataset, testing combinations from 0.01 to 0.99 with 0.01 step increments.

Azure Groundedness Detection Method:

1. Create Content Safety guard on Azure platform following documentation.
2. Generate plausible queries from responses using LLM where queries were missing.
3. Test both QnA and Summarization modes; report results based on better-performing Summarization mode.

NeMo Guardrails Description:

• Processes requests in multiple steps: analyzing user intent, determining appropriate action based on defined rules, and generating policy-compliant responses.
• Requires configuration files specifying models, rules, and action flows.
• Uses GPT-4o as underlying model for generation and verification.
• Appends policy rules to each prompt.
• Performs multiple API calls per request.

Limitations include configuration file updates required for each new policy check, concise policy descriptions due to context length restrictions, higher latency from multiple API calls, and limited ability handling complex multi-document policy scenarios.

NeMo Guardrails Method:

1. Install NeMo Guardrails following documentation.
2. Set up OpenAI API access for GPT-4o.
3. Configure guardrails with appropriate policies and rules.
4. Reframe problem to verify user-provided text against established policies.

Prompt-Based Safety Method:

1. Select state-of-the-art general reasoning models: gpt-4o-2024-11-20 and claude-3-5-sonnet-202410222.
2. Try various prompting approaches including zero-shot, rule learning, and few-shot learning with manually crafted 3-example demonstrations.
3. Run several improvement iterations in prompt styles.
4. Record best results under few-shot setting.

Evaluation Framework

F1 score serves as the evaluation metric, accounting for both precision and recall. Focus centers on the Noncompliant class, as identifying these cases is crucial—missing noncompliance instances could have serious consequences. All results reported are based on the open-source version of the evaluation dataset.

Latency estimations use Average Model Latency in milliseconds, measured on the same machine with time recorded for each example and averaged across all examples.

Results and Analysis

Model	F1 Score	Avg Model Latency (ms)
AWS Bedrock Guardrail	76.5	342.79
Azure Groundedness Detection (summarization)	71.2	232.71
NeMo Guardrails (gpt-4o)	71.2	2669.68
gpt-4o (few-shot)	75.0	904.46
sonnet-3.5 (few-shot)	75.5	2926.69
NAVI	90.4	387.62

Key Findings:

Performance: NAVI achieved 90.4% F1 score, significantly outperforming AWS Bedrock Guardrail (76.5%), gpt-4o/Claude Sonnet (~75%), and Azure/NeMo implementations (71.2%). This demonstrates robust ability to accurately identify policy violations.

Speed: With average processing time of 488ms (note: table reports 387.62ms), NAVI matches AWS Bedrock Guardrail and Azure speeds while proving markedly faster than NeMo (2669ms) and Claude (2926ms) implementations.

Purpose-built systems like NAVI provide substantial advantages over general-purpose guardrails and prompt engineering approaches. The tailored architecture enhances both performance and speed.

Conclusion

The NAVI Safety Model demonstrated "remarkable performance, outperforming existing solutions like AWS Bedrock Guardrail." Results underscore the importance of specialized systems achieving higher accuracy and faster processing compared to general-purpose and factuality-driven models. As organizations seek automated policy compliance solutions, "NAVI stands out as a robust and efficient tool, capable of meeting the demands of enterprise-level applications." Findings pave the way for further advancements in policy verification technology and multi-stage AI applications.

Figures

Figure 1: NAVI Console verification flow

Figure 2: AWS Bedrock Guardrails 3D surface plot

Figure 3: Performance and Latency comparison