Now Assist Guardian

Now Assist Guardian is built on the ServiceNow Small Language Model (SLM) and monitors generative AI interactions to detect offensive content, prompt injection attacks, and sensitive topics.

Now Assist Guardian Overview

Generative AI is an emerging technology. Human interactions are unpredictable, and outputs generated by large language model (LLM) are probabilistic, which means that they're based on probabilities. Running the same input twice may generate two different outputs. Managing this risk is an important consideration when implementing generative AI on your instance. Now Assist Guardian evaluates requests sent to LLMs and their responses in real-time to reduce that risk.

Guardrails

Now Assist Guardian provides three guardrails. Each guardrail has a different scope of applicability:

Note: The scope of each guardrail differs. Prompt injection detection applies to all generative AI applications and features. Offensiveness detection applies only to supported Now Assist skills and workflows. Sensitive topic filters apply only to Virtual Agent conversations and require HR Service Delivery.

• Offensive content

  Due to the probabilistic nature of generative AI, it's possible for an LLM to generate offensive content. If there's offensive content in the input of the request, offensive content can also occur in the response. Examples of offensive content include language that is toxic, defamatory, or fraudulent.

  When offensive content is detected, Now Assist Guardian logs the event by default. You can also configure it to block the content. This guardrail applies to specific Now Assist skills and workflows.

• Prompt injection

  Prompt injection is a type of security attack where someone tries to override the normal instructions of an LLM to access restricted information or cause unintended behaviors. Now Assist Guardian detects prompt injection attempts by using an LLM trained on various types of prompt injection techniques, such as role playing, paraphrasing, repetition, instructions to ignore other instructions, and persuasion.

  Note: Due to the probabilistic nature of the model and evolving attack techniques, Now Assist Guardian may not identify every prompt injection attempt in some cases.

  Prompt injection protection applies to all generative AI applications and features on your instance. You can configure it at the instance level or for individual skills. When a skill has its own setting, Now Assist Guardian automatically applies the more protective of the two settings, the skill-level setting or the instance-level setting.

• Filtered subjects

  Certain subjects, such as workplace safety employee compensation, or personal well-being may not be best suited for generative AI responses. You can activate filters that detect these kinds of subjects in Virtual Agent conversations and redirect users to the Sensitivity Detection: Fallback Virtual Agent topic instead of generating an AI response.

  Note: Sensitive topic filters apply only to Virtual Agent conversational skills. These filters are available only with HR Service Delivery and Customer Service Management.

Logging and blocking

Now Assist Guardian detects and logs events for offensive content and prompt injection attempts by default. You can access logs from Now Assist Admin > Settings > Now Assist Guardian. Log data includes information about the request, the conversation that contains the offensive content, and any user feedback.

In addition to logging, you can configure Now Assist Guardian to block offensive content or prompt injection attempts. When blocking is enabled and content is detected, you see a standard error message instead of the generated response. The standard error message displays that the request couldn’t be completed, and you don't see what the AI generated. Before enabling blocking, review logs for a period of time to understand how frequently these issues occur in your environment.

Redirection for sensitive filtered topics

After a filter detects a sensitive topic, Now Assist Guardian redirects you to the Sensitivity Detection: Fallback topic in Virtual Agent. This topic can redirect you to a live agent or help you create an HR case.

You can override the redirection by selecting Proceed, not sensitive. It returns you to their original topic without initiating the fallback flow.

Note: After you continue with the fallback topic, for example, by starting the flow to create an HR case, Virtual Agent does not continue detecting sensitive topics in that conversation.

Now Assist Guardian at runtime

All skills that use Now Assist Guardian remove personally identifiable information (PII) before the request reaches the LLM. You can configure what type of data is anonymized. For more information see, Configuring Now Assist for Data Privacy.

For conversational skills, semantic search processes requests to determine whether a filter has been detected. If so, the user is redirected to a Virtual Agent topic that asks if they want to make an HR case or speak to a live agent.

For catalog item generation and agent skills, such as summarization and resolution note generation, offensiveness and prompt injection guardrails run on inputs and outputs of requests. If either is detected, Now Assist Guardian logs the request. If you’ve chosen to block this content, then a standard error message appears, and the user doesn’t see the result of the request.

Safety categories captured in Now Assist Guardian logs

Skills that support offensive detection

• Activate offensiveness protection for generative AI
  Activate offensiveness detection to log or block offensive content generated by Now Assist skills and workflows.
• Configure prompt injection attack protection
  Activate or deactivate prompt injection attack detection settings to protect all generative AI interactions on your instance from malicious inputs and unintended model behaviors.
• Configure sensitive topic filters
  Set up filters in Now Assist Guardian to redirect Virtual Agent users to a live agent or HR case when a sensitive subject is detected in a conversation.
• Export Now Assist Guardian logs
  Export logs from Now Assist Guardian to get insights into how often different guardrails are being detected and used.
• Configuring a Guardrail Service Provider
  Now Assist Guardian supports default guardrails and third-party or custom guardrail service providers to extend AI content monitoring with your organization's AI governance policies.

Parent Topic:Agentic AI security and governance