Skip to content

Governance

and roll back changes.

Governance records every answer, every agent run and every settings change. When something looks wrong, you can see why and put it right.

Answers

Every answer, scored

Every question and answer is kept with its confidence and coverage scores. You see the minimum, average and maximum, so the weak answers behind a complaint stand out.

Semantic confidence

console › governance › semantic insights
The Semantic Insights view: tiles for each answer score, with a date picker.
Minimum, average and maximum for each score over the dates you pick.

Read the tiles

What each score tells you

What each tile measures over the dates you choose, and what a high or low value means.

Semantic confidenceHow well each answer matches the sources it came from, shown as lowest, average and highest. Your warning and minimum-confidence levels can use this score, so a falling average warns you early.

Agent runs

Every agent run, recorded

For each agent: how often it ran, its average and peak response times, the guardrails that fired, and the child agents it called. Find a slow agent fast, and open any run step by step.

Runs over time

console › governance › agent details
The Agent Details view under Governance: an agent picker and date range, tiles for invocations, average run time, median, tail and peak latency and guardrail activations, and an invocation tree linking a parent agent to its child agents.
Agent Details: latency per agent, and the invocation tree of the child agents it called.

Settings

Every settings change, reversible

Every change to your setup is logged with the user who made it and shown on a timeline. You can revert any setting from the change log.

Every change, with its author

console › governance › configuration insights
The Configuration Insights view under Governance: a large title above a timeline whose dated axis carries grouped change events.
Configuration Insights: change events grouped along a dated timeline.

What is recorded

Spot changes in quality early

Quality usually slips slowly: a source goes stale, questions change, a model is updated. These readings show it first.

  • Answer quality, as a distribution

    Confidence, coverage, answer length and how many sources an answer used, each with its minimum, average and maximum, so you see the full spread of results.

  • Quality by topic, over time

    Coverage and confidence for each topic, so you can spot when one type of question gets worse.

  • Which sources are actually answering

    Your knowledge base's confidence and coverage, the documents and URLs used most, and user ratings. Use them to tune search and decide which content to keep up to date.

  • The terms that keep appearing ungrounded

    Terms that appear in answers without support in your sources, ranked, so you can act early. Allow-list a term from the same view when it is correct.

  • Tokens, latency and cost

    Input and generated tokens, cost per call, speed and tokens over time, with a model cost comparison. Your finance team gets cost data next to the quality data.

  • Who is using it, and how much

    See who uses it and how much, in the same console as the answers.

Closing the loop

Each of these lets you make the fix in the same place you spotted the problem.

  • Fix an ungrounded term where you found it

    When a term flagged as hallucinated is correct, allow-list it from the same view. The change applies across your instance.

  • Settle a model question with evidence

    Select models, enter a question and compare the answers side by side, with token cost and speed recorded. Your model choice is backed by results you can share.

  • Roll a configuration back

    Open the change log, see who changed which setting, and revert it from the same screen.

  • Full audit trail

    Every question, answer, score and settings change is logged, timestamped and attributed. Export it to S3, Splunk or Datadog.

Questions from risk and security teams

What model-risk, security and finance reviewers ask about the record.

  1. Can we keep the logs in our own systems?

    Yes. Corporate logging sends Seek and Curate activity to an endpoint you run, so the record sits beside your other security data under your own retention policy.

  2. What can we look back on after an incident?

    The question and answer with their scores and sources, replayed as they appeared then. Which agent ran and which child agents it called. And which user changed which setting, and when.

Start building with NeuralSeek today.

Sign up online and start with a two-week trial, or talk to a NeuralSeek expert about running it in your own cloud.