Results

Every text a prism answers is saved as a result, whether it came from the API, the browser extension, a file run or Try. A prism's Results tab shows them. It has three views: Overview, Rows and Report.

This page covers the app. Results explains what a result stores and how the API returns it, and the Lens explains how answers become output columns.

Range and search

The bar at the top of the tab applies to all three views.

• The range picks 24 h, 7 days, 30 days or All kept.
• The search box finds results by their text, their reference or their result id. Press / to jump to it.
• The count on the right says how many results are in scope, and how many filters apply.

The views share the scope, so a filter set in Rows also narrows Report, and a click on a figure in Overview opens Rows with that filter.

Overview

Overview answers how the prism is doing in the chosen range.

The four figures

Each figure has a sparkline of its last 30 days.

• Texts counts the results in scope, split by source: API, Browser extension, File and App. It compares the count with the period before, such as "+12% on the week before", and says when the last text arrived.
• Flagged counts results with an answer you flagged, and its share of answered results. Before any answer is flagged it shows a dash and Choose answers. Review opens those results in Rows.
• Unsure counts answered results that were Unsure on at least one question, and its share.
• Failed counts results that got no answers, and says how many still wait for a retry. Retry all retries up to 200 at once. See them opens them in Rows.

A share is compared with the period before only when both periods have at least 15 answered results. When the change could be chance, it reads "about the same as" the period before.

Per day

The chart stacks each day's results by source. Lines mark when a version was published or saved. Click a day to open its rows.

What it found

Once answers are flagged, What it found lists each flagged question with how often each flagged answer came back. Click an answer to open those results. Change flags opens the Flags dialog.

Unsure, by question

This list ranks the questions by how many of their answers were Unsure, with the count and share. Review opens those results in Rows.

Live version

The Live version box names the published version and when it was published, and any newer version that is saved but not live. Below that it shows the verdict of the newest test of the prism, such as "Version 4 is better on Routing team.", with the test set, case count and date. Open Test opens the Test tab. A prism without a test set says Not tested yet.

Flags

Flags name the answers that need a person, such as Yes on a self-harm question or High on a concern level. Open the dialog with Choose answers or Change flags on Overview.

The Flag answers that need a person dialog lists every question of the published version with a checkbox for each answer. A Choice also offers Other and Not enough information. Tick the answers and click Save flags.

• A result with any flagged answer counts as flagged on Overview and on Home, and Rows can show only flagged results.
• Flags send, block and change nothing. They don't alter answers, API output or the browser extension.
• Flags are counted against the stored answers each time, so new flags count older results too. A Yes / No answer matches Yes or No by the cutoffs saved with its result.
• Flags follow the published version. Publishing a version that lacks a flagged question or answer drops that flag.

What Unsure means

An answer is Unsure when the model was not sure enough to rely on it.

• A Choice or Scale answer is Unsure when its top option or level has a probability under the Unsure threshold. The bar in Rows and Report shows it as Unsure: choice under 60%, and you can pick another value for the view. Save for this prism keeps it, and Reset to prism default goes back. The account default is on Defaults.
• A Yes / No answer is Unsure when its probability of yes falls inside the band between its Lens cutoffs. See Yes / No cutoffs.

A result is Unsure on Overview and Home when any question it answered was Unsure.

Rows

Rows lists the results in scope, newest first, 100 at a time. Load 100 more adds the next page.

The first column shows when each result was saved, the start of its text, where it came from and its reference. Then comes one column per shown question. Each header shows the share of its answers that were Unsure. A failed result shows Failed and its error across the answer columns, with Retry.

By default only the prism's key question, the first question of the published version, has a column. Columns picks others. When some questions have no column, Least sure shows each row's least sure answer among them.

Click a question's header to see its answer mix in the live version, and in the saved version when one is waiting. From there:

• Only unsure keeps the rows Unsure on that question.
• Least sure first sorts by that question.
• Hide column removes it.

Keys: j and k move, Enter opens, x selects, a adds to a test set, r toggles full-width reading and Esc closes.

Filters

Filters opens the rail on the left. It has groups for Status (Answered and Failed), Source, Version and each question, with a count beside every item. A question's group lists Unsure and its answers. Click items to combine them.

The quick chips beside Filters keep only Unsure on the key question, only Flagged results, or only Failed ones. Flagged shows once the prism has flags.

Each active filter shows as a chip with a remove button, and Clear all removes them all. When nothing matches, the view offers to remove a filter, clear the search or look at all kept results.

Select rows

Tick rows, or press x, to select them. A bar at the bottom shows how many, with:

• Retry N failed, for selected failed results that kept their text.
• Add to test set, which steps through the selected rows one by one. Skip passes a row.
• Export N, for up to 200 selected rows.
• Clear.

Export

Export N rows downloads every result in scope as CSV, TSV, JSON, JSON Lines or Raw JSON (every probability). The columns follow the prism's Lens, as the API does. Formats describes each one.

Find by text

A prism that keeps only a fingerprint of each text has Find by text beside the search box. Enter the exact fields and context that were sent, and Find results shows the results with that fingerprint. Only an exact match is found, letter for letter, and only for results saved with the published version's fields. The text you enter is not saved. It does not work for prisms with image fields.

Report

Report looks at one question at a time. Pick it in Focus question.

• A line gives the question text and its Unsure share. See the N unsure opens those rows.
• Per day charts each day's results by Status, Version, Confidence or Source.
• Answers shows each answer's share of answered results, in scope and per version. The amber part is Unsure. A Yes / No question also shows how its probabilities of yes spread around the Lens band. Click a row to open it in Rows.
• Other questions shows a small card for every other question. Click a title to focus it.
• Answer mix by version sets two versions side by side. Changes bigger than sampling noise are bold.

A result

Click a row to open the result beside the list. The pane has buttons for the previous and next result, Add to test set, full-width reading and close. A link to a result opens the prism's Rows with it open.

• The header shows when it was saved, its source, its reference, the version that answered it (such as v3 live or v4 saved) and how long it took. A retried result links to its retry, and the retry links back.
• The fields follow, as the result kept them. A result whose prism keeps only a fingerprint, or nothing, says Text not stored.
• Answers lists every question with its answer and its probability. Unsure answers come first. Click a question to see its text and every option's probability.
• Technical details shows the result id, model, input tokens, latency, source, reference and the date it is Kept until. A browser extension check also shows who checked it, the review, the session and the prism key. When an operator confirmed a warning and sent the text, Sent anyway records the time. See What Send anyway records.
• Copy bug report copies the result's details as JSON. Export... downloads this one result in any format. What the caller received shows the JSON the API returned.

Retry a failed result

A failed result shows Failed, no answers with its error, and Retry. A retry sends the stored text to the same version again and saves a new result. The failed one stays, marked Retried, answered. A result can't be retried when it kept no text or when its prism is archived. Send the text again from where it came from instead.

Add to a test set

Add to test set, or a, makes the result a case in a test set.

1. Pick a Test set, or A new test set with a Name.
2. Check the Reference. It identifies the case in the set.
3. Correct the expected answers. They start from what the result reads as. Not checked leaves a question out.
4. Click Create test set, or Add as revision N for an existing set.

If the set already has a case with that reference, the dialog says so and Update case replaces it. A result can't be added when it failed, kept no text, has other fields than the latest version, or belongs to an archived prism. After you add a result, Rows marks the results already in that set.

No results yet

A prism with no results shows what stands between it and its first one:

• whether a browser review runs it, or how to call it from Integrate or add it to a review,
• whether it has a test set, with Add labelled texts when it has none,
• a link to try it on one of your texts.

How long results are kept

An account keeps its results for 30, 90 or 365 days, 90 by default. An admin sets it under Keep results for on Defaults. Older results are deleted once a day, for everyone in the account. Test runs are not results and are not deleted with them.

What an operator sees

An operator sees only Rows, and only the results of their own checks. They can open and retry a result, but they can't export, add to a test set or save the Unsure threshold. With no checks yet, the tab points them to Try and the browser extension. See Roles.