Try and file runs

The Try tab runs the prism on real text from the app, without code. Every answer it gets is saved as a result, like an API call, so it shows on the Results tab. What a result keeps of the text follows the prism's input storage.

An admin's Try runs the latest saved version, published or not, so you can check a version before you publish it. An operator's Try runs the published version.

Try one text

Under Try your text, the tab shows one box per field of the prism.

1. Paste the text into each field.
2. Optionally, fill in Reference with your own ID, such as a ticket number. It is saved with the result so you can find it later.
3. Choose Run check.

The Answers panel shows each question's answer and the version that gave it. The status line beside Run check says how long the answer took. A failed check is saved as a result too, and the panel shows the error. Open in Results opens the full result. Results explains what a result page shows.

If you edit the text after a check, the panel says Changed since last check, because its answers describe the earlier text. Run it again to check your edits.

If you have unsaved changes on Questions, the tab says so. Try uses the last saved version, not your unsaved edits. Save first to try them.

Image fields

An image field shows a file picker instead of a text box. Choose a PNG, JPEG or WebP file. The app scales a large image down before it sends it. A prism with image fields needs an account model that reads images. See Fields.

Run a file

Further down the tab, Run a whole file answers every row of a file, one row per text. Each row is saved as a result with the file as its source. File format, next to the heading, opens the file format details.

In short:

• The file is CSV, TSV or JSON. A .txt file is read as CSV or TSV.
• In CSV and TSV the first row is the column names. A JSON file is an array of objects.
• A file holds at most 10,000 rows and 200 MB.

Drop the file on the box, or click it to choose one.

Choose the columns

After the file loads, the app shows its name and row count, and asks where each part comes from.

• Text comes from, and one more like it for each field. The app picks a column whose name matches the field's key or name. You must choose a column for every field.
• Reference comes from is optional. Without it, the reference is the row number.
• Context comes from is optional. Each row's cell is sent as that call's context, after the prism's own.

Run 250 rows starts. The app answers eight rows at a time and shows its progress. Try, file runs and retries count against a per-account limit each minute; when the account reaches it, the run waits and goes on. Keep the tab open. Leaving it stops the run. Rows already answered are kept, and running the same file with the same columns and version picks up where it stopped.

Choose another file starts over.

Images in a file

For a prism with image fields, each image cell holds a file name, such as scan-0412.png. Under Images for this run, choose the images with Choose images, or a whole folder with Choose a folder. The app matches cells to files by name, ignoring case. Before it runs, it lists any cell whose file you haven't chosen, and any two chosen files that share a name.

When the prism's only field is one image field, you don't need a table. Choose the photos, and each one becomes a row with its file name as the reference.

After the run

The status line counts the rows answered and failed. A row the app could not send, for example one with a reference over 200 characters, counts as not saved and has no result.

• See these results opens the Results tab filtered to this file.
• Download answered file saves the file with the Lens columns and result_id added after your own columns, in the file's own format.

Check against your own labels

If the file already holds the answers you expect, from a QA team for example, the run can score the prism against them. Under Expected answers (optional), choose a column for each question you want checked. The app picks a column whose name matches the question's key or text. Leave the others on Not checked.

Group results by (optional) splits the figures by a column, such as language, site or team.

The app sends only the field columns, and the context column when you chose one. Your expected answers and the group column stay in your browser unless you save a test set.

The evaluation report

After a run with expected answers, the Evaluation section shows one block per checked question.

• A count line says how many rows were scored, and how many had no expected answer, were not run, failed, had no valid answer or abstained, such as Other or Unsure. Cells the app could not read are counted, with a few examples.
• Yes / No shows agreement, recall and precision for Yes at a range of cutoffs, plus a row for the Lens's own cutoffs with the share that fell in the Unsure band.
• Choice shows overall agreement, precision and recall per option, and agreement when only confident answers are counted.
• Scale shows exact agreement and agreement within one level.
• With a group column, a table splits the figures per group.
• Each row where the answer differs from your label is listed as a miss, with its reference, the expected answer, the answer and its probability. The report shows the first 50.

Every percentage shows its sample beside it, such as 62% (31 of 50), so a figure from three rows never looks like one from 300. Download evaluation saves the figures as CSV. Download misses saves every miss with its result_id.

Changing the expected-answer or group columns after a run updates the report without asking the model again. Checking answers against your own labels explains which cell values count and how each figure is worked out.

Save as a test set

A file run scores one version once. To compare versions, keep the file as a test set and run it from the Test tab on any saved version.

Save as test set is next to the run button once every field has a column. It opens a dialog:

• Save to picks A new test set, with a Name, or an existing set. Saving into an existing set makes its next revision, and older runs keep pointing at the revision they answered.
• Each case keeps its row's fields, reference, context, group and expected answers. Expected answers the app can't read are left out, and the dialog counts them.
• A set holds at most 1,000 cases. Each case needs its own reference, so the dialog stops you if a reference appears twice. Choose another reference column.
• A test set keeps the text of its cases until you delete it, whatever the prism's input storage. If the prism keeps a fingerprint or nothing, a checkbox asks you to confirm that first.

Save test set, or Save as revision 3 for an existing set, saves it. Saving doesn't run anything. Only admins can save test sets.