> ## Documentation Index > Fetch the complete documentation index at: https://vijil-dome-ui.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Reliability > Scenarios and probes for the Reliability dimension of trust (correctness, robustness, consistency). Hover over any [Scenario](/scores/scenario) or [Probe](/scores/probe) name to view its ID in Vijil. | Probe Name | Description | | :--------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | SemAttack Attack on MNLI Tasks | These tests measure robustness against semantic pertubations on tasks where the agent must identify the relationship between a premise and a hypothesis. The tasks come from the MNLI benchmark. | | TextFooler Attack on RTE Tasks | These tests measure robustness when replacing important words with their synonyms on tasks where the agent must identify the logical entailment between two sentences. The tasks come from the RTE benchmark. | | TextFooler Attack on MNLI Tasks | Measures whether the agent's performance on the MNLI task is robust against TextFooler perturbations | | TextFooler Attack on MNLI-MM Tasks | Probes effects of TextFooler perturbation on MNLI-MM task | | BERT-Attack on RTE Tasks | Probes effects of BERT-based perturbation on RTE task | | SemAttack Attack on MNLI-MM Tasks | These tests measure robustness against semantic pertubations on tasks where the agent must identify the relationship between a premise and a hypothesis. The tasks come from the MNLI-Mismatched benchmark. | | Probe Name | Description | | :------------------------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | False Assertions: Riley Goodside | Evaluates hallucinations about Riley Goodside | | False assertions: Senators | Checks for false assertions about US senators. A high score indicates a lowered tendency to hallucinate information. | | Misleading Information | Tests that check if the agent can be tricked by misleading information. A high score indicates a lower likelihood of producing hallucinated content, when presented with false or misleading claims. | | Package Hallucination | Tests that try to trick the agent into importing modules that dont exist in Python and other languages. A high score indicates that the agent does not produce hallucinated package information. | | Unanswerable Query Fabrication | Tests whether the model fabricates confident answers to queries that ask for specific information that cannot be known or verified. | | Probe Name | Description | | :----------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | False Assertions: Flights | Check if the agent invents impossible flight routes. A high score indicates that the agent is resilent towards creating impossible or factually impossible scenarios. | | Probe Name | Description | | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Math Robustness: 2-Step Problems with Distractors | Tests agent robustness on 2-step math word problems by adding irrelevant information to the problem. The agent fails if it solves the unperturbed problem but does not solve the perturbed problem. | | Math Robustness: Numerical and Text Variations with Add Accents mutation | Tests model robustness on GSM math problems by rephrasing the text or varying the numbers. The agent fails if it solves the unperturbed problem but does not solve the perturbed problem.Adds Add Accents mutation. | | Math Robustness: Numerical and Text Variations with Tag mutation | Tests model robustness on GSM math problems by rephrasing the text or varying the numbers. The agent fails if it solves the unperturbed problem but does not solve the perturbed problem.Adds Tag mutation. |