Skip to content

create_automated_reasoning_policy_test_case

Operation

create_automated_reasoning_policy_test_case

Creates a test for an Automated Reasoning policy. Tests validate that your policy works as expected by providing sample inputs and expected outcomes. Use tests to verify policy behavior before deploying to production.

Parameters:

Name Type Description Default
request CreateAutomatedReasoningPolicyTestCaseInput | CreateAutomatedReasoningPolicyTestCaseInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[CreateAutomatedReasoningPolicyTestCaseInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
CreateAutomatedReasoningPolicyTestCaseOutput

An instance of CreateAutomatedReasoningPolicyTestCaseOutput.

create_automated_reasoning_policy_test_case async

Creates a test for an Automated Reasoning policy. Tests validate that your policy works as expected by providing sample inputs and expected outcomes. Use tests to verify policy behavior before deploying to production.

Parameters:

Name Type Description Default
request CreateAutomatedReasoningPolicyTestCaseInput | CreateAutomatedReasoningPolicyTestCaseInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[CreateAutomatedReasoningPolicyTestCaseInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
CreateAutomatedReasoningPolicyTestCaseOutput

An instance of CreateAutomatedReasoningPolicyTestCaseOutput.

Input

Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.

CreateAutomatedReasoningPolicyTestCaseInputDict

Bases: TypedDict

Dictionary form of CreateAutomatedReasoningPolicyTestCaseInput.

Attributes

client_request_token instance-attribute
client_request_token: str | None

A unique, case-sensitive identifier to ensure that the operation completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error.

This field is autopopulated if not provided.

confidence_threshold instance-attribute
confidence_threshold: float | None

The minimum confidence level for logic validation. Content that meets the threshold is considered a high-confidence finding that can be validated.

expected_aggregated_findings_result instance-attribute
expected_aggregated_findings_result: Required[str]

The expected result of the Automated Reasoning check. Valid values include: , TOO_COMPLEX, and NO_TRANSLATIONS.

  • VALID - The claims are true. The claims are implied by the premises and the Automated Reasoning policy. Given the Automated Reasoning policy and premises, it is not possible for these claims to be false. In other words, there are no alternative answers that are true that contradict the claims.

  • INVALID - The claims are false. The claims are not implied by the premises and Automated Reasoning policy. Furthermore, there exists different claims that are consistent with the premises and Automated Reasoning policy.

  • SATISFIABLE - The claims can be true or false. It depends on what assumptions are made for the claim to be implied from the premises and Automated Reasoning policy rules. In this situation, different assumptions can make input claims false and alternative claims true.

  • IMPOSSIBLE - Automated Reasoning can't make a statement about the claims. This can happen if the premises are logically incorrect, or if there is a conflict within the Automated Reasoning policy itself.

  • TRANSLATION_AMBIGUOUS - Detected an ambiguity in the translation meant it would be unsound to continue with validity checking. Additional context or follow-up questions might be needed to get translation to succeed.

  • TOO_COMPLEX - The input contains too much information for Automated Reasoning to process within its latency limits.

  • NO_TRANSLATIONS - Identifies that some or all of the input prompt wasn't translated into logic. This can happen if the input isn't relevant to the Automated Reasoning policy, or if the policy doesn't have variables to model relevant input. If Automated Reasoning can't translate anything, you get a single NO_TRANSLATIONS finding. You might also see a NO_TRANSLATIONS (along with other findings) if some part of the validation isn't translated.

guard_content instance-attribute
guard_content: Required[str]

The output content that's validated by the Automated Reasoning policy. This represents the foundation model response that will be checked for accuracy.

policy_arn instance-attribute
policy_arn: Required[str]

The Amazon Resource Name (ARN) of the Automated Reasoning policy for which to create the test.

query_content instance-attribute
query_content: str | None

The input query or prompt that generated the content. This provides context for the validation.

CreateAutomatedReasoningPolicyTestCaseInput dataclass

Bases: _SerdeShape

Dataclass for CreateAutomatedReasoningPolicyTestCaseInput structure.

Attributes

client_request_token class-attribute instance-attribute
client_request_token: str | None = ...

A unique, case-sensitive identifier to ensure that the operation completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error.

This field is autopopulated if not provided.

confidence_threshold class-attribute instance-attribute
confidence_threshold: float | None = ...

The minimum confidence level for logic validation. Content that meets the threshold is considered a high-confidence finding that can be validated.

expected_aggregated_findings_result class-attribute instance-attribute
expected_aggregated_findings_result: str | None = ...

The expected result of the Automated Reasoning check. Valid values include: , TOO_COMPLEX, and NO_TRANSLATIONS.

  • VALID - The claims are true. The claims are implied by the premises and the Automated Reasoning policy. Given the Automated Reasoning policy and premises, it is not possible for these claims to be false. In other words, there are no alternative answers that are true that contradict the claims.

  • INVALID - The claims are false. The claims are not implied by the premises and Automated Reasoning policy. Furthermore, there exists different claims that are consistent with the premises and Automated Reasoning policy.

  • SATISFIABLE - The claims can be true or false. It depends on what assumptions are made for the claim to be implied from the premises and Automated Reasoning policy rules. In this situation, different assumptions can make input claims false and alternative claims true.

  • IMPOSSIBLE - Automated Reasoning can't make a statement about the claims. This can happen if the premises are logically incorrect, or if there is a conflict within the Automated Reasoning policy itself.

  • TRANSLATION_AMBIGUOUS - Detected an ambiguity in the translation meant it would be unsound to continue with validity checking. Additional context or follow-up questions might be needed to get translation to succeed.

  • TOO_COMPLEX - The input contains too much information for Automated Reasoning to process within its latency limits.

  • NO_TRANSLATIONS - Identifies that some or all of the input prompt wasn't translated into logic. This can happen if the input isn't relevant to the Automated Reasoning policy, or if the policy doesn't have variables to model relevant input. If Automated Reasoning can't translate anything, you get a single NO_TRANSLATIONS finding. You might also see a NO_TRANSLATIONS (along with other findings) if some part of the validation isn't translated.

guard_content class-attribute instance-attribute
guard_content: str | None = ...

The output content that's validated by the Automated Reasoning policy. This represents the foundation model response that will be checked for accuracy.

policy_arn class-attribute instance-attribute
policy_arn: str | None = ...

The Amazon Resource Name (ARN) of the Automated Reasoning policy for which to create the test.

query_content class-attribute instance-attribute
query_content: str | None = ...

The input query or prompt that generated the content. This provides context for the validation.

Output

CreateAutomatedReasoningPolicyTestCaseOutput dataclass

Bases: _SerdeShape

Dataclass for CreateAutomatedReasoningPolicyTestCaseOutput structure.

Attributes

policy_arn instance-attribute
policy_arn: str

The Amazon Resource Name (ARN) of the policy for which the test was created.

response_metadata class-attribute instance-attribute
response_metadata: ResponseMetadata = ...

Metadata about the response that produced this output. Use this to recover the request identifiers a service's support team needs in order to investigate a call. Members of the metadata are individually optional.

test_case_id instance-attribute
test_case_id: str

The unique identifier of the created test.