Skip to content

INFERENCE_PROFILE.md

Latest commit

 

History

History
87 lines (62 loc) · 2.72 KB

INFERENCE_PROFILE.md

File metadata and controls

87 lines (62 loc) · 2.72 KB

Application Inference Profile

AWS Bedrock allows you to copy specific models and inference profiles as user-managed inference profiles (hereafter referred to as "Application Inference Profiles") with custom tagging. These Application Inference Profiles enable detailed cost tracking and allocation for foundation model execution.

📋 Prerequisites

AWS CLI Environment

  • AWS CLI version v2.18.17 or higher is required
  • AWS credentials must be properly configured

Required IAM Permissions

To create and manage Application Inference Profiles, the following IAM permissions are required:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:CreateInferenceProfile",
        "bedrock:GetInferenceProfile",
        "bedrock:ListInferenceProfiles",
        "bedrock:DeleteInferenceProfile",
        "bedrock:TagResource",
        "bedrock:UntagResource",
        "bedrock:ListTagsForResource"
      ],
      "Resource": "*"
    }
  ]
}

🚀 Creating Application Inference Profiles

Basic Creation Command

The copyFrom key value should contain the ARN of a system-defined inference profile or base model.

aws bedrock create-inference-profile --region 'ap-northeast-1' \
  --inference-profile-name 'custom-bedrock-profile' \
  --description 'custom-bedrock-profile' \
  --model-source '{"copyFrom": "arn:aws:bedrock:ap-northeast-1::foundation-model/anthropic.claude-3-5-sonnet-20240620-v1:0"}' \
  --tags '[{"key": "CostAllocateTag","value": "custom"}]'

Verifying Creation Status

To check Application Inference Profiles, filter by inference profile type APPLICATION:

aws bedrock list-inference-profiles --region 'ap-northeast-1' \
  --type-equals 'APPLICATION'

Get detailed information for a specific profile:

aws bedrock get-inference-profile --region 'ap-northeast-1' \
  --inference-profile-identifier 'custom-bedrock-profile'

🖥️ Using Bedrock Engineer

Enabling in Settings

  1. Open Settings Screen

    • Select "Settings" from the menu

  2. Enable in AWS Settings Section

    • Check the "Enable Inference Profiles" checkbox
    • Settings are automatically saved

🔗 References

This documentation enables effective use of Application Inference Profiles to manage Bedrock costs and perform detailed analysis by project and department.