Listing Your Entitled Models in Amazon Bedrock

I have created an assistant application called Quippy. Quippy uses foundational models Bedrock provides to have a nice chat. The provision of foundational models varies between the different AWS regions. As a user, you can choose a region and select a model from available models in that region. Before using a model, you must enable the model in your profile. At least until January 2025, there has been no API to list only the models you are entitled to.

In this blog post, you learn about the problem of listing only the models you are entitled to and the solution to the problem. You’ll see two scenarios. The first is working in your local Python environment. The second is running your Python code in a Lambda.

The problem

I want to select the model to use in Quippy when chatting with him. The screenshot below shows the model selection. Look for models from Meta, and note that they are available.

Next, you’ll be able to see an overview of the models I have been given access to. Note that the two Meta models Llama 3.2 are not selected.

With a few lines of Python code, you can use the boto3 library to interact with AWS and list the models. This code is used for the dropdown in Quippy’s screenshot.

client = boto3.client('bedrock', region_name='eu-central-1')
response = client.list_foundation_models(
  byOutputModality='TEXT'
)

The code shows the API call list_foundation_models. You can include filters, for instance, to filter output modality to be TEXT, so the IMAGE models are omitted. At least until January 2025, there has been no filter to list only the models you have been granted access to. There is a GitHub issue that explains the problem and gives a workaround. Funny enough, the workaround mentioned in the GitHub issue is also used by the screen from AWS itself.

The solution

As the GitHub issue describes, the solution to show only the models you have been granted access to is a call per model to a different URL. To be able to call that URL and be sure it returns information about you having access, you need to be authenticated. When calling an API, using the SDK (boto3) is better. Boto3 handles all the signing and encoding to call a backend service.

The URL needed to find the entitlement to use a model is unavailable through the API. Therefore, we need another manual mechanism. Some libraries are available, but they, in general, are old, look abandoned, and do not work in this scenario. I based my solution on code made available by the AWS team. You can find the entire class here. I had to alter one small thing in the encoding for Bedrock. I turned the code into a class, and now we are good to go to

The following two sections show you how to use the boto3 library to make it work.

The code for your local Python application

First, you can look at the class with the signing code discussed in the previous section on GitHub.

https://github.com/jettro/quippy/blob/main/lambda/util.py

I assume you set up the AWS command line interface and have your connection parameters set up. For my employer’s organisation, we use SSO to log in to AWS. Therefore, I need the profile to use. In my case, the default profile will do. Note in the code below how we obtain the session using boto3. From the session, we obtained the bedrock client and the credentials.

if __name__ == '__main__':
    region = 'eu-central-1'

    # Initialize Bedrock client with SSO profile
    session = boto3.Session(profile_name='default')
    cred = session.get_credentials().get_frozen_credentials()

    client = session.client('bedrock', region_name=region)
    response = client.list_foundation_models(byOutputModality='TEXT')

    models = response['modelSummaries']
    entitlement_access = module.AWS4Access(bedrock_region=region, credentials=cred)
    for model in models:
        entitlement = entitlement_access.get_entitlement(model_id=model['modelId'])
        print(f"Entitlement for model {model['modelId']}: {entitlement}")

We first obtain a list of foundation models that output text in the code. Next, for each model, we call the entitlement function of the class to ask if I am entitled to use the model with my account. Below is the output from the program. Note that I am not entitled to use the meta llama3 models, just like in the screenshot from the AWS console.

Entitlement for model amazon.titan-text-express-v1:0:8k: True
Entitlement for model amazon.titan-text-express-v1: True
Entitlement for model amazon.titan-text-lite-v1:0:4k: True
Entitlement for model amazon.titan-text-lite-v1: True
Entitlement for model amazon.rerank-v1:0: False
Entitlement for model anthropic.claude-instant-v1: True
Entitlement for model anthropic.claude-v2:1:18k: True
Entitlement for model anthropic.claude-v2:1:200k: True
Entitlement for model anthropic.claude-v2:1: True
Entitlement for model anthropic.claude-v2: True
Entitlement for model anthropic.claude-3-sonnet-20240229-v1:0: True
Entitlement for model anthropic.claude-3-haiku-20240307-v1:0: True
Entitlement for model anthropic.claude-3-5-sonnet-20240620-v1:0: True
Entitlement for model cohere.rerank-v3-5:0: False
Entitlement for model meta.llama3-2-1b-instruct-v1:0: False
Entitlement for model meta.llama3-2-3b-instruct-v1:0: False

In the next step, you can see the same functionality running in the Lambda to show a dropdown with only the models we are entitled to.

The code for your Lambda

With the class available to connect to the URL to ask for the entitlement of a model, we can focus on the lambda. The code is almost identical for local execution; obtaining the session is easier. No profiles are needed. Below is the code for the Lambda. It shows how we get the region and the output modality and only return the models to which we are entitled.

# The body of the POST request will be in event['body']
body = json.loads(event.get('body', '{}'))
logger.info(f"The body of the request: {body}")
region = body.get('region', DEFAULT_REGION)
output_modality = body.get('outputModality', DEFAULT_MODALITY)  # TEXT, IMAGE

# Initialize Bedrock client
client = boto3.client('bedrock', region_name=region)

response = client.list_foundation_models(byOutputModality=output_modality)
cred = boto3.Session().get_credentials().get_frozen_credentials()
entitlement_access = AWS4Access(bedrock_region=region, credentials=cred)
models = []
for model in response['modelSummaries']:
    entitlement = entitlement_access.get_entitlement(model_id=model['modelId'])
    if entitlement:
        models.append({
            "id": model['modelId'],
            "name": model['modelName'],
            "provider": model['providerName'],
            "output_modalities": model['outputModalities'],
            "entitlement": entitlement
        })

# Return the response from Bedrock
return {
    'statusCode': 200,
    'headers': {
        'Access-Control-Allow-Origin': '*',
        'Access-Control-Allow-Methods': 'GET, POST, OPTIONS',
        'Access-Control-Allow-Headers': 'Content-Type, Authorization'
    },
    'body': json.dumps({
        'models': models
    })
}

You do have to allow the Lambda to perform the action ‘bedrock:GetFoundationModelAvailability’. This action is added to the policy in the construct. Below is the screenshot showing the dropdown with models; note that the meta llama models are not visible anymore.

That is it. If you have questions or comments, don’t hesitate to contact me. I am easy to find on Linkedin.