Building Smarter AI Applications with Amazon Bedrock’s Data Retrieval

Overview

Businesses rely on structured data to manage operations effectively in today’s data-driven world. Amazon Bedrock Knowledge Bases now supports structured data retrieval, enabling natural language querying to access structured data from various sources. This enhancement allows developers to build custom generative AI applications seamlessly incorporating contextual information from structured and unstructured data. By leveraging natural language processing (NLP), Amazon Bedrock Knowledge Bases can transform user queries into SQL, executing them directly on supported databases without requiring data movement or preprocessing.

Introduction

Integrating structured data into generative AI applications has traditionally been challenging due to the complexities of training large language models (LLMs) to generate SQL queries based on intricate database schemas. Ensuring proper data governance and security adds another layer of difficulty. Amazon Bedrock Knowledge Bases addresses these challenges with a fully managed Natural Language to SQL (NL2SQL) module. Users can now retrieve structured data effortlessly by asking questions in plain language, and Amazon Bedrock Knowledge Bases will automatically generate and execute the corresponding SQL queries.

Currently, this feature supports Amazon Redshift and Amazon Sagemaker Lakehouse, offering a streamlined solution for structured data retrieval in all commercial regions where Amazon Bedrock Knowledge Bases are available.

Amazon Bedrock Knowledge Bases

Amazon Bedrock Knowledge Bases enables users to integrate with structured data stores such as Amazon Redshift and AWS Glue Data Catalog. It translates natural language queries into SQL queries, retrieves relevant data, and generates responses. Key functionalities include:

• Retrieve operation: Fetches data from the knowledge base.
• RetrieveAndGenerate operation: Generates responses based on the retrieved data.
• GenerateQuery operation: Converts user queries into SQL statements without retrieving data.

Businesses can streamline data access and improve decision-making processes by leveraging these capabilities.

Connecting a Knowledge Base to a Structured Data Store

To connect a knowledge base to a structured data store, you must specify the following components:

1. Data Store Selection: You can connect to Amazon Redshift or AWS Glue Data Catalog.
2. Query Engine: Amazon Bedrock currently supports Amazon Redshift for SQL query generation.
3. Authentication Methods: Various authentication methods ensure secure access:
   • AWS IAM Role: Uses an AWS IAM service role with necessary permissions.
   • Temporary Credentials: Authenticates via a database user.
   • AWS Secrets Manager: Utilizes stored credentials for secure authentication.
4. Query Configurations (Optional): Enhance SQL generation accuracy using:
   • Maximum query time limits
   • Metadata descriptions for tables and columns
   • Inclusion/exclusion lists to filter tables or columns
   • Curated queries with pre-defined SQL examples

Steps to Set Up a Knowledge Base

Using AWS Console

1. Sign in to the AWS Management Console.
2. Navigate to Knowledge Bases and select Create Knowledge Base with Structured Data Store.
3. Configure settings such as query engine, AWS IAM role, and authentication method.
4. Choose a data store, enter database details, and modify query configurations if needed.
5. Review and confirm settings to create the knowledge base.

Using Amazon Bedrock API

Send a CreateKnowledgeBase request with the following JSON body:

Syncing a Structured Data Store with Amazon Bedrock

You must sync the knowledge base to ingest metadata once the knowledge base is connected. This allows Amazon Bedrock to process user queries efficiently. You should also re-sync whenever database schema changes occur.

Syncing via AWS Console

1. Open the Amazon Bedrock console.
2. Navigate to Knowledge Bases and select your knowledge base.
3. Under Data Source, click Sync to begin metadata ingestion.
4. Check sync status and view logs for any warnings or errors.

Syncing via API

Use the SyncKnowledgeBase API request to initiate synchronization.

Structured Data Retrieval (SQL Generation) Pricing

Structured Data Retrieval incurs charges per request for generating SQL queries. These queries are used to fetch data from structured databases.

Region: Asia Pacific (Mumbai)

• Feature: Structured Data Retrieval (SQL Generation)
• Pricing: $2.00 per 1,000 queries

Conclusion

Amazon Bedrock Knowledge Bases streamlines the querying of structured data, improving accessibility and efficiency.

Drop a query if you have any questions regarding Amazon Bedrock Knowledge Bases and we will get back to you quickly.

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, AWS GenAI Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner,  AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, Amazon ECS Service Delivery Partner, AWS Glue Service Delivery Partner, Amazon Redshift Service Delivery Partner, AWS Control Tower Service Delivery Partner, AWS WAF Service Delivery Partner, Amazon CloudFront, Amazon OpenSearch, AWS DMS and many more.