The Bedrock Knowledge Base project is a comprehensive Terraform-based solution for setting up and managing an AWS Bedrock agent knowledge base. This project integrates various AWS services and external tools to create a powerful, scalable, and efficient knowledge management system.
- AWS Bedrock Knowledge Base integration
- Pinecone vector database for efficient data storage and retrieval
- S3 bucket for document storage
- Lambda functions for document ingestion and request processing
- IAM role and policy management
- SNS topic for notifications
- GitHub integration for CI/CD
- ECR for Docker image management
- ECS for containerized application deployment
- API Gateway for RESTful API access
- Secrets Manager for secure credential storage
- SSM Parameter Store for configuration management
- Terraform v1.0+
- AWS CLI configured with appropriate permissions
- Pinecone API key
- GitHub token (for CI/CD setup)
- Anthropic API key (for AI model access)
s3
: Manages S3 buckets for document storagesecrets_manager
: Handles secure storage of API keys and credentialsiam
: Configures IAM roles and policiespinecone_vectorDB
: Sets up Pinecone vector databasebedrock_knowledge_base
: Configures AWS Bedrock Knowledge Baselambda
: Manages Lambda functions for document ingestion and processingsns
: Sets up SNS topics for notificationsgithub
: Configures GitHub repositories and CI/CDecr
: Manages ECR repositories for Docker imagesssm_parameter_store
: Stores configuration parametersecs
: Sets up ECS cluster and servicesapi_gateway
: Configures API Gateway for RESTful access
- Clone this repository
- Set up your
terraform.tfvars
file with the required variables - Initialize Terraform:
terraform init
- Plan the deployment:
terraform plan
- Apply the configuration:
terraform apply
Key variables to configure in your terraform.tfvars
:
region
: AWS region for deploymentpinecone_api_key
: Your Pinecone API keypinecone_environment
: Pinecone environmentgithub_token
: GitHub personal access tokengithub_repo_owner
: GitHub repository ownergithub_repositories
: List of GitHub repositories to integrateanthropic_api_key
: Anthropic API key for AI model accessdefault_email_address
: Email for SNS notificationsaws_environment
: Environment tag (e.g., "prod", "dev")
After deployment, you can:
- Upload documents to the created S3 bucket
- Use the API Gateway to query the knowledge base
- Use the web application deployed as ECS container for interacting with the knowledge base: UNDER DEVELOPMENT
- Check SNS topics for important notifications
This project uses AWS Secrets Manager and IAM roles to manage secrets and permissions securely. Ensure that you follow AWS best practices for managing access to your resources.
Contributions to this project are welcome. Please fork the repository and submit a pull request with your changes.
This project is licensed under the MIT License - see the LICENSE file for details.