← Back to overview

Platform Engineering & AI

Platforms I designed and built end to end, using the same architecture patterns as the federal and defense programs I work on.

Kafka · Kubernetes · CDC Infrastructure as Code

Streaming Data Platform, Built End to End

A production-shaped data platform designed and built from the hypervisor up: virtualisation, infrastructure as code, Kubernetes, a three-broker Kafka cluster, schema registry, change data capture from PostgreSQL, object storage, and automated TLS. Every change goes through a merge request and a pipeline. Nothing was clicked into place by hand.

What Was Built
Kafka on Kubernetes
Three brokers via the Strimzi operator in KRaft mode, one per node, replication factor 3 with min.insync.replicas 2. Pod anti-affinity enforces the spread, so losing a node costs one replica rather than two.
Change Data Capture
Debezium reading the PostgreSQL write-ahead log and publishing row-level change events. Many consumers read the same stream; the source database sees one reader and no added query load.
Schema Contracts
Avro schemas in a registry with compatibility enforcement. A breaking change is rejected when it is deployed rather than discovered by a downstream consumer at runtime.
Infrastructure as Code
Terraform provisioning virtual machines against the hypervisor API with remote state and locking, Ansible for configuration, GitLab CI running plan and apply behind a protected branch and a manual gate.
Automated TLS
cert-manager issuing a wildcard certificate by DNS-01, delegated to a separate domain so the token cannot reach production DNS. A scheduled role distributes renewals to services that terminate TLS outside the cluster.
Storage & Data Layer
PostgreSQL 18 with logical replication and pgvector, MinIO for S3-compatible object storage, MetalLB and ingress-nginx for load balancing and ingress on bare metal.
Why It Is Built This Way

Every architectural decision is recorded with the alternative that was rejected and the reason: for_each over count for stable resource addressing, monitoring deliberately outside the cluster it observes, one PostgreSQL instance rather than Patroni, and why the CI runner cannot safely restart the virtual machine that hosts it. Runbooks are written for the person on call at three in the morning, not for the person who built it.

Apache KafkaStrimziKRaftDebeziumSchema RegistryAvroKubernetesk3sHelmTerraformAnsibleGitLab CIPostgreSQLpgvectorMinIOcert-managerMetalLBProxmox
AWS · AI/ML · Personal Project AWS · GenAI · RAG

GenAI Data Assistant, AWS Cloud AI Architecture

Designed and built a full AWS GenAI data assistant architecture from the ground up, integrating AI, voice control, document intelligence, and automated CI/CD into a single cohesive platform. Built using production-grade architectural patterns, AWS services, and data pipelines aligned with real-world federal and enterprise data operations.

What Was Built
AI-Powered Data Assistant Bot
30+ command Telegram bot (1,200+ lines Python) running as a 24/7 EC2 service, integrated with Amazon Bedrock Claude for intelligent data queries, document processing, and inventory management.
RAG Architecture & Vector Search
Bedrock Knowledge Base with vector embeddings over S3 documents, enabling grounded AI responses against private data without hallucination.
Document Intelligence Pipeline
Amazon Textract OCR pipeline converting physical forms and documents to structured data → PostgreSQL. Amazon Rekognition for image analysis and asset visual inspection with confidence scoring.
Predictive Analytics & ML Modeling
SageMaker Jupyter notebooks for data quality dashboards, stock risk classification models using pandas and scikit-learn, and operational analytics over live inventory and infrastructure data.
Bedrock Agents & Autonomous AI
Configured Bedrock Agents with Knowledge Base integration, autonomous AI capable of multi-step reasoning over structured operational data without human orchestration.
Voice Interface, Alexa Skill + Lambda
Custom Alexa Skill backed by AWS Lambda, voice-driven queries for live EC2 status, RDS health, and AWS cost data. Claude AI (Anthropic API) handles open-ended questions via natural language. Smart query router dispatches to boto3 or Claude based on intent.
CI/CD Pipeline, Push to Deploy
GitHub → CodePipeline → CodeBuild → CodeDeploy → EC2 auto-deploy in ~90 seconds. Full production-grade deployment automation with syntax validation, IAM roles, security groups, and environment management.
Technology Stack
Amazon Bedrock Claude Haiku 4.5 RAG / Vector Search Bedrock Agents Amazon Textract Amazon Rekognition RDS PostgreSQL Amazon S3 EC2 · Linux SageMaker CodePipeline · CodeDeploy Python · boto3 IAM · Security Groups GitHub CI/CD pandas · scikit-learn Alexa Skills Kit AWS Lambda Anthropic API · Claude CodeBuild
Architecture

Two views of the same system. The client reference shows the shape of the solution. The engineering view adds the delivery chain, identity boundaries, voice channel and analytics path. Click either diagram to enlarge.

GenAI Data Assistant, client reference architecture on AWS
Client Reference ArchitectureTelegram Bot API into EC2 over HTTPS, cache-aside on ElastiCache for Redis with RDS for PostgreSQL behind it, Bedrock for inference, Textract and Rekognition for document and image intelligence, and Bedrock Knowledge Bases over S3 with OpenSearch Serverless for vector retrieval. Only EC2, ElastiCache and RDS sit inside the VPC. Everything else is a managed service reached by API.

Code & Repositories

Project source is kept in private repositories. Access is granted on request to hiring teams, prime contractors, and clients.

github.com
Private repositories · access granted on request
Request Access
Available on Request
genai-data-assistant
Full GenAI platform source. Roughly 1,200 lines of Python across the Bedrock integration, RAG retrieval layer, Textract pipeline, and the conversational interface.
Infrastructure as Code
Terraform modules covering RDS, EC2, IAM, S3, and networking, alongside the CodePipeline and CodeDeploy configuration that takes a commit to a running instance in under two minutes.
Automation & Tooling
Python and PowerShell used for database provisioning, backup and restore validation, and operational reporting. Sanitized versions available where the original work is client-owned.

Nothing here originates from a client or government environment. Work performed under federal and defense programs stays inside those boundaries and is described only at an architectural level.