Technical Architecture
Runs on YOUR AWS Account
Flustack deploys entirely within your own AWS infrastructure. Your data never crosses account boundaries — full control, zero vendor lock-in on data residency, enterprise security from day one.
System Architecture
End-to-End Data Pipeline
Eight integrated layers working together — from raw source data to business-ready insights, all orchestrated within a single AWS account.
Data Quality Architecture
Medallion Architecture
A five-layer data lake guarantees every record is validated, enriched, and analytics-ready before reaching your dashboards. Bad data is isolated and traceable — never silently discarded.
Raw data as-is from every source. Immutable, append-only.
Deduplicated, parsed, schema-normalized. Glue jobs apply initial transformations.
Validated, enriched, anomalies flagged. SageMaker RCF runs here.
Analytics-ready, aggregated, domain-modeled. Athena and QuickSight consume from here.
Isolated bad records with full lineage. Reprocessable after correction.
Quarantine sits outside the main flow — bad records are routed here automatically and reprocessed after correction.
Full AWS Stack
Every Service, Every Layer
Flustack is an opinionated architecture over 30+ AWS services — the right service for each job, connected and configured out of the box.
| Layer | Service | Role |
|---|---|---|
| Ingestion | S3 | Landing, Bronze, Silver, Gold, Quarantine zones |
| AppFlow | SaaS connectors (Salesforce, ServiceNow, Slack) | |
| Kinesis Data Streams | Real-time streaming source | |
| MSK (Kafka) | Alternative streaming source | |
| Secrets Manager | JDBC credentials, API keys, tenant API keys | |
| Compute | AWS Glue 4.0 | ETL PySpark jobs — batch and streaming |
| EMR Serverless | High-volume jobs and complex joins | |
| AWS Lambda | Triggers, connectors, API handler, GC | |
| Orchestration | Step Functions | Main ingestion flow per source |
| EventBridge | Scheduled triggers per tenant per source | |
| EventBridge Scheduler | Cron schedules per tenant | |
| SNS | Operational alerts (KMS-encrypted) | |
| AI / ML | Amazon Bedrock | Schema inference, semantic mapping, error analysis |
| Amazon SageMaker | RCF anomaly detection, risk scoring, batch inference | |
| Storage | Glue Data Catalog | Iceberg table metadata |
| Apache Iceberg | Format for Silver and Gold layers | |
| Governance | KMS | Customer Managed Key per tenant |
| Lake Formation | Column and cell-level access control | |
| IAM | Least-privilege roles per service | |
| Consumption | Amazon Athena | SQL queries on Iceberg tables |
| QuickSight | BI dashboards | |
| QuickSight Q | NLQ in Spanish and English (optional) | |
| Control Plane | DynamoDB | Control plane Single Table Design — source config + job state |
| Observability | CloudWatch Logs | Structured JSON logs (13 log groups) |
| CloudWatch Metrics | Business KPIs | |
| CloudWatch Insights | Predefined operational queries | |
| API Layer | API Gateway | REST API exposure |
| FastAPI + Mangum | API handler (Lambda) | |
| CI / CD | CodePipeline | CI/CD pipeline per tenant |
| CodeBuild | CDK synth + test execution | |
| Local Dev | LocalStack Pro | Local simulation of all AWS services |
| cdklocal | CDK deploy against LocalStack |
Why it matters
Three Principles That Define Flustack
Your Infrastructure
Every byte of data stays in your AWS account. No shared compute, no shared storage, no data egress to a vendor cloud. Flustack deploys via CDK into your environment and you own everything from day one.
AI-Native by Design
AI is not a bolt-on feature. Amazon Bedrock drives schema inference and semantic mapping at ingest time. SageMaker RCF runs anomaly detection at the Silver layer. Intelligence is baked into the pipeline, not added as an afterthought.
Multi-Tenant by Design
Native tenant isolation from the data model up — KMS Customer Managed Key per tenant, dedicated EventBridge schedules, isolated pipelines. Onboard a new client in minutes without touching existing tenant infrastructure.
Ready to see it in your environment?
Schedule a technical walkthrough with our engineers and see how Flustack maps to your existing AWS setup.