Christian Richter

Ingenieurbüro Christian Richter

Freelance Data & AI Engineer • Jan 2010 – Present

• Independent consultancy for cloud data platforms, ETL and streaming, Kubernetes and infrastructure as code
• 25+ client engagements; see Selected Projects below

Shibui Finance

Founder • Sep 2025 – Present

• Built and operate a public MCP server (mcp.shibui.finance) that makes 60+ years of US stock market data queryable in plain English from Claude; listed in the official MCP Registry
• Own data platform (dbt, PostgreSQL, DuckDB) with governed LLM access (read-only SQL, EXPLAIN validation, rate limiting)
• AI-driven software factory with Claude Code agents (plan → implement → review)

AltusInsight GmbH

Founder & Managing Director • Mar 2013 – Jun 2021

• Founded and ran the company for eight years; responsible for product design and implementation
• Built LambdaNow, a cloud-based platform for fully automated deployment of Apache Hadoop clusters on AWS and OpenStack

MOG Inc.

Backend Software Engineer & Systems Architect (contract) • May 2005 – Oct 2009

• First engineer at this music and media startup; responsible for backend software, databases and infrastructure
• Designed, built and launched the production hosting environment
• Developed C++ backend systems, including a distributed collaborative filtering engine, a search spellchecker and a high-performance XML-RPC server

Fraunhofer IDMT

Research Assistant • Aug 2000 – Mar 2005

• Researched nearest-neighbor and similarity search algorithms for music information retrieval (query by humming)
• Designed and implemented algorithms in C++ and MATLAB; built test environments and trade-show demo applications

ID Analytics Inc.

Data Analyst Intern • Jan 2003 – Jun 2003

• Analyzed ~250 GB of data for feature extraction and developed regression-based algorithms to detect identity theft

Cloud-native data platform on Kubernetes

Data Architect • Feb 2025 – Present

• Create architecture diagrams and documentation (backup/restore, parameterization/validation)
• Define and implement standards for Kubernetes-native services
• Document architecture decisions as ADRs
• Design and document standardized ETL pipelines
• Evaluate high-availability deployment scenarios for PostgreSQL and Kafka brokers against system requirements
• Design disaster-recovery processes for data infrastructure components

Industry: Energy utility

Technologies: Kubernetes, Terraform, Helm, Strimzi, Kafka, PostgreSQL Operator, MongoDB, Argo CD, GitLab, Apicurio, Velero, GitHub Copilot

AI-assisted data extraction and ETL for an SCM platform

Data Engineer • Oct 2024 – May 2026

• Helped build the master data set for a supply chain management application from 8 data sources
• Designed and implemented ~60 ETL pipelines on Google Cloud
• Designed and implemented LLM-based workflows for document data extraction and analysis (Document AI, Vertex AI), processing ~120k documents
• Analyzed and combined the data sources to improve data quality and coverage
• Built a dashboard to monitor data quality
• Set up CI/CD pipelines, monitoring and alerting for production ETL processes

Industry: Supply chain management

Technologies: Google Cloud, GKE, Document AI, Vertex AI, Kubernetes, Terraform, Kafka, Kafka Connect, Kotlin, Pekko, dbt, SQL, GitHub, Claude Code

Data warehouse for research and production data

Data Architect & Engineer • Sep 2023 – Oct 2024

• Advised on the data model, ETL pipelines and cloud architecture
• Designed and implemented multiple ETL pipelines providing access to research and production data
• Integrated SAP PLM to extract and process laboratory research data
• Integrated the manufacturing execution system (MES) for data extraction and processing
• Built an OpenSearch index for semantic search (vector search / k-NN)
• Transferred knowledge to internal staff

Industry: Chemicals

Technologies: AWS, CloudFormation, SAP PLM, SAP MES, OpenSearch, Docker, Java, Spring, Flyway, SQL

Migration of an on-premises data warehouse to AWS

Data Architect & Engineer • Jul 2021 – Oct 2023

• Evaluated Databricks/Delta Lake against AWS managed services/Hudi as the foundation of a data lake
• Designed and built the data warehouse on AWS Lake Formation and the Glue Data Catalog (~1.5 TB data lake)
• Migrated 100+ ETL jobs (Spark, Hive, Pig) from Cloudera to AWS Glue and Amazon EMR, with shorter runtimes through elastic scaling
• Replaced Oozie with Airflow for workflow orchestration
• Designed and implemented several GitLab CI/CD pipelines
• Trained internal staff and transferred knowledge

Industry: Market research

Technologies: AWS (Glue, EMR, Lake Formation), Airflow, Databricks, Terraform, Python, Java, SQL, Cloudera, Hadoop, Hive, Pig, Oozie, GitLab

Cloud-based data warehouse for user data analytics

Data Architect & Lead Engineer • Jun 2020 – Dec 2022

• Designed and built the cloud infrastructure for the data warehouse
• Introduced a decentralized data mesh architecture, including governance guidelines
• Documented architecture decisions as ADRs and presented them to the architecture board
• Integrated Airflow as the workflow engine and the Spark on Kubernetes operator as the ETL runtime
• Built and led a 6-person team developing ETL processes for 4 data sources; implemented several ETL jobs in PySpark
• Implemented an event streaming pipeline for real-time analytics, including a real-time Braze integration with under 1 second latency
• Prepared data for downstream analysis in Power BI and advised on the overall data architecture

Industry: Media

Technologies: AWS, Kinesis, Kubernetes, Istio, Docker, Kafka, Kafka Streams, Braze, Airflow, Python, PySpark, Pandas, Avro, Terraform, Kustomize, Power BI

Sensor data processing platform and extension of a data science environment

Cloud & Data Engineer • Aug 2019 – Dec 2019

• Advised on the design and tooling of a Kubernetes-based infrastructure for sensor data processing
• Designed and built the infrastructure on Kubernetes (Kafka cluster, Spark, ZooKeeper)
• Designed and built the ETL pipeline for data ingestion
• Designed and built a CI/CD pipeline with Bamboo and Kubernetes

Industry: Manufacturing

Technologies: AWS, Kubernetes, Helm, CloudFormation, Kafka, Kafka Connect, PySpark, InfluxDB, Java, Python, Docker, Bamboo

Cloud-based data warehouse for vehicle data; evaluation of a data science environment

Data Engineer • Jan 2019 – Mar 2020

• Extended a prototype for large-scale data processing and brought it into production, improving runtime by a factor of 10
• Built and launched a CI/CD pipeline; set up project structure and release management
• Designed, implemented and deployed a backend API, including a Helm chart
• Built ETL pipelines for vehicle data validation, ingestion and indexing of millions of documents in OpenSearch
• Implemented feature extraction from vehicle data
• Evaluated Kubeflow and Databricks/Delta Lake

Industry: Automotive

Technologies: AWS (Lambda, IAM), Kubernetes, Kubeflow, Docker, Helm, Databricks, Terraform, Python, OpenSearch, Logstash, Kibana

Cloud-based data warehouse and data science environment

Data Architect & Engineer • Oct 2017 – Nov 2018

• Designed a dynamically scalable data warehouse and its data model and storage layout
• Implemented the infrastructure as code with CloudFormation and infrastructure components on Kubernetes
• Built ETL pipelines for data ingestion

Industry: Manufacturing

Technologies: AWS, CloudFormation, Kubernetes, Docker, Spark, R, Apache NiFi, Python, Jupyter

Design and implementation of a microservice architecture

Data Architect • May 2017 – Dec 2018

• Aligned and coordinated several teams on technology choices
• Co-designed core elements: (de)serializers, data pipeline design, error and message handling, database design
• Introduced Kafka as the central message bus for microservices
• Introduced Liquibase for database schema management
• Provided functional and technical ownership of one microservice

Industry: Financial software

Technologies: Java, Kafka, Docker, Liquibase, microservices

Evaluation of big data vendors

Requirements Engineer • Mar 2017

• Gathered and documented technical and business requirements for building and operating a Hadoop-based data warehouse
• Requested vendor proposals and prepared the information for decision-making
• Implemented a data ingestion prototype

Industry: Energy

Technologies: Hortonworks, Cloudera, SAP Cloud, Apache NiFi, Parquet, AWS, Microsoft Azure

Big data warehouse on AWS

Lead Data Engineer • Jan 2017 – Aug 2017

• Led the project technically and designed the AWS cloud infrastructure
• Implemented data pipelines, the data warehouse and workflow management
• Handled data preparation and process management

Industry: Market research

Technologies: AWS, Spark, SparkR, Cloudera, Hadoop, Hive, Avro, Python, R, Jupyter, Bamboo, Terraform

Workshop on big data technologies

Data Engineer (Trainer) • Oct 2016

• Delivered a three-day workshop introducing the big data/Hadoop ecosystem, with hands-on exercises on AWS

Industry: Healthcare

Technologies: Hadoop, Spark, MapReduce, Hive, Pig, ORC, Avro, R, AWS

Architecture review and real-time streaming layer

Data Engineer • Jun 2016 – Dec 2016

• Reviewed the existing architecture and data model; ran a workshop on data management and Lambda architecture
• Designed and implemented a real-time layer with Spark Streaming and integrated it with the batch layer

Industry: Entertainment technology

Technologies: Hadoop, Spark, Scala, Protocol Buffers, MapReduce, JCascalog, Redshift, AWS, CloudFormation

ETL pipelines for a Hadoop-based data warehouse

Data Engineer • Feb 2016 – Jul 2016

• Built a Hive export module and Kafka and Redis export modules for the open-source project Schedoscope
• Implemented an algorithm for clickstream analysis

Industry: E-commerce

Technologies: Hadoop, Hive, HCatalog, Spark, Kafka, Redis, Avro, Scala, Schedoscope

Continuous delivery pipeline for a data-driven application on AWS

DevOps Engineer • Dec 2015 – Aug 2016

• Planned and built a big data infrastructure on AWS and a continuous deployment pipeline
• Led the client's internal team technically

Industry: Market research

Technologies: AWS, Hadoop, Spark, Terraform, Vagrant, Bamboo, Git, InfluxDB

Big data warehouse for OLAP workloads

Data Engineer • Jul 2015 – Oct 2015

• Planned and built the cluster infrastructure, benchmarked input formats for performance, and prepared and ran load tests

Industry: Telecommunications

Technologies: Hadoop, Impala, Hive, AWS

Batch and real-time processing of machine data

Lead Data Engineer • Jul 2014 – Jun 2015

• Implemented a distributed, fault-tolerant, high-throughput messaging and analytics system for machine data (Lambda architecture)
• Planned the deployment environment, evaluated data ingestion and processing technologies, and led the team technically

Industry: Media technology

Technologies: Hadoop, Samza, Spark, Kafka, Java, AWS, CloudFormation

Hadoop-based data warehouse for game analytics

Lead Data Engineer • Mar 2013 – Sep 2014

• Planned and built the data warehouse, including a distributed, fault-tolerant, high-throughput messaging system
• Led and coordinated a distributed team across Germany, China and Canada

Industry: Gaming

Technologies: Hadoop, MapReduce, Kafka, Hive, Java, Linux

Big data infrastructure in virtualized environments

DevOps Engineer • Feb 2013 – Jun 2014

• Built a deployment process for on-demand Hadoop clusters on OpenStack
• Prototyped several algorithms in MapReduce

Industry: Telecommunications

Technologies: Hadoop, OpenStack, Chef, Java, Linux

Big data architecture for telecommunications data

Cloud & Data Engineer • Nov 2012 – Aug 2015

• Planned and built the network (AWS VPC) and a 100 TB Hadoop cluster, including deployment and monitoring
• Implemented a data ingestion framework storing ~300 GB of data per day

Industry: Market research

Technologies: Cloudera, Hadoop, Hive, Flume, Java, Spring, Puppet, Ubuntu, AWS

Design and rollout of a Hadoop cluster

Data Engineer • May 2012 – Dec 2012

• Advised on the cluster design, selected the hardware, set up the deployment process and rolled out the cluster
• Ported existing statistics routines to MapReduce

Industry: Marketing analytics

Technologies: Cloudera, Hadoop, Hive, Pig, Python, Java, Maven, Puppet, Debian

Re-implementation of an analytics tool as a MapReduce application

Data Engineer • Jun 2011 – Mar 2012

• Ported an existing implementation to MapReduce using the Hadoop Streaming API
• Installed and configured a Hadoop cluster with monitoring and set up the deployment process

Industry: Web analytics

Technologies: Cloudera, Hadoop, HBase, Java, Maven, Ganglia, Chef, PHP, Debian

Payment provider integration

Backend Developer • Sep 2010 – Feb 2011

• Integrated a payment provider into the existing backend
• Mapped, converted and imported data, including text matching against the existing database

Industry: Music streaming

Technologies: Ruby on Rails, OAuth, MySQL, Git, Debian

Signature component for an email client

Backend Developer • May 2010 – Sep 2010

• Integrated a digital signature component into KMail, including debug environment setup and testing

Industry: Security software

Technologies: C++, Qt, KDE, Ubuntu

Core Stack

Python, SQL, Spark, Kafka, Airflow, dbt, Terraform, Kubernetes, AWS, GCP, MCP

Data Engineering

Spark/PySpark, Databricks, Delta Lake, Hudi, dbt, Airflow, AWS Glue, Apache NiFi, Pandas, Polars, NumPy, scikit-learn, Power BI

Streaming

Kafka, Kafka Connect, Kafka Streams, Strimzi, Amazon Kinesis, Apache Pekko

Databases & Formats

PostgreSQL, OpenSearch, MongoDB, DynamoDB, DuckDB, TimescaleDB, MySQL/MariaDB, Oracle, Hive, Impala, HBase, Cassandra; Parquet, Avro, ORC

Cloud

AWS (S3, EMR, Glue, Athena, Lake Formation, Lambda, Kinesis, RDS, DynamoDB, ECS/ECR, IAM, CloudFormation, CDK), Google Cloud (GKE, Vertex AI, Document AI), OpenStack

Containers

Kubernetes, OpenShift, Docker, Helm, Kustomize, Argo CD, cert-manager, Prometheus, Grafana, Kubeflow

IaC & DevOps

Terraform, Terragrunt, CDKTF, Ansible, GitOps, GitLab CI, GitHub, Jenkins, Bamboo, Open Policy Agent, Puppet, Chef

AI & LLM

Vertex AI, Document AI, Model Context Protocol (MCP), llama.cpp, embeddings and vector search, AI-driven workflows, Claude Code, GitHub Copilot

Programming

Python, Java, Kotlin, SQL, Scala, Bash, Ruby, C++; Spring Boot, Flask, Liquibase, Flyway, Maven, Gradle

Johannes Kunze - Senior Solution Architect@GfK

"Christian's expertise with distributed and AWS based infrastructures helped us big time when we needed a solution to orchestrate our servers and services, by integrating Terraform into our application lifecycle management."

Christoph Safferling - Head of Game Analytics@Ubisoft

"Thanks to the use of big data technologies for our tracking backend, we are now able to analyze the behavior of our users much more precisely and to significantly improve the game play of our games."

Technische Universität Ilmenau, Germany

Diplom-Ingenieur (Dipl.-Ing.), Electrical Engineering, equivalent to M.Sc. • Oct 1998 – Sep 2004

Major: Theoretical Electrical Engineering

Data platforms

Architecture and delivery of data warehouses, data lakes and lakehouse platforms (AWS, GCP, Kubernetes), from concept to production

Data pipelines & streaming

Design and operation of ETL/ELT, streaming and ML pipelines (Spark, Kafka, Airflow, dbt) at terabyte scale

AI integration

AI-assisted data processing: document extraction, semantic search and workflow automation with LLM tool integration (MCP), including a self-built MCP server in production. See agent-ready data.

Cloud infrastructure as code

Cloud-native infrastructure as code (Terraform, Helm, GitOps), including high-availability and disaster-recovery concepts

Data governance & modeling

GDPR-compliant data management including data residency, data modeling, and requirements and process analysis

Technical leadership

Technical leadership of distributed teams, architecture reviews and knowledge transfer

Workshops

Hands-on workshops for your team on cloud data platforms, data pipelines, streaming and AI integration, tailored to your team's level. See workshops for topics.

Patent

Co-inventor of European patent EP1377924 B1 (Fraunhofer IDMT)

Open Source

caddy-jsonrpc-matcher: Caddy module for selective rate limiting of MCP tool calls; Kafka and Redis export modules for Schedoscope, an open-source Hadoop data warehouse framework

Languages

German (native), English (fluent, full professional proficiency)

Work model

Remote or on-site, hybrid preferred (based in Berlin)

• freelancermap
• malt