Christian Richter

Ingenieurbüro Christian Richter

Data Engineer - Data, Cloud & Container • Jan, 2022 — Present

I support customers in the design and implementation of cloud-based data infrastructure and data-driven process pipelines.

AltusInsight GmbH

Founder • Mar, 2013 — Jun, 2021

AltusInsight has developed the cloud-based software LambdaNow, a deployment platform for data infrastructure. I founded and run the company for 8 years.

Ingenieurbüro Christian Richter

Big Data Engineer • Jan, 2010 — Mar, 2014

My main focus was to enable customers using big data technologies to cater to their data processing needs.

MOG Inc.

Software Developer • May, 2005 — Oct, 2009

First employee of the start-up. Worked on backend software, databases and infrastructure. Implemented algorithms to customize user experience.

Fraunhofer-Gesellschaft

Research Assistant • Aug, 2000 — Mar, 2005

Researched various algorithms on speech recognition.

ID Analytics

Data Science Intern • Jan, 2003 — Jun, 2003

Researched various algorithms for financial fraud detection.

Supply chain management application

Data Engineer • Oct, 2024 — Present

Supply chain management consulting and implementation services.

• Support in creation of master data set for a supply chain management application from multiple heterogeneous sources
• Design and implementation of ETL and data pipelines
• Implementation data quality dashboard for business critical KPIs
• Deployment and monitoring of various data pipelines

Industry: Logistics

Technologies: Google Cloud, GKE, SQL, Kotlin, Pekko, Kafka, GitHub

Data warehouse consulting services

Data Engineer • Sep, 2023 — Oct, 2024

Support and implementation services.

• Consulting services data model, deployment strategies, process design
• Design and implementation of various ETL processes to provide access to research and production data
• Integration of a SAP PLM system to extract and process research data from laboratories
• Stakeholder management to manage and set expectations
• Optimizing job execution and scheduling for faster job processing
• Workshops and knowledge transfer on cloud and data architecture, process design

Industry: Chemistry

Technologies: AWS Cloud, Spring Boot, Java, Docker, GitLab

Data warehouse migration

Cloud Architect • Jul, 2021 — Oct, 2023

Concept and support of an on-premise data warehouse migration towards the cloud.

• Design and implementation of a cloud-based data warehouse infrastructure based on AWS Managed Services
• Help in migrating various data processing jobs from Cloudera to AWS Managed Services
• Design and implementation of deployment pipelines for testing and rollout
• Workshops and knowledge transfer on cloud and data architecture, process design

Industry: Market research

Technologies: AWS Cloud, LakeFormation, Glue, EMR, Athena, Lambda, SQS, S3, Docker, Spark, Hadoop, Airflow, Terraform, Python, GitLab

Cloud based data warehouse

Data Engineer • Jun, 2020 — Dec, 2022

Design and implementation of a cloud-based data ware house for processing user related data.

• Implemented workflows to fetch data from various third-party providers
• Building and enabling a team to create new ETL processes
• Realtime integration of a market automation software suite
• Implemented modern ETL processing environment using Airflow, Spark and Kubernetes
• General advice on data architecture and data management

Industry: Entertainment

Technologies: AWS Cloud, Kubernetes, Docker, Spark, Airflow, Kafka, Terraform, Python, Kustomize, GitLab

Sensor data processing

Solution Architect • Aug, 2019 — Dec, 2019

Design of AWS managed infrastructure platform for sensor data processing, extension of an existing data science environment.

• Advice on design and tools for building a Kubernetes based infrastructure platform for sensor data processing
• Design and implementation of infrastructure components on Kubernetes
• Design and implementation of the ETL pipeline for data collection
• Design and implementation of CI/CD pipeline with Bamboo & Kubernetes

Industry: Consumer Goods

Technologies: AWS Cloud, Kubernetes, Kafka, Spark, Bamboo, Java, Docker, Terraform

Data analytics on car measurement data

Data Engineer • Jan, 2019 — Mar, 2020

Design and implementation of a cloud-based data warehouse for evaluation of vehicle data. Design and implementation of a data science environment.

• Extented a prototype and put into operational readiness for a production environment
• Design and implementation of a CI / CD pipeline
• Setup of project structure and release management
• Implemented various ETL pipelines for car measurement data collection, validation and transformation

Industry: Automotive

Technologies: AWS Cloud, Lambda, IAM, Airflow, Kubernetes, Terraform, Python, Jenkins

Data science environment

Data Engineer • Oct, 2017 — Nov, 2018

Design and implementation of a cloud-based data warehouse & data science environment.

• Designing data warehouse architecture & data storage strategies
• Architecture proposal of a dynamically scalable data warehouse
• Implementation of infrastructure components in Terraform & Kubernetes
• Implementation of infrastructure components in Kubernetes
• Development of ETL pipelines for data collection

Industry: Consumer Goods

Technologies: AWS Cloud, Kubernetes, Spark, R, NiFi, Terraform, Docker, Jupyter NB

Micro service architecture

System Architect • May, 2017 — Dec, 2018

Support conception and implementation/migration of a monolith into a micro service architecture.

• Alignment and coordination of different teams regarding technology usage
• Introduction of Kafka as the central message bus for micro service communications
• Introduction of LiquiBase for database schema management
• Professional / technical support for a specific micro service

Industry: Financial services

Technologies: Micro Services, Java, Docker, Kafka, Liquibase, Jenkins

Big Data vendor evaluation

Requirements Engineer • Mar, 2017

Support in evaluating big data providers

• Acquisition and documentation of the technical requirements for setting up and operating an Apache Hadoop based data warehouse
• Obtaining offers from various providers, preparing information for decision-making
• Implementation of a prototype for data collection

Industry: Energy

Technologies: Hortonworks, Cloudera, SAP Cloud, Apache NiFi, AWS Cloud, MS Azure, Terraform

Big Data warehouse

Data Engineer • Jan, 2017 — Aug, 2017

Design and implementation of a cloud-based big data warehouse in the AWS Cloud for market research analytics.

• Technical project management
• Design of an architecture based on AWS cloud infrastructure and managed services
• Implementation of ETL data pipelines
• Development of data warehouse / workflow management
• Data preparation / process management

Industry: Market research

Technologies: Spark, SparkR, Hadoop, Hive, Jupyter, AWS Cloud, R, Bamboo, Terraform

Workshop Big Data technologies

Data Engineer • Oct, 2016

Workshop Big Data Technologies - Introduction and Getting Started.

• Conducting a 3-day workshop
• Introduction to Big Data / Hadoop ecosystem
• Practical exercise using big data tools in the AWS Cloud

Industry: Education

Technologies: Hadoop, Spark, AWS Cloud, MapReduce, Hive, Pig, R, Terraform

Architecture Review

Data Engineer • Jun, 2016 — Dec, 2016

Architecture review and design and implementation of a realtime aggregator for machine statistics.

• Review and assessment of the existing architecture and data model design
• Implementation workshop data management/Lambda architecture
• Design and implementation of a realtime layer with Spark Streaming

Industry: Entertainment

Technologies: Hadoop, Spark, AWS Cloud, Scala, MapReduce, JCascalog, RedShift

ETL Pipelines

Software Engineer • Feb, 2016 — Jul, 2016

Support in the development of ETL processes on a Hadoop based DWH.

• Planning and implementation of a hive export module
• Implementation of a Kafka & Redis export module as part of an open source project
• Implementation of an analysis algorithm for click stream analytics

Industry: Online retail

Technologies: Hadoop, Hive, Spark, Redis, Kafka, Avro, Scala, HCatalog, Schedoscope

CI/CD Pipelines

DevOps Engineer • Dec, 2015 — Aug, 2016

Design and implementation of a continuous deployment & delivery pipeline for data-driven applications in cloud environments.

• Design and implementation of a big data infrastructure in the AWS Cloud
• Design and implementation of a continuous deployment pipeline
• Technical management of an customer internal team

Industry: Market research

Technologies: AWS Cloud, Hadoop, Spark, Bamboo, Git, Terraform, Vagrant, InfluxDB

Data warehouse

Data Engineer • Jul, 2015 — Oct, 2015

Conception and implementation of a data ware house based on big data technologies - OLAP workload.

• Planning and implementation of the cluster infrastructure
• Evaluation of various input formats with regard to performance
• Preparation, execution and documentation of load tests

Industry: Technology

Technologies: Hadoop, Impala, Hive, ETL, AWS Cloud

Data analytics on wifi device data

Data Engineer • Jul, 2014 — Jun, 2015

Design and implementation of a big data system for batch and real-time data processing of machine generated data.

• Planning and implementation of the deployment environment
• Evaluation of various technologies for data acquisition / data processing
• Implementation of a distributed, fail-safe high throughput messaging and analysis system for machine data (Lambda Architecture)
• Technical management of a team

Industry: Consumer Goods

Technologies: Hadoop, Samza, Spark, Kafka, Java, ETL, AWS

Game Analytics

Data Engineer • Mar, 2013 — Sep, 2014

Design and implementation of Hadoop based data warehouse for online game analytics.

• Planning and implementation of a data warehouse
• Evaluation of different approaches for data collection
• Selection of suitable technologies
• Technical management / coordination of a distributed team (GER, CN, CAN)
• Implementation of a distributed, fail-safe high throughput messaging system

Industry: Entertainment

Technologies: Hadoop, Map / Reduce, Kafka, Hive, ETL, Java, Linux

Hadoop on Demand

DevOps Engineer • Feb, 2013 — Jun, 2014

Design and implementation of a big data infrastructure in virtualized environments.

• Planning and implementation of a big data deployment infrastructure
• Implementation deployment process for Hadoop Cluster on demand in a virtualized environment
• Prototype implementation of various algorithms with the map/reduce framework

Industry: Telecommunication

Technologies: Hadoop, OpenStack, Opscode Chef, Java, Linux

Geo location data analytics

DevOps Engineer • Nov, 2012 — Aug, 2015

Design and implementation of a big data architecture for evaluating telecommunications data.

• Planning and implementation of the network setup
• Planning and implementation of a medium sized Hadoop cluster
• Set up deployment process, including monitoring
• Implementation of a data integration framework for high volume data storage

Industry: Market research

Technologies: Apache Hadoop, Hive, Flume, Java, Spring, Puppet, Ubuntu Linux, AWS

Building data-driven backends

I thrive on building new things from scratch, contributing best practices from previous projects and experience on how to design, build, deploy and operate data processing software and infrastructure.

Requirements engineering

Understanding customer needs and business requirements and translating them into infrastructure and software is one of my core competency used throughout the lifetime of every project. My focus on the essential core value proposition provides value to the customer early on.

Enabling teams

I support and enable teams to efficiently build and operate data-driven processes. Sharing knowledge and best practices on designing and implementing reliable ETL processes and required infrastructure components is part of my daily routine.

Using technology

Having worked on more than 20 projects allowed me to learn and use a broad variety of tools and services to design and build data-driven backend processes. This understanding helps me in choosing the right tools and technologies for the task given.

Johannes Kunze - Senior Solution Architect@GfK

"Chistian's expertise with distributed and AWS based infrastructures helped us big time when we needed a solution to orchestrate our servers and services, by integrating Terraform into our application lifecycle management."

Christoph Safferling - Head of Game Analytics@Ubisoft

"Thanks to the use of big data technologies for our tracking backend, we are now able to analyze the behavior of our users much more precisely and to significantly improve the game play of our games."

Technische Universität Ilmenau

Diplom, Advanced Electromagnetics • 1998 — 2004

Design data-driven processes

I support your organization to design and develop a data strategy that aligns with the business goals and organizational structure and provide guidance in the design and implementation of ETL processes and required cloud infrastructure.

Rapid project setup

If you are planning a new project and need help getting started I'm ready to support you and your team. After drafting an initial concept based on the business requirements, I immediately start setting up cloud infrastructure, CI/CD pipelines and ETL processes.

Architecture review

I help you to improve the performance of an application by reviewing the architecture and identifying weak spots (e.g. how data is partitioned, stored and processed) and provide suggestions for remediation.

Workshops

I offer various workshops touching data engineering topics and cloud infrastructure best practices. Don't hesitate to get in touch to learn more about my workshop offerings.

• freelancermap
• GULP
• malt