As a big data engineer, you will be responsible for developing solutions and implementing requirements around our large-scale data processing platform in the Amazon cloud.
This includes writing automated loaders that ingest data from various sources, developing modules to enrich raw data with additional attributes and designing algorithms to connect millions of data points every week
In addition, you will work on algorithms to derive insights from our data set and implement tools to ensure a frictionless delivery of data to our customers all around the globe.
What You’ll Do
Work as part of our big data processing team to bring our platform to the next level
Enhance our data processing pipeline by implementing new algorithms on Apache Spark
Develop new distance measures and rules to connect data points
At least two years of experience as Java developer
Eligible to work in Germany and willing to re-locate to Frankfurt am Main
Skills in scripting languages like Python
Fluent in English
Nice to Have
Previously worked in agile environments
Experience with AWS infrastructure like S3, Redshift, EC2, etc.
Experience working with big data and technologies like Hadoop, Spark, Oozie, Hive, etc.
Experience with rule based expert systems and machine learning algorithms