Data Engineer (m/f/d)

WHAT YOU CAN EXPECT

We are currently seeking a talented Data Engineer to join our team at BASF's Agricultural Solutions department Digital Factory.

RESPONSIBILITIES

• Assist in the design and development of scalable and efficient cloud-based data pipelines using Databricks and Azure Data Factory.
• Support the integration of real-time streaming solutions with Event Hubs and Kafka for data ingestion and processing.
• Help optimize data storage and transformation using Azure Data Lake Storage Gen2 (ADLS Gen2).
• Write clean and efficient code in Python and PySpark to process and analyze datasets.
• Collaborate with cross-functional teams to gather data requirements and implement solutions that align with business goals.
• Monitor and troubleshoot cloud-based data platforms and pipelines, contributing to performance improvements.
• Ensure adherence to data quality, governance, and security standards across data solutions.

REQUIREMENTS OF THE POSITION

• Basic knowledge of Databricks and Azure Data Factory for building data pipelines and workflows.
• Familiarity with Event Hubs and Kafka for real-time data streaming and messaging.
• Understanding of Azure Data Lake Storage Gen2 (ADLS Gen2) for data management and storage optimization.
• Proficiency in programming with Python and PySpark for data engineering tasks.
• Awareness of data integration, transformation, and optimization best practices in cloud environments.
• Experience or coursework in large-scale data processing and real-time streaming architectures is a plus.
• Familiarity with Azure cloud services and related tools is advantageous.
• Strong analytical and problem-solving skills with a focus on performance and scalability.

NICE TO HAVE

• Relevant coursework or certifications in Azure data engineering, cloud platforms, or related technologies.
• Exposure to CI/CD pipelines for cloud data solutions is a plus.
• Basic knowledge of machine learning concepts and integration within data pipelines (optional).

WHAT WE OFFER

• A secure work environment because your health, safety and wellbeing is always our top priority.
• Flexible work schedule and Home-office options, so that you can balance your working life and private life.
• Learning and development opportunities
• 23 holiday days per year
• 5 additional days (readjustment)
• 2 cultural days
• A collaborative, trustful and innovative work environment
• Being part of an international team and work in global projects
• Relocation assistance to Madrid provided