Data Engineer (m/f/d)

BASF

Festanstellung

Datenbankentwicklung/BI, Datenanalyse, Entwicklung

Bachelor, Master

Nächstmöglicher Zeitpunkt

Deutsch

Estación de Chamartín, Madrid, Spanien



WHAT YOU CAN EXPECT

We are currently seeking a talented Data Engineer to join our team at BASF's Agricultural Solutions department Digital Factory.

RESPONSIBILITIES

  • Assist in the design and development of scalable and efficient cloud-based data pipelines using Databricks and Azure Data Factory.
  • Support the integration of real-time streaming solutions with Event Hubs and Kafka for data ingestion and processing.
  • Help optimize data storage and transformation using Azure Data Lake Storage Gen2 (ADLS Gen2).
  • Write clean and efficient code in Python and PySpark to process and analyze datasets.
  • Collaborate with cross-functional teams to gather data requirements and implement solutions that align with business goals.
  • Monitor and troubleshoot cloud-based data platforms and pipelines, contributing to performance improvements.
  • Ensure adherence to data quality, governance, and security standards across data solutions.

REQUIREMENTS OF THE POSITION

  • Basic knowledge of Databricks and Azure Data Factory for building data pipelines and workflows.
  • Familiarity with Event Hubs and Kafka for real-time data streaming and messaging.
  • Understanding of Azure Data Lake Storage Gen2 (ADLS Gen2) for data management and storage optimization.
  • Proficiency in programming with Python and PySpark for data engineering tasks.
  • Awareness of data integration, transformation, and optimization best practices in cloud environments.
  • Experience or coursework in large-scale data processing and real-time streaming architectures is a plus.
  • Familiarity with Azure cloud services and related tools is advantageous.
  • Strong analytical and problem-solving skills with a focus on performance and scalability.

NICE TO HAVE

  • Relevant coursework or certifications in Azure data engineering, cloud platforms, or related technologies.
  • Exposure to CI/CD pipelines for cloud data solutions is a plus.
  • Basic knowledge of machine learning concepts and integration within data pipelines (optional).

WHAT WE OFFER

  • A secure work environment because your health, safety and wellbeing is always our top priority.
  • Flexible work schedule and Home-office options, so that you can balance your working life and private life.
  • Learning and development opportunities
  • 23 holiday days per year
  • 5 additional days (readjustment)
  • 2 cultural days
  • A collaborative, trustful and innovative work environment
  • Being part of an international team and work in global projects
  • Relocation assistance to Madrid provided

Ähnliche Jobs