Data Engineer (m/f/d)

RESPONSIBILITIES

• Assist in the design and development of scalable and efficient cloud-based data pipelines using Databricks and Azure Data Factory.
• Support the integration of real-time streaming solutions with Event Hubs and Kafka for data ingestion and processing.
• Help optimize data storage and transformation using Azure Data Lake Storage Gen2 (ADLS Gen2).
• Write clean and efficient code in Python and PySpark to process and analyze datasets.
• Collaborate with cross-functional teams to gather data requirements and implement solutions that align with business goals
• Monitor and troubleshoot cloud-based data platforms and pipelines, contributing to performance improvements
• Ensure adherence to data quality, governance, and security standards across data solutions.

REQUIREMENTS

• Basic knowledge of Databricks and Azure Data Factory for building data pipelines and workflows.
• Familiarity with Event Hubs and Kafka for real-time data streaming and messaging.
• Understanding of Azure Data Lake Storage Gen2 (ADLS Gen2) for data management and storage optimization
• Proficiency in programming with Python and PySpark for data engineering tasks.
• Awareness of data integration, transformation, and optimization best practices in cloud environments.
• Experience or coursework in large-scale data processing and real-time streaming architectures is a plus.
• Familiarity with Azure cloud services and related tools is advantageous
• Strong analytical and problem-solving skills with a focus on performance and scalability.

NICE TO HAVE

• Relevant coursework or certifications in Azure data engineering, cloud platforms, or related technologies.
• Exposure to CI/CD pipelines for cloud data solutions is a plus
• Basic knowledge of machine learning concepts and integration within data pipelines (optional).

BENEFITS

• Responsibility from day one in a challenging work environment and "on-the-job" training as part of a committed team.
• Adequate compensation according to your qualifications and experience • A secure work environment because your health, safety and wellbeing is always our top priority.
• Flexible work schedule and Home-office options, so that you can balance your working life and private life.
• Learning and development opportunities
• 23 holidays per year
• Another 5 days (readjustments days) and 2 days (cultural days)
• A collaborative, trustful and innovative work environment
• Being part of an international team and work in global projects
• Relocation assistance to Madrid provided