Azure Data Engineer
Job Summary
The Azure Data Engineer will design, implement, and manage scalable data architecture solutions within the Microsoft Azure ecosystem. This role involves building efficient ETL pipelines, managing data storage, and ensuring data quality through transformation and cleansing processes. You will collaborate with data scientists and stakeholders to deliver high-performance big data processing and analytics solutions.
Responsibilities
- Design and manage Azure data architecture
- Build scalable ETL pipelines
- Perform big data processing and analytics
- Ensure data quality and integrity
Required Skills
- Azure Data Factory, Synapse, Databricks
- SQL and NoSQL proficiency
- Data warehousing and modeling
- English language proficiency
Job Details
Responsibilities:
- Design, implement, and manage data architecture solutions on Microsoft Azure, including data lakes, data warehouses, and data marts.
- Develop efficient data models and schemas for optimal storage and retrieval.
- Build scalable ETL pipelines using Azure Data Factory, Azure Databricks, or SQL Server Integration Services (SSIS) to ingest data from various sources into Azure.
- Ensure data transformation and cleansing processes align with business requirements.
- Deploy and manage Azure data storage solutions such as Azure Blob Storage, Azure Data Lake Storage, and Azure SQL Database.
- Optimize data storage and retrieval performance, ensuring security and compliance.
- Utilize Azure Synapse Analytics and Azure Databricks for big data processing and analytics tasks.
- Perform data aggregation, enrichment, and analysis to support business intelligence and reporting needs.
- Monitor data pipelines and storage solutions for performance bottlenecks and optimize for scalability and cost-effectiveness.
- Troubleshoot data-related issues and ensure data quality and integrity.
- Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver effective solutions.
- Document data architecture, processes, and workflows for knowledge sharing and future reference.
Requirements:
- Bachelor’s degree in Computer Science, Information Technology, or a related field.
- Proven experience as a Data Engineer or similar role, with a focus on Azure cloud platform.
- Hands-on experience with Azure Data Factory, Azure Synapse Analytics, Azure Databricks, and SQL Server Integration Services (SSIS).
- Strong proficiency in SQL and experience with NoSQL databases.
- Familiarity with data modeling, schema design, and data warehousing principles.
- Experience in deploying and managing Azure data storage solutions.
- Knowledge of cloud computing concepts and best practices, particularly in Microsoft Azure.
- Good communication and analytical skills, strong verbal and written communication skills in English are mandatory.
- Azure certifications (e.g., Azure Data Engineer Associate) are a plus.
- European Union Citizenship or Portuguese Working Visa / Portuguese Residency.
- Availability to go to the office, once or twice per week.