Senior Data Engineer
Job Summary
We are seeking a Senior Data Engineer with over 6 years of experience to join our team. The role focuses on large-scale data transformation and processing within distributed environments. You will work in a hybrid setup in Lisbon, utilizing a robust tech stack including Spark, Hadoop, and Kafka.
Responsibilities
- Design and maintain distributed data processing systems
- Develop large-scale data transformation pipelines
- Perform data analysis using SQL
- Implement CI/CD workflows and versioning practices
Required Skills
- Spark, Hadoop, Hive, Kafka
- SQL, Scala, Java
- CI/CD (Git, Jenkins, Jira)
- 6+ years experience in Big Data
Job Details
Responsibilities and Requirements:
- Strong experience working in Big Data environments and distributed data processing systems is essential.
- Previous hands-on experience with Spark for large-scale data transformation and processing.
- Solid practical knowledge of Hadoop, Hive and Kafka within data-driven architectures.
- Strong SQL skills, used transversally across the role for data analysis and processing.
- Familiarity with Scala or Java for backend and data engineering development.
- Knowledge of CI/CD and versioning tools such as Git, Jira, Jenkins, BitBucket and XLDeploy.
- Strong development best practices with a quality-oriented and TDD mindset.
- Contact with Spring Boot, Control-M is nice to have.
- Exposure to financial and/or trading environments is welcomed.
- Autonomous and proactive profile with strong analytical skills and a hands-on problem-solving mindset.
- Fluency in English is mandatory; French is a plus but not mandatory.
- Enjoy a hybrid setup with 8 onsite days each month.