AI Data Engineer
Job Summary
Caixa Mágica is seeking an AI/Data Engineer to focus on the development, maintenance, and evolution of application solutions, data, Machine Learning, and GenAI. The role involves technical leadership in code development, optimization, and managing pipelines within Microsoft Azure. The candidate will also be responsible for developing GenAI solutions, including RAG architectures and REST APIs.
Responsibilities
- Lead code development and optimization
- Manage Azure Data Factory and pipelines
- Maintain ML/regression/classification models
- Develop GenAI and RAG solutions
- Maintain REST APIs and ensure code quality
Required Skills
- Python, PySpark, Databricks
- Microsoft Azure & Azure Data Factory
- Machine Learning, GenAI, & LLMs
- LangChain, LangGraph, RAG
- SQL, PostgreSQL, & Vector databases
- REST APIs, GitHub, & DevOps practices
Job Details
Responsibilities:
- Lead technical activities related to code development, maintenance, and optimization.
- Work with Python, PySpark, Databricks, and application technologies.
- Support the evolution of solutions in Microsoft Azure, including Azure Data Factory.
- Maintain and improve data and Machine Learning pipelines.
- Support the maintenance of regression and classification models in production.
- Develop and maintain GenAI solutions, including integrations with OpenAI APIs.
- Implement and maintain RAG solutions, single-agent and multi-agent architectures.
- Develop and maintain REST APIs.
- Ensure best practices in development, testing, documentation, and code quality.
- Support deployments, environment management, and troubleshooting.
Requirements:
- Up to 5 years of experience in similar roles.
- Bachelor’s degree in Computer Engineering, Software Engineering, or a similar field.
- Experience with Python, PySpark, and Databricks.
- Solid experience with Microsoft Azure and Azure Data Factory.
- Knowledge of Machine Learning, GenAI, and LLM-based architectures.
- Experience with LangChain, LangGraph, RAG and, preferably, Langfuse.
- Knowledge of SQL/PostgreSQL, vector databases, and document indexing.
- Experience with REST APIs, GitHub, pytest, SonarQube, and DevOps practices.
- Good communication skills, critical thinking, and problem-solving ability.
- English required; Portuguese and Spanish are valued.
- Knowledge of Agile, JIRA, and Confluence is valued.