Data Platform Engineer Specialist

Responsibilities of a Data Platform Engineer Specialist at Grupo QuintoAndar:

• Build and maintain a high-performance data platform that meets the company's needs, connects with product solutions, and leads analytical innovation, enabling amazing architectures and efficient platforms;
• Responsible for the entire lifecycle of code development (monitoring deployment, adding metrics and alarms, ensuring SLO budget, and more);
• Align with stakeholders to understand their primary needs, but also have a holistic view of the problem and propose extensible, scalable, and incremental solutions;
• Conduct PoCs and benchmarks to determine the best tool for a given problem, as well as understand whether to use market solutions or develop internally;
• Contribute to defining the strategic vision, crossing team and service boundaries to solve problems;
• Be a reference within the chapter on technical concepts, tools, and/or best coding practices

Here are some amazing projects we've worked on:

• We migrated our entire legacy infrastructure with transactional strategies to distributed processing environments, gaining more scalability and flexibility
• We designed an architecture capable of storing metadata and exposing a data catalog company-wide, further enhancing reliability and understanding of the data
• We also worked on LGPD compliance by connecting anonymization strategies, access control, and metadata management to ensure secure access and sharing
• Additionally, we centralized access to analytical data, separating data storage from processing, retention, and discovery

Requirements

• Specialist in technologies, solutions, and concepts of Big Data (Spark, Hadoop, Hive, MapReduce) and multiple languages (YAML, Python);
• Proficiency in Python, or one of the major programming languages, and a passion for writing clean and maintainable code. You might even be a Software Engineer with a focus or passion for data-oriented solutions;
• Understanding of the data lifecycle and concepts such as lineage, governance, privacy, retention, anonymization, etc.;
• Excellent communication skills, proactively sharing and seeking context to collaborate with different teams