For one our clients based in Eastern Europe, we are currently looking for a Lead Data Engineer.

We are seeking a highly skilled Data Tech Lead to lead a team/pod of 4 Senior Data Engineers and 2 QAs to implement cutting-edge Azure Databricks solution focused on automating tax reporting. This role will involve managing the ingestion of daily data feeds and producing outputs on the scale of billions of forms per client, ensuring high performance and scalability.

Details:

• Role: Lead Data Engineer
• Location:  100% remote
• Working overlap needed: Minimum 2-6pm CET, preferred 2-7pm CET
• Start: asap
• Duration: 6 months+ extension
• HackerRank challenge: Yes

Scope:

• Lead and mentor a team of data engineers and QA specialists to successfully execute data engineering projects.
• Design and implement advanced data solutions utilizing Azure Databricks to automate tax reporting processes.
• Manage and optimize the ingestion and transformation of daily data feeds, ensuring seamless data flow and integration.
• Develop and maintain scalable data pipelines and architectures for high-volume data processing.
• Collaborate with cross-functional teams to ensure alignment and integration with broader IT and business strategies.
• Drive the use of best practices in data engineering, leveraging modern tools and technologies.

Requirements:

• Advanced Python knowledge and software engineering skills. 
• Proven experience with Spark and SQL for data engineering and analysis.
• Experience with Databricks for data ingestion and transformation.
• Hands-on experience with one or more cloud services (Azure - must, AWS, GCP).
• Experience with Azure Data Factory, Azure Cosmos DB, Azure functions, Event store, ADLS, Key Vault.
• Proficiency in RDBMS/NoSQL data stores and appropriate use cases.
• Experience with Data as Code; version control, small and regular commits, unit tests, CI/CD, packaging, familiarity with containerization tools such as Docker (must have) and Kubernetes (plus).
• Solid understanding of the software development life cycle.
• Strong communication, interpersonal and presentation skills.
• Experience leading teams.