Azure Data Engineer with Databricks

Azure Data Engineer with Databricks

THE WORK:

• Develop and implement scalable data processing solutions on Azure, transforming and cleaning large structured and unstructured datasets efficiently.

• Design, build, and maintain robust data pipelines integrating data from diverse sources (databases, APIs, streaming platforms).

• Collaborate with cross-functional teams including data engineers, data scientists, and cloud architects to deliver end-to-end data platform solutions.

• Engage at all critical stages of project development with business analysts, source system experts, data architects, engineers, and product owners.

• Clearly communicate and document technical work, factoring in performance and cost considerations in all design decisions.

HERE’S WHAT YOU’LL NEED:

• Hands-on experience with implementing solutions with data lakes and pipelines (Databricks, Azure Data Factory).

• Proven experience in designing and implementing solutions in Azure cloud (knowledge of AWS, GCP clouds will be an additional advantage).

• Proficiency in SQL, PySpark, data modeling (3NF, star/snowflake schemas) and ELT/ETL design patterns (knowledge of popular ETL tools, including Azure Data Factory).

• Experience with Python or similar scripting languages for data transformation, orchestration, and automation.

• Familiarity with advanced Databricks features (Unity Catalog, MLflow, Jobs & Pipelines).

• Understanding cloud data architecture principles, including data Lakehouse concepts, Medallion architecture, API integration, and event-driven architecture.

• Experience with CI/CD and DevOps practices for data workflows (Git, Azure DevOps) and analytics integration (e.g., Power BI).

• Excellent communication and client-facing skills, with the ability to explain technical concepts clearly to team members and stakeholders.

• Strong collaboration skills across multidisciplinary teams.