Relevant Work Experience : 10+ years
Status – Full Time
Location – Pan India (Remote)
Job Description: Azure Data Engineer (Data Engineering + Application Support + DevOps)
About the Role
We are seeking an Azure Data Engineer who can design, build, and maintain scalable data pipelines on Azure while also providing production support for data applications. The role requires strong coding abilities (PySpark, Python, SQL), hands-on experience with Azure data services, and familiarity with CI/CD pipelines. This is a hybrid role suited for someone who can balance data engineering, application support, and Azure DevOps responsibilities.
Primary Responsibilities
Data Engineering
•Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake Storage (ADLS), and Azure Blob Storage.
•Design, implement, and maintain data ingestion, processing, and transformation pipelines on Azure.
•Build and optimize data models to support analytics and reporting use cases.
•Develop and maintain ETL/ELT workflows using Azure Data Factory (ADF) or equivalent orchestrators.
•Use Azure Data Factory and Azure Databricks (PySpark/Spark Core) to assemble and process large-scale datasets.
•Implement data validation, cleansing, and reconciliation processes to ensure data quality and reliability.
•Enforce data security, access control, and compliance with organizational standards.
Application Support
•Provide L2/L3 support for data pipelines, scheduled jobs, and production data services.
•Monitor pipeline performance, troubleshoot failures, and ensure adherence to SLAs.
•Handle incident management, root-cause analysis, and long-term fixes.
•Coordinate with infra, data engineering, and product teams to resolve platform and pipeline issues.
DevOps & CI/CD
•Create and maintain Azure DevOps pipelines for deployment, testing, and automation.
•Manage source control, branching, pull requests, and release cycles.
•Implement CI/CD for data pipelines, notebooks, and infrastructure components.
•Contribute to IaC (Infrastructure-as-Code) using ARM/Bicep/Terraform (added advantage).
Collaboration & Stakeholder Alignment
•Work closely with data engineers, architects, analysts, and application owners to understand requirements and translate them into scalable data platform solutions.
•Communicate effectively across cross-functional teams with clarity and technical depth.
•Participate in sprint ceremonies, design discussions, and code reviews.
Required Skills & Qualifications
•Strong experience in Azure Data Engineering with hands-on exposure to:
oAzure Data Factory (ADF)
oAzure Databricks / PySpark
oAzure SQL, ADLS, Azure Blob Storage
oAzure DevOps (repos + pipelines)
•Strong coding skills in:
oPySpark
oPython
oSQL (must be very strong—joins, window functions, performance tuning)
•Solid understanding of Spark Core, cluster execution, and distributed processing.
•Experience with big data technologies and data lake-based architectures.
•Experience in application support, pipeline monitoring, and incident handling.
•Understanding of CI/CD, version control, and deployment automation.
•Good communication skills with the ability to explain technical concepts clearly.
Preferred / Nice-to-Have
•Experience with Delta Lake, Unity Catalog, Databricks Workflows.
•Knowledge of Event Hub, Service Bus, Kafka, or streaming pipelines.
•Familiarity with API integrations and REST-based ingestion patterns.
•Knowledge of monitoring tools (Log Analytics, App Insights).
•Understanding of data governance, cataloging, and metadata management.
Ideal Candidate Profile
A technically strong Azure Data Engineer with hands-on coding skills, deep Spark experience, strong SQL, and the ability to manage both data pipelines and production support operations. Someone who is reliable in handling on-call support, debugging issues, optimizing pipelines, and ensuring smooth CI/CD workflows.
