Responsibilities
- Lead and oversee a distributed data engineering team that builds technical assets as code using professional development environments, version control, and automated CI/CD pipelines.
- Ensure the delivery of high-quality data assets that support enterprise analytics, artificial intelligence, machine learning, and scalable data governance initiatives.
- Serve as both delivery and people manager for a team located in Bengaluru, working closely with global platform, business, and IT partners.
- Promote engineering consistency, technical excellence, and reuse across all stages of the data lifecycle—from data ingestion to semantic layer development—while enforcing strong software engineering principles.
- Directly manage a team of approximately 10 to 15 data engineers with diverse skill sets, including generalists and specialists.
- Deliver data and analytics capabilities through a Product Owner model to serve multiple business domains.
- Collaborate with Product Owners, lead architects, senior engineers, data governance teams, infrastructure, cybersecurity, and functional IT units across regions.
- Provide line management for a high-performing, multidisciplinary data engineering group.
- Drive continuous learning, mentorship, and performance evaluation within the team.
- Cultivate an environment rooted in accountability, psychological safety, and mutual trust.
- Own end-to-end delivery of data and analytics assets, spanning data sourcing to semantic modeling.
- Align team output with business objectives and enterprise architectural standards.
- Enforce quality checkpoints and comprehensive documentation practices.
- Act as the central point for issue escalation and coordination across business units.
- Integrate decisions from infrastructure, functional IT, security, and platform teams into cohesive solutions.
- Represent team interests and priorities in global forums and strategic discussions.
- Promote engineering best practices such as test-driven development, continuous integration and deployment, and infrastructure as code; guide implementation of all artifacts as code, including batch and streaming pipelines on Azure Databricks using PySpark or Scala.
- Design and maintain scalable batch and streaming data pipelines in Databricks, handling ingestion from structured and semi-structured sources and implementing governed bronze, silver, and gold layers within a lakehouse architecture.
- Oversee dimensional modeling efforts and development of curated data marts for analytical use cases, ensuring alignment with semantic layers and integration with the enterprise third-normal-form data warehouse.
- Ensure robust engineering standards across data validation, test-driven development with CI/CD, performance optimization, metadata management, and agile collaboration with stakeholders.
- Develop an inclusive, high-performance culture within the Bengaluru-based team.
- Advocate for DevSecOps principles, automation, component reuse, and system reliability.
- Require all code and configuration artifacts to be stored in version control with mandatory peer reviews and integrated into CI/CD workflows.
- Maintain thorough documentation, enable knowledge transfer, and drive ongoing process improvements.
- Design and operate secure, scalable data ingestion services, including change data capture, delta loads, full extractions, and SAP data pulls using tools such as Theobald Extract Universal.
Work Arrangement
Remote (Worldwide) — Bengaluru
Team
- Team size: 10–15
- Structure: Global IT team for data engineering, co-located in Bengaluru, collaborating globally with platform, business, and other IT stakeholders.
