You will work closely with clients, engineering teams, solution architects, delivery leaders, and business stakeholders to design innovative data platforms, guide technical decision-making, and expand the organization's Databricks capabilities. The ideal candidate is both a strong technical leader and a trusted advisor who can bridge business objectives with modern data engineering solutions.
Lead the technical direction and growth of the Databricks Practice, establishing standards, best practices, and reusable solution accelerators.
Design and review enterprise-scale data platform architectures using Databricks, Apache Spark, and Microsoft Azure.
Provide technical leadership and architectural guidance across multiple client engagements.
Partner with clients to understand business challenges and recommend scalable data and analytics solutions.
Lead technical discovery sessions, solution workshops, architecture reviews, and design discussions with clients.
Mentor and coach Data Engineers, Technical Leads, and Solution Architects, fostering technical excellence and career development.
Define best practices for data engineering, software engineering, DevOps, security, governance, and operational excellence.
Establish standards for ETL/ELT development, data quality, monitoring, performance optimization, and production support.
Drive adoption of modern data architectures, including Lakehouse, Delta Lake, and event-driven data processing.
Collaborate with Delivery Managers and Engineering Managers to support project planning, staffing, technical risk management, and solution quality.
Support pre-sales activities, including solution design, technical proposals, effort estimation, client presentations, and proof-of-concept initiatives.
Stay current with Databricks platform capabilities and emerging technologies, identifying opportunities to improve services and expand practice offerings.
Promote the adoption of AI-assisted software development and engineering best practices across the practice.
8+ years of professional experience in Data Engineering, Data Platform Engineering, or related disciplines.
3+ years of experience leading technical teams, architecture initiatives, or data engineering practices.
Deep expertise with Databricks, including workspace administration, notebooks, jobs, workflows, Unity Catalog, Delta Lake, and Lakehouse architecture.
Strong experience developing scalable data pipelines using Apache Spark (PySpark).
Extensive experience with Microsoft Azure, including services such as Azure Data Lake Storage (ADLS), Azure Data Factory, Azure Event Hubs, Azure Key Vault, and Azure DevOps.
Strong programming skills in Python, including clean coding practices and object-oriented programming principles.
Experience with orchestration tools such as Apache Airflow or Dagster.
Experience working with streaming technologies such as Apache Kafka or Azure Event Hubs.
Strong understanding of data architecture, dimensional data modeling, metadata management, and enterprise data platform design.
Experience implementing data governance, data quality, security, and compliance practices.
Experience building and optimizing enterprise ETL/ELT solutions.
Strong knowledge of SQL and relational database concepts.
Experience deploying cloud-native data solutions, preferably on Microsoft Azure.
Excellent consulting, presentation, and stakeholder management skills with experience interacting directly with enterprise clients.
Experience leading architecture discussions, technical governance, and solution reviews.
Ability to lead multiple initiatives simultaneously while providing technical direction across distributed teams.
Experience working in Agile development environments.
