Talent.com
M&T Resources
Lead Engineer - Data & AIM&T Resources • Sydney, AU
Lead Engineer - Data & AI

Lead Engineer - Data & AI

M&T Resources • Sydney, AU
5 days ago
Job description

We're looking for a Lead Data Engineer with deep, hands-on Databricks expertise to join a major enterprise data and AI transformation.

You'll lead the design and delivery of scalable data platforms and pipelines supporting GenAI and agentic AI initiatives.

Strong Databricks experience is essential. We're looking for someone who has delivered production-grade solutions in Databricks, not just exposure to the platform.

What You'll Do

  • Lead data engineering solutions using Databricks and the Lakehouse platform.
  • Build scalable pipelines using Spark, PySpark, Python and SQL.
  • Design Delta Lake / Lakehouse architectures across batch and streaming.
  • Build data foundations for GenAI, RAG and agentic AI.
  • Work with structured and unstructured data.
  • Contribute to Vector Search, embeddings and RAG capabilities.
  • Drive data quality, governance, security and observability.
  • Provide technical leadership and influence architecture and engineering standards.

What We're Looking For

Essential:

  • Extensive commercial Data Engineering experience.
  • Deep hands-on Databricks experience.
  • Strong Spark / PySpark, Python and SQL.
  • Strong Delta Lake / Delta Tables experience.
  • Experience designing complex data pipelines and platforms.
  • Cloud experience, ideally Azure or AWS.
  • Strong technical leadership and architecture skills.

Desirable:

  • GenAI / LLM experience.
  • RAG, Vector Search and embeddings.
  • Agentic AI experience.
  • LangChain, LangGraph, LlamaIndex or MCP.
  • Unity Catalog, MLflow, Databricks Workflows or DLT/Lakeflow.
  • The Key Requirement

If you're a strong Data Engineer but only have limited Databricks exposure, this role is unlikely to be suitable. We're specifically looking for candidates who are genuinely strong in Databricks and can operate at Lead level.

#J-18808-Ljbffr

Create a job alert for this search

Lead Engineer - Data & AI • Sydney, AU

Similar jobs

Senior Data & Insights Architect - AWS AI/ML Leader

Cevo trades as Cevo (VIC) PtySydney, AU

A technology consulting firm in Australia is seeking a Principal Data and Insights Consultant responsible for leading data architecture and engineering initiatives.The ideal candidate will have ext... Show more

 • Promoted

Technical Team Lead (Fullstack+ AI integration)

PeoplebankSydney, AU

Technical Team Lead (Fullstack+ AI integration).Technical Team Lead (Fullstack+ AI integration).Technical Team Lead (Fullstack+ AI integration).Be among the first 25 applicants.Technical Team Lead ... Show more

 • Promoted

Data Analytics Engineer - Hybrid, Nonprofit Tech for Impact

Change RecruitmentSydney, AU

A prominent recruitment agency is seeking a Data Engineering and Analytics Specialist for a non-profit organization in Sydney.This role offers a competitive salary between A$145,000 and A$165,000 w... Show more

 • Promoted

Senior Databricks Specialist Lead | AI & Data | Sydney

DeloitteSydney, NSW, AU

Be part of market-leading projects with global scale and complexity.Have your voice heard through people panels and leader forums.Prioritise in-person connection with your team and clients, with fl... Show more

Principal AI/Data Engineer - Enterprise AI Solutions Lead

DyFlex SolutionsSydney, AU

A leading IT consulting firm in Australia is seeking a Principal Consulting AI / Data Engineer to design and optimize enterprise-scale data and AI solutions.The role requires deep technical experti... Show more

 • Promoted

Senior Data Engineer - AI & MLOps

SONITECSydney, AU

SONITEC Sydney, New South Wales, Australia.SONITEC Sydney, New South Wales, Australia.Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.This range is pr... Show more

 • Promoted

Lead Data Engineer

MAP TALENTSydney, AU

Direct message the job poster from MAP TALENT.Connecting the best data talent to businesses in Australia Data Recruiter Tennis Enthusiast Co-Founder of The Data Map….We’re working with a well-estab... Show more

 • Promoted

Senior Data Engineer - AI-Driven Pipelines, Cloud & Hybrid

Lendi GroupSydney, AU

Lendi Group seeks a Senior Data Engineer to join a multi-disciplinary team developing AI systems for property, mortgage and financial services.You will design, build and deploy scalable data pipeli... Show more

 • Promoted

Senior Data Engineer

Krauss PartnersSydney, AU

Get AI-powered advice on this job and more exclusive features.This range is provided by Krauss Partners.Your actual pay will be based on your skills and experience — talk with your recruiter to lea... Show more

 • Promoted

Lead AI Developer

ELMO SoftwareSydney, AU

AI solutions that automate complex HR processes, enhance employee experiences, and deliver unprecedented business intelligence.This isn't just another AI engineering role; it's an opportunity to pi... Show more

 • Promoted

Senior Data Engineer — Cloud & AI Solutions

PractivSydney, AU

An innovative tech consultancy in Australia is seeking a Data Engineer to join their dynamic team in Sydney.The ideal candidate will have a strong grasp of data concepts, such as data modelling, wa... Show more

 • Promoted

Senior Principal - Lead data and Analytics

Infosys Singapore & AustraliaSydney, AU

Lead–Data&Analytics(DirectorLevel).Sydney,NewSouthWales,Australia(Hybrid).ThoughtLeadership&PracticeDevelopment. Show more

 • Promoted

Lead Data Engineer

WPP MediaSydney, NSW, AU

WPP is the trusted growth partner for the world's leading brands.With exceptional talent, trusted data and intelligence, and world-class partnerships - all united by our pioneering agentic marketin... Show more

Senior Data & AI Engineer, Cloud Data Platforms

SlalomSydney, AU

Slalom is a fiercely human business and technology consulting company that leads with outcomes to bring more value, in all ways, always.From strategy through delivery, our agile teams across 52 off... Show more

 • Promoted

AI-Ready Data Platform Lead

icare NSWSydney, AU

A state government agency in New South Wales is seeking a Manager of Data Platform Engineering to lead their team responsible for developing scalable data products and overseeing the engineering te... Show more

 • Promoted

Principal Consulting AI / Data Engineer

DyFlex SolutionsSydney, AU

Principal Consulting AI / Data Engineer.DyFlex is an established SAP Platinum Partner with offices across Australia.We deliver fit‑for‑purpose SAP solutions and our services range from greenfield a... Show more

 • Promoted

Azure Databricks AI/ML Engineer for GenAI & MLOps

Resource CornerSydney, AU

Get notified about new Emergency Medicine Resident jobs in.Emergency Medicine Resident Jobs in United States.Emergency Medicine and Critical Care Faculty Position.Academic Emergency Medicine Physic... Show more

 • Promoted

Data Architect

Mane ConsultingNorth Sydney, AU

Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.This range is provided by Mane Consulting.Your actual pay will be based on your skills and experience ... Show more

 • Promoted

Lead Engineer

Corto Pty LtdSydney, AU

We are CORTO, a cutting-edge software company dedicated to revolutionising the legal industry.Our mission is to empower legal practitioners with AI-driven solutions that streamline their workflow, ... Show more

 • Promoted

Analytics Engineer

Standards AustraliaSydney, AU

Analytics Engineer role at Standards Australia.This is a unique role that combines the responsibilities of a Data Analyst and a Data Engineer.The Analytics Engineer will be extremely creative and h... Show more