Job Description

About the Role

We are hiring a Lead Data Engineer to help transform a complex, fragmented data ecosystem into a scalable, unified platform that powers analytics, AI, and business-critical workflows.

This is a hands-on leadership role focused on building real, production-grade data systems-not prototypes. You'll play a key role in enabling a modern data platform that supports advanced analytics, AI-driven insights, and improved user experiences across the organization.

If you enjoy solving complex data challenges and building systems that are actually used at scale, this is a high-impact opportunity.

What You'll Do

• Design, build, and own enterprise-grade data pipelines on Microsoft Azure / Fabric
• Integrate data across multiple enterprise platforms (APIs, files, and streaming sources)
• Develop scalable ingestion patterns (REST APIs, SFTP, batch & near real-time pipelines)
• Build curated, analytics-ready datasets (dimensional modeling / Data Vault)
• Ensure data quality, monitoring, SLAs, and alerting
• Implement secure and governed data movement (lineage, access control, compliance)
• Enable data for:
  • AI / ML initiatives
  • RAG (Retrieval-Augmented Generation) pipelines
  • Vectorized and unstructured data processing
• Collaborate with cross-functional teams across Data, Analytics, and Business units
• Provide technical leadership and mentorship (no direct reports initially)

What You'll Bring (Required)

• 7+ years of hands-on data engineering experience
• Strong experience with Microsoft Azure (required)
• Proven track record building production data pipelines across complex systems
• Strong SQL and Python skills (PySpark/Spark preferred)
• Experience with:
  • Data ingestion (API, SFTP, batch, streaming)
  • Data modeling and transformation
  • Pipeline reliability, monitoring, and performance optimization
• Hands-on experience with Microsoft Fabric (end-to-end pipelines)
• Experience supporting AI/ML or advanced analytics use cases

AI / Modern Data Experience (Highly Preferred)

• Experience with RAG pipelines and LLM-based systems
• Knowledge of:
  • Prompt engineering
  • Semantic search / vector databases
  • Retrieval pipelines and chunking strategies
• Familiarity with frameworks like:
  • LangGraph, CrewAI, or similar agent frameworks
• Experience building production AI/data workflows (not just POCs)

Nice to Have

• Experience integrating financial or enterprise SaaS platforms
• Exposure to regulated environments (security, compliance, governance)
• Experience working with offshore or distributed teams

What Success Looks Like

• Data pipelines are reliable, scalable, and monitored
• New systems integrate faster and more efficiently
• Data is easily accessible and trusted across teams
• AI and analytics teams can move faster with clean, usable data
• The platform becomes a core part of business operations

Job Tags

Full time

Similar Jobs