EY - GDS Consulting - AI And DATA -MS Fabric -Senior

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.

EY GDS – AI & Data

Microsoft Fabric Senior Data Engineer

AI-enabled data platform engineering

The opportunity

Our AI & Data practice helps clients build trusted, modern data platforms that power analytics, operational decisions, and responsible AI. As a Microsoft Fabric Senior Data Engineer, you will own end-to-end data products and technical workstreams, from design through production operation. You will combine strong engineering fundamentals with practical use of native AI capabilities on Microsoft Fabric, always prioritising business value, security, and reliable production operation.

Your key responsibilities

Data engineering and platform delivery

• Build governed analytics solutions using OneLake, Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Real-Time Intelligence, notebooks, and Power BI semantic models.
• Develop reliable batch and streaming pipelines using Python, PySpark, SQL, Spark notebooks, eventstreams, KQL where relevant, and deployable engineering practices.
• Design and own scalable batch and streaming data products, selecting appropriate data models, processing patterns, quality controls, and service-level expectations.
• Troubleshoot complex performance, reliability, and cost issues; establish reusable engineering patterns and guide code reviews and delivery standards.
• Translate business requirements into reusable, tested data products with clear ownership, contracts, documentation, data-quality checks, and service-level expectations.

AI-ready data and native AI capabilities

• Use native Fabric AI capabilities—Copilot in Fabric, AI Skills, Fabric data agents, and Azure AI integrations—where they are suitable for governed analytics and AI experiences.
• Design AI-ready data products and implement production patterns for retrieval, semantic search, evaluation, safety, and reliable operation.
• Assess when a governed RAG or agent workflow is justified and implement the supporting ingestion, metadata, access-control, evaluation, and deployment foundations.
• Develop and test scoped system prompts, context-assembly patterns, agent skills, and approved MCP integrations with clear tool contracts, least-privilege access, input/output validation, and traceability.
• Partner with data scientists, analytics teams, security, and business stakeholders to select the right pattern: deterministic analytics, semantic layer, retrieval-augmented generation (RAG), agent workflow, or model-based solution.
• Ensure AI solutions have documented data sources, access controls, quality thresholds, evaluations, human oversight where needed, and clear release controls.

Governance, security, and engineering excellence

• Implement governance across Microsoft Purview, OneLake security, workspace/domain design, sensitivity labels, lineage, data quality, semantic models, and capacity optimization.
• Implement automated testing, source control, code reviews, CI/CD, release controls, monitoring, alerting, incident learning, and clear runbooks.
• Work in Agile teams and communicate progress, dependencies, risks, and design decisions clearly to technical and non-technical stakeholders.
• Lead technical delivery for a workstream, mentor engineers, and communicate design choices and delivery risks to client stakeholders.

Skills and attributes for success

• Hands-on Python, PySpark, SQL, Spark notebook engineering, Delta/Parquet data formats, Power BI semantic modelling, and DAX or Power Query knowledge.
• Microsoft Fabric capacity, workspace, deployment pipeline, Git integration, Azure DevOps, and CI/CD experience.
• Practical experience implementing or supporting RAG, semantic search, agent workflows, context engineering, system prompts, agent skills, MCP integrations, or LLM evaluation would be an added advantage.
• Strong analytical problem solving, written and verbal communication, and a consulting mindset grounded in measurable client outcomes.
• Experience with API-based integrations, data contracts, security-by-design, and cloud-native identity, networking, and secrets-management concepts.
• Ability to explain trade-offs between batch, streaming, SQL, Python, PySpark, platform-native AI, and external AI services.

Experience and qualifications

• 4–6 years of relevant data engineering, analytics engineering, platform engineering, or comparable consulting experience.
• Bachelor’s or master’s degree in computer science, engineering, information systems, data science, or a related discipline; equivalent practical experience will be considered.
• Demonstrable delivery of production data solutions using Python, PySpark, and SQL. Experience with modern data modelling, distributed processing, and cloud data security is essential.
• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600) or comparable certification is preferred.

Ideally, you will also have

• Experience in financial services, consumer, healthcare, public sector, or another regulated industry.
• Experience with data mesh, domain ownership, event-driven architecture, or FinOps practices.
• Evidence of building accessible, inclusive teams and improving engineering practices through reusable accelerators, standards, or coaching.

EY | Building a better working world

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.

Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.

Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.