Data Architect / Senior Data Engineer (Azure Databricks)

About Zoral
Zoral is an IT product and professional services company serving banks, insurers, wealth managers and fintechs. Alongside its flagship product, Zoral fOS (Financial Operating System), Zoral delivers data, analytics and AI programmes for regulated financial-services firms in Europe, the UK, the US and Asia.

About the role
We are looking for a Data Architect / Senior Data Engineer to define the target architecture and actively design, develop and deliver enterprise data platforms on Azure Databricks for regulated financial-services clients. Typical platforms integrate tens of source systems and millions of customer and policy records, support several hundred regulatory, management and operational reports, and are delivered by growing multidisciplinary teams of data engineers, architects, analysts and client specialists.

This is a hands-on architecture and engineering role within a collaborative delivery team. You will assess the client's existing data landscape, help refine the target architecture, establish modelling and engineering standards, and act as a senior technical authority throughout delivery. You will work closely with other architects and engineers, providing direction, reviewing designs and supporting implementation rather than operating as a standalone architect.

At the same time, you will remain close to the engineering work: developing transformation logic and data models, profiling and cleansing data, implementing data-quality controls, and contributing to production engineering where your experience has the greatest impact. The successful candidate should be comfortable moving between architecture, technical leadership and hands-on implementation while enabling the wider team to deliver effectively.

Responsibilities

Assess existing data lakes, warehouses, pipelines, models, jobs, notebooks and tooling; work with client architects and the wider delivery team to confirm or refine the target architecture and document decisions, options, trade-offs and rationale.
Design the layered data architecture (landing, bronze, silver and gold) and guide its implementation across the engineering team, covering ingestion, standardisation, transformation, integration, conformance, data products and reporting marts.
Develop and review production-grade data transformations using SQL and Python/PySpark, including reusable and metadata-driven pipelines that are idempotent, re-runnable, testable and observable.
Define data-modelling standards and contribute to the development of integrated and consumption-layer models, including Data Vault 2.0, Kimball dimensional models, slowly changing dimensions, historical and as-at reporting, and model-generation approaches.
Define the data-quality and data-cleansing approach together with the engineering team: establish profiling and baseline measures; define rule catalogues, quality gates and scorecards; design quarantine, remediation, reconciliation, audit, balance and control processes; and ensure these are implemented consistently.
Guide and contribute to data cleansing, standardisation, de-duplication, customer matching and entity-resolution approaches, including match rules, survivorship and persistent identifiers.
Design how the new platform is separated from and coexists with legacy platforms, including catalogue and environment layout, access boundaries, dependencies, migration, reconciliation and decommissioning.
Define practical standards for layering, naming, ownership, classification, lineage, security, masking, resilience and retention, and work with engineers to ensure these standards are applied consistently in delivery.
Evaluate tools for ingestion, transformation, orchestration, data quality, catalogue, lineage, master data and reporting, working with other technical stakeholders to present recommendations and trade-offs.
Review engineering designs and code, contribute directly to complex or critical implementation areas, troubleshoot difficult data and pipeline issues, and support automated testing and CI/CD practices.
Provide technical leadership to engineers and client staff through design reviews, pairing, mentoring and collaborative problem solving.
Produce architecture documents, decision records, technical standards, runbooks and handover material in collaboration with the wider project team.

Requirements

8+ years in data architecture and/or senior data engineering, with substantial hands-on delivery responsibility, including at least three years designing and engineering Databricks platforms in production.
Demonstrated ability to operate as both an architect and a senior engineer within a delivery team: define target-state designs and standards, guide other engineers, and implement or materially contribute to pipelines, transformations and data models.
Experience providing technical leadership within multidisciplinary data teams, including design reviews, mentoring, engineering standards and collaborative delivery.
Expert SQL and strong Python/PySpark skills, with experience developing, testing, optimising and supporting production data transformations.
Hands-on knowledge of Azure Databricks, including Unity Catalog governance (grants, row filters, column masks, tags and lineage), Delta Lake, Lakeflow Declarative Pipelines, Lakeflow Jobs and Lakeflow Connect.
Proven experience designing and developing layered lakehouse or data-warehouse architectures using Data Vault 2.0 and Kimball dimensional modelling, including slowly changing history and as-at data.
Experience defining and implementing data-quality and cleansing strategies, including profiling, rule design, quality gates, monitoring, scorecards, quarantine, remediation and reconciliation controls.
Experience with master data management, customer matching, de-duplication or entity resolution.
Experience migrating data, pipelines and models from legacy warehouses or lakes, including coexistence, reconciliation, parallel running and decommissioning.
Practical experience with Git, code review, automated testing, CI/CD and production support for data pipelines.
Ability to write clear architecture and engineering documentation and present technical decisions to senior stakeholders.
Excellent written and spoken English; willingness to work on site at client premises, including in the UK, for periods of several weeks.