Senior Data Engineer

Job Description

Number of Positions: 1 Duration: 12 months initial contract

Must be eligible to work in Canada

Hybrid position, 2-3d/w onsite in Toronto mandatory, 7hr/d billing, no exceptions

The Senior Data Engineer needs to have strong cloud technologies and a minimum of 5 years of hands‑on Databricks experience. As well, this individual needs to have strong ETL background. The resource will be working on a major data platform migration and transformation initiative.

Key Responsibilities

• Design and implement robust batch and streaming ingestion pipelines on Databricks using Apache Spark (PySpark/Scala).


• Utilize advanced Databricks features like Delta Live Tables (DLT) or Workflows, and Auto Loader for scalable file ingestion.


• Model and manage datasets with next‑generation open table formats, preferably Apache Iceberg, registered in Unity Catalog.


• Implement end‑to‑end data governance, lineage, and access controls using Unity Catalog.


• Build robust, observable data pipelines, including implementing monitoring, alerting, and logging mechanisms.


• Take a leadership role in defining Software Engineering best practices for code standardization, branching strategy, optimization, and design patterns.


• Collaborate with Solution and Enterprise Architects to design complex data transformation solutions.


• Optimize cluster performance and cost using appropriate cluster types and tuning Spark jobs.

Must have skills/experience/qualifications

• At least 10 years of strong experience in Data Integration and ETL pipeline development.


• Minimum 5 years of hands‑on experience building data pipelines with Databricks and Apache Spark (PySpark/Scala) in cloud environments.


• Strong understanding of open table formats, with a preference for Apache Iceberg (snapshots, time travel).


• Experience with data governance using Unity Catalog.


• Proficiency in orchestration/scheduling and familiarity with Delta Live Tables or similar frameworks.


• Solid SQL and data modeling skills.


• Experience with streaming ingestion (Spark Structured Streaming) and version control (Git) in a CI/CD environment.


• Familiarity with cloud object storage (AWS S3 / Azure Data Lake / GCS).

Seniority level

Mid‑Senior level

Employment type

Contract

Job function

Information Technology

Industries

IT Services and IT Consulting, Insurance, and Financial Services

Toronto, Ontario, Canada CA$80,000.00-CA$130,000.00