Databricks Engineer -Hybrid -Montreal, QC / Mississauga, ON Job at W3Global, Montréal, QC

eE1DT0ZqTC9EeWhVWDRtMnNlQ0tjb20vWnc9PQ==
  • W3Global
  • Montréal, QC

Job Description

Role: Databricks Engineer

Job Summary

We are seeking highly skilled Azure Data Engineer with strong expertise in Databricks to join our data

team. The ideal candidate will design, implement and optimize large-scale data pipeline, ensuring

scalability, reliability and performance. This role involves working closely with multiple teams and

business stakeholders to deliver cutting-edge data solutions.

Key Responsibilities

Data Pipeline Development:

  • Build and maintain scalable ETL/ELT pipelines using Databricks.
  • Leverage PySpark/Spark and SQL to transform and process large datasets.
  • Integrate data from multiple sources including Azure Blob Storage, ADLS and other

relational/non-relational systems.

Collaboration & Analysis

  • Work Closely with multiple teams to prepare data for dashboard and BI Tools.
  • Collaborate with cross-functional teams to understand business requirements and

deliver tailored data solutions.

Performance & Optimization

  • Optimize Databricks workloads for cost efficiency and performance.
  • Monitor and troubleshoot data pipelines to ensure reliability and accuracy.

Governance & Security

  • Implement and manage data security, access controls and governance standards using

Unity Catalog.

  • Ensure compliance with organizational and regulatory data policies.

Deployment

  • Leverage Databricks Asset Bundles for seamless deployment of Databricks jobs,

notebooks and configurations across environments.

  • Manage version control for Databricks artifacts and collaborate with team to maintain

development best practices.

Technical Skills

  • Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table

Triggers, Delta Live Pipelines, Databricks Runtime etc.)

  • Proficiency in Azure Cloud Services.
  • Solid Understanding of Spark and PySpark for big data processing.
  • Experience in relational databases.
  • Knowledge on Databricks Asset Bundles and GitLab.

Preferred Experience

  • Familiarity with Databricks Runtimes and advanced configurations.
  • Knowledge of streaming frameworks like Spark Streaming.
  • Experience in developing real-time data solutions.

Certifications

  • Azure Data Engineer Associate or Databricks certified Data Engineer Associate certification.

(Optional)

Job Tags

Similar Jobs

会社名非公開

Sales Manager Job at 会社名非公開

As a Sales Manager, you will drive sales growth and manage client relationships within the industrial and manufacturing sectors. Based in Tokyo, you will play a pivotal role in expanding the company's footprint across Japan and the APAC re... 

Benesse BE studio

Sunday English instructor in Tokyo Job at Benesse BE studio

 ...rt provided and dedicated mentors to help you on an individual basis. Video lessons offered to students and instructors to practice the lesson content Transportation costs fully covered Visa support is available in some cas... 

Saskatchewan Health Authority

Laboratory Scientist I Job at Saskatchewan Health Authority

 ...Chemistry, Microbiology in Microbiology, etc. - Bachelor of Science Degree with a major in defined science & required courses - Baccalaureate...  ...required - Official at: // About US The Saskatchewan Health Authority (SHA) is the largest employer in Saskatchewan,... 

The Walt Disney Company (APAC)

Assistant Manager, Lifecycle Marketing, Disney+ Job at The Walt Disney Company (APAC)

 ...remium video streaming services in the world. It is the dedicated home for streaming movies and shows from Disney, Pixar,...  ... - Ownership of Lifecycle Content Calendar finding key drivers to increase Disney+ user viewership and customer engage... 

Louis Vuitton

Merchandising Associate Job at Louis Vuitton

Louis Vuitton is seeking a Merchandising Associate for its Women's Leather Goods division in New York City. The role involves supporting the merchandising strategy, generating sales reports, coordinating product launches, and providing administrative support. The ideal...