Senior Databricks Engineer

Зарплата: $6000–$7500

Senior Databricks Engineer
The role requires significant daily overlap with the US team during US Eastern Time (EST) working hours. Candidates must be able to work within the required US EST schedule.

TechBar is a software services company working with clients across the US and Europe. We are investor-backed, growing steadily, and known for putting senior, thoughtful engineers in front of our clients.
We are looking for an experienced Senior Databricks Engineer to join an enterprise-level data modernization initiative for a US-based client.
The project focuses on modernizing a large-scale enterprise data platform as part of a broader cloud transformation journey. You will work with the Databricks Lakehouse Platform to build scalable data solutions supporting advanced analytics, AI/ML initiatives, and enterprise-wide data products.
What you'll do
Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Spark SQL
Build robust ETL/ELT processes for large-scale structured and unstructured data
Implement Medallion Architecture (Bronze, Silver, Gold) and Delta Lake solutions
Develop batch and near real-time data processing solutions using Spark Streaming, Kafka, and Delta Live Tables (DLT)
Integrate data from enterprise sources, including SAP, Teradata, Mainframe, DB2, SQL Server, Oracle, Tibco, Kafka, and REST APIs
Work across both Microsoft Azure and AWS environments as part of the client's cloud data modernization program
Develop and maintain Databricks notebooks, workflows, jobs, and clusters
Implement Unity Catalog, access controls, data governance, and security standards
Support deployments and environment promotion across Dev, QA, UAT, and Production
Optimize workloads for performance, scalability, reliability, and cost efficiency
Implement data validation, reconciliation, monitoring, and automated data quality frameworks
Contribute to data lineage, metadata management, compliance, and enterprise data governance
Build reusable frameworks and automation for data ingestion, orchestration, monitoring, and quality management
Work closely with Data Scientists, BI developers, architects, and business stakeholders to translate requirements into scalable technical solutions
Participate in architecture and solution design discussions
Mentor junior engineers and contribute to engineering best practices
Participate in Agile ceremonies and collaborate with distributed engineering teams
Must-have requirements
7+ years of professional experience in Data Engineering
Strong hands-on experience with Databricks
Databricks certification is mandatory: Databricks Certified Data Engineer Associate or Professional
Strong practical experience with both Microsoft Azure and AWS. Experience in only one cloud platform is not sufficient.
Solid hands-on experience with:Databricks Workflows / Jobs
Delta Lake
Delta Live Tables (DLT)
Unity Catalog
Databricks SQL

Strong programming skills in Python, PySpark, Spark SQL, and SQL
Strong understanding of ETL/ELT, data modeling, data warehousing, and distributed data processing
Experience with Apache Kafka, Spark Streaming, and event-driven architectures
Hands-on experience with Azure, particularly:Azure Databricks
Azure Data Factory
ADLS Gen2
Azure Key Vault

Hands-on experience with AWS, particularly:Amazon S3
IAM

Experience working with enterprise databases such as:Teradata
SQL Server
Oracle
DB2

Experience with Git and CI/CD, preferably Azure DevOps or GitHub Actions
Strong communication skills and the ability to work effectively with technical and non-technical stakeholders
Ability to work independently, take ownership, and contribute to architectural decisions
Nice to have
Azure Data Engineer Associate certification
Experience working on large-scale enterprise data modernization programs
Experience migrating workloads from Teradata or other legacy platforms to Databricks
Experience with SAP Datasphere, SAP BW, or SAP Business Data Cloud (BDC)
Experience with Databricks Asset Bundles
Experience with BigQuery or Snowflake
Exposure to AI/ML workloads and feature engineering on Databricks
Experience working with highly governed or regulated enterprise data environments
What makes this project interesting
This is not a project where you simply maintain existing pipelines.
You will be involved in a broader enterprise data transformation, helping move legacy data workloads toward a modern cloud-based architecture. The platform combines Databricks, Delta Lake, Azure, AWS, streaming technologies, enterprise databases, and modern data governance practices.
You will have the opportunity to work on problems involving large data volumes, complex integrations, real-time processing, data quality, governance, and the enablement of analytics and AI/ML use cases.
Working model
Remote
Collaboration with a US-based team
US Eastern Time (EST) working hours / strong overlap is required
Long-term project
English: Upper-Intermediate / Advanced
Senior-level ownership and direct communication with the client
Why TechBar
At TechBar, we believe strong engineering teams are built around people who can think independently, communicate clearly, and take ownership of what they build.
You will work on challenging international projects, collaborate with experienced engineers and client teams, and have the opportunity to make a meaningful impact rather than simply being another person assigned to a ticket queue.
If you are a Senior Databricks Engineer with strong hands-on experience across both AWS and Azure and hold a Databricks certification, we'd be happy to hear from you.