Senior Data Engineer - Databricks

The Mission

At Caylent, a Senior Data Engineer is an integral part of a cross-functional delivery team implementing data management solutions on AWS for our customers. You will work hands-on to define RDS data sources, develop ETL jobs, and manage data warehouses and lakehouse platforms, including Databricks-based environments. You will participate in daily standups with your team and in regular agile ceremonies with the customer. Your manager will meet with you weekly for a 1:1 to guide your career and help you make the most of your time at Caylent.

Your Assignment

• Deliver complex data platform components or migration solutions independently
• Troubleshoot performance, scalability, and reliability issues
• Contribute to solution design and engineering standards
• Communicate technical topics clearly to peers and stakeholders
• Produce high-quality documentation and implementation reviews

Required Technical Competencies:

Platform Development Focus:

• Strong experience building scalable cloud-native data platforms and pipelines
• Advanced knowledge of modern data lake/lakehouse practices
• Hands-on experience with distributed processing (Spark/PySpark or equivalent)
• Hands-on experience with Databricks, including notebooks, job orchestration, Delta Lake, and working knowledge of Unity Catalog, or a comparable managed lakehouse platform
• Experience with streaming or event-driven ingestion patterns
• Python with solid engineering principles (OOP, testing)

Migration & Modernization Focus:

• Strong expertise in relational databases and schema design
• Advanced query optimization and performance troubleshooting
• Experience executing large-scale migrations and modernization efforts, including migrations onto Databricks
• Knowledge of HA/DR patterns, security, and cost optimization
• Automation and scripting for migration workflows

Across Both:

• Infrastructure-as-code and CI/CD integration
• Cloud security fundamentals
• Cost-aware design

Experience Requirements:

• 4+ years of RDBMS or big data engineering experience
• 2+ years of AWS-based implementations
• 1+ years of hands-on Databricks experience (or equivalent lakehouse platform)

Language:

• Upper-Intermediate English (minimum B2)

Benefits

• 100% remote work
• Generous holidays and flexible PTO
• Competitive phantom equity
• Paid for exams and certifications
• Peer bonus awards
• State of the art laptop and tools
• Equipment & Office Stipend
• Individual professional development plan
• Annual stipend for Learning and Development
• Work with an amazing worldwide team and in an incredible corporate culture