Data Engineer – AI/BI

Job Description

• This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Data Engineer – AI/BI in India. This role is centered on building and scaling a modern data and AI platform powered by Databricks, enabling enterprise-wide analytics, AI-driven insights, and self-service BI capabilities. You will design robust data pipelines and governed data architectures based on Medallion principles to transform raw enterprise data into trusted, analytics-ready assets. The position involves working with large-scale integrations from systems such as PeopleSoft, Salesforce, and D2L, ensuring seamless data flow across the organization. You will also play a key role in enabling AI/BI experiences, including dashboards, semantic models, and natural language analytics through Databricks Genie. Operating in a highly technical and collaborative environment, you will contribute to building the backbone for data-driven decision-making and machine learning initiatives. This is a hands-on engineering role with strong impact across data, AI, and analytics ecosystems.
• Accountabilities: Design, build, and optimize end-to-end data pipelines on Databricks using Apache Spark and Delta Lake, following Medallion Architecture (bronze, silver, gold) principles. Develop and operationalize AI/BI solutions including dashboards, semantic models, and Databricks Genie experiences for self-service analytics. Integrate enterprise data sources such as PeopleSoft, Salesforce, and D2L using APIs, JDBC, and other ingestion frameworks. Implement scalable ELT workflows with orchestration, monitoring, schema evolution, and automated error handling. Establish data quality frameworks, validation rules, and observability mechanisms to ensure accuracy, reliability, and performance. Implement data governance, lineage, and metadata management using tools such as Unity Catalog or equivalent solutions. Support AI/ML teams by delivering curated datasets, feature stores, and ML-ready data pipelines using tools like MLflow. Ensure data security, privacy, and compliance through access controls, encryption, masking, and governance best practices. Maintain technical documentation, architecture diagrams, and enablement materials for platform adoption and scalability. Requirements: Strong hands-on experience with Databricks, Delta Lake, and Apache Spark in building scalable data engineering solutions. Proficiency in SQL and Python (or Scala) for data transformation, pipeline development, and workflow automation. Experience implementing Medallion Architecture and working with large-scale cloud data platforms. Proven ability to build and manage ELT pipelines, data ingestion frameworks, and orchestration workflows. Experience integrating enterprise systems such as Salesforce, PeopleSoft, or similar platforms. Understanding of data governance, metadata management, lineage tracking, and data quality best practices. Familiarity with cloud environments such as Azure or AWS, including storage and security concepts. Experience with BI/AI enablement tools such as Databricks AI/BI, dashboards, or semantic modeling is highly desirable. Strong analytical thinking, problem-solving skills, and ability to work in a fast-paced engineering environment. Benefits: Remote-friendly work arrangement based in India with flexibility in working setup. Opportunity to work on cutting-edge Databricks AI/BI and enterprise-scale data platforms. Exposure to advanced data engineering, AI, and analytics technologies in a global environment. Strong focus on learning, technical growth, and hands-on innovation. Collaborative engineering culture focused on high-quality, scalable data systems. Opportunity to work on impactful enterprise data initiatives driving AI and business intelligence. Access to modern cloud and data stack tools and best engineering practices.
• How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether? Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. #LI-CL1
• We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
• apply for this job