Data Engineer

About Us: LTIMindtree is a global technology consulting and digital solutions company that enables enterprises across industries to reimagine business models, accelerate innovation, and maximize growth by harnessing digital technologies. As a digital transformation partner to more than 700 clients, LTIMindtree brings extensive domain and technology expertise to help drive superior competitive differentiation, customer experiences, and business outcomes in a converging world. Powered by nearly 90,000 talented and entrepreneurial professionals across more than 30 countries, LTIMindtree — a Larsen & Toubro Group company — combines the industry-acclaimed strengths of erstwhile Larsen and Toubro Infotech and Mindtree in solving the most complex business challenges and delivering transformation at scale. For more information, please visit www.ltimindtree.com . Job Title: Azure Databricks Engineer Location: Calgary Canada Job Description: Job Overview CLIENT is seeking a skilled Data Engineer to join our team responsible for providing advanced support troubleshooting and performance optimization for our data platforms including Azure Data Factory ADF Databricks Azure SQL Git and other emerging tools as needed This role will focus on ensuring the efficient operation of data pipelines resolving complex production issues optimizing workflows and implementing longterm fixes You will also contribute to key data engineering projects aimed at enhancing CLIENTs data capabilities Key Responsibilities Advanced Support Troubleshooting Issue Resolution Root Cause Analysis Provide support for troubleshooting and resolving issues related to Azure Data Factory ADF Databricks SQL and Git Pipeline Troubleshooting Diagnose and resolve issues in ADF pipelines including challenges related to data movement transformations and orchestration failures Databricks and ADF Optimization Identify and resolve performance bottlenecks in Databricks Spark jobs and notebooks ensuring optimal execution times efficient resource usage and costeffective operations SQL Performance Tuning Troubleshoot and resolve complex SQL query performance issues applying optimization strategies to improve query efficiency Version Control Support Provide support for Git version control processes resolving conflicts and ensuring proper versioning of code and configuration for seamless collaboration Performance Tuning Optimization Root Cause Analysis Analyze and resolve performance issues in ADF pipelines and Databricks jobs recommending and implementing improvements for speed and reliability Job Optimization Optimize Spark jobs in Databricks and ADF pipelines to enhance performance manage memory efficiently and reduce operational costs Pipeline Scalability Fault Tolerance Ensure that data processing pipelines are scalable faulttolerant and optimized for high performance and resilience Project Support Expert Guidance for Projects Act as a subject matter expert on data engineering projects providing guidance and best practices for designing and implementing data processing workflows Collaborative Solution Delivery Work with crossfunctional teams to develop and deliver endtoend data engineering solutions including the creation of proofofconcepts PoCs and the implementation of new ETL pipelines Data Integration ETL Contribute to the integration of data across various platforms and assist with ETL tasks using Azure Data Factory and Databricks Cost Alignment Optimization CostEffective Solutions Design and implement data processing workflows and pipelines that optimize resource utilization ensuring costeffective solutions without compromising performance or scalability Cloud Resource Management Continuously monitor and assess the usage of cloud resources eg Azure Data Factory Databricks SQL databases to identify opportunities for reducing costs such as scaling resources appropriately or leveraging more efficient services Cost Analysis Reporting Work with management and stakeholders to assess the costs associated with data engineering workflows providing insights on potential savings and ensuring alignment with the organizations budgetary constraints BudgetConscious Development Advocate for and implement practices that prioritize cost savings such as optimizing Databricks Spark jobs for memory management and leveraging Azure Reserved Instances or spot pricing where applicable Documentation Knowledge Sharing Maintain Troubleshooting Documentation Document troubleshooting steps common solutions and best practices for addressing recurring technical issues Internal Knowledge Sharing Create and maintain a knowledge base and technical guides to support best practices in data engineering and ensure teamwide consistency Process Improvements Share insights and lessons learned from troubleshooting and performance optimizations to help refine workflows and improve team efficiency Collaboration Communication CrossFunctional Collaboration Work closely with data engineers developers technical leads and architects to resolve complex technical issues ensuring smooth operation and optimization of data platforms and pipelines Stakeholder Engagement Effectively communicate with internal stakeholders including business teams and management to provide updates on issues progress and resolution timelines Establish clear expectations and action plans Reporting Documentation Prepare detailed reports and status updates for leadership summarizing ongoing issues improvements and upcoming changes to keep stakeholders informed Knowledge Sharing Team Collaboration Foster a culture of continuous learning by sharing knowledge best practices and process improvements with the team enhancing the overall performance and efficiency of the data engineering Benefits/perks listed below may vary depending on the nature of your employment with LTIMindtree (“LTIM”): Benefits and Perks: Comprehensive Medical Plan Covering Medical, Dental, Vision Short Term and Long-Term Disability Coverage 401(k) Plan with Company match Life Insurance Vacation Time, Sick Leave, Paid Holidays Paid Paternity and Maternity Leave The range displayed on each job posting reflects the minimum and maximum salary target for the position across all US locations. Within the range, individual pay is determined by work location and job level and additional factors including job-related skills, experience, and relevant education or training. Depending on the position offered, other forms of compensation may be provided as part of overall compensation like an annual performance-based bonus, sales incentive pay and other forms of bonus or variable compensation. Disclaimer: The compensation and benefits information provided herein is accurate as of the date of this posting. LTIMindtree is an equal opportunity employer that is committed to diversity in the workplace. Our employment decisions are made without regard to race, color, creed, religion, sex (including pregnancy, childbirth or related medical conditions), gender identity or expression, national origin, ancestry, age, family-care status, veteran status, marital status, civil union status, domestic partnership status, military service, handicap or disability or history of handicap or disability, genetic information, atypical hereditary cellular or blood trait, union affiliation, affectional or sexual orientation or preference, or any other characteristic protected by applicable federal, state, or local law, except where such considerations are bona fide occupational qualifications permitted by law. Safe return to office : In order to comply with LTIMindtree’ s company COVID-19 vaccine mandate, candidates must be able to provide proof of full vaccination against COVID-19 before or by the date of hire. Alternatively, one may submit a request for reasonable accommodation from LTIMindtree’s COVID-19 vaccination mandate for approval, in accordance with applicable state and federal law, by the date of hire. Any request is subject to review through LTIMindtree’s applicable processes.