Senior Data Engineer

Senior Data Engineer Salary- Up to £90k Location- London (Hybrid) This is an exciting opportunity to work with cutting-edge cloud-native data platforms (AWS, Azure, Databricks) and technologies like Spark, Delta, Kafka, and Terraform. You’ll collaborate with cross-functional teams, mentor junior engineers, and drive best practices in data engineering. Key Responsibilities: Design & Build Data Pipelines: Develop and optimize batch and real-time data pipelines using Python, SQL, Spark, and Delta. Data Transformation & Processing: Implement ELT workflows to ingest, enrich, and publish data for analytics and AI/ML. Security & Compliance: Ensure data privacy, governance, and regulatory compliance. Performance Optimization: Tune queries and pipelines for cost-efficiency and speed. Monitoring & Observability: Integrate real-time monitoring to detect and resolve issues proactively. CI/CD & DevOps: Implement Terraform, GitHub, and DevOps pipelines for automation and deployment. Collaboration & Mentorship: Work closely with product owners, analysts, and data scientists, while mentoring junior engineers. Research & Innovation: Stay ahead of the curve with the latest data technologies and best practices. What We’re Looking For: Strong experience in Python & SQL for Databricks and Spark (including Delta, Delta Live Tables, PyTest, Great Expectations). Expertise in batch and streaming data processing using Kafka, AWS Kinesis, or Azure Stream Analytics. Hands-on experience in cloud-native data platforms (AWS, Azure, or Databricks). Knowledge of data governance, privacy laws (e.g., GDPR), and security best practices. Experience with DevOps for data engineering – Terraform, GitHub, CI/CD. Agile mindset – experience working in Scrum/Kanban teams. Ability to mentor junior engineers and conduct code reviews (PR reviews). Please reach out to- daniel.wexlerxcede.com for more information Unfortunately, they do not offer sponsorship