Senior Data Engineer

Job Title: Senior Data Engineer Location: Birmingham, UK Employment Type: Full-Time Experience: 7 Years Job Description: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team in Birmingham. This role offers an exciting opportunity to work on challenging projects that drive our data distribution initiatives and contribute to the evolution of our platform architecture. The ideal candidate will have a strong background in managing and optimising databases, deep expertise in Java programming, and the ability to mentor and guide junior team members. Key Responsibilities: Data Management & Optimisation: Design, implement, and optimise robust database solutions, ensuring high performance and scalability. Backend Development: Utilise Java, J2EE, Spring Boot, MVC, and Microservices to develop and maintain backend systems, ensuring seamless data flow and integration. Security Implementation: Apply security principles and frameworks like Spring Security to safeguard data and systems. Data Warehousing: Build and manage data warehousing solutions using technologies like Redshift, Snowflake, and BigQuery. Big Data Technologies: Work with big data technologies such as Hadoop, Spark, and Kafka to process and analyse large datasets. Project Build & Dependency Management: Use Maven and Gradle for efficient project build and dependency management. Cloud Integration: Leverage cloud platforms (AWS, Azure, Google Cloud) and their data services for scalable data solutions. ETL Processes: Design and implement ETL processes using tools like Talend, Informatica, and Apache Nifi. Team Collaboration: Mentor junior team members and contribute to the overall platform architecture's growth and evolution. Agile Methodology: Participate in Agile/Scrum processes, ensuring timely and effective project delivery. Version Control & CI/CD: Utilise version control systems (e.g., Git, Jira) and CI/CD pipelines (e.g., Jenkins, GitLab CI) for efficient development and deployment. Testing & Debugging: Implement unit and integration tests, and apply best practices in debugging, exception handling, and logging. Qualifications & Experience: Minimum of 7 years of hands-on experience in programming languages, with a strong preference for Java (Scala/Python also considered). Extensive experience with backend development using Java, J2EE, Spring Boot, MVC, Microservices, and Rest API. Proficiency in working with relational databases and SQL scripting (e.g., Sybase, Single Store, MySQL, PostgreSQL, SQL Server). Experience working with multiple file formats such as Parquet, JSON, and XML. Familiarity with Docker and Kubernetes is a plus. Strong understanding of big data technologies, data warehousing, and cloud platforms. Experience with Agile/Scrum methodologies. Proficiency in version control systems and CI/CD pipelines. Knowledge of ETL tools and processes. Experience in writing unit and integration tests. Preferred Qualifications: Experience with Spark and Kafka. Experience with Single Store databases. Knowledge of cryptography and authorisation/authentication practices. Expertise in DB session management, transaction handling, and related concepts. Familiarity with Test-Driven Development (TDD) or Behaviour-Driven Development (BDD). If you are a passionate Scripting Developer looking to contribute to innovative projects within a leading organisation, we encourage you to apply. You may apply via LinkedIn or send your CV to Shweta.BhushanApextechno.co.uk. We look forward to welcoming you to our team at Apex UK.