Observability Engineer - ELK Stack Specialist

We are seeking an experienced Observability Engineer specializing in the ELK (Elasticsearch, Logstash, Kibana) stack to join our cross-functional team. In this role, you will be responsible for maintaining and enhancing our observability and monitoring platform, ensuring optimal performance and visibility across our systems. You will work collaboratively with development, operations, and data teams to implement robust logging, monitoring, and alerting solutions that provide actionable insights into our platform's health and performance. Key Responsibilities: Design, implement, and maintain ELK stack deployments for comprehensive observability across our platform Configure and optimize Elasticsearch clusters for efficient log storage, indexing, and querying Develop and maintain Kibana dashboards and visualizations to provide meaningful insights to various stakeholders Implement automated alerting based on log patterns and metrics to proactively identify issues Collaborate with cross-functional teams to define logging standards and implement observability best practices Troubleshoot and resolve complex issues related to the observability platform Optimize resource utilization and manage costs associated with our observability infrastructure Create and maintain documentation for observability systems and processes Continuously evaluate and implement improvements to enhance our monitoring capabilities Requirements: Elasticsearch Expertise: Strong hands-on experience with Elasticsearch, including cluster management, indexing, querying, and optimizing performance Google Cloud Platform (GCP): Proven experience in deploying, managing, and scaling applications and services in GCP, including working with Google Kubernetes Engine (GKE), and Google Compute Engine (GCE) Networking Knowledge: Solid understanding of networking principles, including TCP/IP, VPNs, DNS, firewalls, load balancing, and troubleshooting network-related issues in a cloud environment Linux Administration: Proficiency in administering Linux-based systems, including installation, configuration, and troubleshooting. Familiarity with system monitoring and log management Automation & Scripting: Experience with automation tools (e.g., Terraform) and scripting languages (e.g., Python, Bash) to manage infrastructure and automate routine tasks Security Awareness: Understanding of security best practices in both cloud and on-prem environments, including user access controls, encryption, and vulnerability management Troubleshooting Skills: Strong problem-solving skills with the ability to diagnose and resolve issues across the stack, from networking and systems to Elasticsearch performance Collaboration & Communication: Ability to work effectively in cross-functional teams, with strong communication skills to explain technical concepts to both technical and non-technical stakeholders What you'll be working on: Observability Platform Enhancement : You'll be joining our team at a crucial time as we enhance our observability platform built on the ELK stack. You'll help design and implement improvements to provide deeper insights into our systems' performance and health. Cross-Functional Collaboration: You'll work closely with development teams to implement proper logging standards, with operations and data teams to ensure system visibility, —all while maintaining a balance between comprehensive monitoring and resource efficiency. Real-time Monitoring and Alerting: You'll develop sophisticated alerting mechanisms that help us identify and address issues before they impact our users, using Elasticsearch's powerful query capabilities and Kibana's visualization tools. Cost Optimization: Following our FinOps approach, you'll help optimize our observability infrastructure to maintain high performance while controlling costs, ensuring we get maximum value from our cloud resources. Join our team and help us build a world-class observability platform that empowers our organization with the insights needed to deliver exceptional service to our customers.