Back to jobs

220 - Cloud Tech

Senior Software Engineer, Observability

  • Roku
  • New York, New York
  • Full-time
  • Salary not listed

About the role

Teamwork makes the stream work. Roku is changing how the world watches TV Roku is the #1 TV streaming platform in the U.S., Canada, and Mexico, and we've set our sights on powering every television in the world. Roku pioneered streaming to the TV. Our mission is to be the TV streaming platform that connects the entire TV ecosystem. We connect consumers to the content they love, enable content publishers to build and monetize large audiences, and provide advertisers unique capabilities to engage consumers. From your first day at Roku, you'll make a valuable - and valued - contribution. We're a fast-growing public company where no one is a bystander. We offer you the opportunity to delight millions of TV streamers around the world while gaining meaningful experience across a variety of disciplines. About the team Are you passionate about building and evolving large-scale observability platforms? Do you want to work on a system that processes terabytes of data daily and supports hundreds of millions of active series? Are you excited about using AI to change how engineers build, operate, and debug systems at scale? If so, this role is for you! The Observability team is part of Roku's Cloud Technology Infrastructure organization. We ensure the services and workloads running on our platform have robust observability. We are a high-performing, fast-moving international team that thrives on taking ownership, communicating effectively, and iterating quickly to deliver impactful engineering solutions. Our mission is to advance Roku's observability platform, which ingests terabytes of data daily and manages hundreds of millions of active series. We don't just deploy off-the-shelf tools: we use Go and Rust to build our own world-class metrics stack on top of open-source technologies, and we tailor it to Roku's scale and needs. We work closely with the Cloud Platform team to support Roku's shift toward a unified, cloud-agnostic infrastructure where every team benefits from a common framework with out-of-the-box features. Together, we build and scale both the foundational infrastructure (Kubernetes, Istio, Envoy, operators, and more) and the observability layer (built on OSS/CNCF-supported observability projects). AI is part of how we work. We use AI tools and agents throughout our engineering lifecycle: to design, write, review, and test code; automate operational work; and speed up troubleshooting and incident response. We continuously refine these workflows as a team. We bring the same mindset to the platform, where we provide AI-assisted capabilities to the engineers who rely on our observability stack. What sets us apart is our results-driven culture rooted in ownership, accountability, and fast iteration. We prioritize solving meaningful engineering problems over process for process's sake, and focus our time and energy on high-impact work. Collaboration and open communication are core to how we operate. We share ideas freely and make decisions with purpose. We're continuously improving, not just the systems we build, but how we grow and work as a team. About the role As a Senior Software Engineer on the Observability team, you'll help design, develop, and scale the platform used across Roku engineering teams to alert and monitor thousands of services. You'll write Go and Rust to build and evolve our own metrics stack on top of open-source technologies, and you'll work hands-on with tools such as Prometheus, Grafana, Thanos, Loki, Tempo, and OpenTelemetry, optimizing them for performance, reliability, and usability. AI is central to how you'll work: you'll use AI tools and agents daily to build, test, and operate our systems, and help the team keep improving how we engineer with AI. You'll collaborate with infrastructure and platform teams, automate deployment and configuration, and contribute improvements back to the open-source community. This role is ideal for an engineer who's passionate about systems programming, observability at scale, and AI-driven engineering. For California Only - The estimated annual salary for this position is between $251,000 - $403,750 annually. Compensation packages are based on factors unique to each candidate, including but not limited to skill set, certifications, and specific geographical location. This role is eligible for health insurance, equity awards, life insurance, disability benefits, parental leave, wellness benefits, and paid time off. For Illinois Only - The estimated annual salary for this position is between $220,900 - $325,200 annually. Compensation packages are based on factors unique to each candidate, including but not limited to skill set, certifications, and specific geographical location. This role is eligible for health insurance, equity awards, life insurance, disability benefits, parental leave, wellness benefits, and paid time off. For New York Only - The estimated annual salary for this position is between $268,950 - $403,750 annually. Compensation packages are based on factors unique to each candidate, including but not limited to skill set, certifications, and specific geographical location. This role is eligible for health insurance, equity awards, life insurance, disability benefits, parental leave, wellness benefits, and paid time off. What you'll be doing Building Our Metrics Stack: Design and build high-performance components of our metrics stack in Go and Rust, on top of open-source technologies, to handle terabytes of data daily and hundreds of millions of active series. AI-Driven Engineering: Use AI tools and agents as part of your daily development and operations work (writing, reviewing, and testing code, automating operational tasks, and accelerating troubleshooting and incident response), and help shape how the team works with AI. Scaling and Innovating: Enhance and optimize Roku's observability stack while introducing Roku-specific features and innovations, including AI-assisted capabilities for the engineers who use the platform. Tooling and Standardization: Work with the Platform team to identify, standardize, and improve observability tools as part of a cohesive solution. Stability and Performance: Invest in making the observability platform more stable, performant, and reliable for all users. Automation and Usability: Build automation around observability tools to simplify onboarding for developers (dashboards, alerts, traces, etc.). Open Source Contributions: Contribute new open-source tools and improvements back to the CNCF ecosystem. Collaboration and Engagement: Partner with engineering teams to build solutions that deliver a better out-of-the-box experience and drive innovation. Technical Leadership: Lead design discussions, mentor engineers and interns, and help shape the platform's technical direction. We're excited if you have Significant experience (typically around 8 years) in Infrastructure Engineering and/or DevOps, with a focus on observability platforms. AI-Driven Engineering: Hands-on experience using AI tools and agents in your day-to-day development and operations work, and practical experience building with Large Language Models (for example, tool/function calling, agents, MCP, RAG, and evaluating LLM output). Solid programming experience (typically around 4 years) in Go and/or Rust. Experience with performance-sensitive systems is a strong plus. If you're strong in one, we welcome your interest in growing in the other. Large-Scale Expertise: Proven experience designing and operating observability platforms that process massive amounts of data (metrics, logs, and traces) to provide actionable insights. Expertise in deploying, operating, and extending open-source observability tools at scale, such as Prometheus, Grafana, Thanos, Loki, Tempo, or similar solutions like Cortex, Mimir, or the ELK stack. Familiarity with open standards such as OpenTelemetry and OpenMetrics. Deep understanding of Kubernetes and Istio architectures, and the ability to contribute improvements for scalability and integration. Hands-on experience with AWS, GCP, and/or Azure. Strong ability to engage with customers and stakeholders to deliver tailored observability solutions. Education: B.S. or M.S. degree in Computer Science, Engineering, or equivalent experience. Even if you don't match every point listed, we'd love to hear from you if you have the drive to dive in, learn, and make an impact. #LI-RN1 What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work in the office Monday through Thursday. Fridays are generally flexible for remote work, except for employees whose specific roles or assigned office location require five days' a week attendance. What are some of the benefits? Roku is committed to offering a diverse range of benefits as part of our compensation package to support our employees and their families. Our comprehensive benefits include global access to mental health and financial wellness support and resources. Local benefits include statutory and voluntary benefits which may include healthcare (medical, dental, and vision), life, accident, disability, commuter, and retirement options (401(k)/pension). Employees are supported in taking time off, in accordance with local leave policies and other personal needs to support their evolving work and life needs. It's important to note that not every benefit is available in all locations or for every role. For details specific to your location, please consult with your recruiter. Accommodations Roku welcomes applicants of all backgrounds and provides reasonable accommodations and adjustments in accordance with applicable law. If you require reasonable accommodation at any point in the hiring process, please direct your inquiries to EmployeeRelations@Roku.com . What should I know about Roku's culture? Roku is a great place for people who want to work in a fast-paced environment where everyone is focused on the company's success rather than their own. We try to surround ourselves with people who are great at their jobs, who are easy to work with, and who keep their egos in check. We appreciate a sense of humor. We believe a fewer number of very talented folks can do more for less cost than a larger number of less talented teams. We're independent thinkers with big ideas who act boldly, move fast and accomplish extraordinary things through collaboration and trust. In short, at Roku you'll be part of a company that's changing how the world watches TV.  We have a unique culture that we are proud of. We think of ourselves primarily as problem-solvers, which itself is a two-part idea. We come up with the solution, but the solution isn't real until it is built and delivered to the customer. That penchant for action gives us a pragmatic approach to innovation, one that has served us well since 2002.  To learn more about Roku, our global footprint, and how we've grown, visit https://www.weareroku.com/factsheet . By providing your information, you acknowledge that you want Roku to contact you about job roles, that you have read Roku's Applicant Privacy Notice , and understand that Roku will use your information as described in that notice. If you do not wish to receive any communications from Roku regarding this role or similar roles in the future, you may unsubscribe at any time by emailing WorkforcePrivacy@Roku.com .