Senior Data Engineer

As a Data Engineer on the Capacity Engineering team, you will help design, build, and operate the data foundation that drives capacity, cost, and power-related decisions across Apple’s infrastructure footprint. In this role, you will: Architect, implement, and maintain large-scale batch and streaming pipelines that ingest, process, and model infrastructure telemetry, cost, metering, utilization, forecasting, and power metrics from multiple clouds and bare metal environments. Design and evolve robust data models (with a strong focus on dimensional modeling) and storage patterns that support analytics, internal billing, and efficiency use-cases. Treat data as a product: define quality checks, SLAs, and observability to ensure data is accurate, timely, and trusted by stakeholders across Apple. Integrate and enrich raw signals with metadata and attribution to power use cases such as internal billing/showback, usage understanding, efficiency and optimization, clawbacks, planning, and procurement. Collaborate closely with data scientists, software engineers, platform teams, finance partners, program managers, and leadership to translate requirements into scalable, reliable data solutions and services. Implement standard methodologies for data governance, lineage, metadata management, and security, in alignment with Apple’s standards for data protection and privacy. Build end-to-end data solutions that include logging, anomaly detection, data validation, cleaning, and transformation, with strong emphasis on monitoring, debuggability, and continuous improvement. Contribute to the evolution of our data and platform stack, including tooling, frameworks, and standards for development, testing, deployment, and operations (CI/CD, infrastructure as code, etc.).

Description

Apple’s Capacity data engineering team, within the Apple Services Engineering organization, is building the centralized data backbone that powers how Apple understands, plans, and optimizes its cloud and data center infrastructure. We engineer a unified, trusted data lake that consolidates cost, metering, utilization, forecasting, and power metrics produced by Apple platforms and systems (including bare metal) across both third-party and Apple internal clouds. Enriched with metadata and attribution, this becomes the single source of truth for internal billing, understanding usage and utilization, clawbacks, planning, procurement, and efficiency initiatives. We collaborate with platform engineering, finance, capacity engineering, and leadership teams to build large-scale data pipelines, enable descriptive and predictive analytics, and power dashboards and products that support critical business decisions. This is your opportunity to help design and operate highly visible, global-scale systems processing petabytes of data and supporting hundreds of users across Apple. Come join us to help deliver the next generation of infrastructure insights at Apple.

Minimum Qualifications

Bachelors degree or equivalent experience in Computer Science, Information systems, Software Engineering, Data Science or related field. 5+ years of experience in data engineering (or equivalent practical experience), including: Building and maintaining large-scale ETL/ELT data pipelines Distributed computing (e.g., Spark / PySpark) for data processing and automation Query performance optimization and tuning at scale Hands-on experience with: Apache Spark and Airflow (or similar workflow/orchestration tools) for efficient large-scale data pipelines Data modeling, especially dimensional modeling, and designing schemas optimized for analytics and reporting Big data platforms and/or data lake architectures Experience with CI/CD tooling such as Jenkins (or similar tools) Experience with cloud technologies, AWS/GCP (e.g., S3, EMR, Lambda, Glue, RDS/Redshift, Big-query, Kinesis or similar services)

Preferred Qualifications

Advanced degree in a related field a plus. Experience with data visualization / BI tools, such as Superset or Tableau (other tools like QuickSight, QlikView, Cognos, or Business Objects are a plus) Experience with containerization and orchestration, such as Docker and Kubernetes/EKS is a plus Understanding of authentication and authorization (AuthN/AuthZ) patterns Knowledge of data governance principles, data security best practices, and data privacy regulations