Introduction
You build the data pipelines that AI systems for a safe, connected world are built on.
Tasks
- You design and implement scalable data pipelines for ingestion, ETL/ELT, as well as batch and streaming use cases.
- You develop and maintain data models, schemas, catalogs, and feature stores for reliable and discoverable data access.
- You ensure data quality, validation, lineage, and observability for high-performance real-time sources.
- You integrate and operate storage solutions such as object stores, SQL/NoSQL and vector databases, and optimize access patterns for product systems.
- You ensure reproducibility and metadata tracking in ML workflows through experiment and feature lineage tracking.
- You automate CI/CD, monitoring, alerting, and capacity planning for pipelines, thereby ensuring operational reliability.