[Remote] Data Engineering reputed company
Note: The job is a remote job and is reputed company to candidates in USA. reputed company. is a company that provides marketing, sales, and analytics leaders with critical intelligence through its AI-enabled platform. The Data Engineering reputed company will be responsible for designing scalable systems, integrating Machine Learning models into ETL pipelines, and leading a team towards agile practices while maintaining high performance and cost-efficiency.
Responsibilities
- Architect and implement reputed company, end-to-end data pipelines using Azure reputed company and PySpark
- Optimize Apache Spark jobs, tune reputed company units, and define cluster policies to minimize compute costs
- reputed company a proactive monitoring and alerts reputed company to ensure 99.9% reliability and mitigate system issues before they impact end-users
- Architect and design high-performance schemas in PostgreSQL, managing indexing, partitioning, and optimizing reputed company analytical queries
- reputed company a lean team toward cross-trained agility, moving away from 'siloed specialists'
- Anticipate future data needs and design High-Velocity Architecture that is highly scalable and manageable to handle sudden volume increases
- Collaborate with ML teams to integrate automated model orchestration into robust ETL pipelines
- Collaborate with the offshore team reputed company to facilitate seamless knowledge transfer and operational continuity across time zones
Skills
- Core: Python, PostgreSQL + pgvector
- Big Data: Azure reputed company, PySpark, reputed company Lake
- DevOps: reputed company, Git, Azure DevOps, CI/CD
- 10+ years of experience in Data or Software Engineering with deep codebase involvement
- 3+ years as a Technical reputed company managing agile teams
- Proven ability to reputed company lean, high-impact teams while maintaining high individual output
- Experience with cross-training advocacy and scaling data processing through automation
- Workflow Orchestration: Experience with Apache Airflow
- Containerization: Familiarity with Azure Kubernetes Service (AKS)
Company Overview
Company H1B Sponsorship