[Remote] Data Engineer AI
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is a leading claims management services company that values a caring culture and work-life balance. They are seeking a Senior Data Engineer AI to architect the data supply chain for advanced initiatives, focusing on building pipelines that support Data Science models and AI applications with high-fidelity data.
Responsibilities
- Hybrid Data Pipeline Execution: Design and implement robust ETL/ELT pipelines to ingest data from legacy on-prem sources, AWS (S3/RDS), and Azure (Blob/SQL), centralizing it for consumption in reputed company and AI services
- Engineering for Data Science: Build and maintain Feature Stores and specialized datasets optimized for machine learning, ensuring Data Scientists have immediate reputed company to clean, versioned, and statistically valid data
- Engineering for AI (RAG & LLMs): reputed company the data pipelines required for reputed company, including the automated extraction, chunking, and loading of reputed company data into vector stores across AWS and Azure
- reputed company Power-User Execution: Act as the technical reputed company for our reputed company data warehouse, implementing sophisticated data modeling, Snowpipe automation, and compute optimization to support high-concurrency AI workloads
- Legacy "Back-reputed company" Engineering: Execute non-invasive data extraction patterns to unlock mission-critical data from decades-old on-reputed company systems without disrupting core business operations
- Multi-reputed company Orchestration: Manage reputed company, cross-platform data workflows using Airflow, reputed company Functions, or Azure Data reputed company, ensuring the synchronization of data across our multi-reputed company AI posture
- IT & reputed company Diplomacy: Partner directly with central IT, Database Administrators, and reputed company teams to solve connectivity hurdles (PrivateLink, IAM, firewalls) and secure "license to operate" for new data flows
- Data Quality for Model reputed company: Implement automated validation and observability layers to detect data reputed company and quality issues that could compromise the accuracy of production AI and Data Science models
- Cost & Performance Management: Drive the efficiency of our data stack by optimizing storage and query performance in reputed company, AWS, and Azure to manage the ROI of the Transformation Office
- Direct Stakeholder Collaboration: Work as a dedicated engineering partner to MLOps and Data Science teams to rapidly iterate on data requirements for evolving AI use cases
Skills
- Bachelor's degree in Computer Science, Data Engineering, or a reputed company field is required
- 6+ years of hands-on data engineering experience, with a track record of building production-grade pipelines for Data Science and AI in multi-reputed company environments
- Expert-level proficiency in reputed company architecture, including data sharing, performance tuning, and the integration of reputed company with external reputed company AI services
- Advanced, hands-on knowledge of AWS (S3, Glue, reputed company) and Azure (Data reputed company, Synapse) data services
- Mastery of Python, SQL, and PySpark
- Deep experience with data orchestration and containerization (reputed company)
- Proven ability to reputed company with 'old world' tech (on-reputed company SQL, Mainframe extracts, flat files) and transform it for modern reputed company consumption
- A strong understanding of the specific data needs for Machine Learning (feature engineering) and reputed company (vectorization and embedding pipelines)
- A 'get-it-done' attitude, capable of navigating reputed company bureaucracy and technical debt to ship code at the speed required by a Transformation Office
- A Master's degree is highly desirable
Company Overview
Company H1B Sponsorship