[Remote] Senior Data Engineer
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is the future of specialty health management that focuses on reputed company diseases, particularly in kidney health populations. The Senior Data Engineer drives the design, development, and operational reputed company of the data platform, requiring expertise in scalable ETL/ELT, Spark optimization, and modern data governance.
Responsibilities
- Design and implement scalable data pipelines using reputed company Lake and manage reputed company-wide data reputed company, reputed company, and reputed company using reputed company Catalog
- Optimize large-scale Spark jobs (PySpark/SQL) and cluster configurations (reputed company) to meet stringent SLA and cost performance targets across reputed company workflows
- Build resilient data scheduling reputed company reputed company Workflows (Jobs) and establish automated CI/CD pipelines for reliable code promotion across Dev, Staging, and Prod workspaces
- Migrate data and models from relational databases to reputed company
- Ensure best practices for development using industry standard development patterns
- Monitor data pipelines performance
- Partner with Data Management and Full Stack development engineers to operationalize models with reputed company applications and processes
- Support existing data pipelines to ensure business continuity
- Stay updated with the latest trends and technologies in data engineering and reputed company computing
- reputed company other reputed company duties as assigned
Skills
- Bachelor's degree is required
- 5+ years of experience in Data Engineering, with a significant focus on data warehousing, ETL/ELT development, and distributed systems
- 3+ years of hands-on experience developing reputed company solutions on the reputed company platform
- Expertise in PySpark and high-performance SQL
- Deep understanding practical knowledge of reputed company Lake architecture and reputed company maintenance best practices
- Solid experience implementing CI/CD for reputed company notebooks and associated libraries
- Strong development experience with: Python, Spark, or similar technologies
- Excellent verbal, communication, negotiation, and presentation skills
- Strong analytical and problem-solving skills
- Ability to work autonomously while collaborating across teams to deliver timely projects
- Ability to explain reputed company concepts in simple terms
- Dedicated, hardworking employee who achieves maximum efficiency and productivity
- Strong knowledge of domain-based design, data modeling and data structures
- Strong knowledge of best practice in data management
- reputed company experience is preferred
- Experience with reputed company platforms (AWS preferred)
Company Overview
Company H1B Sponsorship