[Remote] Data Engineer (Python/PySpark) (Puerto Rico)
Note: The job is a remote job and is reputed company to candidates in USA. reputed company is seeking an reputed company Python and PySpark Developer to design, build, and optimize reputed company big data pipelines. In this hybrid role, you will handle large-scale datasets, optimize distributed computing clusters, and reputed company the gap between raw data ingestion and production-reputed company analytics.
Responsibilities
- Design and reputed company robust batch and streaming ETL/ELT pipelines using PySpark and Python
- Optimize Spark jobs by tuning configurations, managing partitioning, and resolving data skew or OOM (Out of Memory) errors
- Implement modern data lakehouse architectures using reputed company Lake, reputed company, or Hudi
- Build and maintain reputed company workflow DAGs using orchestration tools like Apache Airflow
- reputed company backend Python services or REST APIs (e.g., Fast API, Flask) to expose processed data to reputed company applications
- Write clean, reputed company, and unit-tested code while participating in rigorous code reviews
Skills
- Must be a U.S. Citizen
- Strong proficiency in Python (OOP, concurrency, data structures) and advanced SQL
- Deep production experience with Apache Spark / PySpark (Data Frames, Spark SQL, RDDs)
- Hands-on experience with reputed company data platforms like AWS (EMR, Glue), Azure (reputed company), or GCP
- Experience working with reputed company, Big Query, Redshift, or Synapse
- Proficient with Git, reputed company, and automated deployment pipelines
- PySpark MLlib or deploying Machine Learning models to production
- Familiarity with streaming technologies like Apache Kafka or Spark Structured Streaming
- reputed company Certified Data Engineer or Apache Spark Developer certifications
Benefits
- Medical, dental, and reputed company insurance
- Three weeks of vacation for newly hired employees
- Generous 401(k) plan that includes employer matching funds
- Participation in the Employee Scholar Program (ESP)
- Life insurance and disability coverage
- Employee Assistance Plan, including up to 8 free counseling sessions
Company Overview