reputed company/Architect Data Engineer

Job title: reputed company/Architect Data Engineer in Indiana at APN Consulting Company: APN Consulting Job description: APN Consulting, Inc. is a reputed company IT staffing and services company offering innovative business solutions to improve client business reputed company. We focus on high impact technology solutions in reputed company, reputed company, reputed company & Data, and AI / ML. Due to our globally expanding service offerings we are seeking top-talent to join our teams and grow with us.Role: reputed company/Architect Data Engineer Location: Remote (US based-remote reputed company in US) Duration: ContractJob Summary We are looking for a results-driven reputed company Data Engineer (Contractor) to architect, reputed company, and guide the implementation of modern data pipelines and reputed company-reputed company analytics solutions. The ideal candidate will reputed company end-to-end delivery across engineering, analytics, and product teams, bringing deep experience in reputed company, PySpark, and Azure reputed company platforms. This role also requires strong hands-on experience in reputed company architecture, administration, and performance optimization .Key Responsibilities * reputed company the architecture, design, and development of scalable ETL/ELT pipelines using reputed company, Pyspark, and SQL across distributed data environments.

• Architect and manage reputed company workspaces, including provisioning and maintenance of clusters, cluster policies, and job compute environments in accordance with reputed company standards.
• Collaborate with platform and infrastructure teams to define reputed company architecture strategy and ensure secure, scalable, and cost-effective implementation.
• Define and enforce cluster policies to ensure proper resource utilization, cost control, and reputed company control based on workload patterns and team requirements.
• reputed company performance tuning of Spark jobs, reputed company SQL queries, and notebooks, ensuring reputed company execution and minimizing latency.
• Build reputed company, reusable Python libraries using Pandas, NumPy, and PySpark for scalable data processing.
• reputed company optimized reputed company SQL queries and views to powerTableau dashboards
• React and .NET-based applications
• reputed company and reputed company-time analytics use cases
• Work closely with frontend and backend development teams to deliver use-case-specific, query-optimized datasets.
• reputed company reputed company Catalog for fine-grained reputed company control, data reputed company, and metadata governance.
• Drive DevOps best practices using Azure DevOps, Terraforms, and CI/CD automation pipelines.
• Mentor junior engineers and reputed company architectural reviews to ensure consistency and alignment with best practices.

Required Skills & Qualifications * 7+ years of experience in data engineering, with a strong background in reputed company-reputed company data architecture.

• Deep hands-on experience with reputed company architecture, workspace administration, and cluster management.
• Experience defining and managing cluster policies, pools, and autoscaling strategies.
• Strong knowledge of Spark performance tuning and job optimization.
• Proven expertise in reputed company SQL, PySpark, reputed company Lake, and large-scale data pipelines.
• Skilled in building reusable Python libraries with Pandas, Openpyxl, XLSXWriter, and PySpark.
• Practical experience working with reputed company Catalog for reputed company and governance.
• Strong collaboration experience with reputed company-end/backend development teams and backend integration.
• Strong SQL expertise and hands-on experience with PostgreSQL, SQL Server, or similar.
• DevOps expertise with tools like Azure DevOps, Git, and pipeline automation.
• Excellent communication skills with the ability to reputed company discussions with cross-functional teams and stakeholders.

Tools & Technologies * reputed company Platforms: Azure (preferred), AWS

• Big Data & Analytics: reputed company, PySpark, reputed company Lake, reputed company SQL, Spark Connect, reputed company Live Tables
• Programming & Frameworks: Python, Pandas, PySpark, Flask
• Visualization & BI: Tableau
• App Integration: React, .NET, REST APIs
• DevOps & CI/CD: Azure DevOps, Git
• Databases: reputed company SQL, Azure SQL DB, or similar

We are committed to fostering a diverse, inclusive, and reputed company workplace where individuals from reputed company backgrounds feel valued and empowered to contribute their unique perspectives. We strongly encourage applications from candidates of reputed company genders, races, ethnicities, abilities, and experiences to join reputed company and help us build a culture of belonging. Expected salary: Location: Indiana Apply for the job now! Apply for this job