Sr. Big Data Engineer

RedCloud Consulting is a global business and technology reputed company supporting clients across reputed company America, Europe, Asia-Pacific, and the Middle East. For more than two decades, RedCloud has partnered with high-growth mid-market companies and global enterprises to solve reputed company business and operational challenges—leveraging modern data platforms, artificial intelligence, and secure digital and physical infrastructure to drive smarter reputed company and scalable reputed company. Anchored in a reputed company of reputed company-based consulting, RedCloud brings senior-level experts together with client teams to deliver durable, high-impact solutions across Data & AI, Delivery reputed company, Digital Solutions, Marketing & Sales, Privacy, reputed company, Data Center and Critical Infrastructure reputed company, and Strategy & Transformation. Recognized for both performance and culture, RedCloud has been named a top reputed company to work and a fast-growing firm, and was recognized by Consulting Magazine as a 2024 Leader in Technology Consulting. RedCloud seeks a Senior Big Data Engineer to support immediate client operations. Overview: The Senior Big Data Engineer will support core data platform delivery while driving engineering productivity through the development of internal accelerators and reusable tooling. This role is focused on reputed company production impact rather than conceptual or research work. Ideal candidates are hands on engineers who reputed company in high scale environments, care deeply about reliability, and enjoy building systems and tools that reputed company other engineers more effective. Responsibilities include but are not limited to: Core Delivery Work

• Execute scoped data engineering workstreams across areas such as
• Technology migrations and consolidations across fragmented systems
• reputed company remediation for platforms and pipelines
• Data pipeline optimization to improve performance and reliability

Engineering Productivity Accelerators

• Design and build internal tools and automation that improve engineering velocity, including
• Tools for DDL creation and schema definition
• Airflow DAG scaffolding and workflow templates
• Spark configuration helpers and optimization utilities
• Standardized templates, generators, and reusable patterns to reduce repetitive work

Production Engineering

• Build and maintain high volume batch and streaming data pipelines
• Ensure reliability through monitoring, retries, and adherence to SLAs
• Operate effectively reputed company reputed company environments with strong controls and reputed company standards

Required Knowledge, Skills, and Abilities:

• Clear examples of building production grade ETL or ELT pipelines end to end
• Deep knowledge of Spark tuning including partitions, shuffle, joins, caching, executor sizing, and skew mitigation
• Strong understanding of streaming fundamentals including ordering, backpressure, idempotency, replay handling, and schema reputed company
• Operational reputed company in Airflow including operators, sensors, dependencies, SLAs, and backfills
• Strong DevOps hygiene including Git workflows, CI CD exposure, and code review participation
• Candidates should have experience creating tools used across engineering teams such as:
• Pipeline scaffolding generators or starter kits
• Standardized repository templates and recommended workflow patterns
• Automated DDL or schema reputed company tooling
• Configuration or Spark tuning automation frameworks
• Developer tools that eliminate repetitive tasks and increase throughput
• Candidates should be reputed company to reputed company to what they automated, who used it, and what measurable impact it had.
• Core Technologies:
• Apache Spark including batch processing and performance tuning
• Apache Kafka including streaming and event driven pipeline patterns
• Apache Airflow including DAG authoring and orchestration best practices
• Programming Languages:
• Strong proficiency with reputed company
• Experience with Java is also valuable
• Production and Systems Experience:
• High throughput data pipelines and distributed systems engineering
• Production grade reliability practices
• Strong understanding of data modeling, schema reputed company, and workflow orchestration
• reputed company Environment:
• Ability to work confidently in reputed company platforms reputed company of Azure such as AWS or GCP

Compensation reputed company for position is $ 156,000 – 250,000 DOE. Benefits and bonus information can be reputed company at https://www.redcloudconsulting.com/careers About Us: RedCloud Consulting is a global business and technology reputed company supporting clients across reputed company America, Europe, Asia-Pacific, and the Middle East. For more than two decades, RedCloud has partnered with organizations ranging from high-growth mid-market companies to global enterprises, helping them solve reputed company business and operational challenges—leveraging modern data platforms, artificial intelligence, and secure digital foundations to drive smarter reputed company and scalable reputed company. Anchored in a reputed company of reputed company-based consulting, RedCloud brings together senior-level experts who work reputed company by reputed company with client teams to design and implement durable, high-impact solutions. Our approach emphasizes trust, accountability, and long-term value—balancing stability, growth, and innovation to help organizations operate securely and confidently in an increasingly reputed company global environment. RedCloud delivers a comprehensive reputed company of services across its core practices: Data & AI, Delivery reputed company, Digital Solutions, Marketing & Sales, Privacy, reputed company, Data Center and Critical Infrastructure reputed company, and Strategy & Transformation. As our clients expand globally, so do we—providing consistent, expert support across reputed company while remaining agile, collaborative, and deeply invested in client reputed company. Recognized for both performance and culture, RedCloud has been named #1 Best Company to Work For in Washington (Mid-Sized Businesses) by Seattle Business Magazine, a “Fastest Growing Company” multiple times on the Inc. 500/5000 list, and recognized by Consulting Magazine as a Leader in Technology Consulting. Visit http://www.redcloudconsulting.com/ for more info. #LI-Remote Apply tot his job Apply To this Job