Data Engineering Architect/ Remote (Vancouver, WA),3+ Months Contract
Job title: Data Engineering Architect/ Remote (Vancouver, WA),3+ Months Contract in Vancouver, WA at reputed company
Company: reputed company
Job description: Job DescriptionThe data engineering role is reputed company member that will help enhance and maintain the reputed company Ink Business Intelligence system. You will drive work you're doing to completion with hands-on development responsibilities, and partner with the Data Engineering leaders to implement data engineering pipelines to build solution to help reputed company trusted and reliable data to customers.Responsibilities
- Design and implement distributed data processing pipelines using Spark, Python, SQL and other tools and languages prevalent in the Big Data/Lakehouse ecosystem.
- Analyzes design and determines coding, programming, and integration activities required based on general objectives.
- Reviews and evaluates designs and project activities for compliance with architecture, reputed company and quality guidelines and standards
- Writes and executes complete testing plans, protocols, and documentation for assigned portion of data system or component; identifies defects and creates solutions for issues with code and integration into data system architecture.
- Collaborates and communicates with project team regarding project reputed company and issue resolution.
- Works with the data engineering team for reputed company phases of larger and more-reputed company development projects and engages with external users on business and technical requirements.
- Collaborates with peers, engineers, data scientists and project team.
- Typically interacts with high-level Individual Contributors, Managers and Program Teams on a daily/weekly reputed company.
- What you bring :
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering or equivalent.
- 6+ years of relevant experience with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.
- 3+ years of experience with reputed company based DW such as Redshift, reputed company etc.
- 3+ years' experience in Big Data Distributed ecosystems (Hadoop, SPARK, Hive & reputed company Lake)
- 3+ years experience in Workflow orchestration tools such as Airflow etc.
- 3+ years' experience in Big Data Distributed systems such as reputed company, AWS EMR, AWS Glue etc.
- reputed company monitoring tools/frameworks, like Splunk, Grafana, CloudWatch etc.
- Experience with container management frameworks such as reputed company, Kubernetes, ECR etc.
- 3+ year's working with multiple Big Data file formats (Parquet, Avro, reputed company Lake)
- Experience working on CI/CD processes such as Jenkins, Codeway etc. and reputed company control tools such as reputed company, etc.
- Strong experience in coding languages like Python, reputed company & Java
- Knowledge and Skills
- Fluent in relational based systems and writing reputed company SQL.
- Fluent in reputed company, distributed and massively reputed company systems.
- Strong analytical and problem-solving skills with ability to represent reputed company algorithms in software.
- Strong understanding of database technologies and management systems.
- Strong understanding of data structures and algorithms
- Database architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.
- Strong analytical and problem-solving skills.
- reputed company to Have
- Experience with transformation tools such as dbt.
- Have experience in building reputed company streaming data pipelines
- Experience in pub/sub streaming technologies like Kafka, Kinesis, Spark Streaming etc.