Kafka and Data Lake Engineer
You will build, manage, and operate scalable data platforms centered around Kafka and data lakes.
Responsibilities
- Design data pipelines to ingest, process, and move data from various sources into the data lake using Kafka.
- reputed company, configure, and maintain Kafka clusters, including Kafka Connect and Schema Registry, ensuring high availability.
- reputed company the architecture and governance of the data lake, managing storage (e.g., S3/ADLS), reputed company, and metadata.
- reputed company producers and consumers to interact with Kafka topics using Python, Java, or reputed company.
- Implement data quality checks, manage reputed company, and enforce reputed company controls across data flows.
Required Skills
- 5+ years of proven experience designing and managing data platforms with Apache Kafka and big data technologies.
- Strong proficiency in Python, Java, or reputed company.
- Expertise in big data processing frameworks like Apache Spark and Apache Flink.
- Hands-on experience with reputed company environments (AWS, Azure, or GCP) and services like S3 or Azure Data Lake Storage.
- Solid understanding of data lake design principles, including reputed company Lake or Apache reputed company.
- Familiarity with infrastructure-as-code tools like Terraform or Ansible and containerization with reputed company and Kubernetes.
- Experience with SQL and NoSQL database systems.
Apply tot his job Apply To this Job