Senior Data Engineer (366 views)

Woodstock, Cape Town
30 Jul 2020

Senior Data Engineer

 

Location: Woodstock, Cape Town

Salary: R960k – R1.2m per annum negotiable depending on experience

 

About the Role:

Opportunity available for a Senior Data Engineer reporting to the Head of Search and Data

 

Requirements:

2+ years’ experience working with server-less services such as GCP or AWS

4+ years’ experience building and optimizing ‘big data’ data pipelines, architectures and data sets

A successful history of manipulating, processing and extracting value from large disconnected datasets

Build processes supporting data transformation, data structures, metadata, dependency and workload management

Strong analytic skills related to working with unstructured datasets

Working knowledge of message queuing, stream processing and highly scalable ‘big data’ data stores

Technologies used:

AWS: S3, EC2, RDS, Redshift, Lambdas, CloudWatch etc

Python

Jenkins

Github

Terraform

 

Duties:

Create and maintain optimal data pipeline architecture

Designs data integrations and data quality framework

Assemble large, complex data sets that meet functional/ non-functional business requirements

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Keep our data separated and secure across national boundaries through AWS security services

Writes unit/integration tests, contributes to engineering wiki, and documents work

As a team member:

Continuously keeping the team up to date with possible changes

Report progress in daily stand-ups

Weekly sprint meetings

Reporting:

Weekly meeting with the Head of Search and Data to align, share learnings and suggest improvements

Participate in the scrum process and daily stand-ups

Seasoned communicator as regards to technical and business/ functional aspects of assigned work

Be comfortable working in an agile and fast-paced environment

Salary: Negotiable

Don't have time now?
Get a reminder in your inbox