Skip to content
Contractor on Demand Network

Senior Data Engineer

Absa

n/a - n/a
R900–950
15 Troye Street, City of Johannesburg Metropolitan Municipality, 2001
Data Engineer
Cloud Engineer
Data Engineering
Databricks
Python
SQL
Apache Spark

Role

Role Overview
We are seeking a Senior Data Engineer (senior individual contributor) to design, build, and operate Databricks & Lakehouse data platforms that support analytics, AI, and Generative AI applications.
This role works within product-aligned squads and focuses on delivering high-quality, governed, and scalable data assets consumed by analytics platforms, machine learning models, and GenAI applications including LLM- and agent-based systems.
Key Responsibilities
  1. Data Engineering & Lakehouse Delivery
  2. Build, and maintain data pipelines and lakehouse structures
  3. Deliver data solutions that support:
    1. Analytics and BI
    2. Machine learning workloads
    3. Generative AI applications and agents
  4. Apply enterprise data lake and lakehouse principles to ensure data is:
    1. Reliable
    2. Well-governed and aligned to Absa’s governance
    3. Secure
  5. Fit for downstream consumption
  6. Translate business and analytical requirements into production-ready data solutions
 
Databricks & Platform Usage
  1. Build and operate solutions using Databricks, including:
    1. Delta Lake
    2. Databricks Jobs and Workflows
    3. Unity Catalog
    4. Notebooks and shared libraries
  2. Enable data consumption by:
    1. GenAI use cases (e.g. RAG, AI services, agent workflows)
    2. Analytics and reporting tools
    3. Downstream operational systems
  3. Support feature-style and curated data access patterns required by AI and GenAI workloads
 
Generative AI Enablement
  1. Build data pipelines that feed Generative AI applications, including:
    1. Curated knowledge datasets
    2. Structured and semi-structured data sources
    3. Metadata and lineage required for AI consumption
  2. Enable data patterns commonly used in GenAI, such as:
    1. Retrieval‑Augmented Generation (RAG)
    2. Context and prompt data preparation
    3. Model input, output, and feedback data flows
  3. Work closely with AI Engineers and Product Owners to align data engineering deliverables to GenAI use cases. Note: you will also be involved in AI Engineer development.
 
Engineering Practices
  1. Develop production-grade pipelines using Python, SQL, and Apache Spark
  2. Implement automated testing and CI/CD practices for data workloads
  3. Ensure data solutions are:
    1. Observable
    2. Resilient
    3. Performant
    4. Cost-efficient
  4. Contribute to improving data quality, reliability, and operational stability
 
Collaboration & Ways of Working
  1. Work as a senior engineer within a cross-functional product squad
  2. Collaborate closely with:
    1. Product Owners
    2. AI / ML Engineers
    3. Analytics teams
    4. Platform and security teams
  3. Provide engineering input into design discussions and delivery decisions
  4. Support peer reviews and shared engineering standards
 
Risk, Governance & Run
  1. Ensure data solutions comply with enterprise security, risk, and governance standards
  2. Support operational stability of data pipelines used by analytics and AI workloads
  3. Participate in incident resolution and root cause analysis
  4. Maintain appropriate documentation and runbooks
 
Required Skills & Experience
  1. Proven experience as a Senior / Lead Data Engineer
  2. Hands-on experience working in Databricks environments
  3. Strong understanding of enterprise data lake and lakehouse architectures
  4. Proficiency in:
    1. Python
    2. SQL
    3. Apache Spark
  5. Experience building and operating production-grade data platforms
  6. Experience working in enterprise or regulated environments
 
Desirable Experience
  1. Experience enabling AI, ML, or Generative AI use cases from a data engineering perspective
  2. Familiarity with:
  3. RAG data patterns
  4. Feature-style or AI-serving datasets
  5. Vector or embedding-ready data workflows
  6. Experience working in Agile, product-aligned squads
  7. Exposure to cloud-native data platforms (AWS or Azure)
 
Role Clarification
  1. This is a senior individual contributor role
  2. The role does not include formal people management or technical lead accountability
  3. The focus is on delivery, quality, and enabling AI and GenAI outcomes
Apply
Share

Refer a friend

Enter their email below to share this role with them