Data Engineer - Cairo, مصر - Andela

وصف

• Design, build, and optimize data engineering pipelines to integrate and stream data from different sources and implement data ingestion routines using ETL/ELT processes.
• Implement methods to improve data reliability and quality.
• Develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling.
• Defining and setting development, testing, release, update, and support processes for data engineering operation, troubleshooting techniques, and fixing code bugs
• Develop the Big Data models/use cases based on the architecture and make them ready for data operations end users.
• Able to create and execute queries to structure and unstructured data sources to identify process issues or to perform mass updates, preferred.
• Ensures that batch production scheduling and report distribution are accurate and timely.
• Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
• Performs ad hoc requests from users such as data research, file manipulation/transfer, research of process issues, etc.