Data Engineer - Contract

Data Engineer – Security Clearance, Redshift, AWS, Glue, Python, SQL, Remote

 

A multi award winning Data Consultancy is for several contract Data Engineers to join their growing team. Working on a multitude of different projects across the public sector you will be assisting our client in a key growth period.

You will help lead the development of a secure real-time data platforms in getting data from a variety of different sources, getting it into the right format, assuring alignment to target models, and ensuring that their users can get the data quickly for analysis.

As a Data Engineer your duties will include:

 

• Create and maintain optimal data pipeline architecture
• Write unit/integration tests, contribute to documentation via collaboration tools,
• Conduct logical and physical database design
• Conduct data analysis, identifying feasible solutions and enhancements to data processing challenges
• Ensures that data models are consistent with the data architecture
• Performing root cause analysis on internal and external
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.

 

Core skills for the role:

 

• Postgres
• Python
• AWS Redshift
• AWS Glue
• Security clearance is must
• Experience with troubleshooting, root cause analysis and issue resolution in data pipelines.
• Experience with data pipeline and workflow management tools
• Experience with stream-processing systems
• Experience with designing, building, testing and deploying systems in Python.

• Experience with big data tools: Hadoop, Spark, Kafka, etc.