The IT Hub South Africa is seeking an AWS Data Engineer to drive innovation in global IT solutions. The role involves building and maintaining advanced Big Data pipelines, ensuring data integrity and security. The ideal candidate should have expertise in Terraform, Python, SQL, PySpark, Boto3, and ETL processes, as well as experience in Docker, Linux/Unix, and Big Data platforms. Proficiency in handling data formats like Parquet, AVRO, and JSON is essential, along with experience with Data Quality Tools like Great Expectations and REST APIs. Familiarity with enterprise collaboration tools and an Agile working environment is also required. Experience in AWS Glue, Data Pipeline, and components like S3, DynamoDB, Lambda, and CloudWatch is preferred. Strong analytical skills, data modeling expertise, and precise documentation are crucial for handling large data sets and ensuring data validation.
