Data Engineer

JOB DESCRIPTION

The Role

• Work closely with AI engineers to understand infrastructure needs and bottlenecks.

• Implement and maintain cloud infrastructure solutions in accordance with good practice and departmental policy.

• Monitor and understand resource utilization, make recommendations, manage optimization and scaling to ensure resources are used efficiently and costeffectively.

• Design infrastructure solutions using Kubernetes, Terraform, Serverless functions like lambdas, SQS, S3 and other services.

• Develop and maintain data pipelines that enable efficient processing and storage of data for AI applications.

• Ensure data security, compliance, and governance standards are met across all infrastructure.

About You

• Deep experience in infrastructure engineering, DevOps or large scale data platforms.

• Expert in Infrastructure-as-code (Terraform, CloudFormation) and experience with moving legacy infrastructure into state.

• Strong AWS experience including creating and maintaining VPCs, security groups, network ACLs, subnets, & NAT gateways with a focus on creating private infrastructure with controlled access.

• Knowledge of containerization, scripting languages (Bash) including Python

Experience with database technologies and data warehousing solutions relevant to AI applications.

Understanding of data governance principles and experience implementing data security measures.

• Ability to collaborate effectively with cross-functional teams and communicate technical concepts clearly.

• Problem-solving mindset with a focus on creating scalable, resilient infrastructure solutions.