- Collaborating with team members and stakeholders to design data integration solutions that meet business requirements, utilising Microsoft Azure tools such as Azure Data Factory and SQL.
- Develop, implement, and maintain ETL pipelines to ensure efficient and secure data transfer between various systems, adhering to best practices for data processing and transformation.
- Design and maintain data models, ensuring data accuracy, consistency, and availability while considering scalability and performance optimizations.
- Implement data validation and quality checks to identify and rectify data anomalies, errors, and inconsistencies within integrated systems.
- Continuously monitor and optimise data integration processes to enhance performance, reduce latency, and minimize resource consumption.
- Create and maintain comprehensive documentation for data integration solutions, including technical specifications, data flow diagrams, and operational procedures.
- Work closely with cross-functional teams, including developers, and system analysts, to understand data requirements and ensure integrations align with business objectives.
- Troubleshoot and resolve data integration issues, working collaboratively with the team to implement effective solutions and prevent recurrence.
Skills and experience required:
- Strong Microsoft Azure Data Factory skills and experience
- Proven experience in data engineering, including data integration, ETL processes, and data modelling.
- Strong SQL skills and experience with data warehousing concepts.
- Knowledge of data governance, security, and compliance best practices.
- Excellent problem-solving skills and the ability to work effectively in a collaborative team environment.
- Effective communication skills for collaborating with technical and non-technical stakeholders.
- Demonstrated ability to build and maintain effective working relationships.
- Ability to work independently and be a self-motivated individual.
Desirable
- Microsoft Azure tools and services, Azure SQL Database.
- Bachelor’s degree in Computer Science, IT or related field (or equivalent experience).
- Relevant certifications (e.g., Microsoft Azure Data Engineer, AWS Certified Data Analytics) are advantageous.
Job Type: Full Time