Senior Data Engineer

Company Description

Your mission

• End-to-End Development: Design, develop, and deploy scalable, maintainable, efficient, and reliable applications using Python 3, FastAPI or Flask, with optional use of Angular. Ensure seamless integration of back-end components.
• Big Data Management: Handle (near) real-time big data environments, managing and processing datasets starting from terabytes with precision and efficiency.
• ETL Processes: Design, implement, and manage ETL (Extract, Transform, Load) processes to ensure efficient data handling through its entire lifecycle.
• Performance Optimization: Implement caching services like Redis to improve application performance.
• Data Pipelines: Develop and manage data pipelines using Airflow.
• Mentorship: Provide guidance and mentorship to junior developers, fostering a culture of continuous learning and improvement within the team.
• Architecture & Design: Collaborate with architects and other developers to design robust, high-performance, secure application architectures.
• Collaboration: Work closely with product managers, designers, and other stakeholders to understand requirements, provide technical insights, and ensure the successful delivery of features.
• Code Quality & Best Practices: Write clean, efficient, and well-documented code while setting and maintaining high standards for code quality through code reviews, testing, and automated tooling.
• Containerization: Containerize applications using Docker and manage container orchestration with Kubernetes.
• Infrastructure Management: Create and manage infrastructure using Helm Charts and Terraform.
• Cloud Deployment: Deploy and manage applications on AWS cloud platforms, including S3, ECR, EKS, RDS, and EFS.
• Data Visualization: Create data visualizations using Plotly.
• CI/CD Implementation: Implement CI/CD pipelines using GitHub.
• Troubleshooting & Optimization: Troubleshoot and resolve application issues while optimizing performance.