Job Details

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field (Engineering or Math preferred).

Technical Skills:

• Programming & Tools:

10+ years of experience in SQL, Python, .NET is a plus.

• 5+ years of experience in Azure cloud services, including:
• Logic Apps

• 5+ years of experience in GIT and deploying code using CI/CD pipelines.

• Certifications (Preferred):

Databricks Certified Data Engineer Associate or Professional.

Soft Skills:

• Strong analytical and problem-solving skills.
• Excellent communication and interpersonal skills.
• Ability to work independently and collaboratively within a team.
• Attention to detail and a commitment to delivering high-quality work.

Responsibilities:

• Create and manage scalable data pipelines to collect, process, and store large volumes of data from various sources.
• Data Integration:

Integrate data from multiple sources, ensuring consistency, quality, and reliability.

• Database Management:

Design, implement, and optimize database schemas and structures to support data storage and retrieval.

• ETL Processes:

Develop and maintain ETL (Extract, Transform, Load) processes to ensure accurate and efficient data movement between systems.

• Data Warehousing:

Build and maintain data warehouses to support business intelligence and analytics needs.

• Performance Optimization:

Optimize data processing and storage performance for efficient resource utilization and quick data retrieval.

• Documentation:

Create and maintain comprehensive documentation for data pipelines, ETL processes, and database schemas.

• Monitoring and Troubleshooting:

Monitor data pipelines and systems for performance and reliability, troubleshooting and resolving issues as they arise.

Stay updated with emerging technologies and best practices in data engineering, evaluating and recommending new tools and technologies as appropriate.