Job Description

Project Overview

• Our client is embarking on a significant project to transition from our legacy Informatica system to a new data strategy tool, DataConnect, built on Spark and Hadoop technologies. This transition aims to enhance our data processing capabilities and streamline our operations.

Need Behind the Project

• Our current Informatica system, while functional, is not meeting our evolving data processing needs. We require a more robust and scalable solution to handle large volumes of data efficiently. DataConnect will provide the necessary infrastructure to support our growing data requirements and improve performance.

Primary Responsibilities

• Building Data Pipelines: Develop and maintain data pipelines using Informatica, ensuring seamless data transfer and integration.
• Data Transfer: Facilitate the migration of data from Informatica to DataConnect, ensuring data integrity and consistency.
• Oracle and Data Storage: Support the transition to Oracle 19c (Exadata) for enhanced data storage solutions.
• Performance Tuning: Optimize queries and data storage to handle large volumes of data efficiently.
• Legacy System Support: Maintain and support legacy Python code and Informatica systems as needed.
• Interim Development: Work on interim solutions involving Hadoop and Spark, ensuring continuity during the transition.
• Data Processing: Manage large datasets, tuning performance of queries, and ensuring efficient data processing.

Requirements

• Data Pipeline Development: Proven experience in building and maintaining data pipelines.
• Informatica Knowledge: Familiarity with Informatica and its functionalities.
• Oracle Expertise: Experience with Oracle 19c and performance tuning.
• Python Skills: Ability to work with legacy Python code.
• ETL Background: Strong background in ETL processes and tools like Alteryx.
• Data Handling: Experience in managing and processing large volumes of data.
• Day-to-Day Activities
• Pipeline Development: Design, develop, and maintain data pipelines using Informatica & DataConnect.
• Data Migration: Facilitate the migration of data from Informatica to DataConnect.
• Performance Optimization: Tune queries and optimize data storage solutions.
• Legacy Support: Provide support for legacy systems and interim development projects.
• Collaboration: Work closely with team members to ensure smooth transition and integration of new systems.
• Monitoring and Maintenance: Continuously monitor data pipelines and systems to ensure optimal performance and address any issues promptly.

Compensation:

$55/hr to $65/hr.

Exact compensation may vary based on several factors, including skills, experience, and education.

Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.

Job Tags

Similar Jobs