A day in my data Engineering life

Updated: Jun 1, 2022

Identify the Business mission and insights expected for a minimum viable product for an initiative.
Obtain As-Is & To-Be Scenarios ; must-have and good-to-have features of the product.
Gather current challenges, pain points and devise mitigation strategies.
Understand Timeline criticality, propose Estimate and communicate the go-forward plan of execution.

Interrogate the data: Gap analysis to map customers’ business and technical requirements with reality , present quality reports of the findings and signoff on the mockup or data definition template of the end product.
Define and design an end-to-end deterministic, robust ,scalable and reliable solution architecture.
Design Best Practices including boilerplates setups to speed the development.
Lead/implement the data integration pipelines to collect, manage, and convert raw data into usable information for data scientists and business analysts.
Automate , orchestrate, monitor, error log and alert setup defined per the agreement.
Build and unit Test data ingestion and processing pipelines.
Look for ongoing improvements that drive new insights for the customer.
Shape the product for customer readiness.
Support and maintain data pipelines, and provide expertise and best practices recommmendation.

Work closely with Business partners, customers, stakeholders, professional services, supporting groups, Data Scientist and other engineering groups for defining, designing, and delivering solutions
Design/Implement change strategies of deviations, assumptions, process measures, based on new discoveries.
Track delivery timelines and quality standards while providing responsive customer support
Recurring meet ups with involved teams for Integration and show stoppers.
Ongoing assessment of risks to customer success; measure and communicate the impact.
Take part in software engineering lifecycle activities including sprint planning, backlog grooming, code reviews etc.
Interface with customer, Internal team and Enterprise on Technical presentations, Demos , Learning sessions to enable efficient utilization of the built solutions.

Consistently seeking opportunities where I can leverage my existing skills
Look for opportunities to learn new skills in the ever evolving Tech stack
Product evaluations and POV: Comparative study to understand the unique features of the vendor products that offer similar functionality with the focus on priority of organization , support of customer use cases and the tool set that complement and add values to the strategic visions.
Questions the how and why to look for improvements.
Strive to get better with the process and methodologies.
Quickly adapt to new tools and methodologies.
Experiment the same design with different products and options to combine products for faster and efficient results.

Furnish Guidance on projects over technology, processes & applications
Provide Technical Leadership in establishing plan, sequence of operations with the supporting team.
Facilitate knowledge sharing and gathering within the team and Technology partners
Aid Business integrate strategy and align with the roadmap

Recent Posts