Sr. Data Engineer

SENIOR DATA ENGINEER DATA OVERVIEW OF RESPONSIBILITIES:

The Senior Data Engineer role is responsible for overseeing the data engineering activities, including but not limited to building the business’ data collection systems and processing pipelines, building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data Science and Analytics departments.

The Senior Data Engineer is involved in various phases of data science initiatives, starting from data acquisition, data collection, building and configuring the data connectors to funnel data from various heterogenous sub-systems to the corporate’s data warehouse, data cleansing and merging, creating and maintaining the dimensional models on the data warehouse, creating and maintaining necessary database constructs (such as views, functions, stored procedures), providing the necessary pipelines to feed the corporate’s visualization tools.

The Senior Data Engineer works closely with the data science leadership as well as other data and analytics teams in leveraging data with reporting and scientific tools, for instance, Tableau, Python, SnowFlake, among other tools and technologies. The Senior Data Engineer strives to continuously develop new and improved data engineering capabilities.

DESCRIPTION OF DUTIES:

• Maintain and build on top of our data warehouse and analytics environment, the home for almost all of the data.
• Design, implement, test, deploy, and maintain stable, secure, and scalable data engineering solutions and pipelines in support of data and analytics projects, including integrating new sources of data into our central data warehouse, and moving data out to applications and affiliates.
• Build reports and data visualizations, using data from the data warehouse and other sources.
• Produce scalable, replicable code and engineering solutions that help automate repetitive data management tasks.
• Perform one-off data manipulation/munging and analysis on a wide variety of data.
• Implement and monitor best in class security measures in our data warehouse and analytics environment, with an eye towards the evolving threat landscape.
• Help troubleshoot other data engineers’ SQL, Python, or R code.
• Other duties as assigned.

QUALIFICATIONS:

• BS in related field or equivalent work experience (MS is preferred for this role).
• Strong command of relational databases and SQL. Extract, Transform, and Load (ETL) data into a relational database.
• Proficiency with Python or R, especially for data manipulation and analysis, and ability to build, maintain and deploy sequences of automated processes with these tools.
• BS in related field or equivalent work experience (MS is preferred for this role).
• General data manipulation skills: read in data, process and clean it, transform and recode it, merge different data sets together, reformat data between wide and long, etc.
• Demonstrated ability to learn new techniques and troubleshoot code without support, ex. find answers to common programming challenges on Google. In other words, be able to learn on the job.
• Demonstrated ability to write clear code that is well-documented and stored in a version control system (we use Git).
• Demonstrated ability to work independently and be a self-starter.
• Excellent listening, interpersonal, communication and problem solving skills.
• Demonstrated ability to work effectively in teams, in both a lead and support role.
• Use APIs to push and pull data from various data systems and platforms.
• Experience working with cloud infrastructure services like Amazon Web Services and Google Cloud is preferred but not required.
• Effective time management skills, including demonstrated ability to manage and prioritize multiple tasks and projects.
• Experience with SnowFlake is required.
• Experience with advanced data visualization and mapping are required.