52 day(s) ago

Data Integration Engineer

Negotiable Salary


Worldwide
English: Advanced, Upper Intermediate, Native Speaker
Experience: 3+ years
Employment: Full-time

Syndigo

Waukesha, WI
Our story begins in 1973 when Gladson was founded to help independent retailers arrange their products on the shelf to generate greater sales. As an early adopter of “space management” concepts Gladson demonstrated an entrepreneurial spirit that continues today in Syndigo. During the past 45 years, we have evolved into the leading product information management and digital asset management solution, enabling clients to increase in-store and eCommerce sales using detailed product information, and optimize in-store and shelf layouts to facilitate path to purchase.



Beginning in 2017, under a new leadership team working to deliver on an updated vision, we expanded our offerings to serve the broader brand, distributor, and retailer ecosystem, including industrial and retail foodservice, grocery, consumer goods, hardlines and automotive. This journey included building and acquiring additional detailed and verified product information solutions to help consumers and operators in their buying decisions; store optimization services for effective in-store layouts and shelf merchandising; syndication of data to GS1 global standards via GDSN; publishing of enhanced product content integrated into retail sites globally; and interactive tools to allow restaurant and foodservice brands to organize and share nutrition data with their customers.

Our clients all benefit from Syndigo’s integrated platform, Content Experience Hub – which enables them to collect, store, manage, audit, syndicate and publish their content through our solutions across the largest trading network in the world.

General overview of the project(s)

The Data Integration Engineer is responsible for implementing data ingestion, validation, and transformation pipelines at Syndigo. In collaboration with the Product, Development, and Enterprise Data teams, the Data Integration Engineer will design and maintain batch and streaming integrations across a variety of data domains and platforms. The ideal candidate is experienced in big data, cloud architecture, and is excited to advance innovative analytics solutions!

The preference is that this role be based in one of Syndigo's offices in Brookfield WI, Nashville TN, Chicago IL, or Lisle IL, but well qualified candidates in other parts of the United States can be considered as fully remote employees.

Responsibilities

Work with stakeholders to define and develop data ingest, validation, and transform pipelines
Participate in solution and architecture design & planning
Troubleshoot data pipelines and resolve issues in alignment with SDLC
Ability to diagnose and troubleshoot data issues, recognizing common data integration and transformation patterns
Estimate, track, and communicate status of assigned items to a diverse group of stakeholders

Requirements

3+ years experience in developing large scale data pipelines in a cloud environment
Demonstrated proficiency in Scala (Object Oriented Programming) / Python (Scala preferred), SPARK SQL
Experience with Databricks, including Delta Lake
Experience with Azure and cloud environments, including Azure Data Lake Storage (Gen2), Azure Blob Storage, Azure Tables, Azure SQL Database, Azure Data Factory
Experience with ETL/ELT patterns, preferably using Azure Data Factory and Databricks jobs
Fundamental knowledge of distributed data processing and storage
Fundamental knowledge of working with structured, unstructured, and semi structured data
Excellent analytical and problem solving skills
Ability to effectively manage time and adjust to changing priorities
Bachelor’s degree preferred, but not required

Similar Jobs