Senior Data Engineer

New York Full Time Live

Required Skills

My Compatibility Score

Choose Match Score option:

hadoop Java infrastructure Kafka production Python rdbms Software Engineer Spark SQL
Automatch with LinkedIn
As a Senior Data Engineer, you'll support data infrastructure services, like Hadoop, Kafka, Vertica, etc., from building and configuration to maintenance and curation. You'll be a technology evangelist, helping people throughout the organization use data to make informed decisions and draw accurate conclusions. You will work closely with analysts, data scientists, and developers to make data processing transparent and provide data services that help drive and support business goals.

The most exciting part about working here is the enormous potential for personal and professional growth. We are always seeking new and better tools to help us meet challenges such as adopting proven open-source technologies to make our data infrastructure more nimble, scalable and robust. Some of the cutting edge technologies we are currently implementing are Kafka and Spark.

What you'll be doing:
• Design and build efficient, reliable and scalable enterprise level distributed transactional data processing systems
• Optimize jobs to utilize Kafka, Hadoop, Vertica, Spark Streaming resources in the most efficient way
• Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc.)
• Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build / deploy tools and datasets that fit their use cases)
• Collaborate within a small team with diverse technology backgrounds
• Provide mentorship and guidance to junior developers

What you'll need:
• BA / BS degree in Computer science or related field
• 5+ years of software engineering experience developing software, including 3+ years for distributed production systems i.e., Hadoop
• Fluency in Java, experience in Python is a huge plus
• Strong understanding of RDBMS, SQL
• Passion for engineering and computer science around data
• Willingness to participate in 24x7 on-call rotation for production support read more