10 days old
2018-01-082018-02-05

Big Data Engineer

Hoboken, NJ 07030
  • Job Code
    594905
  • Payrate
    $70 To $85

What you'll do


We need super smart engineers from all levels to help us build one of the best engineered e-commerce platform in the world (big talk we know, but that is our goal!). Our engineers combine creativity, curiosity, and drive to continuously perfect and revolutionize us from the inside out. We are looking to bring more intellectually curious engineers who are passionate about technology in general (we are a technology first company and prides itself on its culture of learning and knowledge sharing and we want all our engineers to be as passionate as we are!)


The Environment


Our platform is largely an event driven platform implemented via a microservice architecture. The platform runs on Microsoft Azure and uses a large number of technologies and middle ware. The bulk of backend code is written in a functional style (F#) and our system of record is an event sourced system (essentially a log of all actions on the platform). We use a mix of Kafka, Redis, Elastic Search, Azure SQL, Event Store and Azure Storage for our data stores. About 50% of the platform runs on Linux and the rest Windows (although we move into more containerization we will see a shift towards Linux).


About the Job


We are looking for an exceptional Data Engineer to help build a world class analytical platform to collect, store and expose both structured and un-structured data generated by a vastly growing system landscape.


You can expect a freewheeling, informal work environment, populated by a combination of folks from top companies that have produced many successful products, as well as some PhD's that have escaped the ivory tower.


Because we work on cutting edge technologies, we need someone who is a creative problem solver, resourceful in getting things done, and productive working independently or collaboratively.


What you will do:



  • Design, implement and manage a near real-time ingestion pipeline into a data warehouse and Hadoop data lake.
  • Gather and process raw data at scale - collect data across all business domains (our functional-first, event sourced, micro services backend) and expose mechanisms for large scale parallel processing
  • Process unstructured data into a form suitable for analysis and then empower state-of-the-art analysis for analysts, scientists, and APIs
  • Support business decisions with ad hoc analysis as needed
  • Evangelize an extremely high standard of code quality, system reliability, and performance.
  • Influence cross functional architecture in sprint planning


What you will have:



  • Minimun four years of experience in running, using and trouble shooting the Apache Big Data stack i.e. Hadoop FS, Hive, HBase, Kafka, Pig, Oozie, Yarn
  • Programming experience, ideally in Scala or F# but we are open to other experience if you're willing to learn the languages we use
  • Proficient scripting skills i.e. unix shell and/or powershell
  • Experience processing large amounts of structured and unstructured data with MapReduce


We use Azure extensively, so experience with cloud infrastructure will help you hit the ground running

Categories

  • Information Technology

Randstad utilizes a technology-driven focus with a human touch to provide better staffing and business solutions to organizations around the world. Our team of experts match professionals with available career opportunities in a variety of fields.

Featured Jobs

Career News

Before you go...

Our free job seeker tools include alerts for new jobs, saving your favorites, optimized job matching, and more! Just enter your email below.

Share this job:

Big Data Engineer

Randstad Technologies
Hoboken, NJ 07030

Share this job

Big Data Engineer

Randstad Technologies
Hoboken, NJ
US

Separate email addresses with commas

Enter valid email address for sender.

Join us to start saving your Favorite Jobs!

Sign In Create Account
Powered ByCareerCast