17 days old
2017-11-072017-12-05

.NET Developer - Big Data/Data Lake

Boston, MA 02114
  • Job Code
    586448

Summary of the Position


This position is with the Massachusetts Executive Office of Energy and Environmental Affairs (EEA). The Senior Software and Data Engineer will be part of an agile development team, building, maintaining and expanding our Enterprise Data Lake that comprised of data from multiple data sources from multiple Departments. The Engineer will work to maintain data flow in the Data Lake and work with other team members to expose data to different applications and data portals.


Primary Responsibilities ? Maintain, clean, improve and manipulate data in Hadoop based data-lake, which comprised of multiple business operational and analytical databases. ? Develop and maintain ETL processes importing data from multiple Operational Data Sources, including Oracle, Microsoft SQL Server and others. ? Analyze and develop data models that fit into enterprise business processes. ? Process unstructured data into suitable forms for analysis. ? Apply machine learning, natural language processing or data mining techniques to analyze structured and unstructured data from multiple sources. ? Use Sqoop, Flume, Hive, Pig/MapReduce. ? Develop and maintain Python scripts working with data in Hadoop and S3. ? Working in team environment and coordinate activities with development team. ? Learn new technologies if requested. ? Exhibit excellent verbal and communication skills.


Qualifications ? Minimum BS/CS with 8+ years or MS/CS with 10+ years of hands-on related work experience. ? 3+ years of hands-on experience with Hadoop ecosystem, including HDFS, EMR, Sqoop, flume, Hive, Pig, MapReduce. ? 3+ years of experience with Hadoop data ingestion and Hadoop transformation. ? 2+ years of proven experience working with, processing and managing large data sets. ? 3+ years of Python programming in conjunction with Big Data. ? 4+ years of experience with C#/.NET/ASP.NET framework. ? Strong experience with data modeling and problem solving skills. ? Deep knowledge of data mining, machine learning, natural language processing or information retrieval. ? Strong knowledge of and experience of statistics and other advance math. ? Prior experience of analysis and resolution of data quality and integration issues. ? Hands-on experience with AWS cloud services. ? Experience in data ingestion technologies with reference to relational databases (Microsoft SQL Server, Oracle). ? Expertise in relational database design and writing stored procedures using T/SQL or PLSQL. ? Middle-tier RESTful web services using .NET Web API framework is a plus. ? Front End development with AngularJS and/or Angular 2-4 is a plus. ? Excellent written and oral communication skills.

Categories

  • Information Technology

Randstad utilizes a technology-driven focus with a human touch to provide better staffing and business solutions to organizations around the world. Our team of experts match professionals with available career opportunities in a variety of fields.

Featured Jobs

Career News

Share this job:

.NET Developer - Big Data/Data Lake

Randstad Technologies
Boston, MA 02114

Share this job

.NET Developer - Big Data/Data Lake

Randstad Technologies
Boston, MA
US

Separate email addresses with commas

Enter valid email address for sender.

Join us to start saving your Favorite Jobs!

Sign In Create Account
Powered ByCareerCast