Big Data Engineer
108 w 39 New York, New York H1A 0
Job Title: Lead Big Data Engineer
Location: New York, NY
Salary: Commensurate with Experience
- Develop and implement a highly scalable Hadoop cluster for enterprise applications
- Build secure, highly-scalable and reliable big data platforms to consume, integrate and analyze complex data
- Work closely with developers and architects to deploy new Hadoop infrastructure, resolve cluster and job performance issues.
- 5-7 years of experience with the Hadoop and Big Data technologies
- Hands-on experience with Hadoop i.e. HDFS, Hbase, Hive, Apache Spark, Scala, Java, Phoenix
- Experience building secure, highly-scalable, reliable and big data platforms to consume, integrate and analyze complex data using a variety of open-source platforms and tools
- Experience with Kafka and scripting and automation skills in Java or Python
- Knowledge of handling large data loads and implementing OLAP and OLTP systems.
- Experience with implementation of distributed computing systems and architecture and tools such as Apache Storm, Akka
- Experience developing Hadoop integrations for data ingestion, data mapping and data processing capabilities