Data Engineer

Philadelphia, PA 19103

Posted: 07/29/2020 Job Category: Big Data Engineer Job Number: 143891

Job Description


Title: Data Engineer
Company: Health Tech
Location:  Philadelphia, PA

WHAT YOU WILL BE DOING:
  • Load data into data warehouse
  • Troubleshoot and resolve issues relating to data integrity
  • Help establish procedures and best practices for transforming and storing data
  • Work with some of the most exciting open-source tools like Spark, Hadoop, Docker, Airflow, Zeppelin
  • Lead requirements gathering around data pipeline automation improvements
  • Leverage distributed computing and serverless architecture such as AWS EMR & AWS Lambda, to develop pipelines for transforming data
  • Solve complex problems related to the real-time discovery of large data
  • Watch  your creation make  it into production quickly
  • Research and implement new technologies with a team of developers to execute strategies and implement solutions
  • Produce peer reviewed quality software

WHAT WE WANT TO SEE:
  • Experienced in writing scalable applications on distributed architectures
  • Experience with your work making it to production
  • Comfortable on the command line and consider it an essential tool
  • Extremely confident in SQL, You know it like the back of your hand!

QUALIFICATIONS:
  • 5+ years of work experience
  • 3+ years of experience with Python
  • 3+ years of experience with PySpark and Spark-SQL (writing, testing, debugging spark routines)
  • 1+ years of experience with AWS EMR, AWS S3 service. Comfortable using AWS CLI and boto3
  • Comfortable working in remote environments
  • Comfortable using *nix command line (shell scripting, AWK, SED)
  • Experience with MySQL and Postgres

BONUS POINT FOR EXPERIENCE WORKING WITH:
  • Apache Airflow
  • Apache Zeppelin
  • Healthcare data

Send an email reminder to:

Share This Job:

Related Jobs:

Login to save this search and get notified of similar positions.