Senior Data Engineer

San Francisco, CA 94107

Posted: 06/23/2020 Job Category: Engineering Job Number: 138622

Job Description


Title: Senior Data Engineer
Location: San Francisco, CA
Company: PaaS- Beauty & Wellness

** Candidates MUST be a US citizen of have a green card ** 
** No C2C**


As a Senior Data Engineer, you will join an impactful, multi-functional team of data scientists, analysts, data engineers and backend engineers dedicated to creating a data-driven culture.  The Senior Data Engineer will utilize their experience and create appropriate solutions and tools to solve complex data engineering problems. This positions is with  a rapidly scaling PaaS company in the beauty and wellness intustry!

What you will  be doing:
  • Working on  Data & Infrastructure Projects and experience with Big Data
  • Developing  Big Data processing pipelines for data sources containing structured and unstructured data
  • Monitor and optimize key infrastructure components such as Databases, EC2 Clusters, Containers and other aspects of the stack
  • Act as a bridge between Data Engineering  and the wider Engineering organization
  • Partner closely  with business leaders,  Data Analysts and Data Scientists to understand and discover the potential business value of new and existing data sets and help push them to production
  • Analyzes requirements and architecture specifications to create detailed design

What we want to see:
  • Experience at a bigger startup where you’ ve worked with big data architecture at scale
  • 5+ years of experiene in data engineering:
    • 2+ years of experience with AWS Data Infrastructure, including RDS, RedShift & S3
    • 2+ years building data pipelines in a high ingestion environment with varied forms of data infrastructure technologies
  • Experience designing, developing, and owning ETL pipelines 
  • Proficiency with Python, SQL and other scripting languages
  • Extensive  SQL experience  to scale and optimize schemas, and performance-tune ETL pipelines
  • Identifying and resolving pipeline issues, and discovering opportunities for improvement in in complex designs or coding schemes
  • Monitoring existing metrics, analyzing data, and partnering with other internal teams to solve difficult problems creating a better customer experience

Bonus points if you have (NOT required):
  • Experience with data streaming technologies e.g. Spark, Storm, Flink
  • Experience with message queue systems e.g. Kafka, Kinesis
  • Experience with any of the following message / file formats: Parquet, Avro, Protocol Buffer
  • Experience with Redis, Cassandra, MongoDB or similar NoSQL databases
  • Experience with containerization frameworks and tools e.g. Docker, Terraform, Kubernetes
  • Gathering requirements, scoping, architecting, developing, building, releasing, and maintaining data-oriented projects for different parts of the organization, while taking into account performance, stability and an error-free operation
  • Architecting scalable and reliable data solutions to move data across systems from multiple products in nearly real time

Send an email reminder to:

Share This Job:

Related Jobs:

Login to save this search and get notified of similar positions.