Big Data Engineer

Advantage Tech is seeking an experienced Data Engineer. The Data Engineer’s primary responsibility is to write and implement code to build infrastructure, tools, frameworks, apps, and services to deliver and support data and analytics solutions. The Data Engineer partners with the Business Intelligence team and participates in the full software development life cycle including requirements, technical design, development, testing, and deployment of high-quality data integration, business analytics, and enterprise reporting solutions.  
 
The successful candidate will be expected to work on one or more projects simultaneously that will include a variety of data lake development and administration, data source ingestion, data processing (ELT/ETL), data modeling (machine learning), data warehouse design/development, as well as data integrations. The Data Engineer designs, develops, and maintains the entire Enterprise Data Platform.

Responsibilities:

  • Build and administer the Data Lake & Data Warehouse 
  • Develop and code solutions to ingest, clean, transform, enrich, and model data 
  • Build and implement data pipelines and ELT/ETL processes 
  • Create data collection jobs to ingest and merge new data with existing structures 
  • Construct distributed computing code to process data at scale 
  • Develop data models to best serve data to meet business needs 
  • Create and implement data integration and data modeling standards 
  • Construct and manage monitoring and security controls across the environment 
  • Monitor data pipelines and manage performance of data ingestion and processing 
  • Identify, troubleshoot, and resolve pipeline and code errors 
  • Analyze business requirements, logical data models, and system functions to best create and utilize data architecture to enable business analytics 

Qualifications:

  • 3-5 years of experience in programming and big data or distributed system engineering 
  • Strong experience and passion for data management 
  • Experience managing data lake and data warehouse environments 
  • Proven ability to construct distributed data processing to ingest and process data at scale 
  • Must have strong working technical experience with SQL and relational databases 
  • Experience building ELT/ETL in multiple tools/platforms and languages 
  • Knowledge of a data lake ecosystem, such as Hadoop, Spark, MPP databases 
  • Analytical thinking and structured approach to problem solving 
  • Ability to work in a team environment and communicate effectively 
  • Self-motivated with the ability to work in a fast paced and ever-changing environment 
  • Must have excellent written and verbal communication skills 
  • Must be detail oriented, organized, and able to handle multiple assignments 
  • Bachelor’s degree in IT, Analytics, Computer Science/Engineering or related field, or equivalent relevant work experience 

 

Preferred Qualifications: 

  • AWS Data Lake Technologies (S3, EMR/EC2, Athena, Glue, etc.)
  • MPP Databases such as Snowflake or AWS Redshift
  • Knowledge and experience with Data Warehouse modeling methodologies
  • Apache Spark, Scala, or Java programming experience
  • Apache Airflow or Python programming experience
  • Ability to write and tune Apache Spark jobs for data ingestion and processing
  • Experience creating APIs and data services
  • Hands on experience with AWS or Azure cloud data analytics services
  • Knowledge of Revenue Cycle process and claims datasets 
Submit Resume