We are seeking a Big Data Engineer to design, develop and scale our big data platform that is vital to the core of our business. You will be a part of the team responsible for developing and refining tools that enable our media agency to leverage data effectively for both internal and external reporting, advanced analysis and applications, enabling a data driven decision making culture. The ideal candidate has a passion for big data and quality to join us as we revolutionizes the way consumers and enterprises store data in the cloud.Responsibilities:Responsible for design, develop and implement our big data platform including big data infrastructure, data warehouse, data processing pipelines and system integrationsData modeling and metadata managementStreaming data processing and analysisPay meticulous attention to end-to-end data quality, validation and consistencyEnsure scalable, highly available and robust big data platform architecture to meet service level agreementsEnd-to-end data processing trouble shooting and problem diagnosisPerformance benchmark, load testing and code reviewsAcross team collaborations and effective communicationsRequirements:10+ years of software development experience on large scale distributed systemsExpertise with SQL scripting, data warehousing, business intelligence and ETL developmentFamiliarity with Hadoop open source stack including YARN, Kafka, Hive, Pig, Sqoop and RDBMS such as MySQLHands on experience with AWS EMR, Dynamo, RDS, Redshift, cloud storage such as S3, Glacier, EBS, etc.Expert level proficiency in Programming languages like Python, Perl, Ruby, Scala, etc.Experience building large-scale distributed applications and servicesExperience with other cloud storage provider platform such as Google Cloud Engine, Azure bonus points.Excellent written/oral communication skillsProficiency in statistical analysis and data mining techniques a plusBS/MS - Computer or relevant engineering discipline strongly desired
Important Safety Tips: You should find out as much as you can about the company. Meet your prospective employers/employees face to face. You should NOT have to provide your bank account details. Stay safe - read our safety tips.