Key Duties & Responsibilities:
Build world class high-volume real-time data ingestion frameworks and automate ingesting various data sources into Hadoop.
Provide leadership and expertise in the development of new products/services/processes, frequently operating at the leading edge of technology.
Lead the transformation of existing data and analytic infrastructure to a highly scalable, flexible, and performant big data platform using appropriate open source technologies.
Articulate architectural differences between big data solution methods and the advantages/disadvantages of each.
Interpret and deliver impactful plans that specify strategy and improve data integration, data quality and data delivery in support of big data business initiatives and roadmaps to achieve results.
Collaborate with end users, development staff, and business analysts to ensure that prospective data architecture plans maximize the value of client data across the organization.
Other duties as assigned.
Required Experience & Qualifications:
Understanding the concepts and technology ecosystem around both real-time and batch processing in Hadoop.
Experience with various messaging systems, such as Kafka or RabbitMQ.
Expert level experience with the Hadoop ecosystem – HDFS, MapReduce, Hive, HBase, Oozie, Sqoop, Storm, and Kafka.
Experience with Cloudera/MapR/Hortonworks.
8+ years of experience with progressively increasing responsibilities in the areas of software analysis, design, development and architecture.
3+ years of hands-on experience designing and implementing data architecture, modeling, processing batch and real time data streams with relational and non-relational databases.
2+ years experience installing and administering one or more Hadoop distributions such as Hortonworks or Cloudera.
Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS and/or knowledge of NoSQL platforms.
Development knowledge of J2EE and design patterns in a distributed, multi-platform, heterogeneous computing environment.