Our client is currently seeking a Hadoop Developer/DBA to join our team in Austin TX.
Position Functions or Responsibilities:
- Design and Build automated data ingestion pipelines utilizing Hadoop components like, Kite, AVRO, Sqoop, Falcon, Oozzie, and Custom UDF.
- Design and build an automated data delivery frameworks to provide data to downstream systems and users.
- Design and build post-load processing pipelines(via MapReduce, Pig Scala, Etc.) to support end-user reporting and visualization.
- Design and build data access into secondary and trinary storage technologies like S3 and Glacier.
- Design and build data tagging and classifier models to actively monitor PII security data which is coming into the system.
- Design and build an automated analytics / supervised machine learning pipelines utilizing Mahout, Jubatus and R.
- Design and implement Big Data Dev/Ops frameworks utilizing Jenkins. Develop testing stories to support ingestion and deliver pipelines.
- Design and build integration between Hadoop ecosystem components and non-Hadoop components such as elasticsearch and Solr.
- Design and build Geo-spatial processing pipeline for vector, raster, LiDAR type files utilizing GPU co-processing via GPU clusters.
- Design and build Batch Control Framework to manage all aspects of processing mentioned above.
- 5 years' experience in Data Architecture
- Deep understanding of Data Warehousing concepts and technologies
- Excellent understanding of Big Data Analytics platforms
- 3+ years' experience working on successful Hadoop projects which leverage Hadoop components and non-hadoop components such as, HDFS, Sqoop1/2, Kite, AVRO, HIVE2, org.apache.hadoop.mapreduce*, AWS API, Presto, DataNuculus,, HBASE, Spark, SOLR, Elasticsearch, Flume, Logstash, and Data Frames.
- Experience with Hadoop cluster administration and maintenance. Hadoop Admin and Developer certification will be a big plus.
- Experience of analytics, reporting tools (Microstrategy, Tableau etc), NoSQL data stores, data modeling, analytical tools, libraries
- Experience writing R application for modeling simulation, and forecasting.
- Strong experience with, Java 7/8, J2EE, Struts, Scala, Python, Ruby
- Strong traditional data, including the full DW lifecycle, ODS ,DW, MART, OLAP – MDM, Governance.