Your Responsibilities :
- Using Azure – DataLake tools , you will be doing hands-on role on Data Fabric Tools (Informatica Suite BDM) and Big Data Technology (Handoop – Hortonworks/Cloudera)
- Responsible on DevOps Tools and best practice across organization including automated testing, provisioning, configuration management, service orchestration and dynamic monitoring.
- You will work with Development and Infrastructure activities.
- Supervising Hadoop jobs using scheduler & optimizing it.
- Yarn Queue management to improve the performance
- Assist Map Reduce/Spark/Hive/Informatica programs running on the Hadoop cluster.
- Preserve security and data privacy.
- Develop highly scalable and services with exceptional performance, for data tracking.
- Conduct POC independently and responsible to build, test & performance tuning in Hadoop ecosystem.
- Responsible for troubleshooting Hadoop technologies like HDFS, Hive, Sqoop, Zookeeper, Spark2, MapReduce2, YARN, HBase, Tez, Kafka, kibana, solr, elastic search.
- Responsible for Authorization and Audit tools management with Hadoop Technologies like Ranger/Sentry.
- Fine tune applications and systems for high performance and higher volume throughput.
- Administration of Hadoop Cluster (Cloudera/Hortonworks ) including adding or removing nodes
- Translate, load and exhibit unrelated data sets in various formats and sources like JSON, text files, Kafka queues, and log data.
Your Profile :
Qualification & Experience:
- Total 5+ years of total IT experience working
- 3+ years on Data Sourcing, Quality, Warehousing, Mining & ETL tools
- 3+ years into Big Data platform like Hadoop like Hortonworks HDP 3.5.x / Cloudera 6.x / CDP
- Hands on experience on Hadoop Data Platform preferably Hortonworks (HDP)
- Hands on experience on any big data ETL workloads (Sqoop, Flume, Spark Core/SQL API (Scala), Hive (Tez/LLAP) and Phoenix)
- Hands on experience on Storage covering Schema layout, partitioning & Read/Write API for Relational, MPP & NoSQL stack including File Format (Avro, Parquet), Hive, HBase, MongoDB & Oracle.
- Job Monitoring, debugging, Scheduling and Performance Tuning using Data Platform Operations Tools like Oozie, YARN Settings, Spark tuning, Ambari Config & Grafana
- Hadoop Administrator skills covering Hadoop cluster operation, tuning, troubleshooting in multi-tenant environment, strong Linux Shell, java and python scripting is preferred
Good to have
- Informatica Administrator skills covering Informatica 10.2.2/10.4.x suite BDM/EDL/EDC/BDQ/TDM/PDM integration with external Hadoop cluster operation, tuning, troubleshooting in multi-tenant environment, strong Linux Shell, java and python scripting is preferred
- Knowledge on data Encryption algorithms and creation of encryption zones and encryption key management tools like Ranger KMS or KMS
- Experience with RDBMS like SQL/Oracle
- Experience in Informatica administration skills for Informatica tools like BDM/EDL/EDC/BDQ/DDM/TDM
- Experience with SAML/SMTP/TCP/HTTPS protocols
- Experience with TLS and Keystore/Trust store creation.
- Hands on experience with Dev Ops tools like Git/Jenkins/Nexus
- Experience in setting up data governance, data security, metadata management, lineage tracking on Hadoop Platform using Kerberos, Ranger Policy, Atlas & Ambari
- Networking and DMZ setup.
- Knowledge of Autosys Scheduling tool or the like, GIT, Ansible, and Docker
- ITIL practices.
- Agile methodologies with focus on Scrum Framework
Powered by JazzHR