Hadoop Administrator

Hadoop Admin

• Primary skills:Technology->Big Data - Hadoop->Hadoop Administration Experience with cloud-based Hadoop (AWS EMR, Azure HDInsight, GCP Dataproc) Knowledge of DevOps practices (CI/CD, Docker, Kubernetes) Experience in high availability (HA) and disaster recovery (DR) setups Cluster Management Install, configure, and maintain Hadoop clusters (Cloudera, Hortonworks, or Apache Hadoop) Manage core components including HDFS, YARN, MapReduce Perform cluster scaling, upgrades, and patching Administration & Monitoring Monitor cluster performance, health, and usage Troubleshoot and resolve cluster issues, failures, and bottlenecks Implement alerting and monitoring tools (Ambari, Cloudera Manager, Grafana) Security & Governance Configure and manage Kerberos authentication Implement data security policies using Ranger / Sentry Handle user access controls and permissions Data Management Manage data storage, replication, and backup strategies Optimize HDFS performance and storage utilization Handle data ingestion and integration workflows Performance Optimization Tune Hadoop cluster performance (memory, CPU, storage) Optimize YARN resource allocation and scheduling Identify and resolve performance issues in jobs Integration & Support Support tools in Hadoop ecosystem: Hive Spark HBase Sqoop / Flume / Kafka Work closely with data engineers and developers Required Skills & Qualifications Core Technical Skills 3–6 years of experience in Hadoop Administration Strong knowledge of: HDFS, YARN, MapReduce Hadoop ecosystem tools Experience with Linux system administration (mandatory) Big Data Ecosystem Familiarity with: Hive, Spark, HBase Kafka, Sqoop, Flume Understanding of distributed computing concepts Tools & Platforms Experience with: Cloudera Manager / Ambari Monitoring tools (Nagios, Prometheus, Grafana) Basic scripting in Shell / Python