YarnClusterManager有什么作用

本篇内容介绍了"YarnClusterManager有什么作用"的有关知识，在实际案例的操作过程中，不少人都会遇到这样的困境，接下来就让小编带领大家学习一下如何处理这些情况吧！希望大家仔细阅读，能够学有所成！

YarnClusterManager很关键，它决定了Yarn集群下使用什么TaskScheduler，以及使用什么SchedulerBackend。

代码不多，直接看代码就可以了：

private[spark] class YarnClusterManager extends ExternalClusterManager {  override def canCreate(masterURL: String): Boolean = {    masterURL == "yarn"  }  override def createTaskScheduler(sc: SparkContext, masterURL: String): TaskScheduler = {    sc.deployMode match {      case "cluster" => new YarnClusterScheduler(sc)      case "client" => new YarnScheduler(sc)      case _ => throw new SparkException(s"Unknown deploy mode '${sc.deployMode}' for Yarn")    }  }  override def createSchedulerBackend(sc: SparkContext,      masterURL: String,      scheduler: TaskScheduler): SchedulerBackend = {    sc.deployMode match {      case "cluster" =>        new YarnClusterSchedulerBackend(scheduler.asInstanceOf[TaskSchedulerImpl], sc)      case "client" =>        new YarnClientSchedulerBackend(scheduler.asInstanceOf[TaskSchedulerImpl], sc)      case  _ =>        throw new SparkException(s"Unknown deploy mode '${sc.deployMode}' for Yarn")    }  }  override def initialize(scheduler: TaskScheduler, backend: SchedulerBackend): Unit = {    scheduler.asInstanceOf[TaskSchedulerImpl].initialize(backend)  }}

"YarnClusterManager有什么作用"的内容就介绍到这里了，感谢大家的阅读。如果想了解更多行业相关的知识可以关注网站，小编将为大家输出更多高质量的实用文章！