Aloha: A Scala‑Based Distributed Task Scheduling and Management Framework

Aloha is a distributed task scheduling and management framework implemented in Scala, extending Spark's scheduling module to support various application types (e.g., Spark, Flink, ETL) through a plugin architecture. It offers a unified entry point for task submission and status monitoring.

The core consists of Master and Worker components with HA support; the Master manages Workers, receives application submissions via REST, and schedules them using a FIFO strategy. Workers register with all Masters, send heartbeats, and execute applications in isolated threads, handling start, shutdown, and cleanup via the Application interface.

Custom applications implement the Application trait, which defines start() returning a Promise[ExitState] and shutdown(reason: Option[String]) . Example implementations show launching external processes and handling their lifecycle.

trait Application {
  //启动
  def start(): Promise[ExitState]
  //强制停止
  def shutdown(reason: Option[String]): Unit
  def withDescription(desc: ApplicationDescription): Application
  def withApplicationDir(appDir: File): Application
  def withAlohaConf(conf: AlohaConf): Application
  def clean(): Unit
}

Aloha provides an event‑listener mechanism ( AlohaEventListener ) to react to application state changes, relaunches, or other events, allowing multiple listeners to be registered dynamically.

trait AlohaEventListener {
  def onApplicationStateChange(event: AppStateChangedEvent): Unit
  def onApplicationRelaunched(event: AppRelaunchedEvent): Unit
  def onOtherEvent(event: AlohaEvent): Unit
}

The RPC subsystem replaces traditional IDL with Scala pattern matching. Key classes are RpcEndpoint , RpcEndpointRef , and RpcEnv . A simple client‑server example demonstrates creating an RPC environment, registering endpoints, and sending messages.

// Server side
object HelloWorldServer {
  def main(args: Array[String]): Unit = {
    val host = "localhost"
    val rpcEnv: RpcEnv = RpcEnv.create("hello-server", host, 52345, new AlohaConf())
    val helloEndpoint: RpcEndpoint = new HelloEndpoint(rpcEnv)
    rpcEnv.setupEndpoint("hello-service", helloEndpoint)
    rpcEnv.awaitTermination()
  }
}

class HelloEndpoint(override val rpcEnv: RpcEnv) extends RpcEndpoint {
  override def onStart(): Unit = println("Service started.")
  override def receiveAndReply(context: RpcCallContext): PartialFunction[Any, Unit] = {
    case SayHi(msg) => context.reply(s"Aloha: $msg")
    case SayBye(msg) => context.reply(s"Bye :), $msg")
  }
  override def onStop(): Unit = println("Stop hello endpoint")
}

case class SayHi(msg: String)
case class SayBye(msg: String)

// Client side
object HelloWorldClient {
  def main(args: Array[String]): Unit = {
    val host = "localhost"
    val rpcEnv: RpcEnv = RpcEnv.create("hello-client", host, 52345, new AlohaConf, true)
    val endPointRef: RpcEndpointRef = rpcEnv.retrieveEndpointRef(RpcAddress("localhost", 52345), "hello-service")
    val future: Future[String] = endPointRef.ask[String](SayHi("WALL-E"))
    future.onComplete {
      case Success(value) => println(s"Got response: $value")
      case Failure(e) => println(s"Got error: $e")
    }
    Await.result(future, Duration.apply("30s"))
  }
}

The RPC implementation uses RpcEndpoint for message handling, RpcEndpointRef as a reference for sending requests, and RpcEnv as the runtime environment. Under the hood, NettyRpcEnv provides a Netty‑based transport with components such as TransportServer , TransportClient , Dispatcher , Inbox , and Outbox to manage message routing, serialization, and network I/O.

In summary, Aloha combines a Spark‑based scheduling core, extensible application interfaces, event‑driven monitoring, and a lightweight Scala‑centric RPC layer to deliver a flexible, HA‑capable distributed task management system.