SparkSQL中DataFrame registerTempTable源码浅析

dataFrame.registerTempTable(tableName);
? 最近在使用SparkSQL时想到1万条数据注册成临时表和1亿条数据注册成临时表时，效率上是否会有很大的差距，也对DataFrame注册成临时表到底做了哪些比较好奇，拿来源码拜读了下相关部分，记录一下。

?

临时表的生命周期是和创建该DataFrame的SQLContext有关系的，SQLContext生命周期结束，该临时表的生命周期也结束了
?

DataFrame.scala相关源码
?/**
?? * Registers this [[DataFrame]] as a temporary table using the given name.? The lifetime of this
?? * temporary table is tied to the [[SQLContext]] that was used to create this DataFrame.
?? *
?? * @group basic
?? * @since 1.3.0
?? */
? def registerTempTable(tableName: String): Unit = {
??? sqlContext.registerDataFrameAsTable(this, tableName)
? }

??
?DataFrame中的registerTempTable调用SQLContext中的registerDataFrameAsTable,
?SQLContext中使用SimpleCatalog类去实现Catalog接口中的registerTable方法.
?

SQLContext.scala相关源码
? @transient
? protected[sql] lazy val catalog: Catalog = new SimpleCatalog(conf)
? /**
?? * Registers the given [[DataFrame]] as a temporary table in the catalog. Temporary tables exist
?? * only during the lifetime of this instance of SQLContext.
?? */
? private[sql] def registerDataFrameAsTable(df: DataFrame, tableName: String): Unit = {
??? catalog.registerTable(Seq(tableName), df.logicalPlan)
? }
????
??? 在SimpleCatalog中定义了Map，registerTable中按tableIdentifier为key，logicalPlan为Value注册到名为tables的map中

? Catalog.scala相关源码
? val tables = new mutable.HashMap[String, LogicalPlan]()
? override def registerTable(
????? tableIdentifier: Seq[String],
????? plan: LogicalPlan): Unit = {
??? val tableIdent = processTableIdentifier(tableIdentifier)
??? tables += ((getDbTableName(tableIdent), plan))
? }
?
? protected def processTableIdentifier(tableIdentifier: Seq[String]): Seq[String] = {
??? if (conf.caseSensitiveAnalysis) {
????? tableIdentifier
??? } else {
????? tableIdentifier.map(_.toLowerCase)
??? }
? }
?
? protected def getDbTableName(tableIdent: Seq[String]): String = {
??? val size = tableIdent.size
??? if (size <= 2) {
????? tableIdent.mkString(".")
??? } else {
????? tableIdent.slice(size - 2, size).mkString(".")
??? }
? }

? 阅读以上代码，最终registerTempTable是将表名(或表的标识)和对应的逻辑计划加载到Map中，并随着SQLContext的消亡而消亡