Is it possible to join TempView in spark, as it is throwing cannot resolve symbol error

6507 views apache-spark apache-spark-sql
10

I am trying to join 3 view in Apache Spark with scala

df1.createOrReplaceTempView("customer")
finaldf.createOrReplaceTempView("product")
areadf.createOrReplaceTempView("sales")

Libraries:

import org.apache.spark.sql.{DataFrame, SaveMode, SparkSession}
import org.apache.spark.sql.functions._

Joining view below

val result = customer.join(product, customer("keycol")===product("keycol"))

but not sure why it is not recognize registered view Any miss.... Please suggest

do i need to import any specific library

answered question

1 Answer

2

No need to import any library, Once we create temp view then we need to use SQL api to join syntax the temp views.

Example:

spark.sql("select * from customer c join product p on c.keycol=p.keycol")

//join using dataframe api
df1.join(finaldf, df1("keycol")===finaldf("keycol"))

posted this

Have an answer?

JD

Please login first before posting an answer.