0 votes
in Technology by (323k points)

Which of these code snippet is the correct format to run SQL queries?

1. display(spark.sqlContext("select * from Fresco"))
2. print(spark.sql("select * from Fresco"))
3. display(spark.sql("select * from Fresco"))
4. show(spark.sql("select * from Fresco"))

2 Answers

0 votes
by (323k points)

Which of these code snippet is the correct format to run SQL queries?
1. display(spark.sqlContext("select * from Fresco"))
2. print(spark.sql("select * from Fresco"))
3. display(spark.sql("select * from Fresco"))
4. show(spark.sql("select * from Fresco"))

🔗Reference : W3school.com

🔗Source: Interview Questions and Answers

🔗Reference: Javatpoint.com

answer is display(spark.sql("select * from Fresco"))

0 votes
by (54.6k points)

Spark SQL is a new module in Apache Spark that integrates relational processing with Spark’s functional programming API.

Spark SQL lets Spark programmers leverage the benefits of relational processing (e.g., declarative queries and optimized storage), and lets SQL users call complex analytics libraries in Spark (e.g., machine learning).

// Define the schema using a case class
case class Person(name: String, age: Int)

// you could read people from a CSV file
// It's been a while since you saw RDDs, hasn't it?
// Excuse me for bringing you the old past.
import org.apache.spark.rdd.RDD
val peopleRDD: RDD[Person] = sc.parallelize(Seq(Person("Jacek", 10)))

// Convert RDD[Person] to Dataset[Person] and run a query

// Automatic schema inferrence from existing RDDs
scala> val people = peopleRDD.toDS
people: org.apache.spark.sql.Dataset[Person] = [name: string, age: int]

// Query for teenagers using Scala Query DSL
scala> val teenagers = people.where('age >= 10).where('age <= 19).select('name).as[String]
teenagers: org.apache.spark.sql.Dataset[String] = [name: string]

scala> teenagers.show
+-----+
| name|
+-----+
|Jacek|
+-----+

// You could however want to use good ol' SQL, couldn't you?

// 1. Register people Dataset as a temporary view in Catalog
people.createOrReplaceTempView("people")

<span style="-webkit-font-smoothing:antialiased; -webkit-tap-highlight-color:transparent; box-sizing:border-box; color:#8e908c; font-si
...