0 votes
48 views

Which of these code snippet is the correct format to run SQL queries?

1. display(spark.sqlContext("select * from Fresco"))
2. print(spark.sql("select * from Fresco"))
3. display(spark.sql("select * from Fresco"))
4. show(spark.sql("select * from Fresco"))

in Technology by (189k points)

2 Answers

0 votes

Which of these code snippet is the correct format to run SQL queries?
1. display(spark.sqlContext("select * from Fresco"))
2. print(spark.sql("select * from Fresco"))
3. display(spark.sql("select * from Fresco"))
4. show(spark.sql("select * from Fresco"))

🔗Reference : W3school.com

🔗Source: Interview Questions and Answers

🔗Reference: Javatpoint.com

answer is display(spark.sql("select * from Fresco"))

by (189k points)
0 votes

Spark SQL is a new module in Apache Spark that integrates relational processing with Spark’s functional programming API.

Spark SQL lets Spark programmers leverage the benefits of relational processing (e.g., declarative queries and optimized storage), and lets SQL users call complex analytics libraries in Spark (e.g., machine learning).

// Define the schema using a case class
case class Person(name: String, age: Int)

// you could read people from a CSV file
// It's been a while since you saw RDDs, hasn't it?
// Excuse me for bringing you the old past.
import org.apache.spark.rdd.RDD
val peopleRDD: RDD[Person] = sc.parallelize(Seq(Person("Jacek", 10)))

// Convert RDD[Person] to Dataset[Person] and run a query

// Automatic schema inferrence from existing RDDs
scala> val people = peopleRDD.toDS
people: org.apache.spark.sql.Dataset[Person] = [name: string, age: int]

// Query for teenagers using Scala Query DSL
scala> val teenagers = people.where('age >= 10).where('age <= 19).select('name).as[String]
teenagers: org.apache.spark.sql.Dataset[String] = [name: string]

scala> teenagers.show
+-----+
| name|
+-----+
|Jacek|
+-----+

// You could however want to use good ol' SQL, couldn't you?

// 1. Register people Dataset as a temporary view in Catalog
people.createOrReplaceTempView("people")

<span style="-webkit-font-smoothing:antialiased; -webkit-tap-highlight-color:transparent; box-sizing:border-box; color:#8e908c; font-si
by (49.4k points)

Related questions

0 votes
1 answer 26 views
0 votes
2 answers 36 views
0 votes
2 answers 33 views
33 views asked Oct 22, 2020 in Technology by JackTerrance (189k points)
0 votes
1 answer 22 views
22 views asked Oct 22, 2020 in Technology by JackTerrance (189k points)
0 votes
1 answer 14 views
14 views asked Oct 22, 2020 in Technology by JackTerrance (189k points)
0 votes
1 answer 63 views
63 views asked Oct 22, 2020 in Technology by JackTerrance (189k points)
0 votes
1 answer 19 views
19 views asked Oct 22, 2020 in Technology by JackTerrance (189k points)
0 votes
1 answer 37 views
37 views asked Dec 4, 2020 in Technology by JackTerrance (189k points)
0 votes
1 answer 36 views
0 votes
1 answer 46 views
46 views asked Oct 22, 2020 in Technology by JackTerrance (189k points)
0 votes
1 answer 25 views
0 votes
1 answer 17 views
17 views asked Oct 22, 2020 in Technology by JackTerrance (189k points)
+1 vote
1 answer 95 views
0 votes
1 answer 90 views
0 votes
1 answer 16 views
16 views asked Nov 10, 2020 in Technology by JackTerrance (189k points)
+1 vote
1 answer 36 views
...