in Education by
so I am constructing a recommedation model using ALS package And make all user-product list by cartesian product. And I finally predict all the ratings. But I want to group the ratings by users. and the final format used to be (user, (product, ratings)) and i have to sort by decending order. THIS IS MY CODE val ratings = sc.textFile(new File("/user/ubuntu/kang/0829/rawRatings.csv").toString).map { line => val fields = line.split(",") (Rating(fields(0).toInt,fields(1).toInt,fields(2).toDouble))} val model = ALS.train(ratings,10,10,0.1) val numUsers = ratings.map(_.user).distinct val numMovies = ratings.map(_.product).distinct val usersProducts = numUsers.cartesian(numMovies) val recommendations = model.predict(usersProducts) BUT IN HERE value recommendations is not a Pair-RDD and it's a Rating(_)format so I cannot apply groupByKey... Could anybody solve this problem...? JavaScript questions and answers, JavaScript questions pdf, JavaScript question bank, JavaScript questions and answers pdf, mcq on JavaScript pdf, JavaScript questions and solutions, JavaScript mcq Test , Interview JavaScript questions, JavaScript Questions for Interview, JavaScript MCQ (Multiple Choice Questions)

1 Answer

0 votes
by
It's quite simple: recommendations.map (x => (x.user, (x.product, x.rating))).reduceByKey(here put reduce function)

Related questions

0 votes
    Definition says: RDD is immutable distributed collection of objects I don't quite understand what does it mean. Is ... one please help. Select the correct answer from above options...
asked Jan 30, 2022 in Education by JackTerrance
0 votes
    I'm trying to run a spark application using bin/spark-submit. When I reference my application jar inside my local ... version 1.2.1 Select the correct answer from above options...
asked Jan 29, 2022 in Education by JackTerrance
0 votes
    I want to know in simple language what are all the differences between rdd and dataframes? Select the correct answer from above options...
asked Jan 21, 2022 in Education by JackTerrance
0 votes
    Is spark dependent on Hadoop? If not, then I can run Spark without Hadoop right? Will I miss any features if I do Select the correct answer from above options...
asked Jan 21, 2022 in Education by JackTerrance
0 votes
    How can I convert an RDD to a dataframe? I converted a data frame to rdd using .rdd. After processing it I ... convert it back to rdd Select the correct answer from above options...
asked Jan 21, 2022 in Education by JackTerrance
0 votes
    Hi guys simple question for experienced guys. I have a spark job reading files under a path. I wanted ... Questions for Interview, JavaScript MCQ (Multiple Choice Questions)...
asked May 6, 2022 in Education by JackTerrance
0 votes
    I am trying to write code for converting data in Java RDD to a histogram so that I can bin the data in a certain ... (EntryWithSize entry) { long size = entry.getSize(); if (size...
asked Apr 9, 2022 in Education by JackTerrance
0 votes
    I am working on a project with spark and scala and I am new to both but with lot of help from ... , JavaScript Questions for Interview, JavaScript MCQ (Multiple Choice Questions)...
asked Jun 19, 2022 in Education by JackTerrance
0 votes
    I am loading a (5gb compressed file) into memory (aws), creating a dataframe(in spark) and trying ... JavaScript Questions for Interview, JavaScript MCQ (Multiple Choice Questions)...
asked Jun 12, 2022 in Education by JackTerrance
0 votes
    Recommendations are done in __________. A. Commerce Cloud B. Marketing Cloud C. Community Cloud D. Sales Cloud...
asked Nov 1, 2022 in Education by JackTerrance
0 votes
    Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers. Want ... Questions for Interview, JavaScript MCQ (Multiple Choice Questions)...
asked Feb 27, 2022 in Education by JackTerrance
0 votes
    'People who bought this also bought…' recommendations seen on Amazon are a result of which algorithm?...
asked Apr 27, 2021 in Technology by JackTerrance
0 votes
    May be i am asking very basic question apology for that, but i didn't find it's answer on ... JavaScript Questions for Interview, JavaScript MCQ (Multiple Choice Questions)...
asked Jul 20, 2022 in Education by JackTerrance
0 votes
    May be i am asking very basic question apology for that, but i didn't find it's answer on ... JavaScript Questions for Interview, JavaScript MCQ (Multiple Choice Questions)...
asked Jul 14, 2022 in Education by JackTerrance
...