Login using Social Account
Login using your credentials
Overview of SparkR
Want to create exercises like this yourself? Click here.
No hints are availble for this assesment
Answer is not availble for this assesment
Looks like some of these are deprecated commands. Please update pdf:
> df = createDataFrame(spark, faithful) Warning message: 'createDataFrame(sqlContext...)' is deprecated. Use 'createDataFrame(data, schema = NULL)' instead. See help("Deprecated") > df = createDataFrame(faithful, schema=NULL) > res = select(df, df$eruptions) > head(res)
> people = read.df(spark, "/data/spark/people.json","json") 22/10/21 20:37:19 WARN DataSource: Error while looking for metadata directory. Warning message: 'read.df(sqlContext...)' is deprecated. Use 'read.df(path = NULL, source = NULL, schema = NULL, ...)' instead. See help("Deprecated")
Thank you!
In which version have you tested these?
spark2.4.3 (I see only 2.3,2.4.3 in usr directory)
Ohh,
So it may be due to the version mismatch as commands in pdf are tested on spark 2.0.1.
Correct, thats the reason I was requesting to update pdf if old versions are not supported by cloudxlab. Thanks
Thanks for the info Sarika. We will be looking into it shortly.
1 Spark SQL - Introduction
2 Spark SQL - Dataframe Introduction
3 Spark SQL - Getting Started
4 Spark SQL - Create Df from Json
5 Spark SQL - Dataframe Operations
6 Spark SQL - SQL Queries On Dataframes
7 Spark SQL - Understanding Datasets
8 Spark SQL - Rdd And Dataframe Interoperability
9 Spark SQL - Infer Schema Using Reflection
10 Spark SQL - Converting RDD to Dataframe Using Programmatic Schema
11 Spark SQL - Handling Avro
12 Spark SQL - Loading XML
13 Spark SQL - Handling Various Data Sources
14 Spark SQL - Using Hive tables
15 Spark SQL - Working With JDBC
16 Spark SQL - Resources
17 Overview of SparkR
18 SparkR - Resources
Please login to comment
6 Comments
Looks like some of these are deprecated commands. Please update pdf:
> df = createDataFrame(spark, faithful)
Warning message:
'createDataFrame(sqlContext...)' is deprecated.
Use 'createDataFrame(data, schema = NULL)' instead.
See help("Deprecated")
> df = createDataFrame(faithful, schema=NULL)
> res = select(df, df$eruptions)
> head(res)
> people = read.df(spark, "/data/spark/people.json","json")
22/10/21 20:37:19 WARN DataSource: Error while looking for metadata directory.
Warning message:
'read.df(sqlContext...)' is deprecated.
Use 'read.df(path = NULL, source = NULL, schema = NULL, ...)' instead.
See help("Deprecated")
Thank you!
Upvote ShareIn which version have you tested these?
Upvote Sharespark2.4.3 (I see only 2.3,2.4.3 in usr directory)
Upvote ShareOhh,
So it may be due to the version mismatch as commands in pdf are tested on spark 2.0.1.
Upvote ShareCorrect, thats the reason I was requesting to update pdf if old versions are not supported by cloudxlab. Thanks
Upvote ShareThanks for the info Sarika. We will be looking into it shortly.
Upvote Share