Spark Streaming

6 / 20

Apache Spark - Streaming - DStream

==DStream==

Discretized stream or DStream represents a continuous stream of data. DStream can be created either from input data streams or by applying higher-level operations to other DStreams. We will learn higher-level operations later in the course.

DStream is represented as a sequence of RDDs internally. We write Spark Streaming programs using DStreams.

Apache Spark Streaming - Slides


No hints are availble for this assesment

Answer is not availble for this assesment

Please login to comment

4 Comments

While executing ssc.start() thrownthe beolow errors continuously..

Could you pl suggest, which Port number should be used ( should be free )


23/02/13 05:19:41 ERROR ReceiverTracker: Deregistered receiver for stream 0: Restarting receiver with delay 2000ms: Error connecting to localhost:9999 - java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)

  Upvote    Share

Hi Karthikeyan,

To resolve this issue, you can either try the following:

Ensure that the server is running and is listening on the specified port.

Check if the port 9999 is already in use by another process on your machine. If it is, you can choose a different free port and update the configuration accordingly.

  Upvote    Share

DStreams ? I thing Structured Streaming is the newer Spark api for streaming

  Upvote    Share

DStreams are unstructured streams.

Yes, the structured streaming is new spark API but is for the cases where your data is structured.

  Upvote    Share