Не удалось импортировать искровые пакеты

Question

Я пытаюсь внести изменения в образец кода Java, предоставленный Spark JavaKafkaWordCount.java, и попытаться построить его с помощью Maven. Я работаю над автономным модулем Spark. Я использую следующий pom.xml:

<project> 
    <groupId>com.mycompany.app</groupId> 
    <artifactId>my-app</artifactId> 
    <modelVersion>4.0.0</modelVersion> 
    <name>Simple Project</name> 
    <packaging>jar</packaging> 
    <version>1.0</version> 
    <dependencies> 
     <dependency> <!-- Spark dependency --> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-core_2.10</artifactId> 
      <version>1.5.1</version> 
     </dependency> 

     <dependency> 
      <groupId>junit</groupId> 
      <artifactId>junit</artifactId> 
      <version>4.8.2</version> 
      <scope>test</scope> 
     </dependency> 

     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-streaming-kafka_2.10</artifactId> 
      <version>1.5.1</version> 
     </dependency> 
    </dependencies> 
</project> 

Когда я бегу

mvn package 

я получаю следующие ошибки:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project my-app: Compilation failure: Compilation failure: 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[17,34] package org.apache.spark.streaming does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[18,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[19,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[20,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[21,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,5] cannot find symbol 
[ERROR] symbol: class JavaStreamingContext 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,37] cannot find symbol 
[ERROR] symbol: class JavaStreamingContext 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,73] cannot find symbol 
[ERROR] symbol: class Duration 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol 
[ERROR] symbol: class JavaPairReceiverInputDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol 
[ERROR] symbol: class JavaPairReceiverInputDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[49,23] cannot access org.apache.spark.streaming.dstream.ReceiverInputDStream 
[ERROR] class file for org.apache.spark.streaming.dstream.ReceiverInputDStream not found 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[51,5] cannot find symbol 
[ERROR] symbol: class JavaDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[58,5] cannot find symbol 
[ERROR] symbol: class JavaDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[65,5] cannot find symbol 
[ERROR] symbol: class JavaPairDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] -> [Help 1] 
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. 
[ERROR] Re-run Maven using the -X switch to enable full debug logging. 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles: 
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException 

Кажется, что я должен изменить свою переменную PATH, но Я не могу найти подходящую банку.

Answer 1

сообщение об ошибке вы получаете это:

пакет org.apache.spark.streaming.api.java не существует

Это означает, что вам не хватает следующей зависимости:

<dependency> 
    <groupId>org.apache.spark</groupId> 
    <artifactId>spark-streaming_2.10</artifactId> 
    <version>1.5.1</version> 
</dependency>