2014-10-10 3 views
3

Я новичок в Apache Spark. Я попытался установить Apache искру 1.0.2 с помощью Scala 2.10.4 на Windows 7 в соответствии со следующим правилом:Ошибка при запуске искровой оболочки: ERROR Remoting: Ошибка удаления: [Ошибка при запуске]

http://sankalplabs.wordpress.com/2014/08/25/installing-apache-spark-on-windows-step-by-step-approach/

При запуске искру Shell я получаю следующее исключение:

ERROR Remoting: Remoting error: [Startup failed] [ 
akka.remote.RemoteTransportException: Startup failed 
     at akka.remote.Remoting.akka$remote$Remoting$$notifyError(Remoting.scala 
:129) 
     at akka.remote.Remoting.start(Remoting.scala:194) 
     at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala: 
184) 
     at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:579) 
     at akka.actor.ActorSystemImpl._start(ActorSystem.scala:577) 
     at akka.actor.ActorSystemImpl.start(ActorSystem.scala:588) 
     at akka.actor.ActorSystem$.apply(ActorSystem.scala:111) 
     at akka.actor.ActorSystem$.apply(ActorSystem.scala:104) 
     at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:10 
4) 
     at org.apache.spark.SparkEnv$.create(SparkEnv.scala:152) 
     at org.apache.spark.SparkContext.<init>(SparkContext.scala:202) 
     at org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala: 
957) 
     at $line3.$read$$iwC$$iwC.<init>(<console>:8) 
     at $line3.$read$$iwC.<init>(<console>:14) 
     at $line3.$read.<init>(<console>:16) 
     at $line3.$read$.<init>(<console>:20) 
     at $line3.$read$.<clinit>(<console>) 
     at $line3.$eval$.<init>(<console>:7) 
     at $line3.$eval$.<clinit>(<console>) 
     at $line3.$eval.$print(<console>) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 
     at java.lang.reflect.Method.invoke(Unknown Source) 
     at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala: 
788) 
     at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala: 
1056) 
     at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:614 
) 
     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:645) 
     at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:609) 
     at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:7 
96) 
     at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.sca 
la:841) 
     at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:753) 
     at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply 
(SparkILoopInit.scala:121) 
     at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply 
(SparkILoopInit.scala:120) 
     at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:263) 
     at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoop 
Init.scala:120) 
     at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:56) 

     at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$apply$mc 
Z$sp$5.apply$mcV$sp(SparkILoop.scala:913) 
     at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.s 
cala:142) 
     at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:56) 
     at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkIL 
oopInit.scala:104) 
     at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala: 
56) 
     at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(Spar 
kILoop.scala:930) 
     at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop. 
scala:884) 
     at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop. 
scala:884) 
     at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClass 
Loader.scala:135) 
     at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:884) 
     at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:982) 
     at org.apache.spark.repl.Main$.main(Main.scala:31) 
     at org.apache.spark.repl.Main.main(Main.scala) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 
     at java.lang.reflect.Method.invoke(Unknown Source) 
     at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:303) 
     at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55) 
     at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: Oleander 
/192.168.1.7:0 
     at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:2 
72) 
     at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(Ne 
ttyTransport.scala:391) 
     at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(Ne 
ttyTransport.scala:388) 
     at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) 
     at scala.util.Try$.apply(Try.scala:161) 
     at scala.util.Success.map(Try.scala:206) 
     at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) 
     at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) 
     at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(Ba 
tchingExecutor.scala:67) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(Batc 
hingExecutor.scala:82) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExe 
cutor.scala:59) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExe 
cutor.scala:59) 
     at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72 
) 
     at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) 
     at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) 
     at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(Abst 
ractDispatcher.scala:386) 
     at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) 
     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool 
.java:1339) 
     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:19 
79) 
     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThre 
ad.java:107) 
Caused by: java.net.BindException: Cannot assign requested address: bind 
     at sun.nio.ch.Net.bind0(Native Method) 
     at sun.nio.ch.Net.bind(Unknown Source) 
     at sun.nio.ch.Net.bind(Unknown Source) 
     at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) 
     at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source) 
     at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(Nio 
ServerBoss.java:193) 
     at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQue 
ue(AbstractNioSelector.java:366) 
     at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNi 
oSelector.java:290) 
     at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.ja 
va:42) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
     at java.lang.Thread.run(Unknown Source) 
] 
org.jboss.netty.channel.ChannelException: Failed to bind to: Oleander/192.168.1. 
7:0 
     at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:2 
72) 
     at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(Ne 
ttyTransport.scala:391) 
     at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(Ne 
ttyTransport.scala:388) 
     at scala.util.Success$$anonfun$map$1.apply(Try.scala:206) 
     at scala.util.Try$.apply(Try.scala:161) 
     at scala.util.Success.map(Try.scala:206) 
     at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) 
     at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235) 
     at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(Ba 
tchingExecutor.scala:67) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(Batc 
hingExecutor.scala:82) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExe 
cutor.scala:59) 
     at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExe 
cutor.scala:59) 
     at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72 
) 
     at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58) 
     at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:42) 
     at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(Abst 
ractDispatcher.scala:386) 
     at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) 
     at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool 
.java:1339) 
     at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:19 
79) 
     at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThre 
ad.java:107) 
Caused by: java.net.BindException: Cannot assign requested address: bind 
     at sun.nio.ch.Net.bind0(Native Method) 
     at sun.nio.ch.Net.bind(Unknown Source) 
     at sun.nio.ch.Net.bind(Unknown Source) 
     at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) 
     at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source) 
     at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(Nio 
ServerBoss.java:193) 
     at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQue 
ue(AbstractNioSelector.java:366) 
     at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNi 
oSelector.java:290) 
     at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.ja 
va:42) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
     at java.lang.Thread.run(Unknown Source) 

Мне кажется, что описание неполное. Какие шаги отсутствуют? Есть ли где-то полный (и краткий) список шагов по установке Spark в Windows 7?

Ваша помощь очень высоко ценится, Феликс

ответ

5

Исключение ушел, когда я добавил конфигурационный файл «искрового env.cmd» в папку «конф». В файле я указал локальный IP-адрес (который я, полученный с помощью IPCONFIG -all) следующим образом: установить SPARK_LOCAL_IP = 192.168.1.111

В качестве альтернативы установить: SPARK_LOCAL_IP = LocalHost

+1

Спасибо. Ты спас мой день! –

0

Для этого с подобной ошибки может быть полезно проверить конфигурацию переменных окружения here

Подводя итог, conf/spark-env.sh (или conf/spark-env.cmd на Windows) по умолчанию не существует, когда установлена ​​искра. Поэтому, чтобы изменить переменные среды, сначала скопируйте файл шаблона conf/spark-env.sh.template в conf/spark-env.sh. В spark-env.sh файле найти строку

# - SPARK_LOCAL_IP, to set the IP address Spark binds to on this node 

и изменить его на

SPARK_LOCAL_IP=LOCALHOST 
Смежные вопросы