Исключение в задаче 0.0 на этапе 0.0, ошибка в PySpark

Исключение в задаче 0.0 на этапе 0.0, ошибка в PySpark ⇐ Python

1 сообщение • Страница 1 из 1

Anonymous

Исключение в задаче 0.0 на этапе 0.0, ошибка в PySpark

Цитата

Сообщение Anonymous » 05 ноя 2024, 19:58

I was trying to run a basic simple code to test if pySpark is running on my pc properly but I keep getting this error

Setting default log level to "WARN". To adjust logging level use
sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
24/11/05 11:22:16 ERROR Executor: Exception in task 0.0 in stage 0.0
(TID 0) java.net.SocketException: Connection reset at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source) at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331) at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93) at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141) at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583) 24/11/05 11:22:16
WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0)
(LOAN68-9HH3LS3 executor driver): java.net.SocketException: Connection
reset at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source) at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331) at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93) at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141) at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
24/11/05 11:22:16 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1
times; aborting job Traceback (most recent call last): File
"C:\Users\student\Dropbox (Personal)\Codes\pyspark_test.py", line 15,
in
df.show() File "C:\pyspark\spark-3.5\python\pyspark\sql\dataframe.py", line 947, in
show
print(self._show_string(n, truncate, vertical))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\pyspark\spark-3.5\python\pyspark\sql\dataframe.py", line 965, in
_show_string
return self._jdf.showString(n, 20, vertical)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\pyspark\spark-3.5\python\lib\py4j-0.10.9.7-src.zip\py4j\java_gateway.py",
line 1322, in call File
"C:\pyspark\spark-3.5\python\pyspark\errors\exceptions\captured.py",
line 179, in deco
return f(*a, **kw)
^^^^^^^^^^^ File "C:\pyspark\spark-3.5\python\lib\py4j-0.10.9.7-src.zip\py4j\protocol.py",
line 326, in get_return_value py4j.protocol.Py4JJavaError: An error
occurred while calling o42.showString. :
org.apache.spark.SparkException: Job aborted due to stage failure:
Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0
in stage 0.0 (TID 0) (LOAN68-9HH3LS3 executor driver):
java.net.SocketException: Connection reset at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source) at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331) at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93) at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141) at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Driver stacktrace: at
org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2856)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2792)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2791)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at
org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2791)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1247)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1247)
at scala.Option.foreach(Option.scala:407) at
org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1247)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:3060)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2994)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2983)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
at
org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:989)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2393) at
org.apache.spark.SparkContext.runJob(SparkContext.scala:2414) at
org.apache.spark.SparkContext.runJob(SparkContext.scala:2433) at
org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:530)
at
org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:483)
at
org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:61)
at org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:4333)
at org.apache.spark.sql.Dataset.$anonfun$head$1(Dataset.scala:3316)
at
org.apache.spark.sql.Dataset.$anonfun$withAction$2(Dataset.scala:4323)
at
org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:546)
at
org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:4321)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:125)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:201)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:108)
at
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:66)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:4321) at
org.apache.spark.sql.Dataset.head(Dataset.scala:3316) at
org.apache.spark.sql.Dataset.take(Dataset.scala:3539) at
org.apache.spark.sql.Dataset.getRows(Dataset.scala:280) at
org.apache.spark.sql.Dataset.showString(Dataset.scala:315) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:75)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:52)
at java.base/java.lang.reflect.Method.invoke(Method.java:580) at
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) at
py4j.Gateway.invoke(Gateway.java:282) at
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79) at
py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.base/java.lang.Thread.run(Thread.java:1583) Caused by:
java.net.SocketException: Connection reset at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source) at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331) at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93) at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141) at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
... 1 more
Process finished with exit code 1

I have tried changing the contents in the bin file but still getting the error.
Code:

Код: Выделить всё

import pyspark
from pyspark.sql import SparkSession

spark = SparkSession.builder \
.appName("PySpark Test") \
.getOrCreate()

data = [("Alice", 25), ("Bob", 30), ("Charlie", 35)]
columns =["Name", "Age"]
df = spark.createDataFrame(data, columns)

df.show()
filtered_df = df.filter(df.Age > 30)
filtered_df.show()
spark.stop()

Любая помощь приветствуется.

Подробнее здесь: https://stackoverflow.com/questions/791 ... in-pyspark

1730825904

Anonymous

I was trying to run a basic simple code to test if pySpark is running on my pc properly but I keep getting this error

Setting default log level to "WARN". To adjust logging level use
sc.setLogLevel(newLevel).  For SparkR, use setLogLevel(newLevel).
24/11/05 11:22:16 ERROR Executor: Exception in task 0.0 in stage 0.0
(TID 0) java.net.SocketException: Connection reset    at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491)   at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source)   at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331)     at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)    at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)  at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583) 24/11/05 11:22:16
WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0)
(LOAN68-9HH3LS3 executor driver): java.net.SocketException:  Connection
reset     at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491)   at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source)   at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331)     at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)    at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)  at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
24/11/05 11:22:16 ERROR TaskSetManager: Task 0 in stage 0.0 failed 1
times; aborting job Traceback (most recent call last):   File
"C:\Users\student\Dropbox (Personal)\Codes\pyspark_test.py", line 15,
in
df.show()   File "C:\pyspark\spark-3.5\python\pyspark\sql\dataframe.py", line 947, in
show
print(self._show_string(n, truncate, vertical))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "C:\pyspark\spark-3.5\python\pyspark\sql\dataframe.py", line 965, in
_show_string
return self._jdf.showString(n, 20, vertical)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "C:\pyspark\spark-3.5\python\lib\py4j-0.10.9.7-src.zip\py4j\java_gateway.py",
line 1322, in [b]call[/b]   File
"C:\pyspark\spark-3.5\python\pyspark\errors\exceptions\captured.py",
line 179, in deco
return f(*a, **kw)
^^^^^^^^^^^   File "C:\pyspark\spark-3.5\python\lib\py4j-0.10.9.7-src.zip\py4j\protocol.py",
line 326, in get_return_value py4j.protocol.Py4JJavaError: An error
occurred while calling o42.showString.  :
org.apache.spark.SparkException: Job aborted due to stage failure:
Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0
in stage 0.0 (TID 0) (LOAN68-9HH3LS3 executor driver):
java.net.SocketException: Connection reset    at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491)   at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source)   at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331)     at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)    at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)  at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Driver stacktrace:     at
org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2856)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2792)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2791)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at
org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2791)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1247)
at
org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1247)
at scala.Option.foreach(Option.scala:407)   at
org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1247)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:3060)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2994)
at
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2983)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
at
org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:989)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2393)    at
org.apache.spark.SparkContext.runJob(SparkContext.scala:2414)     at
org.apache.spark.SparkContext.runJob(SparkContext.scala:2433)     at
org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:530)
at
org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:483)
at
org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:61)
at org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:4333)
at org.apache.spark.sql.Dataset.$anonfun$head$1(Dataset.scala:3316)
at
org.apache.spark.sql.Dataset.$anonfun$withAction$2(Dataset.scala:4323)
at
org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:546)
at
org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:4321)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:125)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:201)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:108)
at
org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:900)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:66)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:4321)  at
org.apache.spark.sql.Dataset.head(Dataset.scala:3316)     at
org.apache.spark.sql.Dataset.take(Dataset.scala:3539)     at
org.apache.spark.sql.Dataset.getRows(Dataset.scala:280)   at
org.apache.spark.sql.Dataset.showString(Dataset.scala:315)    at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)   at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:75)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:52)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)   at
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  at
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)    at
py4j.Gateway.invoke(Gateway.java:282)     at
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)   at
py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.base/java.lang.Thread.run(Thread.java:1583) Caused by:
java.net.SocketException:  Connection reset    at
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
at
java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at
java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at
java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at
java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at
java.base/java.io.DataInputStream.readFully(DataInputStream.java:208)
at
java.base/java.io.DataInputStream.readInt(DataInputStream.java:385)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:774)
at
org.apache.spark.api.python.PythonRunner$$anon$3.read(PythonRunner.scala:766)
at
org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:525)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491)   at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)    at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source)   at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331)     at
org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)    at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)  at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
... 1 more
Process finished with exit code 1

I have tried changing the contents in the bin file but still getting the error.
Code:
[code]import pyspark
from pyspark.sql import SparkSession

spark = SparkSession.builder \
.appName("PySpark Test") \
.getOrCreate()

data = [("Alice", 25), ("Bob", 30), ("Charlie", 35)]
columns =["Name", "Age"]
df = spark.createDataFrame(data, columns)

df.show()
filtered_df = df.filter(df.Age > 30)
filtered_df.show()
spark.stop()
[/code]
Любая помощь приветствуется. 

Подробнее здесь: [url]https://stackoverflow.com/questions/79159948/exception-in-task-0-0-in-stage-0-0-error-in-pyspark[/url]