Столбец даты со значением «-» в кадре данных Spark ⇐ Python

[Гость]

I have a date column in data bricks, which has a value '-' for 1 record. I am unable to either filter it out or select this row. How can i use this value '-' in where clause?
I have tried

Код: Выделить всё

spark_df_2023_fil=spark_df_2023.filter(spark_df_2023["navdt"] == '-')
spark_df_2023_fil=spark_df_2023.filter(spark_df_2023["navdt"] != '-')

but both these give me 0 records.


Источник: https://stackoverflow.com/questions/781 ... -dataframe