В соответствии с документацией в библиотеке Petl Python: «Этот пакет широко использует отложенные вычисления и итераторы. Обычно это означает, что конвейер фактически не будет выполняться до тех пор, пока не будут запрошены данные». Конвейеры ETL
Я попытался реализовать это в следующем коде:
`import petl as etl
import re
# Functions to be used for conversions
def convert_commissioning_year(year):
match year:
case None:
return 0
case "NA":
return 0
case str.find("s"):
# If the year is a string with an s at the end, we assume it indicates decade (like 2970s)
middle_year = int(str(year)[:-1]) + 5
return middle_year
case _:
return int(year)
# Clean 'max_speed_warp_factor' by extracting the numeric portion
def extract_numeric_speed(speed):
print("JOLL")
if isinstance(speed, str):
match = re.search(r"[\d]+[.][\d]", speed)
if match:
return float(match.group())
return None
# Load the CSV data using petl
table = etl.fromcsv('/Documents/star_trek_table.csv')
# Standardizing column names to snake_case
table2 = etl.rename(table, {
'Ship Name': 'Name',
'Registry Number': 'RegNo',
'Captain(s)': 'Captain',
'Launch Date': 'Commissioning',
'Notable Appearances (Series/Movies)': 'Appearances',
'Ship’s Complement': 'Complement',
'Max Speed (Warp Factor)': 'Warp'
})
# Convert 'launch_date' and 'ships_complement' to integers (handling any errors)
table3 = etl.convert(table2, 'Commissioning', convert_commissioning_year)
table4 = etl.convert(table3, 'Complement', int)
table5 = etl.convert(table4, 'Warp', extract_numeric_speed)
# Strip any leading/trailing whitespace in string columns
table6 = etl.convertall(table5, lambda v: str.strip(v))
# Display the cleaned data (in a limited format for brevity)
print(etl.lookall(table6))
# Save the cleaned data back to CSV
etl.tocsv(table6, '/Documents/cleaned_star_trek_table.csv')
print("END")
Распечатка:
`/petl/.venv/bin/python /petl/main.py
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| Name | RegNo | Class | Captain | Commissioning | Appearances | Complement | Warp |
+================================+==============+=======================+======================================+===============+=======================================================+============+======+
| 'USS Enterprise (NCC-1701)' | 'NCC-1701' | 'Constitution-class' | 'Capt. James T. Kirk, Capt. Pike' | None | 'Star Trek: TOS, Movies I-IV, VI' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Enterprise (NCC-1701-A)' | 'NCC-1701-A' | 'Constitution-class' | 'Capt. James T. Kirk' | None | 'Star Trek IV-VI' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Enterprise (NCC-1701-B)' | 'NCC-1701-B' | 'Excelsior-class' | 'Capt. John Harriman' | None | 'Star Trek: Generations' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Enterprise (NCC-1701-C)' | 'NCC-1701-C' | 'Ambassador-class' | 'Capt. Rachel Garrett' | None | 'Star Trek: TNG (Episode: "Yesterday\'s Enterprise")' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Enterprise (NCC-1701-D)' | 'NCC-1701-D' | 'Galaxy-class' | 'Capt. Jean-Luc Picard' | None | 'Star Trek: TNG, Movies VII, VIII' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Enterprise (NCC-1701-E)' | 'NCC-1701-E' | 'Sovereign-class' | 'Capt. Jean-Luc Picard' | None | 'Movies VIII-X' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Voyager (NCC-74656)' | 'NCC-74656' | 'Intrepid-class' | 'Capt. Kathryn Janeway' | None | 'Star Trek: Voyager' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Defiant (NX-74205)' | 'NX-74205' | 'Defiant-class' | 'Capt. Benjamin Sisko' | None | 'Star Trek: Deep Space Nine' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Discovery (NCC-1031)' | 'NCC-1031' | 'Crossfield-class' | 'Capt. Gabriel Lorca, Capt. Pike' | None | 'Star Trek: Discovery' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Cerritos (NCC-75567)' | 'NCC-75567' | 'California-class' | 'Capt. Carol Freeman' | None | 'Star Trek: Lower Decks' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Shenzhou (NCC-1227)' | 'NCC-1227' | 'Walker-class' | 'Capt. Philippa Georgiou' | None | 'Star Trek: Discovery' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Excelsior (NCC-2000)' | 'NCC-2000' | 'Excelsior-class' | 'Capt. Hikaru Sulu' | None | 'Star Trek VI, Star Trek III' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Saratoga (NCC-31911)' | 'NCC-31911' | 'Miranda-class' | 'Capt. J.G. Sanders' | None | 'Star Trek IV, DS9 (pilot)' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Prometheus (NX-59650)' | 'NX-59650' | 'Prometheus-class' | 'Various (Multivector Assault Mode)' | None | 'Star Trek: Voyager (Episode: "Message in a Bottle")' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Thunderchild (NCC-63549)' | 'NCC-63549' | 'Akira-class' | 'NA' | None | 'Star Trek: First Contact' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Yamato (NCC-71807)' | 'NCC-71807' | 'Galaxy-class' | 'Capt. Donald Varley' | None | 'Star Trek: TNG' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Equinox (NCC-72381)' | 'NCC-72381' | 'Nova-class' | 'Capt. Rudolph Ransom' | None | 'Star Trek: Voyager (Episode: "Equinox")' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Reliant (NCC-1864)' | 'NCC-1864' | 'Miranda-class' | 'Capt. Clark Terrell' | None | 'Star Trek II: The Wrath of Khan' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Stargazer (NCC-2893)' | 'NCC-2893' | 'Constellation-class' | 'Capt. Jean-Luc Picard' | None | 'Star Trek: TNG (Episode: "The Battle")' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
| 'USS Titan (NCC-80102)' | 'NCC-80102' | 'Luna-class' | 'Capt. William Riker' | None | 'Star Trek: Lower Decks, Star Trek: Picard' | None | None |
+--------------------------------+--------------+-----------------------+--------------------------------------+---------------+-------------------------------------------------------+------------+------+
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
JOLL
END
Process finished with exit code 0
Почему функция extract_numeric_speed оценивается в конце и почему во всех преобразованных столбцах я получаю значения None? Спасибо.
Я ожидал получить преобразованные значения в соответствующих столбцах.
Я переименовал таблицу в последовательность таблиц (таблица1, таблица2, и т. д.) принять во внимание ленивые вычисления Petl.
Исходная таблица выглядит следующим образом:
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| Ship Name | Registry Number | Class | Captain(s) | Launch Date | Notable Appearances (Series/Movies) | Ship’s Complement | Max Speed (Warp Factor) |
+================================+=================+=======================+======================================+=============+=======================================================+===================+===========================+
| 'USS Enterprise (NCC-1701)' | 'NCC-1701' | 'Constitution-class' | 'Capt. James T. Kirk, Capt. Pike' | '2245' | 'Star Trek: TOS, Movies I-IV, VI' | '430' | 'Warp 8' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Enterprise (NCC-1701-A)' | 'NCC-1701-A' | 'Constitution-class' | 'Capt. James T. Kirk' | '2286' | 'Star Trek IV-VI' | '500' | 'Warp 9' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Enterprise (NCC-1701-B)' | 'NCC-1701-B' | 'Excelsior-class' | 'Capt. John Harriman' | '2293' | 'Star Trek: Generations' | '750' | 'Warp 9.6' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Enterprise (NCC-1701-C)' | 'NCC-1701-C' | 'Ambassador-class' | 'Capt. Rachel Garrett' | '2344' | 'Star Trek: TNG (Episode: "Yesterday\'s Enterprise")' | '700' | 'Warp 9' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Enterprise (NCC-1701-D)' | 'NCC-1701-D' | 'Galaxy-class' | 'Capt. Jean-Luc Picard' | '2363' | 'Star Trek: TNG, Movies VII, VIII' | '1014' | 'Warp 9.6' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Enterprise (NCC-1701-E)' | 'NCC-1701-E' | 'Sovereign-class' | 'Capt. Jean-Luc Picard' | '2372' | 'Movies VIII-X' | '885' | 'Warp 9.99' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Voyager (NCC-74656)' | 'NCC-74656' | 'Intrepid-class' | 'Capt. Kathryn Janeway' | '2371' | 'Star Trek: Voyager' | '150' | 'Warp 9.975' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Defiant (NX-74205)' | 'NX-74205' | 'Defiant-class' | 'Capt. Benjamin Sisko' | '2370' | 'Star Trek: Deep Space Nine' | '50' | 'Warp 9.982' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Discovery (NCC-1031)' | 'NCC-1031' | 'Crossfield-class' | 'Capt. Gabriel Lorca, Capt. Pike' | '2256' | 'Star Trek: Discovery' | '136' | 'Warp 9.99 / Spore Drive' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Cerritos (NCC-75567)' | 'NCC-75567' | 'California-class' | 'Capt. Carol Freeman' | 'NA' | 'Star Trek: Lower Decks' | '80' | 'Warp 7' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Shenzhou (NCC-1227)' | 'NCC-1227' | 'Walker-class' | 'Capt. Philippa Georgiou' | '2256' | 'Star Trek: Discovery' | '215' | 'Warp 8' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Excelsior (NCC-2000)' | 'NCC-2000' | 'Excelsior-class' | 'Capt. Hikaru Sulu' | '2287' | 'Star Trek VI, Star Trek III' | '750' | 'Warp 9.5' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Saratoga (NCC-31911)' | 'NCC-31911' | 'Miranda-class' | 'Capt. J.G. Sanders' | 'NA' | 'Star Trek IV, DS9 (pilot)' | '36' | 'Warp 8' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Prometheus (NX-59650)' | 'NX-59650' | 'Prometheus-class' | 'Various (Multivector Assault Mode)' | '2374' | 'Star Trek: Voyager (Episode: "Message in a Bottle")' | '4' | 'Warp 9.99' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Thunderchild (NCC-63549)' | 'NCC-63549' | 'Akira-class' | 'NA' | 'NA' | 'Star Trek: First Contact' | '500' | 'Warp 9.8' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Yamato (NCC-71807)' | 'NCC-71807' | 'Galaxy-class' | 'Capt. Donald Varley' | 'NA' | 'Star Trek: TNG' | '1014' | 'Warp 9.6' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Equinox (NCC-72381)' | 'NCC-72381' | 'Nova-class' | 'Capt. Rudolph Ransom' | '2370s' | 'Star Trek: Voyager (Episode: "Equinox")' | '80' | 'Warp 8.6' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Reliant (NCC-1864)' | 'NCC-1864' | 'Miranda-class' | 'Capt. Clark Terrell' | '2285' | 'Star Trek II: The Wrath of Khan' | '281' | 'Warp 8.5' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Stargazer (NCC-2893)' | 'NCC-2893' | 'Constellation-class' | 'Capt. Jean-Luc Picard' | 'NA' | 'Star Trek: TNG (Episode: "The Battle")' | '500' | 'Warp 9' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
| 'USS Titan (NCC-80102)' | 'NCC-80102' | 'Luna-class' | 'Capt. William Riker' | 'NA' | 'Star Trek: Lower Decks, Star Trek: Picard' | '350' | 'Warp 9.99' |
+--------------------------------+-----------------+-----------------------+--------------------------------------+-------------+-------------------------------------------------------+-------------------+---------------------------+
Подробнее здесь: https://stackoverflow.com/questions/790 ... sing-grief
