Перенос моделей/таблиц Pentaho в PostgrestDB

Перенос моделей/таблиц Pentaho в PostgrestDB ⇐ Python

1 сообщение • Страница 1 из 1

Anonymous

Перенос моделей/таблиц Pentaho в PostgrestDB

Сообщение Anonymous » 04 окт 2024, 19:44

Добрый день.
У меня возникли проблемы с pentaho – интеграцией данных – чайником

api.sh:
#!/bin/bash

# ------------------------------------------------- EJERCICIO Y TRIMESTRE ------------------------------------------------- #
# Definir valores predeterminados
EJERCICIO=""
TRIMESTRE=""
POSTGRES_PASSWORD="2134"
echo "Configuración de conexión a la base de datos:"
echo "Host: $POSTGRES_HOST"
echo "Puerto: $POSTGRES_PORT"
echo "Nombre de la base de datos: $POSTGRES_NAME"
echo "Usuario: $POSTGRES_USER"
echo "Contraseña: $POSTGRES_PASSWORD"  # Cuidado al imprimir contraseñas

# Procesar los argumentos del script
while [[ $# -gt 0 ]]; do
key="$1"

case $key in
--ejercicio)
EJERCICIO="$2"
shift
;;
--trimestre)
TRIMESTRE="$2"
shift
;;
*)
# Argumento desconocido o mal formado
echo "Argumento desconocido: $1"
exit 1
;;
esac
shift
done

# Verificar si se proporcionaron los parámetros requeridos
if [ -z "$EJERCICIO" ] || [ -z "$TRIMESTRE" ]; then
echo "Error: Falta uno o más parámetros requeridos."
echo "Uso: ./eph_descaga_csv.sh --ejercicio EJERCICIO --trimestre TRIMESTRE"
exit 1
fi

# ------------------------------------------------- KETTLE CONFIGURATION ------------------------------------------------- #

# Setear el path al ejecutable de kitchen
[ -z "$KITCHEN" ] && export KITCHEN="/opt/pentaho/data-integration/kitchen.sh"
# Setear el path al directorio de ETL
[ -z "$ETL_PATH" ] && export ETL_PATH="/opt/tdf/datalaketdf/etl/eph/kettle"
ETL_FILE="$ETL_PATH/JobEPHDescargaCSV.kjb"

# Setear log level
[ -z "$LOG_LEVEL" ] && export LOG_LEVEL="Basico"
#(Basico, Detallado, Debug, NivelFila, Error, Nada)

# jdbc.properties
export JNDI_DIR="$ETL_PATH/jndi/"

if [ ! -d "$JNDI_DIR" ]; then
mkdir "$JNDI_DIR"
fi

fileName="$JNDI_DIR/jdbc.properties"
# Recreate  file
rm "$fileName"
touch "$fileName"

# Write properties to file
echo "$KETTLE_DB_NAME/type=javax.sql.DataSource" > $fileName
echo "$KETTLE_DB_NAME/driver=org.postgresql.Driver" >> $fileName
echo "$KETTLE_DB_NAME/url=jdbc:postgresql://$POSTGRES_HOST:$POSTGRES_PORT/$POSTGRES_NAME" >> $fileName
echo "$KETTLE_DB_NAME/user=$POSTGRES_USER" >> $fileName
echo "$KETTLE_DB_NAME/password=$POSTGRES_PASSWORD" >> $fileName

# ---------------------------------------------------- KETTLE RUN ------------------------------------------------------- #
KETTLE_JNDI_ROOT=$JNDI_DIR $KITCHEN -file=$ETL_FILE -level=$LOG_LEVEL -param:EJERCICIO=$EJERCICIO -param:TRIMESTRE=$TRIMESTRE

этот sh - один из тех, которые мы используем для связи py API с Ketter и Ketter с postgestdb.
Все работает нормально, но проблема в Ketter сначала спросил меня о несуществующей базе данных.
Хорошо... Я создал с помощью sql.init.
А теперь спроси меня об этом >

Код: Выделить всё

2024/10/04 16:41:30 - Insert Log.0 - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Unexpected batch update error committing the database connection.
2024/10/04 16:41:30 - Insert Log.0 - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseBatchException:
2024/10/04 16:41:30 - Insert Log.0 - Error updating batch
2024/10/04 16:41:30 - Insert Log.0 - Batch entry 0 INSERT INTO "public".logs_etl (etl_name, "state", step) VALUES ( 'Descarga CSV Indec',  'Inicio',  'Iniciando descarga de CSV') was aborted: ERROR: relation "public.logs_etl" does not exist
2024/10/04 16:41:30 - Insert Log.0 -   Position: 13  Call getNextException to see other errors in the batch.
2024/10/04 16:41:30 - Insert Log.0 -
2024/10/04 16:41:30 - Insert Log.0 -    at org.pentaho.di.core.database.Database.createKettleDatabaseBatchException(Database.java:1440)
2024/10/04 16:41:30 - Insert Log.0 -    at org.pentaho.di.core.database.Database.emptyAndCommit(Database.java:1429)
2024/10/04 16:41:30 - Insert Log.0 -    at org.pentaho.di.trans.steps.tableoutput.TableOutput.dispose(TableOutput.java:590)
2024/10/04 16:41:30 - Insert Log.0 -    at org.pentaho.di.trans.step.RunThread.run(RunThread.java:97)
2024/10/04 16:41:30 - Insert Log.0 -    at java.base/java.lang.Thread.run(Thread.java:829)
2024/10/04 16:41:30 - Insert Log.0 - Caused by: java.sql.BatchUpdateException: Batch entry 0 INSERT INTO "public".logs_etl (etl_name, "state", step) VALUES ( 'Descarga CSV Indec',  'Inicio',  'Iniciando descarga de CSV') was aborted: ERROR: relation "public.logs_etl" does not exist
2024/10/04 16:41:30 - Insert Log.0 -   Position: 13  Call getNextException to see other errors in the batch.
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:169)
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.core.ResultHandlerDelegate.handleError(ResultHandlerDelegate.java:52)
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2285)
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:520)
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.jdbc.PgStatement.internalExecuteBatch(PgStatement.java:878)
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:901)
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1644)
2024/10/04 16:41:30 - Insert Log.0 -    at org.pentaho.di.core.database.Database.emptyAndCommit(Database.java:1416)
2024/10/04 16:41:30 - Insert Log.0 -    ... 3 more
2024/10/04 16:41:30 - Insert Log.0 - Caused by: org.postgresql.util.PSQLException: ERROR: relation "public.logs_etl" does not exist
2024/10/04 16:41:30 - Insert Log.0 -   Position: 13
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2552)
2024/10/04 16:41:30 - Insert Log.0 -    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2284)
2024/10/04 16:41:30 - Insert Log.0 -    ... 8 more
2024/10/04 16:41:30 - InitLogDescarcarCSVIndec - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Errors detected!
2024/10/04 16:41:31 - InitLogDescarcarCSVIndec - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Errors detected!
2024/10/04 16:41:31 - Kitchen - ERROR (version 9.4.0.0-343, build 0.0 from 2022-11-08 07.50.27 by buildguy) : Finished with errors

Я не знаю, как выполнить миграцию или сделать что-то подобное, например makemigrations, и мигрировать, как python/django
Спасибо

Подробнее здесь: https://stackoverflow.com/questions/790 ... ostgrestdb

Anonymous

1 сообщение • Страница 1 из 1

Вернуться в «Python»