Почему моя модель градиентного спуска дает постоянную (очень высокую) функцию стоимости квадратной ошибки и не сходится ⇐ Python

[Anonymous]

Итак, я сделал многократную регрессионную модель и попробовал обучение на ней. Код для этого ниже. При запуске кода после нескольких начальных итераций стоимость стоимости застряла на уровне 5818179,55 и не уменьшается. Я также сделал нормализацию Z в наборе данных. Это минимума для набора данных или я делаю что -то не так? < /P>

Код: Выделить всё

#The program aims to make a model for predicting the price of the car

# Car Model Year    Mileage (km)    Engine Size (L) Fuel Type   Transmission    Price ($)
# Toyota Camry  2018    60,000  2.5 Petrol  Automatic   22,000
# Honda Accord  2020    30,000  1.5 Petrol  Automatic   25,500
# Ford Focus    2017    75,000  2.0 Petrol  Manual  15,200
# Nissan Altima 2016    85,000  2.5 Petrol  CVT 14,800
# BMW 3 Series  2019    40,000  2.0 Diesel  Automatic   ???
# Audi A4   2018    50,000  2.0 Petrol  Automatic   28,000
# Mercedes C-Class  2020    35,000  2.0 Petrol  Automatic   38,500
# Hyundai Elantra   2017    70,000  1.8 Petrol  Automatic   13,500
# Kia Optima    2016    90,000  2.4 Petrol  Automatic   12,800
# Chevrolet Malibu  2019    45,000  1.5 Petrol  Automatic   21,000
# Volkswagen Jetta  2018    55,000  1.4 Petrol  Automatic   18,500
# Subaru Legacy 2017    65,000  2.5 Petrol  Automatic   17,000
# Lexus IS  2019    40,000  2.0 Petrol  Automatic   34,000
# Tesla Model 3 2021    20,000  Electric    Electric    Automatic   42,000
# Honda Civic   2015    100,000 1.8 Petrol  Manual  10,500
# Ford Mustang  2020    25,000  5.0 Petrol  Automatic   45,000
# Mazda 6   2018    58,000  2.5 Petrol  Automatic   19,200
# Volvo S60 2019    37,000  2.0 Petrol  Automatic   29,500
# Infiniti Q50  2018    48,000  3.0 Petrol  Automatic   32,000
# Hyundai Sonata    2017    68,000  2.4 Petrol  Automatic   16,500

import numpy as np
import math
import pandas as pd
import copy
import matplotlib.pyplot as plt

data = {
"Year": [2018, 2020, 2017, 2016, 2018, 2020, 2017, 2016, 2019,
2018, 2017, 2019, 2021, 2015, 2020, 2018, 2019, 2018, 2017],
"Mileage": [60000, 30000, 75000, 85000, 50000, 35000, 70000, 90000, 45000,
55000, 65000, 40000, 20000, 100000, 25000, 58000, 37000, 48000, 68000],
"Engine_Size": [2.5, 1.5, 2.0, 2.5, 2.0, 2.0, 1.8, 2.4, 1.5,
1.4, 2.5, 2.0, 0, 1.8, 5.0, 2.5, 2.0, 3.0, 2.4],  # 0 for Electric
"Fuel_Type": ["Petrol", "Petrol", "Petrol", "Petrol",  "Petrol", "Petrol",
"Petrol", "Petrol", "Petrol", "Petrol", "Petrol", "Petrol", "Electric",
"Petrol", "Petrol", "Petrol", "Petrol", "Petrol", "Petrol"],
"Transmission": ["Automatic", "Automatic", "Manual", "CVT", "Automatic",
"Automatic", "Automatic", "Automatic", "Automatic", "Automatic",
"Automatic", "Automatic", "Automatic", "Manual", "Automatic",
"Automatic", "Automatic", "Automatic", "Automatic"],
"Price": [22000, 25500, 15200, 14800,  28000, 38500, 13500, 12800, 21000,
18500, 17000, 34000, 42000, 10500, 45000, 19200, 29500, 32000, 16500]
}

df = pd.DataFrame(data)
df_encoded = pd.get_dummies(df,columns=["Fuel_Type","Transmission"],drop_first=True)

print("The training data is : ")
print(df_encoded)

price_column = df_encoded["Price"]
df_encoded = df_encoded.drop(columns=["Price"])

def z_score_normalization(X):
mu = np.mean(X , axis=0)
sigma = np.std(X, axis=0)
X_norm = (X-mu)/sigma
return X_norm,mu,sigma

X_train = df_encoded.astype(float).to_numpy()
print(X_train)
X_norm,mu,sigma = z_score_normalization(X_train)
print("Normalized data:  ",X_norm)
y_train = price_column.to_numpy()

def compute_cost(X,y,w,b):
m = X.shape[0]
cost = 0
for i in range(m):
f_wb_i = np.dot(w,X[i])+b
err = (f_wb_i - y[i])**2
cost += err
cost /= 2*m
return cost

def compute_gradient(X,y,w,b):
m = X.shape[0]
n = X.shape[1]
dj_db = 0
dj_dw = np.zeros(n)
for i in range(m):
f_wb_i = np.dot(w,X[i])+b
err = (f_wb_i - y[i])
dj_db += err
for j in range(n):
dj_dw[j] += err * X[i,j]
dj_db /= m
dj_dw /= m
return dj_dw, dj_db

def gradient_descent(X,y,w_in,b_in,cost_function,gradient_function,alpha,num_iters):
J_history=[]
iters=[]
w=copy.deepcopy(w_in)
b=b_in
for i in range (num_iters):
dj_dw,dj_db = gradient_function(X,y,w,b)
w=w-alpha*dj_dw
b=b-alpha*dj_db
J_history.append(cost_function(X,y,w,b))
iters.append(i)
if i%1000 == 0:
print(f"Iteration: {i:4d} Cost: {J_history[-1]:8.2f}")
return J_history,w,b,iters

n = X_train.shape[1]
m = X_train.shape[0]
initial_w =  np.random.randint(low=1, high=100, size=n)
initial_b = -6546
iterations = 10000000
alpha = 5e-2

J_hist,w_final,b_final,iters=gradient_descent(X_norm,y_train,initial_w,initial_b,compute_cost,compute_gradient,alpha,iterations)

print(f"w found is : {w_final} and b found is : {b_final}")

plt.plot(iters, J_hist, linestyle='-', color='b', label="Cost Reduction")
plt.xlabel("Iterations")
plt.ylabel("Cost")
plt.show()

# x = np.array([2019, 45000, 2.0, 1, 0, 1, 0])
# x = (x-mu)/sigma

# prediction = np.dot(w_final,x)+b_final
# print(f"The model predicted the output as: {prediction}")```

ожидание состояло в том, что стоимость составит значение около нуля.

Подробнее здесь: https://stackoverflow.com/questions/795 ... error-cost