Claude/Sonnet Python API - больше токенов замораживает, меньше токенов усечны ⇐ Python

[Anonymous]

My prompt was to create a blog with 5 sections covering 5 different individuals in an industry niche.
When I had max_tokens= 300 (and then 1000), it ran quickly, but the result was not complete.
When I tried max_tokens=1500 (or 3000), then the program freezes, and doesn't come back after even about 5 minutes. Я спросил Клода, что делать, и он предоставил этот код для повторения, но в любое время, когда я более 1500 токенов, я получаю тайм -аут. < /P>
def get_claude_response(prompt, max_retries=3, initial_max_tokens=2000, timeout=60):
for attempt in range(max_retries):
try:
message = client.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=initial_max_tokens * (attempt + 1), # Increase max_tokens on each retry
messages=[
{"role": "user", "content": prompt}
],
timeout=timeout # Set a timeout for the API call
)
return message.content[0].text
except anthropic.APITimeoutError:
print(f"Timeout occurred. Retrying with increased max_tokens. Attempt {attempt + 1}/{max_retries}")
time.sleep(2) # Wait for 2 seconds before retrying
except Exception as e:
print(f"An error occurred: {str(e)}")
return None

print("Max retries reached. Unable to get a complete response.")
return None

# Example usage
prompt = "details here..."

print("Prompt:")
print(prompt)

response = get_claude_response(prompt)

if response:
print(response)
else:
print("Failed to get a response from Claude.")

I asked Claude itself for a solution and it came up with a method to call and get back chunks. Although it worked, it was very slow and was 5 calls, which might be costing me more in the long run than one call with more tokens.

def call_claude_sonnet_in_chunks(prompt, max_chunks=10, tokens_per_chunk=500, timeout=55):
full_response = ""
messages = [{"role": "user",
"content": prompt}]

for i in range(max_chunks):
try:
print ("Call Claude:Sonnet i=" + str(i) + " tokens_per_chunk=" + str(tokens_per_chunk))
message = client.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=tokens_per_chunk,
messages=messages,
timeout=timeout
)

chunk = message.content[0].text
full_response += chunk

# Check if the response is complete
if chunk.strip().endswith('.'):
break

# Prepare for the next chunk
messages.append({"role": "assistant", "content": chunk})
messages.append({"role": "user", "content": "Please continue your previous response."})

time.sleep(1) # Small delay between requests
except anthropic.APITimeoutError:
print(f"Timeout occurred on chunk {i + 1}. Moving to next chunk.")
except Exception as e:
print(f"An error occurred: {str(e)}")
break

return full_response.strip()


Подробнее здесь: https://stackoverflow.com/questions/789 ... -truncates