In this project, I updated 3 examples from https://github.com/joaomdmoura/crewAI-examples/ to use the latest version of CrewAI (0.30.11).
To track the performance of each example, I added AgentOps to each project.
Once everything was setup, I ran each example project with ChatGPT4 and ChatGPT4o.
Check the results below!
- AgentOps Id: 6763a693-542b-47eb-bd51-40a5c09f86c0
- LLM Cost: $0.18
- Time: 1m 33s
- Prompt Tokens: 14,487
- Completion Tokens: 6,948
- AgentOps Id: a60b39c0-ae51-48be-b430-08383f95d6fc
- LLM Cost: $0.23
- Time: 1m 33s
- Prompt Tokens: 9,416
- Completion Tokens: 4,557
- Cost: ChatGPT4o is about 21.7% cheaper than ChatGPT4-Turbo.
- Response Time: Both ChatGPT4o and ChatGPT4-Turbo have the same response time.
- Token Usage: ChatGPT4o uses about 41.9% more tokens overall compared to ChatGPT4-Turbo, indicating potentially more detailed responses.
- AgentOps Id: c826fdea-8710-4a60-b1a4-5825acb41246
- LLM Cost: $0.36
- Time: 2m 45s
- Prompt Tokens: 44,736
- Completion Tokens: 9,370
- AgentOps Id: 55da4c75-8c76-4e19-946b-0abc94929c84
- LLM Cost: $0.48
- Time: 4m 40s
- Prompt Tokens: 33,699
- Completion Tokens: 6,277
- Cost: ChatGPT4o is about 25% cheaper than ChatGPT4-Turbo.
- Response Time: ChatGPT4o is about 41% faster than ChatGPT4-Turbo.
- Token Usage: ChatGPT4o uses about 32% more tokens overall compared to ChatGPT4-Turbo.
- AgentOps Id: f5d5f942-3b0f-47a2-8cd3-b843eef72116
- LLM Cost: $0.33
- Time: 3m 14s
- Prompt Tokens: 38,838
- Completion Tokens: 9,051
- AgentOps Id: fbe4b206-12ce-49eb-b86b-53ed56b6ec85
- LLM Cost: $0.52
- Time: 3m 36s
- Prompt Tokens: 40,777
- Completion Tokens: 3,793
- Cost: ChatGPT4o is about 36.5% cheaper than ChatGPT4-Turbo.
- Response Time: ChatGPT4o is about 10.2% faster than ChatGPT4-Turbo.
- Token Usage: ChatGPT4o uses about 11.3% fewer tokens overall compared to ChatGPT4-Turbo.