AI Finding Logical Fallacies
A note on one model test
A small test compared Gemma 2, Qwen 2, and Mistral Nemo with more than ten other models on logical fallacy detection.
1 Test for AI
- to detect informal logic errors
New gemma2 and qwen2 are good, but in their test phi3 showed the best result/
See for details how and what was tested https://www.glukhov.org/post/2024/08/gemma2-qwen2-mistral-nemo12b/
Here is the table with the test results
2 Their Conclusion
They expected LLMs to find in sample text the Strawman fallacy, Ad Hominem, Appeal to authority and Oversimplification. The text was from some journalist article about politics.
Most new models can see in our sample text Appeal to Authority, and some see Ad Hominem, some Oversimplification. Good results showed qwen2:7b-instruct-q8_0 and gemma2:9b-instruct-q8_0.
But still, winers remain the same.
- phi3:14b-medium-4k-instruct-q6_K
- phi3:3.8b-mini-4k-instruct-q8_0
and only then go qwen2:7b-instruct-q8_0 and gemma2:9b-instruct-q8_0
