Adrian Johnson
TLDR:
ChatGPT and Google Gemini were evaluated by Prasad Calyam at the University of Missouri on their performance in passing the Certified Ethical Hacker (CEH) exam. Both AI models demonstrated high accuracy rates, with Google Gemini slightly outperforming ChatGPT. However, ChatGPT excelled in providing detailed, clear, and concise explanations. The study also introduced confirmation queries to enhance accuracy further.
The study led by Prasad Calyam at the University of Missouri, in collaboration with Amrita University, India, investigated the effectiveness of AI-driven tools like ChatGPT and Google Gemini in enhancing cybersecurity defenses. The study evaluated how AI models perform when challenged with questions from the Certified Ethical Hacker (CEH) exam.
Key findings from the research include:
- ChatGPT and Google Gemini successfully passed the CEH exam, showcasing potential in ethical hacking practices.
- Google Gemini slightly outperformed ChatGPT in overall accuracy, while ChatGPT excelled in providing detailed explanations.
- Confirmation queries were introduced to refine accuracy further, highlighting the potential for iterative query processing in cybersecurity applications.
The study emphasizes the role of AI tools as complementary rather than substitutive to human expertise in cybersecurity. While the AI models showed promising performance, caution was advised against over-reliance on AI tools for comprehensive cybersecurity solutions, underlining the criticality of human judgment and problem-solving skills in devising robust defense strategies.
Looking ahead, the study advocates for further research to enhance the reliability and usability of AI-driven ethical hacking tools, including improving AI models’ handling of complex queries, expanding multi-language support, and establishing ethical guidelines for their deployment. The study serves as a benchmark for evaluating AI performance in ethical hacking and advocates for a balanced approach that leverages AI’s strengths while respecting its current limitations in cybersecurity applications.
