weber.st.michael
New member
We've been experimenting with a few AI agents for test automation lately. While the speed of code generation is impressive, the real struggle is the 'non-deterministic' output.
I was reading a good breakdown on generative ai testing challenges here: https://testomat.io/blog/challenges-of-generative-ai-for-software-testing/ and it perfectly describes the reliability issues we're facing.
How do you guys handle LLM hallucinations when they generate test assertions? Do you just use human-in-the-loop, or is there a smarter way to automate the validation of the AI itself?
I was reading a good breakdown on generative ai testing challenges here: https://testomat.io/blog/challenges-of-generative-ai-for-software-testing/ and it perfectly describes the reliability issues we're facing.
How do you guys handle LLM hallucinations when they generate test assertions? Do you just use human-in-the-loop, or is there a smarter way to automate the validation of the AI itself?