Meet John George, CTO at CaseGen.ai, who joined the BotCast # 6 to answer our usual questions on building with Voice AI.
Who are you?
I'm John George, CTO of CASEGEN.AI at the moment. We're building a voice AI intake platform for law firms. It handles full legal intake, routes to the right staff, and converts the conversation into structured information firms can use and import in their CRM. I'm responsible for the full stack essentially.
How did you end up using voice AI?
I started experimenting with Voice AI a couple of years ago and I really loved it. I was watching a lot of tutorials on YouTube and I just had a lot of ideas for products I could build on there. So, I started building in public about six months in on Pipecat and based on that, got a lot of incoming interest, started doing consultancy, and building products. Ultimately, that content connected me with CASEGEN. And here we are today.
A moment of joy and pain using voice tech.
For joy, definitely a couple of months ago, we had to redo our voice pipeline and I just got everything tuned in really well. We were just getting great results, everything improved actually: the latency, transcription accuracy, turn detection, the model's intelligence and reliability, the naturalness of the voice, and also, to top it all off, the costs came down by 60%. I had great results when I was testing it out. I got this working in the morning in the UK and I was waiting for them to wake up to tell them about this. We got this in production and immediately, call after call, this thing was just performing so much better. Even though we say that it's a voice AI to start off the call, people were still asking, "Is this really an AI?" And some people were asking for the bot's name, as if they wanted to tell the office that it was doing a good job. So, that was a good week.
In terms of pain, I would say this bot works so well because everything works together in harmony. That means there are many points of failure; so many things can go wrong. There is the telephony, the transcription, the VAD, the smart turn detector, the LLM itself, and the TTS. Anything can go wrong in those, and that is just during the live call. That is where the pain comes in. There are lots of points of failure.
What lesson would you share with other builders?
I would recommend anyone starting out to learn lower-level voice orchestration and build on a stack you would be comfortable building in production. These open-source orchestration frameworks are very accessible these days and you can build quite rapidly on them with the modern AI tools that we have. This will help you build a good mental model of how Voice AI works, and then be able to debug from first principles and understand where in the orchestration you should solve certain problems. I would recommend that, and also try to get an understanding of the limits of the tech these days and where it is going. It is good to read about full duplex models, how they work, and compare those to the cascading pipeline, half cascade, and speech to speech.
Where do you see the industry in 12 months and in 5 years?
I think in five years, that is where we are going to see speech-to-speech and full duplex have their day. We need to be able to solve the problem of how to get accurate backchanneling in response to what people are saying, and also have the assistant barge in with tool calls and interruption when people are going off track. Backchanneling makes things much more engaging and natural, so I think that will become very popular in the next five years. I also think that multilingual, fully polyglot bots are going to have their day as well and become table stakes in that sort of time. They are going to catch up with the quality of English bots in that time, essentially. I think it will be even across the board. A lot of this depends on how consumers react to the product, with how AI is going and the regulatory environment. We might see an increase in the discrepancy between adoption and capability.
