The Context
A multi-billion machinery manufacturer was building a digital layer on top of its hardware products. With a strong IT team in place, they invested heavily in copilots and advanced LLM pipelines (RAG, fine-tuned embeddings, reranking, hybrid search). Despite the effort, experts were frustrated—answers were incomplete and unreliable.
The Challenge
The deeper issue wasn’t the models. It was knowledge itself. Much of the expertise was undocumented, buried in chaotic Jira inquiries.
-
70+ categories, dozens of labels, and low categorization accuracy (~35%).
-
1 full-time equivalent (FTE) spent each month manually routing tickets before real work even started.
-
Experts drowning in follow-ups: clarifying incomplete tickets, resending wrongly categorized issues, and chasing unreachable requesters. Clarification often took longer than solving the issue itself.
Our Approach
We broke the problem into phases, starting with routing and triage.
-
Analyzed 250,000+ Jira issues across the product IT landscape.
-
Identified that 2 key components drove ~50% of the workload, broken into ~120 distinct patterns.
-
Built an AI/ML classification system to automatically map these patterns to required fields, workflows, and approval steps.
-
Introduced a human-in-the-loop pipeline to push accuracy beyond 85% and continuously improve.
Results
-
Automated categorization improved from 35% → 75% out-of-the-box, and beyond 85% with feedback loop.
-
Freed up the equivalent of 1 FTE per month previously wasted on manual routing.
-
Reduced ticket reopens and follow-ups by guiding users at intake—ensuring required info was captured upfront.
-
Deployed a chatbot to handle repetitive questions and steer users through correct workflows.
Impact
The system became a translation layer between IT complexity and user goals. Instead of users struggling with categories and fields, the system understood intent and routed accordingly. This clarity not only accelerated response times but also laid the groundwork for scalable automation—approval workflows, extended knowledge bases, and intent-driven user support.
