AI-Powered HR Assistant

The Challenge

A large enterprise software company had a familiar problem in their HR department: a Slack channel where employees asked questions, and a team that was drowning trying to keep up.

The reality on the ground:

Constant interruptions — The HR team maintained an on-call rotation specifically to monitor and respond to the Slack channel. Someone always had to be watching.
Repetitive questions — The vast majority of inquiries were things like "Where do I find the benefits documentation?" or "What's the time-off policy?" — questions with answers that existed somewhere in the company's documentation.
Scattered documentation — HR policies and information were spread across Confluence, Google Docs, PDFs, and various other locations. Employees couldn't find what they needed, so they asked in Slack instead.
Outdated docs — Even when documentation existed, it was frequently out of date, making self-service unreliable.
No bandwidth for real work — The team was so busy answering routine questions that they had no time for the work that actually required their expertise — updating policies, improving processes, handling sensitive situations.

The HR team asked a simple question: could an AI chatbot handle the routine questions and only notify them when something genuinely required a human?

The Approach

Rather than deploying a generic chatbot, we took a ground-up approach that prioritized two things: accuracy and not being annoying.

Building the Test Foundation

Before writing a single line of bot code, we worked with the HR team to create a comprehensive test dataset:

Standard questions and expected answers — The most common questions employees asked, paired with the correct responses and source documents
Edge cases — Questions that should be referred to a human (sensitive topics, complex situations, benefits specifics that change frequently)
Adversarial cases — Attempts to misuse the bot, get it to say inappropriate things, or otherwise go off the rails
Ignore cases — Messages the bot should recognize as not requiring a response: someone saying "thanks," replying to another person, casual conversation

This test dataset became the benchmark for every iteration of the system, allowing us to measure accuracy objectively and catch regressions immediately.

Document Processing

We ingested and processed the HR team's entire documentation corpus:

Confluence pages organized by topic
PDFs of formal policies
Google Docs with supplementary information
Frequently asked questions and their canonical answers

Each document was processed using semantic search and vector representations, allowing the system to find relevant content based on meaning rather than just keyword matching. Individual pages were analyzed for topic coverage so the system could efficiently retrieve the most relevant context for any given question.

The Solution

The Slackbot operated through a multi-layered decision process:

Layer 1: Should It Respond?

Before attempting to answer anything, the system first determined whether a response was appropriate:

Is this actually a question? — Casual conversation, thank-you messages, and replies to other people were ignored
Is this an emergency? — Situations involving safety (the system was specifically trained to handle scenarios like active threats) received immediate direction to contact appropriate authorities, not documentation links
Is this too sensitive for AI? — Topics requiring privacy, nuance, or human judgment were routed directly to the HR team with a compassionate acknowledgment

This layer turned out to be one of the most valuable parts of the system. The company's IT department had deployed a different chatbot that replied to every single message in their support channel, making it nearly unusable. By contrast, the HR bot's ability to stay quiet when it had nothing useful to add was consistently cited as its best feature.

Layer 2: Find Relevant Information

For questions the system determined it should answer:

Semantic search identified the most relevant pages across all documentation sources
A secondary AI review confirmed that retrieved pages actually contained answer-relevant content (not just topically related text)
The most relevant excerpts were assembled as context for the response

Layer 3: Generate and Cite

The response was generated with:

A carefully tuned "character" that was helpful, professional, and — for sensitive situations — genuinely compassionate
Direct links to source documentation so employees could verify and read further
Clear acknowledgment when a question fell outside its capabilities, with a warm handoff to a human

Review System

Initially, every response went through a private Slack channel where the HR team could approve or deny answers before they were posted. This served two purposes:

Quality control — Ensuring the bot wasn't providing incorrect information
Training data — Denied answers with corrections fed back into improving the system

As accuracy improved, the team gradually shifted to spot-checking rather than reviewing every response.

The Results

After deployment:

70% of questions handled entirely by the AI without human intervention
96% accuracy on responses as measured through the HR team's review process
30% of questions appropriately routed to humans for sensitive, complex, or nuanced topics
On-call burden eliminated — The HR team no longer needed someone constantly monitoring the channel

Unexpected Wins

Documentation audit for free — During development and testing, the AI consistently flagged instances where it gave incorrect answers because the underlying documentation was outdated. This gave the HR team a fast, systematic way to identify which policies and documents needed updating — something they'd never had time to do proactively.

Compassionate responses — One of the most meaningful pieces of feedback came when an employee posted about a family medical emergency, asking about leave policies. The bot recognized this as a situation requiring human support, but its initial response — expressing genuine sympathy before directing the employee to the right person — drew positive feedback from both the employee and the HR team. The investment in tuning the bot's character for sensitive situations paid off in ways we hadn't fully anticipated.

Channel usability — By not cluttering the channel with responses to every message, the Slack channel actually became more useful as a communication space, not less.

Before and After

| Metric | Before | After | |--------|--------|-------| | HR on-call for Slack | Constant rotation | Minimal monitoring | | Routine question response | Hours (depending on availability) | Seconds | | Documentation freshness | Unknown, rarely audited | Continuously surfaced by AI | | Channel usability | Functional but overwhelming | Cleaner, more focused | | HR team bandwidth | Consumed by routine answers | Freed for strategic work |

Key Learnings

"Not annoying" is a feature — The decision to optimize heavily for knowing when not to respond was one of the highest-value investments in the project. A bot that's helpful when needed and invisible when not is dramatically more valuable than one that responds to everything.
Test data is the foundation — Building a comprehensive test dataset before building the bot meant we could measure progress objectively, catch regressions, and make confident decisions about when the system was ready for production.
Character matters — Investing time in how the AI communicates — especially around sensitive topics — paid dividends that were hard to predict in advance. The difference between a robotic referral and a compassionate one is significant.
AI surfaces documentation problems — An AI system is only as good as its source data. Rather than being a weakness, this became a feature: the bot effectively became a continuous documentation audit tool.
Start with human review, then relax — The approve/deny workflow gave the HR team confidence in the system and provided invaluable training data. Gradually moving to spot-checks was a natural progression as trust was earned.

What's Next

The pattern of AI-assisted internal support — handling routine questions, routing complex ones to humans, and continuously surfacing documentation gaps — is applicable to virtually any department drowning in repetitive inquiries: IT support, facilities, finance, legal, and beyond.

Is your team spending more time answering the same questions than doing their actual work? Let's talk about getting that time back.