Analysis at scale for a nonprofit research team, in a secure environment
Six months with the analytics team at the Center for the Helping Professions, a nonprofit working in partnership with the University of Kentucky, building a language-model classification workflow their team now extends on its own.
The analytics team at the Center for the Helping Professions needed to categorize a large body of written records. Reviewing them by hand limited which questions they could ask, and the data had to stay in a highly secure environment.
We started with discovery: what each category actually meant, and how we would know a label was right. With those definitions agreed and expert-labeled examples as the reference, we built a workflow that classified the records with language models inside the secure environment, at a scale that hadn't been possible in their work before.
- 0.91Night staff found out about the new schedule only when they arrived for their shift.R-0912Communication
- 0.88The evening team learned about the change from a note on the whiteboard.R-1047Communication
- 0.84The update went out by email, and half the team does not check email at work.R-1133Communication
- 0.81Decisions from the morning meeting never reached the weekend team.R-0764Communication
Comparing the models against the expert labels proved useful in both directions. Some disagreements were model errors to fix. Others led the team to rethink how they had been labeling, which helped them understand their own research questions in different ways.
After six months, Dr. Elizabeth Riley, the team's Chief Analytics Officer, took over the work and has continued to extend it for the team's needs.