Challenge
Motivation to change
Slovenská sporiteľna decided to verify ability of Open AI model to effectively understand banking product and process knowledge.
Target was to realize POC to verify how exact answers we can get by using as is process and product documentation and how much customer support time can be saved by implementing it to production.
Solution
Change delivery
Prepare stand alone chat application connected to Microsoft Azure GPT models. Embed knowledge in areas of mortgages, consumer loans, current accounts, deposit and corporate products. Verify quality in Pilot test with branch employees.
Tools & means
- React FE
- Python BE app
- Usage of Langchain
- Chroma vector BD
- Azure open AI LLM
Outcomes
Change outcomes
We have created complex process and product documentation with lots of tables, set up environment on premise and implemented establishment of evaluation metric to measure quality of the solution. Additionally, we identified lessons learned for product and process documentation creation to be more LLM proof.
Used services
-
27bank process and product documents embedded into the chat application
-
1pilot rollout across all bank branches regions
-
76%of fully correct answers