Recently, my software development team inherited a project that had been in flight for over a year. With it came a mountain of documentation, including more than 40 hours of recorded video calls with valuable information about complex, required system functionality. We were faced with a challenge: as we onboard into the project and build out required functionality, how can team members find the answers they need quickly as questions arise?
Just a few years ago, indexing this information to make it searchable, usable, and useful for team members would have been expensive and time-consuming, representing weeks of work. But with modern AI tools, I was able to build out a knowledge base in a few hours, using safe and secure tools to handle our clients’ sensitive information. Here’s how I did it.
1. Getting Video Transcripts
While much of the documentation was spreadsheets and text files, the videos contained important information that wasn’t duplicated anywhere else. Extracting this information would create huge productivity gains for the team and minimize repetition for our client.
All of the video calls were saved in MP4 format. So, the first thing I wanted to do was get text transcriptions of the video, so we could search for terms and concepts as they came up in real time in the course of the team’s work.
I used MacWhisper’s batch transcription feature to get high-quality transcripts with timestamps. At first, I tried MacWhisper’s local model but quickly realized it wasn’t going to work well. My computer is underpowered for the volume of data that I needed to process, so it ended up being painfully slow and the quality of the output wasn’t stellar. The key to success here was to use Deepgram’s Nova 3 model via MacWhisper’s cloud transcription feature. Deepgram is PCI and HIPAA compliant. That meant my client’s information stayed secure and private throughout the whole process.
To keep things organized, I had each transcript output to a separate text file. Then, each text file name corresponds to the filename of the source video.
2. Knowledge Base
Once I had my transcripts, I needed an easy way to search through them. I have many traditional knowledge management tools at my disposal (Confluence, Google Docs, Obsidian, etc.), but I decided this would be a good chance to try out NotebookLM. It turned out to be a nice experience for the team.
Uploading the video transcripts and other documentation files to NotebookLM was easy (NotebookLM calls them “Sources” and allows many different common file types). Now my team can interact with the knowledge base as with any other AI chatbot. For example, “What can you tell me about concept X?” or “Find all instances where Concept Y is discussed.” NotebookLM can summarize information and, importantly, it cites its sources (and links back to the original). This enables the team to check for accuracy and dig deeper into the source material before they take action or build anything.
Now, when a team member begins working on something new, their first stop is often our NotebookLM Knowledge Base to find out everything we can about the concepts involved. Sometimes the tool can provide answers about functionality or decisions made in the past. Other times, we’ll use it to find the right timestamp and review the video to find the information in context. But it’s been an invaluable way to cruise through the information.
Over time, we can also add to our knowledge base with additional video transcripts, meeting notes, and updated documentation. The tool has enterprise-grade data security and does not use customer data to train AI models.
Staying Productive When the Tools Get Messy
One thing I’ve learned throughout all my experiments with AI tools is that success requires both persistence and creativity. The technology is still early and can be buggy. For example, sometimes my transcriptions would stall and need to be restarted. I’ve found that, with AI tools, if something isn’t working, I can ask a different way or try a different approach to the problem. Sometimes the best thing to do is to turn it off and come back later! It can be frustrating at times, but if you can approach the tools with curiosity and persistence, you’ll usually find a path forward.
The post How I Used AI Tools to Create a Software Project Knowledge Base appeared first on Atomic Spin.
