Blog

Introducing New Discovery: AI-driven theme discovery in VT Docs

At VisibleThread, we know our users need solutions that simplify the analysis of complex documentation. That’s why I’m excited to announce our latest VT Docs 4.0 release featuring New Discovery.

New Discovery is a reimagining of how you approach gap analysis in complex documentation.

Driven by artificial intelligence (AI), New Discovery automatically isolates key themes across multiple files spanning hundreds of pages to reduce risk—a tedious task that can be error-prone when done manually.

 But the reality for most organizations today is that this analysis remains a completely manual process. Contract managers, proposal managers, quality managers, and program managers all need to identify risk and compliance issues in documents. The risk itself may vary but the result of introducing risk to a proposal is always costly.

A big part of what drove me to found VisibleThread was an unmet need to illuminate misaligned elements and reduce risk across multiple documents. 

Missing a single Federal Acquisition Requirement (FAR) clause for a US Government contractor can render you non-compliant and cost you a contract, while catching this type of error post-award can lead to scope creep and damage the profitability of your programs.

A big part of what drove me to found VisibleThread was this unmet need to illuminate misaligned elements and reduce risk across multiple documents. I believed it would ease the burden of the manual review process for time-strapped proposal, contract, and quality teams. So, creating New Discovery has been a long-term ambition of mine since we started out as a company.

But it’s been a long and winding road to get here.

The evolution of New Discovery

Every founder of a SaaS growth company like VisibleThread battles with conflicting product priorities and finite resources. Despite our focus on expanding our language analysis platform in the early releases of VT Docs, I kept returning to our approach to the Discovery feature. The original Discovery was meant to easily identify and capture common themes across multiple documents. I was frustrated by it, especially when it came to how it visualized the information.

But it seemed like there was never really a good time to fix it. Instead, we focused on designing highly valuable product extensions, like Compare Docs (launched in 2016), Acronym Checking (launched 2015), and Excel Compare (Launched 2017). All of which are still loved by customers. Despite this success, I couldn’t shake the nagging feeling that we were within striking distance of a new Discovery that would fulfill our promise of true automatic gap analysis across complex documents.

The original Discovery had clear issues:

  • The data orientation was unintuitive. Users had to begin at the right side of the view to see document themes, then they had to navigate to the left side of the view to see variations of these themes. This violated the basic principles of good UI/UX design. Users tend to navigate left to right and we knew our design was an issue.
  • Users were overwhelmed by the amount of data. We saw that many customers were exporting CSV files to conduct their actual analysis outside of VT Docs. This reinforced the fact that the visual paradigm was not working.
  • Users found saving a view or converting it to a search dictionary difficult. Even when you had items checked, these tasks proved tricky for users. Search dictionaries are the backbone of many VT Docs capabilities and this one negatively impacted customer experience.

Here’s the original view:

Iterating towards the perfect data visualization

We attempted this second prototype in 2017 but it still had information overload issues, and the interaction model was too complex. We were close, but I knew we weren’t quite there yet. Once again, we shelved it.

Getting the visualization right remained our greatest challenge. The current version only became clear to us after a virtual offsite in mid-2020.

We dusted off the 2017 prototype and set about producing a design that would resolve current issues for users. Considering how long these ideas had been swirling around in my head, the visualization for New Discovery came about surprisingly quickly during this offsite.

We knew we had nailed it, it was one of those “ah-ha” moments. Armed with fresh prototypes in Q4 2020, we road tested the mock-ups with a cohort of strategic accounts. Many of these customers have been with us for close to 10 years, and we owe them a huge debt of gratitude for their support. You know who you are. 😉

These customers keep us grounded, helping us to deliver solutions that are genuinely useful. Selfishly, as a CEO, this also ensures we don’t invest engineering dollars in the wrong areas.

The simple but often forgotten step of talking to customers has been critical to VisibleThread’s design process

As an aside, every company has strategic accounts, but I’m always surprised by how many companies don’t talk to them. That’s a huge mistake. The simple but often forgotten step of talking to customers has been critical to VisibleThread’s design process.

New Discovery – one view full of rich capabilities

So, what does this new feature mean for our users? Put simply, New Discovery uses AI to group keywords together by theme. This allows you to easily investigate multiple documents and unearth misalignment or risky elements before it costs you a bid, without the stress of manually searching.

Customers now see a clean overview of reoccurring and emerging document themes all from one central view

It provides the power of analyzing complex documentation sets without the need for a pre-defined search dictionary to extract reoccurring themes. And it’s fast, close to 40% of engineering time has gone towards optimizing our load time for large sets of documents.

Customers now see a clean overview of reoccurring and emerging document themes all from one central view. In this new view, themes are itemized in the left panel. You can see the frequency of these thematic elements against your documents in the middle panel.

This at-a-glance overview saves time for busy teams when preparing for their pursuit

And you can see where specific references occur in your documents in the right-hand panel. This at-a-glance overview saves time for busy teams when preparing for their pursuit.

You can easily navigate the content associated with key terms that could impact your proposal or contract based on industry-specific stipulations. Rich data helps you identify and create search dictionaries to quickly search for terms that are relevant to the success of your proposal or the compliance of your contract.

This streamlined way to create a dictionary of search terms as you work through it, completely removes the friction of manually creating new dictionaries specific to each proposal.

There are multiple use cases for this type of powerful AI:

  • Bid/no-bid team analysis for strategic sales and capture teams
  • Competitive analysis for sales teams looking to identify gaps in offerings and conduct win-loss analysis
  • Requirements analysis for program and project managers identifying conflicting requirements and gaps
  • Resume analysis to identify credentials and capabilities across sets of resumes
  • Past performance analysis for sales teams
  • Consistency analysis for legal and contracts teams
  • Alignment analysis for proposal teams to make sure you’re meeting requirements

As New Discovery sheds light on potential misalignment across all documentation, the use cases are basically endless.

The future of complex document analysis

From the in-depth conversations with our customers to truly understand their needs, to the many iterations from the first early prototypes, to validating the product with customers last year—a massive amount of effort has gone into developing New Discovery.

I want to thank everyone who played a part in this journey, from engineering to customer success to our sales team. Most of all, I want to thank our amazing customers, we are proud to serve you on your mission

Our engineering team has worked 15 development sprints across 30 weeks to deliver this feature, and that’s after much time spent prototyping to get the visualization right, and scaling for large data sets.

I want to thank everyone who played a part in this journey, from engineering to customer success to our sales team. Most of all, I want to thank our amazing customers, we are proud to serve you on your mission. We truly value your input to help shape the evolution of our Language Analysis Platform.

We’re excited to see how you leverage New Discovery to grow your business, win more bids, and reduce risk.

Any questions or feedback, please reach out to me directly at any time on: fergal.mcgovern@visiblethread.com.

Join us on Tuesday November 23rd, 11:30 am EST for more insights on New Discovery. If you’re a contract, proposal, quality, or program manager working with complex documentation, you will not want to miss this webinar. Register below today to reserve your spot!

Register Now