Unstructured Data: Making it Work for the Human-in-the-Loop


Over 80% of the world’s information is trapped in documents, photos, videos, text streams, and voice communications. Trapped because it cannot be processed by automated systems. People need to process that information and that’s slow, expensive, and risky.

Unlocking the business value of unstructured data is the biggest automation challenge of the next decade

Anthony Murphy, CEO

Commonly called unstructured data, this information represents a huge untapped resource with the potential to create business opportunity and competitive advantage for companies that discover how best to work with it.

How can we unlock the full business value of unstructured data? What’s the role of Artificial Intelligence (AI)? And what’s the remaining role of people?

What is Unstructured Data?

Unstructured data is information that cannot be directly processed by automated systems. Those systems can only work with data that is highly structured, data that conforms to a well-defined format. But most information is unstructured.

A simple unstructured data example is a photograph of the damage caused in a car accident. That photograph contains a lot of information but it’s trapped in the visual image and unavailable to systems for automated processing.

The insurance company’s claim system needs to record the vehicle registration number, make and model, damage report, and more. The photographic image contains much of that information but not in a well-structured format the claim system can immediately gather and process from. The data is unstructured.

People need to look at the photo, decide how the information it contains relates to the claim, assess what to do next, and make some decisions. In other words, people are required in business processes that involve unstructured information.

Businesses already process unstructured information in huge amounts daily and that information is vital to them. Day-to-day operations regularly process multiple types of unstructured data, such as:

  • Documents (paper mail, emails or email attachments)
  • Text streams (social media or web chat sessions) 
  • Static images (photographs or branch scans) 
  • Videos (social posts or CCTV film) 
  • Voice interactions (recorded conversations or messages)
What is Unstructured Data? Graphic

Currently, the processing of this information is achieved almost entirely through human effort. How can we deal more efficiently with unstructured data and unlock its business value more effectively?

That’s an urgent question for insurers, banks, logistics companies, manufacturers, healthcare organizations, government departments – in fact, for any information-intensive business where unstructured data is part of everyday working.

The Role of Artificial Intelligence (AI)

Many observers view Artificial Intelligence (AI) as the great panacea. It will transform all unstructured information into structured data so that automated systems can process it.

Machine learning, natural language processing, computer vision, facial recognition – these AI technologies are quickly advancing to deliver on that promise. Intelligent Document Processing (IDP) technologies, for example, use AI to recognise a utility bill document and automatically extract a householder’s name and address.

The investment in AI capabilities is enormous. IDC expects the market for AI software, hardware, and services to exceed $500b in 2023. AI promises to accelerate the pace of digital transformation by releasing digital data from unstructured information, such as that car accident photograph.

Will there be anything left for human workers to do? Will all those people currently processing unstructured information no longer need to be involved?

The Role of People

Just as Robotic Process Automation (RPA) relieved millions of people of repetitive work, so AI will relieve them of the tedious work of translating unstructured information into digital data.

But early fears that RPA would result in widespread job losses proved mostly unfounded. People were still required to process insurance claims, open new accounts, onboard new customers, control supply chains, manage government and healthcare cases, and more. RPA was helpful but the Human-in-the-Loop (HITL) remained essential.

AI similarly will boost productivity, but the Human-in-the-Loop (HITL) will continue to perform an essential role in completing and managing work.

While AI releases data from unstructured information, people engaged in processing insurance claims, opening new accounts, onboarding new customers, etc. still need to be able to:

  • Access: Receive documents, photos, videos, and text streams anywhere, anytime, in its original format, so they can get work done.
  • View and consume: See the document, view the photograph, watch the video, read the text stream, listen to the recorded conversation. People must be able to view and consume unstructured information, even though AI has released data from the document, photo, video, or other unstructured format.
  • Organize: Collect, combine, sort, filter, and search for documents, photos, videos, etc., so they can work efficiently and effectively.
  • Distribute and share: Direct relevant documents, photos, videos, and text streams to the right individuals or teams.
  • Secure: Restrict access to comply with company policy or government regulation.
  • Enrich: Add annotation, comment, or instructions to a document, photo, video, audio, or text stream.
  • Process: Complete tasks related to the unstructured information, e.g., approve a Proof of Identity or Address, or escalate a voice conversation for further assessment.
  • Store: Keep unstructured information in its original format, for use on-demand or as archived or exported assets, in accordance with compliance rules.

People play a vital role in business processes that necessarily deal with unstructured information. AI will change the nature of human work, just as RPA has done, and the content of people’s jobs will change. But as companies invest in AI, they must in addition enhance the automation that enables people to engage with unstructured information more efficiently and effectively.

Combine AI and People to get the most out of Unstructured Data

The quantity of unstructured information in the world is growing fast. Some industry experts believe the growth rate exceeds 50% per year.

Legions of AI technology companies, from the giants to the well-funded start-ups, are investing immense sums in the task of releasing structured data from unstructured information. From video analysis to facial recognition, to behavioural pattern detection, to Intelligent Document Processing (IDP) – the scope of AI application is vast.

Yet, as the evolution of RPA technology has illustrated, new automation changes the role of people but does not displace them. As AI increasingly delivers on its promise, people will continue to play a vital role in business processes where unstructured data is used. And they need better automation to help them.

Solve your Unstructured Data Problem Now

Lithe is the #1 provider of flow and processing software for unstructured data. We empower people as they work with unstructured information, complementing the role of AI.

Lithe provides a secure platform for people to work with documents, photos, and other unstructured information within your organization. Our cloud-based solution empowers people to receive, access, organize, share, enrich, and process unstructured information anytime, anywhere, and on any device.

To explore further how we can help you unlock the business value of unstructured data,  contact us now via the chat on this page or through our other contact options.

Increase the productivity of your hybrid teams, improve responsiveness to your customers, reduce processing times, and fast-track your digital transformation.

Subscribe for blog updates


Connect with us

You might also like


Generative AI: Your New Digital Assistant

In the world of digital assistants, RPA used to reign supreme, revolutionizing routine tasks and boosting productivity. But now, there's a new contender in town:...

Mailroom Automation: A Hybrid Working Solution

The last several years have been a turbulent time for organizations, following the COVID-19 pandemic. Teams that were once in the office five days a...

Process Automation Software: What is it?

“Every company is a software company” says Microsoft CEO, Satya Nadella. He means, of course, that software is crucial to every company. But choosing the...

Sign up for Lithe Insights

Get all the latest news first

This field is for validation purposes and should be left unchanged.

Discuss your unique needs

Talk to our experts

Contact Us Chat now