Since the turn of the millennium, digital assets of all kinds has become an increasingly significant part of our daily experience. Every day, we consume and interact with photos, audiovisual media, text documents, E-Mails and a multitude of other digital formats. Navigating all of these digital assets creates challenges for Enterprises and end-users alike. Users want to organize the assets and make their search more efficient. They want to be able to find them, categorize them, and use them when and where they want. With the substantial increase of digital assets every day, manual labelling of them has become a tedious task. There are several Document Asset Management (DAM) tools in the market. Most of those tools depend on manual tagging and the process depends on an individual perspective, which takes a lot of human effort. Through this blog, we will share how we leveraged Google AI and it’s infrastructure to solve the problem by auto-tagging intelligent content from the files.
Background
With the introduction of AI and Cloud based technologies to address the Storage and Compute needs, Biarca developed a solution, which automatically tags all file types including documents (pdf, word and text), Images, Audio and Video files and make them searchable to the users in a matter of minutes.
At Biarca, we have developed a product ‘IntelliTag’ on Google Cloud Platform (GCP). IntelliTag helps alleviate the manual-tagging problem for the end-sers, there by saving their time & effort. IntelliTag is built with rich Cloud Infrastructure and provides high-end security and storage options to the user at a minimal cost.
To develop IntelliTag, we have used subset of Google Cloud Platform components like Google AI for tag generation, Cloud Storage for saving the user files and Cloud Firestore for storing the generated tags and user information.
Problem Statement
DAM has been a welcoming development while presenting with its own challenges. Once all the files are moved into DAM, the challenge is how effectively they can be managed? Most of these are stored in typical folder/file management format, where the user depends on their memory and/or several clicks to get to what they want.
Users need a software which can automatically tag their files and organize them with a few clicks. Biarca’s solution would serve the purpose which minimizes human intervention on tagging & retrieving assets.
Biarca Solution
“If you define the problem correctly, you almost have the solution.” ~ Steve Jobs
Biarca’s IntelliTag will allow Enterprise Users or Individuals to upload assets in batch mode and use AI to tag each of these files intelligently based on the digital content. IntelliTag develops a customized intelligent algorithm to continuously improve the tagging process, by improving accuracy over a period of time. It runs on GCP and has a tag search to retrieve the files based on the added metadata as part of the import.
- The solution is built as a Microservice on top of the Flask framework to help in Application Scalability.
- Organizes the digital assets based on intelligent tags generated by Google AI and provides an efficient way to search.The following diagram illustrates the functionality
In the current model, the digital assets of the user comes from the local device/GDrive/Dropbox and will be uploaded to the corresponding Cloud Storage bucket on GCP. Each file from the storage is taken and processed to generate the tags/metadata.
It consists of 2 stages:
- Content Extraction from files
Content extraction from various file types is done by Google AI in the following ways:- Image :Image files are processed using Google Vision AI. It returns the text and labels from the images.
- Audio :Audio files are processed using Cloud Speech-to-Text translation API and returns the transcript with time frame.
- Video :Video files are processed by Video Intelligence API to retrieve the labels and objects present in the video at different time frames.
- Text :The PDF/docx/text files are processed with python libraries (pdfminer3,docx) which returns the text from the files.
- Tags Creation
- The extracted content from the files is given as an input to Cloud Natural Language API. It is used to get the entities from the content which in general are termed as tags and also provides a classification label which helps to organize the assets into different categories.
- Content Extraction from files
- After the metadata creation process, the metadata will be saved to the database and files will be uploaded to the processed buckets on Cloud Storage.
- The search option in IntelliTag, will retrieve the files from the Cloud Storage.
Conclusion
Biarca’s IntelliTag solution, provides an efficient way to search the uploaded digital assets and can locate the “needed” content for repurposing/using the information. Our approach is very modular and can scale with ever growing content. It serves the need in real time and can be adopted across any domain.
In the meantime, if you are looking for Cohesive Cloud Services – DevOps, Infrastructure Modernization, Application Re-Engineering – Cloud Native Application, Data Analytics + AI, Secure + Compliant Environments, Open Source Engineering – ONAP, OpenStack, Kubernetes, CNCF, Blockchain, Continuum Services – DevSecOps, SRE, Cloud Security, Managed Security & High Availability please contact us
References
https://cloud.google.com/products/ai/
https://cloud.google.com/natural-language/
https://cloud.google.com/speech-to-text/
https://en.wikipedia.org/wiki/Talk%3ADocument_management_system
Contributor: Swathi Tatavarthy