The idea is to transcribe and to index the audio signal. We have developed a large vocabulary automatic speech recognizer for broadcast news in Greek. The difficulty of the task stems from the variety of the acoustic and linguistic environments that must be processed. Our system comprises of algorithms and techniques drawn from many disciplines, including statistical modeling techniques referred to as Hidden Markov models, communication theory, signal processing, mathematics and linguistics. A speech/non-speech detector separates acoustic events of interest from noise and music. We use a significant amount of training data in order to build our acoustic and language models.
Technology
QUALIA's technology is based on advanced multimedia analytics. We have built a platform for collecting and processing video, audio and text. Data are indexed by intelligent machines drawn from various fields. Here follows a description of our technologies:
Speech Recognition
Image Processing
What we need is to process video frames based on their content. We segment the video into shots and extract characteristic frames (so called key-frames) from the video sequence. In this way, we are able to summarize a long video sequence into a few representative images. Another advanced technology that we apply is video text recognition and it is used for localization, segmentation and recognition of artificial text in video sequences. Such text occurrences mostly appear in newscasts, commercials, sports, etc. and are often the important carriers of information and herewith suitable for indexing and retrieval.
Text Mining
The Web has grown at a phenomenal pace in recent years. However, the absence of structure, combined with the heterogeneity of sources have limited the exploitation of useful on-line information. At QUALIA, we have developed a sophisticated information extraction system that crawls web pages and transforms them into structured data that are tailored for information retrieval. We have programmed vertical information extraction wrappers in order to extract data from specific locations in the pages and to analyze them. Valuable information is converted into structured form and is integrated in the system’s database for further processing.
Web Crawlers
We are not a general purpose search engine that indexes the enormous dynamic collection that constitutes the Web. We rather focus on thousands of pages that convey news, current affairs, opinions and reviews with the aim of providing clean and useful content to our clients. We have implemented different types of crawlers with different strategies, depending on the target set of pages (news, blogs, social networking, forums, etc.). The repository that we build is used for indexing, mining and clustering and we constantly work on improved filtering and cleaning algorithms. In addition, our engines are polite and respectful of the robots' exclusion protocol.
Information Retrieval
Metadata generated from speech, image and text processing are integrated in a unified XML representation that is developed in order to support flexible access and efficient reuse of harvested media. Our technologies segment hours of TV and Radio material into small coherent pieces that are indexed according to their content. Audio and video components are synchronized under a common model that integrates the different indexing methods without information loss. Our search engine is based on an extended vector-based model, augmented with proprietary similarity metrics that are optimized for multimedia search. Advanced features are used for search space reduction, for stemming and capturing inflectional linguistic variations.
Clustering & Classification
By performing cluster analysis, we partition web news into topics. The idea is that similar information will tend to be relevant to the same search queries and user profiles. We apply agglomerative hierarchical methods and we obtain excellent results. Basically, we start with every data input as a cluster and then we merge the closest clusters into new ones. We thus bundle news items together and index them accordingly. Then we can track them and examine how they evolve over time. We are also working on bundling together heterogeneous information, like tweets, broadcast news and blog posts.
Machine Learning
We work a lot on this. We develop data-intensive approaches to a variety of problems including speech processing, text categorization and clustering. We try to solve the ambiguities that are inherent in natural language and accurately learn the user profile. There are plenty of algorithms that we apply: bootstrapping in order to learn from few partially annotated data, high dimensional linear classification and simple bayesian classification, maximum entropy recognition, kernel methods and support-vector machines. Technically, we develop methods and algorithms for learning and prediction on the basis of past experience. Not so technically, we try to develop things like those you see in futuristic sci-fi movies.
Architecture
Our service constitutes the front-end of a powerful software architecture that we have developed in-house. Modern design approaches are combined with innovative ways of multimedia content processing and analysis. This fusion resulted in the construction of a fully expandable Multimedia Processing Framework. The backbone of the system is based on a multi-tier distributed software architecture. Designed in a service-oriented way (SOAP Web Services), the system offers the greatest level of flexibility and interoperability. Fast performance, scalability and robustness gain come from the incorporation of a centrally administered, yet self-organized grid of miscellaneous video, audio and textual processing nodes, all seamlessly integrated into the general service-oriented framework.
