 |
| |
 |
Approach |
|
|
|
|
CASAM introduces the concept of effort-optimized knowledge aggregation as the task of reaching the desired result by requiring the least effort from the user. The project aims at developing an annotation tool that will feature a human-machine interaction loop targeting a fast convergence of human and machine knowledge towards the goal of multimedia content annotation. We divide the annotation task into three main operations: Reasoning for Multimedia Interpretation (RMI), Knowledge-Driven Multimedia Analysis (KDMA) and Human-Computer Interaction (HCI). The best way to visualize the tagert approach of the project is to describe the aforementioned information flow (interaction loop):
- KDMA analyzes the multimedia content (video, image, natural language) and extracts the low level information. This information consists of basic image characteristics like key objects, presence and number of people, context identification etc, as well as speech recognition and simple concepts derived from text processing.
- At the same time, the user enters a small number of keywords that describe the high-level concepts present in the multimedia content. Note that the keywords need not belong to a predefined set.
- RMI augments the KDMA-derived and user-provided information, instantiating appropriately an ontology. Moreover, RMI infers new concept instances and reassesses the context and previous input from KDMA. Results are then fed back to the KDMA for multimedia analysis driven by the renewed information. This RMI-KDMA internal information exchange loop continues until nothing more can be inferred by RMI or recognized by KDMA.
- RMI reasons about what information is needed in order to add missing instances to the ontology or resolve any ambiguities that have arisen in the previous step. If the annotation target has been achieved the loop exits, otherwise the information requirements are fed to HCI.
- HCI transforms the information requirements to input requests towards the user. HCI optimizes the user interaction using an effort-cost model, possibly by augmenting requirements to a single input request, using user-modelling to adapt to user information input patterns, etc. Furthermore, the user interface provides the user with the opportunity to alter the knowledge acquisition path devised by the system.
- HCI input is passed to the RMI and the loop continues from step 3.
The annotation tool will be able to function within the modelled domain of news production of News Agencies and Broadcasters. However, the methods that will be developed will not be bound to the chosen domain, but will be also applicable for the annotation of multimedia documents in a variety of contexts, ensuring generality of the system’s usage.
|
|
|
 |
|
|
 |
| Events |
 |
|
 |
| |
|
Follow our discussions and news at:
|
|
 |
|