Summer06 plan

eCHASE Milestones

Prototype 2.1 (End of June)

Little input required from IAM, using same content as current eCHASE prototype.

No media engine, or perhaps the current eCHASE media engine.

Prototype 2.2 (End of July)

New eCHASE data sets.

This will require the underlying admin functionality to import and manage media collections into the media engine, and provide an API for querying for basic information (web rep URI, thumb URI...)

Prototype 2.3 (End of September)

Full Integration of media engine with eCHASE system.

Full admin functionality and interface, integration of SRW and media engine, media engine web interface widgets etc.

---

Media Engine Admin Interface [cah]

Admin interface - for CBR and maybe taverna data acquisition

Achieved:

• Experiments with javascript/xml-rpc, e.g. a basic help system
• Interface for importing media/collections
• Easy interface for generating./regen of media descriptors and maintaining/checking them

Current:

• Stats on quality/completeness/errors, current status

Future:

• Clean up of old temporary content and stored queries
• Run classifiers (Aniza & Simon's?) - link to database/media engine
• Ability to update an FV generator/classifier/auto run?
• Add video importing to mediaengine
  • probably requires changing jmimemagic for the mimetypes if not already in place
  • Keyframe linking?

Media Engine Internals [ss]

Achieved:

• Discovir feature generators -- this is really simple, just a matter of writing a thin wrapper class to map the differences in the java interfaces.
• FV generation queue
• Parallel processing for FV generation

Current:

• Code review/documentation, improve logging, open source headers...
• The original code comments are a bit sparse! The plugin interfaces at the bare minimum need some javadoc...
• Query by sketch feature vectors into FVS/media engine

Future:

• More advanced plugin loading
  • currently, mediaengine loads plugins from a directory. We also want to be able to dynamically register new plugins:
    • Load plugin from URL - keep a record of these plugins in the preferences. (How) do we handle dependencies?
    • Upload new plugins to mediaengine (mediaengine will put it the correct directory) - part of admin interface.
    • Rescan plugin dir and load new plugins.
• mpeg7 feature generators (if open source one is OK)
  • link-up open src code with our "media engine".
  • Could also export fvs descriptors in MPEG-7 format

these include things like processing media descriptors to mark up images as "colourful" or "mono" or 3D as "pot shape". I've (JH) made a start on this, and will continue to work on it. Basically a media classifier will be able to instantiate a new media descriptor upon training that can be used to classify and search the other items in the collection. The semantic space stuff will be integrated with this too.

• Distance / similarity score normalisation. Basic code in FVS needs testing and extending. Some work in media engine required.
• Allow generation arguments to be passed to FVS. Need to make sure generated features are kept sepearate from those of the same type but different arguments.

CBIR User Interface [pa4]

• interface for CBIR

dyn html (I've been using php for the xmlrpc webservice interface and dojotoolkit for presentation, but there is a native javascript xmlrpc library... alternatively we could knock together a json interface plugin.... -Jon) probably rather than Java. need to be able to run tests for research into descriptors and see thumbnails, sort by attributes etc sits on top of media engine and/or SRW? Working with JH

• make HCI which works within openMKS but with a view to making new interface if needed
• Make an interface design from scratch to include CBRN & browsing
• User interface widgets for querying:
  • a colour picker (http://www.colourmod.com/)
  • cropper (http://www.speedingrhino.com/cropper.htm)
  • drag and drop query by similar image, as with Jon's media engine demo
  • query by sketch? (Sina's)

OpenMKS/SRW/Media Engine Integration [sg]

Achieved:

• Open sourced FVS: http://libfvs.sourceforge.net
• Integrating CBR into OpenMKS through SRW (needs CVS commit)
• Media engine: maven 2 build, expose services

Current:

• Expose API for maintaining the link between metadata and mapped data/triples
• Server-side components for CBIR UI widgets
• Clean up FVS if necessary

Future:

• Make classification/annotation area in the system - for Simon and Aniza's classes

Other plans

SPARQL end point for media engine

A SPARQL interface to the media engine would permit other semantic web systems to interact with it in a seamless manner, e.g. Photocopain, eCHASE metadata import system.

Interesting reads:

Extensions in SPARQL: http://jena.sourceforge.net/ARQ/extension.html

D2R-server to map from the internal mediaengine database to SPARQL: http://www.wiwiss.fu-berlin.de/suhl/bizer/d2r-server/

natural language quiery - pick out meaning using thesauri?

plus there may be general things we can look into: Getting V&A data set imported OAI crawl of AHDS/VADS? documented here