Behold Image Search

New feature: Grouping search results by visual similarity

2008-02-04T07:10:00.000-08:00

It's been a busy few weeks for me while I was working on a new feature. Having neglected the blog for way too long I decided that it is time to update you on what I've been working on.

One of the key differences between Behold and other image search engines is Behold's ability to search not only by matching tags, but also by specifying what the results should look like. For this Behold has 28 popular visual filters, e.g. 'looks like picture of a building' or 'picture of a face', and uses computer vision to reorder the image search results accordingly. These filters can be turned on and off by the user during the search.

With a newly introduced feature, Behold goes one step further and automatically suggests these filters after analysing the words in your query. It then shows you what your search results would look like if you applied one of these filters, so you save time on finding the right one. This can be useful when your search results contain the desired subject in a number of different contexts.

For example, when searching for eagle, Behold offers the visual filters animal, face and bird to refine the search further. When such suggestions are available, they are placed in the top right corner of the screen:

How does Behold know which filters to suggest? After carefully studying and visualising the most popular tags and search requests, a set of a few thousand rules was generated that links query words with Behold's visual filters. Extra effort went into ensuring that these rules suggest visual filters that usefully refine most queries.

This makes Behold similar to the Clusty web search engine that automatically groups search results into subcategories based on their summarisations. The key difference, however, is that Behold uses computer vision to group images by visual similarity. This feature is now live and, as always, I appreciate your feedback!

Here is a link to two demo videos of this new feature.

Early feedback from the blogosphere

2007-10-16T10:06:00.000-07:00

One of the greatest things about having Behold online is the amount of feedback that the users give. For example, Lee Stranahan uses Behold to add high quality Creative Commons photos to his blog posts to enhance readers' experience. A number of Flickr users recently spotted that Behold occasionally does not keep up with the changing image licenses; Behold now tries to address this by re-indexing images from Flickr more frequently. Behold was also recently mentioned in the SMARTBoard podcast, where Ben Hazzard and Joan Badger discussed online tools that are useful for teachers. They highlighted the simple user interface and also the ability to narrow down search results through Behold's image content analysis. In a similar vein, Anthony Evans uses Behold's visual fitlering to find images that are suitable for educational purposes. All of this is enormously useful feedback that will be used to improve the search experience further. You can also leave your comments by going to Behold and clicking on the 'Chat to the developer' link at the bottom of the page, or by sending an e-mail to

New address: Behold.cc

2007-09-21T05:46:00.000-07:00

Behold can now be found at the brand spanking new address behold.cc. Why .cc? Not because Behold has moved to the Cocos Islands in the Indian Ocean. Instead, it was picked to reflect Behold's focus on indexing high quality Creative Commons images :-). However, given the previous experience with moving the site about (see below), all the old links will continue working for quite some time, until the PageRank issue is fully resolved.

Zero PageRank. A guide to annihilating your site's ranking in 3 easy steps

2007-09-20T14:09:00.000-07:00

This post is for webmasters. Recently, I noticed that Behold's PageRank went from 5 to zero. I was very surprised by this and decided to investigate. Previously, I had heard that this is one way that Google can penalise your site for trying to 'spam' it, that is, to deceive it into thinking that the site is more popular/linked to than it really is. Although I had never had this intention, I decided to find out what might have raised Google's alarm. My first port of call was Google's own webmaster guidelines. I found nothing in my actions that could have violated these guidelines, with the possible exception of serving what appeared to be duplicate content from different sub-domains. However, it then dawned on me that Google was not penalising me, but, most likely, I did so myself. Here is what happened:

A long time ago, when Behold did not have a dedicated server, I set up a website at www.beholdsearch.com. The index page used a meta-refresh tag to redirect to the location at which Behold happened to be hosted at the time. Soon I obtained the first dedicated server for Behold. I named it go.beholdsearch.com and changed the meta-refresh tag at www.beholdsearch.com to an HTTP 301 permanent redirect to this address. 9 months later, I purchased a more powerful server to host what is now the Flickr version of the search engine. I named this server photo.beholdsearch.com. Noticing that the Flickr service became much more popular than the university image search that was still located at go.beholdsearch.com, I changed the 301 redirect from www.beholdsearch.com to point to photo.beholdsearch.com instead. Soon I had discontinued the university search and closed the old go.beholdsearch.com domain.

Bad idea. It looks like you should never change a 301 redirect once you have it in place. And not just because it goes against the very idea of a permanent redirect. My best guess is that this is all to do with the way PageRank is assigned. While no one knows how Google really works, it is likely that when Google sees a 301 redirect from site A to site B, it associates all the links pointing to site A with site B. In other words, it transfers the PageRank from A to B. Site B starts showing up in search results instead of A. However, when you change the redirect so that A now points to C, A has no more PageRank to give. Otherwise one could keep redirecting to new sites and increasing their PageRank at will. Meanwhile, B is now not associated with A. If B itself is not linked to from anywhere (and why would other people link to it when it's easier to link to and remember the www version of the site), on subsequent crawls Google realises this and removes all the PageRank from B that was handed over to it previously from A. So, you end up with having no PageRank at all on any of your landing pages, old or new. It will now take some time for Behold to regain its old PageRank. If you have to change your redirects, it looks like it is much safer to do so with the appropriately named 'temporary redirect' (HTTP 302).

Getty's $49 per image price plan

2007-09-12T11:13:00.000-07:00

Getty Images announced a new price plan, allowing to use most of their images online for just $49 per image. This is a drastic price reduction, considering that previously this would cost as much as $200 per image. Perhaps this is the first sign of the image sales market reacting to the increasing availability of free high quality images on sites like Flickr.

Searching for creative commons content

2007-09-12T10:20:00.000-07:00

The number of searches has sharply risen this week, thanks to a blog post by Cameron Parkins on creativecommons.org describing Behold's mission of finding high quality images that can be freely used. He rightly points out the importance of online resources such as Flickr: "Flickr is not only inspiring in terms of the sheer amount of photos available, but even more so in terms for its ability to allow interesting and innovative resources, such as Behold, to be built." I could not agree more. It is the openness of sites like Flickr that makes Behold's job possible. It is encouraging that Behold's approach to image search has found resonance within the Creative Commons community.

Behold's mission

2007-08-09T05:58:00.000-07:00

Major news! Behold is no longer just a research project. It now has a mission statement that concerns you, the user. It reads: "to offer search over images of highest possible quality that are freely available on the Internet". To this end Behold has now indexed over 1 million high quality images from Flickr and is giving you the option to search them using tags as well through Behold's image analysis. While Flickr has nearly 400 million images, only a small proportion of them are professional, artistic and aesthetically pleasing. Behold's aim is to bring you only images of this calibre. Of course, many of you want to use such images on your websites and in your printed materials. By utilising Flickr's license information, Behold can restrict search results to contain only the images that can be freely used.

Behold can continue growing and offering you more high-quality images, thanks to a parallel computation service provided by Amazon Elastic Compute Cloud. In the mean time, work will continue on improving Behold's visual search capability to enhance your search experience.

Revamp

2007-06-12T17:19:00.000-07:00

Two weeks ago Google quietly slipped face-filtering into its image search, as reported by GoogleBlogoscoped. This is a very good sign: search engine companies are now paying attention to image content analysis. Meanwhile, Behold has once again been upgraded. A separate option has been added to search around 100,000 photos from a Flickr group called the Unofficial JPEG Magazine and simplified the user interface. There are also two new demo videos on the new 'about' page. Stay tuned for more updates!

Why focus on the web?

2006-11-29T01:19:00.000-08:00

Some of my friends and colleagues have asked me why, in this age of photo-sharing websites such as Flickr, the focus Behold's research are images from the web. There is no doubt that Flickr is a very exciting platform for developing image search engines of all kinds, Retrievr being one excellent example. Retieving images from the web, however, can pose two additional challenges. On average, the pictures tend to be of pretty bad quality (for example see some random images in Behold). And, importantly, website owners are not incentivised to tag their photos properly. The first problem forces one to explicitly model junk images likely to be irrelevant for any query, which is an interesting research problem in itself. The second shortcoming can be compensated by looking at the interplay between image content and html metadata (a feature that was recently added into Behold), and an interesting question is how to find the right way to combine the two. That being said, there is no reason why we shouldn't apply Behold to images from Flickr one day.

New feature!

2006-11-25T21:49:00.000-08:00

You can now search in Behold using both the regular metadata a la traditional image search engines like Yahoo and the automatic image annotations. Here's how it works: you enter a text query into the traditional text search: box and then pick one or more keywords from the visual vocabulary to refine your search in refine with image analysis:. A few encouraging examples:

#1. Search for beach using only metadata:

then refine with the visual keyword beach:

#2. Search for bus using only metadata:

refine with the visual keyword car:

This can be useful when you want only a certain type of pictures from a given location, e.g. london buldings or parisian towers. It can also be helpful when metadata for a particular word is poor and you would like to filter your results, e.g. metadata lake vs. combined lake and metadata car vs. combined car.

Essentially this new feature lets you see how automatically assigning annotations to images based on their contents can improve standard text-based image search on the web.

Give it a go!

And while we are at it here's a link to a short report outlining how Behold indexes images and summarising the search engine's current features.

Automatic image annotation

2006-11-23T18:44:00.000-08:00

This Wikipedia aricle is a useful literature primer for anyone interested in automatic image annotation. But did you know that one of the first papers describing an attempt to recognise image content from a statistical perspective dates back to 1974? This was recently pointed out to me by Zdenek Zdrahal during my visit to Open University's Knowledge Media Institute lab. The paper is called A Semantics Based Region Analyzer by Jerome A. Feldman and Yoram Yakimovsky, in Journal of Artificial Intelligence 5(4):349-371 1974. It's interesting what they managed to achieve with the very limited computational resources they had back then.

What can Behold recognise?

2006-11-20T07:52:00.000-08:00

After some months being spent on improving the search quality a new version of Behold has been rolled out. The set of keywords that it recognises has somewhat shrunk but hopefully the ones that remain should work better. As well as querying by keywords it is now possible to browse images based on their content, using the algorithm of Daniel Heesch. As this page explains, Behold uses very simple techniques to assign keywords to images so each individual keyword search can give rather vague results. However combining related keywords in your query often helps, e.g. boats vs. boats and water and car vs. car and car_park. Once you have located a close enough image to your target using keywords you can start browsing the database for more similar images. What can you find in Behold by combining keywords and browsing?

Welcome

2005-07-03T14:03:00.000-07:00

Welcome to the Behold blog. Behold is a search engine that allows users to find images from the web based on their visual content using queries consisting of simple keywords.

This blog is maintained by the project developer, Alexei Yavlinsky, and is intended to be the source of latest project news and a place for lively discussion. Most of all, we are interested in your feedback about experiences you have with the search engine as we gradually improve its quality. A quick description of the project can be viewed here.