Instructions

You can use this demo to predict situations for images of your choice. Start by clicking on the example images. The demo provides the nearest neighbors in the imSitu training set and a list of predicted situations and associated probability. The demo has a 2 second time limit, so please do not try to very large images because they will take the server too long to download. The demo is an implemenation of the Compositional Conditional Random Field, pre-trained using semantic data augmention on 5 million web images, found here.

Click on images to try these examples

Recognize Situations

Paste a url

Upload a file

Results


Query

Similar in imSitu train set

Predicted situations

fixing
agent object objectpart tool place
man roof shingle hand outside
0.30590
repairing
agent item problem tool place
man roof shingle hand outside
0.23353
constructing
agent createditem tool components place
man roof beam outdoors
0.13213
mending
agent item tool place
man roof hand roof
0.09530
painting
agent item tool place
man roof brush outdoors
0.06129
stripping
agent removeditem source tool place
man shingle roof hand rooftop
0.05668
installing
agent component destination tool place
man solar array roof hand outdoors
0.04116
building
agent goalitem components tool place
man roof wood hand outdoors
0.02805
nailing
agent item1 item2 tool place
man board roof machine outside
0.01684
tearing
agent item tool place
man roof sword outdoors
0.00580