Task
Assist users in deciding whether a real place is suitable for their accessibility requirements.
Benchmark
Evaluation design and data
MAP evaluates multimodal systems on accessibility planning in an open-world setting with maintained information on places and their relevant features.
Benchmark Overview
The first version of MAP focuses on the city of Zurich, Switzerland. Systems are assessed on providing useful and accurate information for requests with specifications related to accessibility in relation to real locations such as restaurants, cafes, museums, local government, offices, and other places of interest.
Evidence
Responses are assessed against feature-level ground truth data on places and features related to accessibility requirements.
Outputs
System results will be published through the leaderboard after submission and scoring.