Main Page

From Serendipity Searcher
Revision as of 09:09, 7 January 2024 by Boris (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Searcher: The Art of Associative Searching

Prototype of Searcher presenting ever-changing relational maps of the MAO collection. MAO collection © 2023 Museum of Architecture and Design, Ljubljana. All rights reserved.

Searcher is an artistic installation that explores new participatory/co-curatorial interfaces within museum spaces that leverage the networked potential of digital museum collections. The installation consists of a large glass panel and an arrangement of twelve e-paper displays. As a whole, the installation forms a hybrid image atlas that can be interacted with to construct ‘visual maps’ and stimulate new narrative readings in museum collections. Given that the museum's frontend grants access to only a fraction of its holdings through fixed displays and long-serving narratives, Searcher positions the necessity for more open circulation of collections, stories, and their sharing within the social context of museums.

Background Context

Inspired by the image atlas of art historian Aby Warburg (1866-1929) – a project that mapped visual themes across time through ‘alliances of attraction’ – Searcher remediates the visual map as a curatorial ‘thinking space’ where searches and findings are spatial, situated, and social acts. The aim is to challenge the notion that the search of the collection has defined boundaries and, instead, embrace the idea of serendipitous encounters —finding something you were not initially searching for. This concept is integral to the installation’s performative function, where any chosen set of objects, semantic queries, or final recontextualization of one’s findings in the collection is considered a new avenue for inquiry. As a result, new search queries are triggered and relationships that were not initially visible start to emerge and change over time.

Background Principles

To facilitate associative and serendipitous exploration within vast 'big data' collections, this project employs multiple techniques from machine learning to tease out the massive and complex interconnectedness among collection items. These techniques analyze the proximity of collection items, forming relational 'neighbourhoods' across three distinct layers: visual closeness (similarities in the image data via visual embeddings), closeness in how objects are described (relationships in the unstructured textual data via language model embeddings), and co-occurrences of categorical meta-data (which induce a knowledge graph). A more complex search model is formed by pooling these together and allows us to take into account the diverse ways in which relationships between objects in large and rich museum collections can be found.

Backend Implementation

We implement search in this way to both follow users’ intuitions and enhance potential serendipity by bringing more curious and underrepresented results to the fore. Every search initiation triggers three distinct actions:

1) A reordering of the collection in terms of the proximity to (a user's) selected objects of interest. Based on the three layers of data and pooling.

2) A diversification algorithm that disrupts the “relevance” orderings from action 1) in order to prioritize relationships spanning various meta-data categories, such as different sub-collections, media/materials, or historical periods.

3) An enduring search function that continues to explore and display relationships long after the initiation of a search. This function, conceptualized as a 'lingering lunar search,' draws inspiration from the natural dynamics of lunar phases and tidal flows, influencing the rhythm and pace of the associative display of results. Consequently, whether one accesses a saved visual map, queries the collection with the same selected set of objects, or leaves a search ongoing for some time, the sequence of displayed results will vary, creating a dynamic experience influenced by temporal and spatial factors.

Frontend Use

The principal interaction is based on users selecting/fixing objects of interest on Searcher’s display and initiating an associative search to call up more inter-related objects. The data attached to the selected items are used to query the collection, subsequently interjecting inter-related items in any of the unselected/unfixed e-paper displays. Selecting and searching is an iterative action that enables users to construct their own visual map of the collection over several searches.

The Searcher installation provides 4 key built-in affordances to explore and construct these visual maps:

  1. Rotary dial: a turnable dial that can be rotated in ten positions. Position 1 – brings in a random collection of objects to the display. Positions 2-10 each call up a saved visual map that has been pre-constructed by the museum to highlight interesting routes/readings into the collection. From any position users can further interact and change the composition to construct their own personal map by selecting and searching through objects of their interest.
  2. Info switch: On the bottom right side of each e-paper display is a small silver line that acts as a touch sensor to toggle between the display of a digital collection object or its metadata information in the form of an object label.
  3. Select/fix switch: On the bottom center of each e-paper display, a small silver circle acts as a touch sensor to toggle between the selection or deselection of a digital collection object. A selected object is held on the screen, keeping the image in the visual map throughout any subsequent searches.
  4. Search lever: The search lever is a two-way toggle switch. Pushing down initiates a search query using all of the currently selected items to query the digital collection. Lifting the lever upwards acts as a back button, displaying the previous state of the visual map.

Additional affordances will be brought to the Searcher through a web-based extension. These include the explicit introduction of collection items into the display by inputting museum record identifiers, a method for searching the collection with a semantic/text query, and the ability to title and save visual maps of the collection for later retrieval, printing, or to save as one of the pre-set maps within the physical Searcher installation.

Searching for (Walburgarian) References


Searcher Installation Components

Stand-Alone Searcher Application

DMG Digital Collection Feilds of Interest

Workshops

Design Museum Ghent