This article presents results of an exploratory Digital Humanities study focused on researchers working with digitized audio-visual (AV) sources, particularly regarding cases of mediated and historical events.1 In this article, we reflect on narrative creation processes, specifically research, writing and story composition by Media Studies and Humanities scholars as well as media professionals, and the pivotal ways in which digital tools inform these processes of search and storytelling around crossmedia AV sources. Whilst our study is concerned with supporting media research from beginning to end, we take a particular interest in exploratory search2 for supporting the first – exploratory and initial – stages of doing research, because during the initiation of a search researchers “may be in most need of support”.3 We argue that this is especially prevalent for researchers working with AV and crossmedia sources, due to the complex, dynamic and multifaceted nature of this data type.
Sonja de Leeuw has discussed the history and challenges for European television history since the dawn of its archival turn in the opening article of VIEW, arguing that “institutions and digital libraries are challenged to meet the needs of users, to construct new interfaces not only in-house but also through online platforms. This requires fresh conceptual thinking about topical relations and medium-specific curatorial approaches as well as user-led navigation and the production of meaning”4 (our emphasis). In this article we study how contemporary digital tools and platforms of cultural heritage institutions adapt and react to this challenge, in interaction with curatorial approaches and user perspectives. Here, we pay particular attention to research with AV sources via audio-visual archival institutions, and the impact on narrative creation around mediated events.
This article analyses and questions the ‘translation’ of AV data on different platforms into the narratives that we, as researchers working with AV sources, can tell – and by doing so, informs on the conclusions about the role of digital tools in meaning-creation processes. The study’s theoretical and methodological starting point is that narratives5 should be viewed in terms of their socio-technical context. Digital tools – used to search for, annotate, and analyse events – frame and afford the narratives that both media scholars and professionals as researchers can form around their research question. In their work, researchers study and integrate cultural and political meanings connected to media events.6 They delve into how said meanings – often disruptive, and long-term – are reproduced and made sense of via television and connected media platforms. In turn, we have studied how researchers search for narratives (cases) surrounding ‘disruptive’ events (such as natural disasters, terrorist attacks, and ‘breaking news’ marathons of disaster and terror) from a cultural-historical perspective, drawing upon archival and Linked Data materials from the Netherlands Institute of Sound and Vision and the digital search database and tool Media Suite (CLARIAH). Special attention is paid here to the functionalities of DIVE+, a Linked Data event-based browser, based on the simple event data model, where users can browse and explore different heritage collections simultaneously, which supports the creation of browsing narratives.
Outline and Method
This study integrates the research areas Media Studies, Information Studies and Science and Technology Studies. It connects research and search practices to data quality enhancement, to realize a cross-disciplinary project that seeks both technical and academic innovation. This study therefore takes a cross-disciplinary digital hermeneutics approach. By integrating digital technology for interpretation support, we provide insight into the roles of narratives in digital hermeneutics – the encounter of hermeneutics and web technology7 – and how events (and in what form) help interpretation.
Our theoretical and methodological framework connects ideas about user studies in Digital Humanities to our own user-centred design and mixed methodology: a co-creative design approach that includes focus groups, research diaries, and questionnaires with open questions, to learn about the role of narrative creation and exploratory search in media research practices. Furthermore, the framework brings prior work on media events and narratives into focus, in relation to our research on understanding user-technology interactions. Our analysis is focused on how researchers use and reflect on the use of exploratory search tools, and how exploratory search informs narrative creation practices. The collected data provides insights into how researchers search and explore digital audio-visual archives to form narratives.
Through user studies, we were able to focus on, first, how researchers construct navigation paths via exploratory search, and, second, evaluate the role of narratives in learning about historical mediated events and doing research into these events. In this process, DIVE+ (see §3 and Video 1) was also compared to other online search tools, such as Google Explore. Ultimately, studying working with AV can provide specific insights into the different perspectives that define the course and framing of mediated events, and our study offers a critique of digital tools’ socio-technical affordances in terms of support for search, retrieval and narrative creation by researchers working with AV materials.
Digital Humanities centres on humanities questions that are raised by and answered with digital tools. At the same time, the DH-field interrogates the value and limitations of digital methods in Humanities’ disciplines. While it is important to understand how digital technologies can offer new venues for Humanities research, it is equally essential to understand and interpret the ‘user side’ and sociology of Digital Humanities. Our overarching research question is concerned with how media researchers (scholars and professionals) appropriate search tools to ask and answer new questions, and apply digital methods when working with AV sources. To answer this question, we relate it to a concrete search practice and digital tool, and ask the sub question: how does exploratory search support researchers to study (disruptive) media events across media, and how these events are instilled with specific cultural or political meanings? As a result, we can consider the implications on how researchers interpret and negotiate AV sources and affordances of digital tools, in their own research practices.
User studies observe technology use in practice, and can therefore show how users appropriate technologies.8 User studies can serve to evaluate technologies in UI/UX testing (i.e. User Interface Design and User Experience testing) and pre-conceived use cases.9 They may also help us understand how technologies are increasingly becoming part of disciplinary practices.10 Whilst previous user research in Digital Humanities concentrates on assessing how and why Digital Humanities benefits from studies into user needs and behaviour11 – on user requirement research12 and on participatory design research13 – our article proposes to add to this body of research by presenting insights of a cross-disciplinary user study that involves researchers studying AV materials, in an iterative co-creative design process14 set to fine-tune and further develop a digital tool that supports audio-visual research through exploratory search. We employed a user-centred design methodology15 to analyse researchers’ engagement when using exploratory search, and more specifically, how users and technologies co-construct meaning and meaning-making practices.
We studied how media researchers use digital search technologies in their daily work practices, to discover and explore digital AV archival material. Our study includes three user types: (1) Media Studies researchers, who are generally more experienced in working with AV sources; (2) Humanities researchers that use AV materials as a source for research or are interested in doing so, with varying degrees of expertise; and (3) media professionals who need to retrieve AV materials for audio-visual text productions, such as television programmes, journalistic productions or other creative endeavours. In group 1 and 2 we met with both university students (advanced levels) and lecturers. Humanities researchers (group 2) include scholars with academic backgrounds such as history, international studies, digital humanities, communication studies, languages and culture studies, whilst media professionals (group 3) include journalists, television/image researchers, documentalists, documentary filmmakers, digital storytellers, and media innovation experts. These user types are the foreseen end users of DIVE+ and the overarching Media Suite tool and database, because they create audio-visual narratives for their respective work purposes.
We set up co-creative design sessions (see §5) with 89 researchers in both academic as well as professional settings, across different cities and institutions in the Netherlands (group 1: 21 participants; group 2: 57 participants; group 3: 11 participants) to observe and reflect on how they interact with search tools to explore, access and retrieve digitized AV material for narrative creation, and in some cases, creative re-use of this material in new audio-visual productions. From this micro-analysis, we extrapolate insights at the meso level: to relate insights gained about user interactions with one exploratory search tool (DIVE+) to more overarching ideas about user-technology interactions, and what such interactions imply about the role of digital tools in Humanities and Media Studies.
This research study is CLARIAH-centric, from the perspective of DIVE+. The latter is integrated in the national CLARIAH (Common Lab Research Infrastructure for the Arts and Humanities) research infrastructure in the Netherlands – as part of the CLARIAH Media Suite – and is aimed at providing researchers with access to digitized audio-visual data as well as tools for research and analysis. DIVE+ is an event-centric Linked Data digital collection browser, which offers intuitive or exploratory browsing and exploration of media events at different levels of detail. It connects media objects (images or movies retrieved from cultural datasets), places (geographical or descriptive), actors (people or organizations) and concepts that are depicted or associated with particular collection objects, to contextualize search paths into overarching narratives and timelines.16 This tool is the result of collaboration between computer scientists, Humanities scholars, cultural heritage professionals and interaction designers.17
Events are a central part of this data enrichment: giving context to objects in collections by linking them in events. DIVE+ builds on the results of DIVE by expanding this digital hermeneutics approach for interaction, interpretation and exploration of digital heritage via different and linked online collections, providing a basis for interpretation support in the searching and browsing of heritage objects, with semantic information from existing collections plus open Linked Data vocabularies.19 This browser offers events-driven exploration of digital heritage material, where events are prominent building blocks in the creation of narrative backbones20 and links a variety of different media sources and collections. Whilst DIVE+ is continually updated, including through crowdsourcing, at the time of our research the browser contains entities from Delpher (scanned radio bulletins from KB/National Library of the Netherlands), Amsterdam Museum, Tropenmuseum and the Netherlands Institute for Sound and Vision (news broadcasts of the Open Images collection).
Our research study aids in answering the question how such a browser – technically based on linked open data, supporting event-centric exploration or context analysis – can support a scholar/researcher from beginning to end, and therefore this study helps to improve DIVE+ (as part of the overarching Media Suite) as a browser. To do so, our research study draws upon the exploration of narratives (narrative centric approach) instead of other types of search (for instance more traditional or document centric approaches, such as faceted search). Moreover, this study addresses the purpose and usefulness of narratives for scholarly research.
Exploratory Search: A Basis for Tool Criticism and Researching ‘Disruptive’ Media Events
Users’ ideas and practices with exploratory search and retrieval technologies can not only shape AV narratives and productions, but can also enhance the development of exploratory search tools. Our study contributes to ideas about tool criticism in Digital Humanities research. According to David Berry,21 the first wave of Digital Humanities focused on digitization and realizing technological infrastructures, whilst the second wave was generative, creating environments and tools to interact with data that is born digitally. The third wave, which Berry refers to in terms of a third layer, should concentrate on “the underlying computationality of the forms held within a computational medium (…) to look at the digital component of the Digital Humanities in the light of its medium specificity, as a way of thinking about how media changes produce epistemic changes”.22 We advocate a research approach as a part of which questions such as why specific data is collected, for what purpose, and within what context – the so-called politics of archiving – are addressed from a critical (Humanities) perspective.
In line with Berry, we aim to understand (research) culture through digital technology, and even more specifically, the ways in which digital tools facilitate everyday research practices.23 We interrogate the underlying assumptions about how media researchers explore AV materials online. This is in line with Berry’s argument that one should understand culture through the use of (and through working with) digital technology; with a focus on how people use software in their everyday practices.24 Moreover, a reflection on the use of a digital search tool designed to afford both exploration and narrative creation, allows us to draw user-validated conclusions about how this particular tool reshapes an understanding of what it means to explore and create narratives via digital tools. It may well turn out that the ways in which the tool designers translated ideas about exploring and narrativizing digital material, do not match how the foreseen users understand exploration and narratives.
We argue that exploratory search is crucial for Humanities researchers who draw upon audio-visual materials in their research. Recognizing relevant multi-platform sources and bringing these to attention – in an iterative fashion – greatly supports scholars in their research. Supporting researchers’ explorations is especially relevant in the case of scholars studying complex mediated and/or historical events. In the first place, because audio-visual, online and digital sources are in abundance, scattered across different platforms and changing daily in our contemporary landscape. Second, disruptive media events are difficult to interpret due to the challenges of grasping the immediate story. A media event is an event with a specific narrative that gives the event its meaning, and is in contemporary societies increasingly recognized as non-planned or disruptive. Disruptive media events,25 such as the sudden rise of populist politicians, terrorist attacks or environmental disasters, are shocking and unexpected, making them especially difficult to interpret. One can even argue that in today’s crossmedia landscape, disruption has become a marker of the way in which news narratives are continually told, circulated and shared across media, formatted as breaking news.26
This leads to problems for researchers who analyse how narratives construct different political, economic or cultural meanings around such events. Previous research argues that media events should always be viewed in relation to their wider political and socio-cultural contexts. Events, as they unfold in the media, may correspond to long-term social phenomena, and the way in which such events are constructed has particular connotations. Specific actors (newscasters, governments, institutions, political interest groups) use media events to build narratives in line with their own political, economic or cultural purposes – examples are stories of empathy, fear and change in relation to international media events.27 We argue that researchers, in turn, also build event narratives, and can therefore said to be storytellers. Yet, disruptive media events, such as live broadcasting marathons of disaster, terror, and war, have not yet been researched in the context of exploratory search strategies.
Searching for stories, shapes stories.28 Prior research underlines the importance of visualizing, constructing and storing of narratives during information navigation to contextualize retrieved materials.29 Our own research study further illuminates the role of media researchers as storytellers, and their processes of selection and interpretation when working with audio-visual sources and learning about mediated events, especially regarding search, retrieval and narrative creation.
Co-Creative User Sessions
Our case study approach combines grounded theory – that fosters an understanding of how researchers interpret and create narratives – with usability methodologies, such as work task evaluations. First of all, this allows us to draw conclusions about how search tools and digital technologies co-construct the researcher’s professional practice. Second, the data helps us probe the question how the kind of digitality of search and retrieval shapes the practice of media research, and, in extension of this, creative and storytelling processes. The research takes an interdisciplinary approach: it combines insights from Media Studies, as well as from Information Studies and Science and Technology Studies and integrates ideas about narrative creation, search practices, and overarching notions about how users and technologies co-construct meaning.30 Therefore, the presented research does not necessarily focus on how Digital Humanities’ tools have an impact on researchers’ practices, but rather analyses how researchers make use of search tools.
In our user study, we collected qualitative data to answer our main question; in keeping with our user-centred approach, we (A) observed how users used the search browser by giving users search tasks31; (B) asked users specific written and verbal feedback about their user experience (questionnaires with open questions and research diaries)32, and (C) collected user perspectives on the role of digital search technologies in Humanities research in the shape of user-generated posters. The user study observes media researchers as they use DIVE+ to explore media events, across three stages: (1) research question formulation; (2) DIVE+ use; and (3) comparative user evaluations of the DIVE+ browser, compared to other online search tools such as Google Explore, resulting in specific search narratives. While interacting with the search browser, users were observed, and asked to provide feedback on their search experience, talking aloud about their search journeys. They were subsequently asked to export the navigation paths that were generated in the DIVE+ browser and provide written or verbal feedback on their experiences in terms of how DIVE+ supports narrative creation about historical events. This feedback was, then, discussed during a focus group session, in which we asked participants to reflect on their experiences.
Users were introduced to the DIVE+ search browser, the overarching Media Suite, as well as Google Explore and selected online audio-visual repositories, and subsequently asked to perform a search task. Search tasks are “goal-oriented activities carried out using search systems”.33 We developed exploratory search tasks in line with recommendations for task design.34 This means we tailored tasks to research situations. An example task given to the users of the DIVE+ browser was:
Example task 1: Imagine that a media company is going to produce programmes about Jakarta, Beatrix (former Queen and now Princess of the Netherlands), Islam, or Watersnoodramp (1953 North Sea flood). Your goal is to propose an interesting angle for one of the programmes.
For an exploratory search task such as described in example task 1, with a specific focus on the keyword Watersnoodramp (referring to the 1953 North Sea flood, a natural disaster in the Netherlands with 1836 casualties), this could result in an exploration path and search narrative as visualized in Video 2.
Another example task given to users to research a long-term media event was:
Example task 2: Try looking for sources about the representation of the social acceptance of migrants, refugees and migration as a long-term event, and its impact on (Dutch) society. What research questions are sparked by what you find? How do the search affordances of the online repository/ies shape your research question and your understanding of the topic? Reflect on your own role as a storyteller, and how you think the tool you are using influences this role.
For an exploratory search task such as described in example task 2, with a specific focus on the keyword vluchteling (refugee), this could result in a navigation path and search journey such as visualized in Video 3.
The Research Journey
Research into mediated events represented on multiple media platforms (including crossmedia or multi-platform audio-visual texts) can then take the following general steps when circling around a research question and specifying your research topic. This process of the research journey follows, in an iterative fashion, the steps of Explore – Refine – Analyse – Tool Criticism – Write – Disseminate. These are presented below in a model for grounded analysis, which answers to our discussed need for hermeneutic approaches in Digital Humanities, to closely consider practices and mechanisms conditioning narrative creation and for researchers to include a self-reflexive approach.
Step 1. Explore
Exploring the topic to acquire contextual information about the topic (exploratory search, context acquisition):
- searching for videos;
- access academic databases to explore topics, read historical overviews and articles;
- searching for AV-material using faceted search to search for names, date, genre (news, documentary, current events programmes), and by broadcaster
- visiting archives physically to read newspapers on the days of the event, and the weeks/months after.
Making a decision about which collections/archives are of interest to search, has ramifications: are these collections accessible, in terms of (1) their location: does the researcher need to visit the collection/archive in person, is there a digital point of access; (2) materiality of the collection: is the collection retrievable, and in what material form (physical objects, or digitized, or digital) and (3) contextualization, such as accessible metadata and other forms of contextualization that gives research value to the collection items.
Step 2. Refine
Refining ideas about the topic by sorting and relating sources. There are different ways to connect materials on paper (a researcher may use a mind map to draw out how sources relate) to primarily piece together:
- sequence of events;
- the different sources that are found (is it a primary source, is it a secondary source);
- draw out storylines: what do the sequences of events (described by the different sources) show in terms of a narrative, what is the story that is being told? Is this a description of events. Is it an interpretation of said facts? In other words, how are the disruptive events translated into a story (short or long-term narrative)?
When searching in collections, the researcher can refine search for instance by title and keywords of the event or implicated persons (to see whether data is available about the event) (1) as close to the event in time and (2) media content that discusses the event (for example, political talk shows more distant to the event than directly after) to collect discourses surrounding the event for analysis. For each collection, the researcher should apply and source criticism. These include questions such as what media objects, subjects, places, and actors are part of the event, or what information is available to be able to study the event? And what is the position of the retrieved object in context of the larger collection it belongs to? Researchers therefore critically reflect on the role of provenance, novelty, and diversity of objects and collections.
Step 3. Analyse
After selecting a corpus, the researcher analyses this corpus to gain insight into processes of construction and manipulation of meanings: analysing selected materials, looking specifically at how each item tells a story, or trying to piece together what is happening or has happened, per:
- Type(s) of material and medium: television broadcasts, radio broadcasts, online articles (when archived), newspaper articles, interviews, scholarly articles;
- Narrative discourse(s): how is the story about the event being told, what are the central keywords used in the descriptions – because this helps creating insight into the discourse(s) surrounding the event;
- What stories/narratives are told about the event? How are media trying to understand what is happening? And what do these narratives signify in terms of how we interpret media events?
- Integrating findings.
Step 4, 5 and 6. Tool criticism, writing, and dissemination
Finally, the researcher integrates findings and writes these up. During the writing process, including recording findings for dissemination, the researcher also demonstrates tool criticism – also in relation to the aforementioned step of source criticism – explicitly reflecting on and demonstrating awareness of:
- How the archive and search tools used, constrain or shape the outcome of the research process. During the writing process, the researcher therefore also needs to have access to or be able to gather information about the selection and interpretation process of the used tool and repository/database;
- How the research and dissemination practices of the researcher (contextualization, re-mix, re-use) could possible add to further contextualization of cultural heritage objects;
- And finally, during the writing up of findings for dissemination, the researcher can pay particular attention to how the research gives insight into how media create lucid narratives about events that are inherently complex and chaotic, as well as scattered.
This form of grounded analysis leaves room for scholars to discover unexpected insights, new narratives and discourses. Discovering a multitude of narratives around the event can also be just as interesting, as it grants insight into the multi-interpretability of past events.
Analysis of Search Practices and Tool Criticism
Using exploratory search during research question formulation
Exploratory search tools are not used very often.35 Media Studies researchers did indicate enjoyment at the freedom that exploratory search offered them, especially in terms of how it triggered research questions. For instance, Media Studies students with advanced experience in working with audio-visual sources and digital search tools (BA level 3 Research Seminar) seemed to associate a clear research question with rigorous and intent heavy search, and exploratory search is regarded as more free flowing, aiding them in learning about facts that they would not have learned about when using more traditional sources. Exploratory search in this way can help with further focusing or defining the scope of one’s research, and even with developing a research question: “Exploratory search can result in new perspectives and approaches which in turn benefit the initial research” – Media Studies researcher [respondent no. 55]
Humanities researchers further indicated how the randomness of source selection opened up chances for researchers to find sources that other methods might not reveal. In particular, collections that offer the possibility to search Linked Data (related entities) from a singular entry point, were considered to have the potential to illustrate context more than a historical account might provide. Contextual understanding is also central: respondents identify quite often that exploratory search does not necessarily add to the actual research project, but to the understanding of the topic they are researching. On the one hand, this seems to be valued quite highly, but on the other hand, it does not seem to be a priority during research in general, as a group of Media Studies researchers concluded after collecting user perspectives on the role of digital search technologies in Humanities research in the shape of user-generated posters: “Overall, we do believe that exploratory search is useful but perhaps to create a general understanding of the topic you are researching, rather than to find specific information that could answer your research question” – Media Studies researcher [respondent no. 56]
Exploratory search then seems to function more as a creative stimulus. Makri et al. have argued that digital information environments need to support serendipity strategies to allow users to “make mental space or draw on previous experiences”.36 In this context, the co-creative design sessions practically point to how exploratory search during research question formulation and information retrieval offers potential for serendipitous browsing. Serendipitous search encounters are generally characterized as fortuitous accidental findings that are the outcome of a creative act,37 which is either afforded by the personality type of the seeker (e.g. ‘super encounters’38 have prepared minds and are open to recognize serendipitous findings) or by triggers embedded in the search system. Our user studies bring into view what organization and management theorist Miguel Pina e Cunha has described as: “[W]hile unexpected sources of knowledge are by definition impossible to locate (…) serendipitous discoveries may result from intentional exploratory search processes”.39 However, although finding new, unexpected narratives is important – in terms of new discoveries (!) – discovering insights serendipitously is not a goal in itself. Rather, eliciting serendipity is part of the skillset of a researcher, implicitly.
Angles for creative content
In this process, for media professionals specifically, the research question is translated into searching for an angle on a topic: from macro (the bigger idea or angle) to micro. The ‘angle’ is something that depends on the perceived audience of the programme or text the professionals are creating. For instance, an informative programme for a young target audience requires a different take on the Watersnoodramp (North Sea flood) disruptive event, then a documentary for adults would. Exploration is guided by expectations about the audience and the researcher’s own domain knowledge: how much does the professional personally know about, and how much are they personally interested in, the topic? How much exploration is afforded, also in terms of time and budget? Exploration is impacted by the professionals’ poetics,40 meaning the practices, conditions and unwritten rules of thumb guiding the selection and interpretation processes of media professionals with different genres, programmes (for instance television history programming) and target audiences,41 which in turn guide practices of creative retrieval as well.42 The institution of the archive, and the documentalists working there, need to be included here as agents of historical knowledge, as they also reveal such particular aims, strategies and conditions regarding the providing of access, contextualization and circulation of AV sources.43
Our user studies further reveal how media professionals (journalists, television/image researchers, documentalists, documentary filmmakers, digital storytellers, and media innovation experts) often search Wikipedia and YouTube to find inspiration for an angle, while newspapers (in databases) are reviewed for more detailed information about and around a topic. Previously made productions are also revisited: what was already made and searched for regarding the topic? Professionals’ search also includes various search tricks: the use of words that will lead to interesting material (such as the search term curiosa, which is a term that only expert users of the archive system would think of). Offering a browser that invites users to find inspiring and interesting material for a new angle on a topic becomes relevant for AV narratives that need original content, such as documentaries or the news. Sometimes the sheer amount of material is daunting, however, the media professionals feel that this needs to be browsed through. Digital tools offering better support for this, is something that is met with certain enthusiasm.
Making meaning, creating lucid narratives
All three user groups demonstrate deeper reflections about how the tools that they used for (re)search and retrieval, inherently provide narrative elements. On an individual level, this is regarded as crucial in relation to the subjectivity of research. Users reflected further on how they – meaning the user as a researcher – are not the only influential factor regarding their produced research narrative, rather their tools and their own use of these tools also impacts the way in which their research is shaped and narrated:
Meaning is attributed to the way one searches and conducts research – Media Studies researcher [respondent no. 58]
The meaning is formed by the search tools you use and the way that you search – Media Studies researcher [respondent no. 64]
Real connections still have to be made in an old and traditional way… in the mind of the researcher – Humanities researcher [respondent no. 14]
Subsequently, the resulting search or narrative path, which represents a mediated event as a (more or less) lucid narrative, is also not regarded as neutral: “Narrative is a framing tool that helps shape information” – Media Studies researcher [respondent no. 59] (our emphasis). Our research study offers practical examples of how exploratory search can, then, support interpretation and narrative creation of events, through the visualization of the navigation path. “Exploratory research lets you see connections and thus shows you the meaning of AV content” – Media Studies researcher [respondent no. 75]. “When AV content is put together and looked at as a set, it can become a part of a narrative with a variety of meanings” – Media Studies researcher [respondent no. 73].
Trust versus hidden agendas
On an institutional and cultural-historical level, prior media research has argued how the transmission and portrayal of any event is necessarily dependent on the attitude or demeanour of the broadcasting institution.44 The large-scale comparative research of the European Television History Network has demonstrated how “[n]o event is value-free and neither is its mediation or interpretation. Historically, and across cultures and borders, values change”.45 However, this seems problematic when investigating and generating narratives in an exploratory search tool such as DIVE+. This is the case because currently, despite the fact that exploratory search and the visualization of the search path in DIVE+ can support narrative creation, researchers do not grasp how the tool mediates an attitude or demeanour.
Based on our studies, we argue that trust in the search engine, browser and archive, is usually based on prior experience. Prior experience regarding search and retrieval determine the user’s expectations, their skills (for example in investigating signposts, such as the About page, for clues on the politics of archiving) and therefore their attitude towards retrieving dependable search results. As a respondent describes: “Even a database has a hidden agenda (…) Can I trust the algorithm?” – Media professional [respondent no. 3] (our emphasis). It is also relevant to note how in DIVE+, a search for Watersnoodramp leads to material that is dated before the time of the flood. This is interesting to the user, because it triggers curiosity about what the browser suggests. The results encountered through exploratory search are regarded as directionless in the sense that their usefulness depends on the researcher, and the project. The direction and value of the results are thus heavily dependent on the way in which they are used. In relation to our previous point that researchers currently do not grasp how the tool mediates attitude or demeanour, when it is difficult to gauge where materials and entities come from, this makes it problematic for the user to assess the usefulness of the source. In addition, crossmedia audio-visual sources are changing daily, and hence sometimes brings forward a different result due to removal of data from the database. Improvements for the DIVE+ browser are then specifically directed towards how more transparency about entities and relations should be added:
There [in another search tool] the data-triples were shown, the entities, the relation between them, these were explicitly shown. And that already gave me more inspiration … where does this relation stem from, you could find out very quickly, by directly clicking on it … and it was revealed that the birth place is Ghent (…) It’s not immediately clear at a glance what the link is between the entities when you look at your search results – Media professional [respondent no. 3] (our emphasis)
The experience was OK, but the interface is very cluttered. There is too much visible on the screen – Media professional [respondent no. 7] (our emphasis)
Need-to-haves or nice-to-haves?
Media professionals describe fine-grained selection functionalities as the ‘need-to-haves’, especially to easily refine search results beyond entity categories: clear, well-defined search fields and more filter options, including per medium to make a distinction between text, audio and video in search results. Respondents argue that when such need-to-haves are lacking, the functionalities offered for exploring and linking are only ‘nice-to-haves’. Professionals especially request more direct insights into in-depth relationships, stating that this is now deemed too shallow:
You will quickly find relations (connections) based on general search terms, but unfortunately, I did not find the depth of the relation between Beatrix and woningnood [housing shortage] – Media professional [respondent no. 4] (our emphasis)
Very broad results, it’s often unclear why something is shown. You see few other relations between the results except the keyword Jakarta. I had expected a concept such as the independence act’ – Media professional [respondent no. 6] (our emphasis)
The dataset on the background is missing critical mass to deliver sufficient results – Media professional [respondent no. 8]
Professionals argue that their expressed need to give users more control over search filters, stems directly from the fact that in their professional practice, they are used to using search interfaces with many, many search fields. “The useful thing about many search fields is that you can focus very nicely on where you start and end in the definition of the field” – Media professional [respondent no. 3]. Prior experience, again, is thus a key factor impacting the interpretation and selection experience.
Analysis of Search Narratives: The Steep Learning Curve
Exploration routes and meta-structures as narratives
The search engines most often used for exploratory search by our respondents were DIVE+ and Google Explore, the Google Trends explore functionality. While DIVE+ is designed for working with audio-visual sources, the lay-out of Google Explore was deemed more user friendly and easier to navigate by our users. The learning curve of using DIVE+ made it less attractive for use from the outset, compared to Google Explore. This was made especially clear in our studies by respondent commentary about the difficulty in assessing both how connections between entities are established by the tool, as well as the unclear depth of the relation between entities (see also the commentary made above by respondent no. 4 regarding that they did not find the depth of a particular relation).
Across all user groups, respondents expressed how the DIVE+ platform and exploratory search can help in guiding the user, and even aid in raising new research questions. Platform functionalities and affordances can help steer or guide the researcher and at the same time can push to formulate new questions. First, exploratory search is considered by our users to demand narrow research questions. “The added value is that you can determine (…) what your topic is going to be about based on the available research data” – Media Studies researcher [respondent no. 54]. Second, exploratory search is regarded as iterative. For example, one respondent (Media Studies researcher, respondent no. 57) described the process of exploratory search in DIVE/DIVE+ as constant revisioning of the research question based on the retrieved results. Here, a search narrative is defined as a route which indicates different phases. This underscores the learning curve of exploratory search, and different phases of narrative creation for the researcher: narrative creation as an exploration route.
The users’ responses show that narratives in general, and in particular research narratives, are not a fixed entity but fluid. The attached meanings are ever-changing, based on the conditions in which discourses are encountered and constructed via individuals or events. It is noteworthy that both exploratory search and narratives are classified by respondents as non-fixed. Narratives are seen as to be composed of other narratives, in the sense that texts are constructed from other texts:
[Narrative is] a way of framing information and events, that makes certain elements strange and normalizes others, creating something like a story – Media Studies researcher [respondent no. 65]
During the process of collecting data, the narrative might change, for media researchers might find information that changes their research question and primary focus – Media Studies researcher [respondent no. 59]
Importantly, users indicate here how the practice of telling narratives is, as we saw earlier for the practice of searching, based in prior experiences, and narratives are shaped by prior experiences. Research itself was not considered as a narrative by all respondents. For Humanities researchers especially, research was strongly considered as not a narrative: “I believe that the narrative metaphor does not really apply to my research, because I do not produce sequential data, but rather a meta-structure, which cannot be told as a story” – Humanities researcher [respondent no. 38] (our emphasis). Media professionals were most critical whether the DIVE+ search path resulted in a narrative:
The list of narratives is very helpful, but does not really yield a story. More like a storage of the search process – Media professional [respondent no. 2] (our emphasis)
I mainly found general information and a further search for a relationship with an event did not offer a satisfactory outcome – Media professional [respondent no. 4] (our emphasis)
Subsequently, visualization of more in-depth relationships is requested by users as an improvement of the exploratory search browser.
Media users as storytellers in control?
Professionals also found that not every click should be saved in the exploration path, which not only points to giving the user more control over search functionalities like filters, as discussed above, but also more control over the lucid narrative that is generated (in the form of the exploration or search path), which can be exported offline and saved on the researcher’s own desktop:
Ideally this functionality [saving the search log] will not simply save my entire click history, but will retain only relevant results– Media professional [respondent no. 7] (our emphasis)
It would be more useful if DIVE[+] did not save everything itself, but only on therequest of the user – Media professional [respondent no. 2] (our emphasis)
Media researchers (scholars and professionals) are, in fact, storytellers. Our research outlines how researchers build narratives, and makes the role of the researcher and digital search tools in the construction of narratives explicit. This highlights the interpretative aspects of research, and research is always being interpreted in certain (social) contexts. Practices of search, research and retrieval, too, frame a certain version of reality through the construction of a narrative.
The researcher is framing the narrative by choosing which sources to use and not to use – Media Studies researcher [respondent no. 61] (our emphasis)
Media researchers acquire information from multiple searches and piece this information together in order to find similarities, patterns, and discrepancies. These are then put together in a storytelling format – Media Studies researcher [respondent no. 71] (our emphasis)
Such skills, on the one hand, seem to be something that people in modern societies are more and more used to, as well as actively developing: in our current association society46, individuals function as experienced as a kind of information hunters and gatherers47 that collect information from different platforms or databases in logical narratives for themselves. On the other hand, our research also indicates how these skills, as well as awareness of how such skills contribute to understanding for both learning and doing research, can be better supported.48
Across all user groups, user explorations underline the difficulty for users to create narratives about media events, due to the fact that there is a learning curve when it comes to understanding how to inspect collections for metadata, how to compare collections, and even how to explore collections. The features and interface of DIVE+, especially, offers a steep learning curve. Each of the tools in the Media Suite supports users in a particular way, but it is a challenge for users to synthesize found source materials into an overarching narrative. The ideal place for this synthesis would be the Media Suite’s workspace functionality, where a user can create a workspace for a particular (shared) project and to collect and inspect bookmarked materials.
Reflection: The Researcher in a Split Position
In this study, we have argued how narrative creation occurs during the encounter and interaction of digital search apparatuses’ attitudes, with those of the researcher. We have also pointed out differences between research fields in terms of prior skills in search and retrieval, and the expectations regarding search and retrieval that arise during the research journey. As we learned in our study, researchers themselves can also be made more aware of how, through their own search and research practices, they build narratives around events, and how this impacts the meaning making process. Offering researchers the ability to explore and create lucid narratives about media events, including bringing relevant (multi-media and multi-platform) AV sources to their attention, therefore greatly supports their interpretative work.
We argue that this is especially prevalent in the first exploratory search stage of typical media and humanities research.49 Exploratory search is crucial for researchers who draw upon media materials in their research, because audio-visual, online and digital sources are in abundance, scattered across different platforms, and change daily in the contemporary landscape. Supporting researchers’ explorations becomes even more important when scholars study disruptive media events via audio-visual sources, due to the complexity of the narrative and the audio-visual text’s representation – including re-presentation in digital heritage and memory institutions.
In today’s association society, we then find the media researcher in what could be described as a split position. On the one hand, there are important new opportunities and new types of questions that can be asked, which encourages the (re)use of television archives and European audio-visual heritage, promoting engagement with cultural memory on national and international levels. The increased access and more direct availability of high-quality material, with connected metadata and contextualization makes and keeps AV material valuable for research. Digital tools offer significant research opportunities to identify useful data faster and over a longer research period. This also includes important multimedia and crossmedia perspectives, such as searching and linking various data sets from different collections via a singular entry point.
On the other hand, there are also new challenges and new types of questions that should be asked. One challenge is that practices of crossmedia and transmedia storytelling, for instance television programme websites and social media platforms with relevant contextual information are highly susceptible to change. Often there is no structural archiving of such contextual information, regarding online (web-archiving), printed and digital production documentation for a complete memory of production. Furthermore, media literacy remains a considerable issue for the skill sets of both digital natives and non-digital natives. New critical questions to be asked concern the so-called politics of archiving. Audio-visual sources represent a construction and selection of our reality, and their (un)availability in a database is again a selection: curators adding a further interpretative layer. In short, in the digital age, more people are part of the selection processes of the media representations we reuse and encounter as researchers.
Exploratory search can support researchers’ explorations of difficult to interpret disruptive media events, potentially offering serendipitous browsing and discovery of event narratives, helping users to better assess the quality of sources. However, this serendipitous browsing needs to be anchored to situated search practice of the researcher – thus, creating a tool that affords both exploration and anchoring of narratives. However, such opportunities of linked open data do require a shift in search cultures.
It is therefore relevant to deconstruct how exploratory search and digital tools afford narrative creation, giving insight into the constructed quality and key perspectives that define the course and framing of mediated events. But also, how they shape narratives due to technological affordances and constraints. Creating narratives whilst exploring adds understanding plus creative insights to research and learning through audio-visual materials. This process also highlights the constructed nature of narratives in general, making users aware of their own storytelling practices. Based on gained experiences, respondents often expect to find exactly what they were looking for, but this is not what exploratory search offers: users thus had to open themselves up to new search learning curves and expectation management. Across all user groups, exploratory search was understood as a kind of loose concept, or as research without a direction. If searching – especially in the early phases of research – produced unexpected results, it could already be regarded as exploratory, and successfully serendipitous. Moreover, the respondents stressed the importance of a ‘refine step’ in the research journey, when both research questions and search queries are revisited, repeated and revised.
Opportunities of linked open data, then, seem to require a shift in search attitude or even search cultures. Moreover, as results are interlinked across data types, platforms and databases, free association is supported by exploratory search. Links which redirect users from a certain source to another were also often associated with exploratory search, and its functionality to make interconnections more visible. Once users were able to recognize the value of the lack of directionality and of meandering AV that exploratory search offers, they loosened their expectations of finding what they wanted to find, and rather started to focus on the value of what they happened to find whilst roaming the archive – allowing for unexpected insights into topics.
We have used different methods to gain insight into users’ search behaviour, contributing to an understanding of users’ “non-purposive information practices”50, as well as to the development of digital tools. Reflecting on tool usage with researchers grounds the research in the professional, daily practice of the end user, and strives to embrace the complexity of Digital Humanities projects: balancing Humanities’ and Computer Science concerns.51 Digitization has changed work practices of media scholars and media professionals, and in their research practices they increasingly use digital archives to create media texts. This means that retrieving audio-visual material requires an in-depth knowledge of how to find sources digitally. Our studies show how in interaction we can perhaps learn most, and more effectively, about this.