Public organizations, in the role of data providers, are releasing open government data (OGD) (Davies, 2010). OGD focuses on public sector information associated with agencies at federal, state, and local levels that collect and record data in order to monitor or inform social arrangements relating to human populations, economics, and public services (Open Data Handbook, 2015). The providers produce and provide the data to others without any restrictions on its use or redistribution (Janssen, Charalabidis, & Zuiderwijk, 2012). The providers hope that OGD will lead to better transparency, citizen engagement, and innovation (Charalabidis et al., 2018). On the other side, intermediaries develop information products or services (solutions) on the data for others (Davies, 2010; Janssen & Zuiderwijk, 2014; van Schalkwyk, Willmers, & McNaughton, 2016). Users who are seeking information can use these solutions to satisfy their information needs, which can be experienced when they attempt to satisfy a primary need and encounter a gap in their knowledge (Belkin & Vickery, 1985; Wilson, 1981). Users can draw facts from data provided by data providers (Davies, 2010). However, the rawness of the data can make it difficult to use it for any meaningful purpose relating to the life events or decisions of users (Weerakkody, Irani, Kapoor, Sivarajah, & Dwivedi, 2017). Intermediaries play a vital role in assisting users in interpreting and acting on information drawn from the data (Hunnius & Krieger, 2014), adding value to the data (Janssen & Zuiderwijk, 2014), and increasing its accessibility and utility (van Schalkwyk et al., 2016). However, OGD is not reaching its expected level of reuse (Zuiderwijk, Susha, Charalabidis, Parycek, & Janssen, 2015), which can come from a lack of awareness or interest (Hellberg & Hedström, 2015; Huang, Lai, & Zhou, 2017). Data providers attempt to tackle the low awareness and engage intermediaries to develop innovations with OGD via innovation contests, like Hackathons (Johnson & Robinson, 2014). However, satisfying users’ needs is the developers’ fourth motivation to participate in such events, after the fun and enjoyment, intellectual challenges, and status and reputation (Juell-Skielse, Hjalmarsson, Johannesson, & Rudmark, 2014). Therefore, it is possible that the methods used to stimulate the use of OGD do not meet the information needs of the crowd and carve a gap between intermediaries’ information solutions and real users’ information needs.
To solve the above problem, the purpose of this paper is to develop design principles that can help data providers, intermediaries, and users design methods to capture information needs of users that can be satisfied by building information products based on OGD. A design principle is a statement that prescribes what and how to build an artifact in order to achieve a predefined design goal (Chandra, Seidel, & Gregor, 2015). Providers can use the information needs to identify sought-after data, intermediaries can use the information needs as an input to the development of information solutions to satisfy these needs, while users can follow the design principles to help them express information needs of their community. The design principles are a first step towards bridging the gap between a group of users, their information needs, and information solutions in a specific everyday context of information use. The information needs expressed by implementing the design principles could cover three functions of information use for the users: fact-finding (answering specific questions), staying aware (keeping up to date in a topic) and briefing (refreshing one’s knowledge and understanding in more depth) (Erdelez, 1999; Nicholas & Herman, 2010). This research is guided by the following research question:
In this paper, the presentation of the research process and results follow the recommendations of Gregor and Hevner (2013). The paper is structured as follows. Section 2, the background, introduces the concept of information needs, approaches and methods to capture information needs, and the design principles as prescriptive knowledge for designing methods. It has contributed to the design and development of our design principles. Section 3 develops the research process, anchored in Design Science Research. Section 4 presents the final design principles, with examples taken from the last instantiations, and the evaluation. Then, Section 5 discusses the results and implications, which is followed by a conclusion.
People gain information when they make sense of data, where data can be defined as a representation of objective facts or unprocessed information (Hey, 2004). Open datasets can offer many different reuses, but first aim to produce information (Hey, 2004; Portal, 2019). When there is an extra processed layer added to information, it can become a digital information solution (e.g., dashboards and journalists’ blogs (Davies, 2010)), an improved service (e.g., waste collection optimized by the real-time data of sensors placed in the trash bins), or an aggregated information service (e.g., an optimized route planner based on different data sources and public transport data) (Berends, Carrara, Engbers, & Vollers, 2017). These solutions can be used to satisfy information needs, which Wilson (1981) considers to be a secondary need that can emerge when people try to satisfy primary needs. When people recognize a gap in their knowledge, information needs arise (Belkin & Vickery, 1985) that are personal and contextual. Information needs are determined by the roles an individual fills in social life, the environment within the role is performed, and depend on his level of knowledge on the matter (Wilson, 1981).
Information can help people in their work, with problem-solving, or pursue hobbies (Nicholas & Herman, 2010). Information is used to find facts, stay aware, research, obtain background understanding, stimulate thoughts, and recreation (Nicholas & Herman, 2010). Its value does not decrease as it is used. Rather, the value can increase as one piece of information is added to the other (Moore, 1997). The search for information follows these stages: (1) needing (the first inkling that some information might be needed), (2) starting (attempting to understand their need and preparing for a search), (3) working (to identify the information), (4) deciding (on the value of any results), and (5) closing (and wrapping up the work) (Westbrook, 1993). Any of these stages can be the final one, skipped, iterated between, and lead to a new effort to seek information. People can encounter information when they are searching for information related to another topic or bump into it as they are working (Erdelez, 1999).
Information needs are difficult to understand for the person experiencing them and the people who study them. They can be perceived as a hologram that the person walks around and through and may have difficulty putting it into words and seen as an anomaly (Westbrook, 1993). If the situation is unclear, confusing, or difficult to comprehend, the person may be unable to formulate questions that can help them identify information for their need (Daft & Lengel, 1986). This situation can be accompanied with a feeling of uncertainty and apprehension (Kuhlthau, 1991). The person might need some initial gathering of information to make a general, internal expression of their need (Westbrook, 1993). They can discuss their need with friends or relate it to previous experiences and knowledge (Kuhlthau, 1991). New pieces of information can cause the frame of reference to change (Westbrook, 1993), which can change their understanding of the need or the need itself. On the other hand, people might lack awareness of their need (dormant or unrecognized), know about it, but not work to satisfy it (unexpressed) (Erdelez, 1999; Nicholas & Herman, 2010), or misunderstand their expressed needs (misguided) (Shenton, 2007).
From an external perspective, the actions of a person attempting to satisfy their information need can appear chaotic. They can jump between past, current, and future information needs and different information-seeking processes, which can lead to cross-pollination. These behaviors can make the person seem to behave irrational, change focus, and not follow search directions provided by experts (Erdelez, 1999). At the same time, encountered information (1) can help a person satisfy another information need in the present, (2) could have helped them or other persons in the past, and (3) can help them satisfy a future information need (Erdelez, 1999). Consequently, other people, experiences, and situations are all sources of information and knowledge, which means information can be found in unexpected ways and places (Erdelez, 1999).
People could want, demand, or need information (Nicholas & Herman, 2010). Information wants are similar to a dream scenario and are often based on a bit of wishful thinking. In a perfect world, it would be the same as their information need. Lack of time, resources, motivation, and knowledge can mean people are tempted to obtain information that they in fact do not need. A want can lead to a demand for information, which is formulated as a request for an item of information based on what they think they want. However, there can be a mismatch between their understanding and the information, which causes them to demand information they do not need (Nicholas & Herman, 2010). The above qualities of information needs add to the difficulty of identifying and satisfying information needs of users for data providers and intermediaries.
Previous research has explored how to capture needs and information needs of users, within different schools of thoughts. It is often framed as a form of users involvement that has become a common practice in, for example, Information System, Human Computer Interaction, Service Design, Innovation, New Product Development, and Marketing. A better understanding of the needs can increase the user’s satisfaction and acceptance of the designed solutions, by improving the solutions all along the process (Kujala, 2003). The active involvement of users is a growing innovation strategy to develop internet-based applications since it can help intermediaries to understand user requirements, access to useful information about their context of uses, habits, preferences, get new ideas, and define the scope of a project (Komninos, Pallot, & Schaffers, 2013). Therefore, understanding the users’ needs can serve multiple purposes and the used methods vary according to the objectives of the intermediaries. But, information needs are specific needs, requiring specific methods.
The methods that focus on the identification of information needs originate in Information System and Information Science literature. They have developed strategies to overcome the difficulties for the users to express information needs. One such method is to involve the users in collaborative workshops (Barbosa Tavares, Hepworth, & De Souza Costa, 2011; De Tuya, Cook, Sutherland, & Luna-Reyes, 2017; Hogan et al., 2017; Staron, Meding, & Baniasad, 2019). Barbosa Tavares et al. (2011) used techniques including scene setting, brainstorming, cards for people to express ideas, individual and group work, and discussions as they are efficient to help people identify their information needs. In fact, they combine different strategies to enable the participants to express their needs and constraint them to follow a structured mental framework to unravel the unexpressed or misguided ones. Given that information needs are secondary needs and are identified when people experience a knowledge gap (Belkin & Vickery, 1985), the users need to be guided in an information seek workflow (Hörold, Mayas, & Krömker, 2012; Timmerman, Boer, Hisschemöller, & Mulder, 2001). Scenarios (Carrol, 1999), or vignettes (short scenarios) (Urquhart, 2001), and the critical incident technique (Urquhart, 2001) can be used to create a situation where participants experience knowledge gaps. They can help the user to project himself in a clear context, role, and tasks. In term of group dynamics in workshops that involve several stakeholders, to enhance the active participation of everyone, Staron et al. (2019) used the brainwriting and similarly, Hogan et al. (2017) used the idea writing technique of Warfield (1994).
Information needs are complex to capture as they are temporary, personal, contextual, and sometimes misguided or unexpressed. To tackle such a complexity, Devadason and Lingam (1997) have developed a set of methods, which are extensive and time-consuming by nature. They argue that no single method can identify information needs. They use their methods to study users (role, tasks, behaviors by direct involvement and indirect user studies, observation), their environment (available channels, tools, organization by case study, desk research) and information use (which need, information form, frequency of need, by interviews, surveys). Methods to capture information needs can have different focuses (Westbrook, 1993): (1) understanding the information needs and the problem (knowledge gap and its reasons (e.g., Staron et al., 2019)), (2) understanding the information self (what the user is looking for performing specific tasks, its form (e.g., Hess, Diebold, & Seyff, 2017)) and (3) designing the interface between the two (the solution, information system, its functionalities, that will deliver the information (e.g., Paulus, Meesters, & Van de Walle, 2018)).
Evidently, methods developed in innovation and service design for the capture of users’ needs can be applied, to some extent, to capture information needs (e.g., Al-Shboul & Abrizah, 2014). Information needs are difficult to express in words (Shenton, 2007), just as the so-called latent needs of users (Sanders, 2002; Sanders & Stappers, 2008). As in service design, the fulfillment of information needs is embedded in the development of solutions. In service design, co-design and empathetic design, two approaches to participatory design, are particularly relevant to the capture of unexpressed or misguided information needs. They involve users to imagine or envision a future practice or product, and to seek inspiration together with the designers (Steen, Kuijt-Evers, & Klok, 2007). In participatory design (Abras, Maloney-Krichmar, & Preece, 2004), users are partners with the intermediaries, their involvement is considered as central in the process. Therefore, a special effort is put to the development of tools and techniques to enable the expression of the participants, and the communication between the users, experts in their usage experience, and the intermediaries, experts in their own field and technologies (Sanders, 2002; Sanders & Stappers, 2008). The generative tools guide participants in small steps to construct and express deeper levels of knowledge about their experiences, in order to get to know tacit knowledge and latent needs, often difficult to express in words (Visser, Stappers, Van der Lugt, & Sanders, 2005). The activities are intended to engage users in the group cognition, leading to the emergence of new ideas, concepts and solutions (Pallot, Trousse, Senach, & Scapin, 2010; Sanders, 2002; Sanders & Stappers, 2008).
Previous OGD research that study methods to capture information needs are scarce, and have focused on how OGD can empower citizens (Barbosa Tavares et al., 2011), how to engage intermediaries to create value (De Tuya et al., 2017; Hogan et al., 2017), or how to develop an OGD solution that solves a specific issue (Paulus et al., 2018). The users’ information needs are to elicit to reuse OGD, but the researchers’ aim was not the development of methods or guidelines that tackle this challenge. In conclusion, several schools of thought have explored and developed methods to capture needs difficult to express in words (information needs and latent needs), but to our knowledge, no method has been developed to capture the information needs of users for OGD. This paper aims at developing design principles as a first step towards filling this gap.
Design Science Research (DSR) is an approach to research which output, the artifact, aims to contribute to the knowledge base (rigor and scientific legitimacy) and be useful in its environment (relevance) (Dresch, Lacerda, & Antunes, 2015). In other words, DSR develops artifacts that should achieve a goal and solve a problem, which are presented as the result of the research. The artifact is designed and developed based on problems, motivations, and objectives identified by researchers in practice and previous research. This artifact is then demonstrated and evaluated in practical contexts to observe how well it supports a solution to the problem (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). Artifacts can be of four types or classes: classified constructs, models, methods, or instantiations (March & Smith, 1995). Given the practical orientation of the artifact, the contribution to knowledge can be either more limited and specific (e.g., one instantiation), or more abstract (e.g., design theories) (Gregor & Hevner, 2013). Design principles belong to the latter and are considered as prescriptive knowledge. A design principle is a statement that prescribes what artifact and how to build it in order to achieve a predefined objective (Chandra et al., 2015). Its purpose is to capture knowledge about the creation of other instances of artifacts belonging to the same class (Sein, Henfridsson, Purao, Rossi, & Lindgren, 2011). It moves the knowledge contribution from the specific case to the generalization. Design principles should be evaluated based on their reusability and utility in their environment (Iivari, Hansen, & Haj-Bolouri, 2020).
An effectively formulated design principle contains three kinds of information: (1) information about the actions made possible through the use of an artifact, (2) information about the material properties making that action possible, and (3) the boundary conditions under which the design will work (Chandra et al., 2015). Gregor, Kruse, and Seidel (2020) add three components that should make a design principle understandable and useful in real-world design contexts: the specification of the actors (implementer vs. user of the artifact), the aim of the user (its problem to be solved) and the rationale (why a mechanism is recommended).
In the end, the conceptual schema of a design principle by Gregor et al. (2020) is made of seven components: the aim, the implementer, the context, the users, the mechanisms, the enactor, and the rationale. The aim is what is to be achieved. The implementer is the actor that instantiates abstract specifications in a concrete design context. The users are those whose aims are to be achieved. The mechanisms are human actions, the use of artifact, or the combination of both, that lead or allow users to achieve an aim. The mechanisms may require the help of enactors, who perform actions as part of the mechanisms that are used to accomplish the aim. The rationale is a justification for believing that the mechanisms will lead to achieving the aim. Gregor et al. (2020) suggest the following formulation for design principles: “For the implementer, to reach the aim of the user, in a context, use the mechanisms, because of the rationale”.
Despite design principles having their roots in the Information System literature to develop socio-technical artifacts (e.g., tools, computer systems, interfaces), they have also been used to prescribe co-creation methods (e.g., Kohler, Fueller, Matzler, Stieger, & Füller, 2011).
The development of the design principles followed the design science research (DSR) methodology and the steps suggested by Peffers et al. (2007): (1) problem identification and motivation, (2) definition of the objectives for a solution, (3) design and development, (4) demonstration, (5) evaluation, and (6) communication. We iterated between the steps, and the result is a set of design principles.
The design principles have been communicated to practitioners in a popular science article (description of the design principles with examples of implementation based on the instantiations, addressed to practitioners without prior knowledge and accessible at http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-181126).
The problem identification for this study is that the data providers and intermediaries’ low awareness of the users’ information needs, adding to the problems of which OGD to publish and what solutions to develop. Previous attempts in the area, such as hackathons, are rarely driven by the information needs of the users, more built on existing datasets, technology, and the experiences of the intermediaries (Carr & Lassiter, 2017; Johnson & Robinson, 2014; Juell-Skielse et al., 2014). In reality, this situation means that the user’s information needs are most likely not met and that the use of OGD continues to be low. We understand that the three actors are different and separated by cultural values, possibly geography, and goals, but there is also a focus on data and technology-driven solutions over actual needs. There is a misalignment between the three parts: providers’ data, intermediaries’ solutions, and users’ needs (Belkin & Vickery, 1985; Davies, 2010; Wilson, 1981).
At the same time, we know that information needs emerge when people try to satisfy primary needs and experience a knowledge gap (Wilson, 1981), but they can be dormant, unrecognized, or unexpressed (Erdelez, 1999; Nicholas & Herman, 2010). They are personal, contextual (Wilson, 1981), and can be difficult to put into words (Westbrook, 1993). Situations that are unclear, confusing, or difficult to comprehend, can make it impossible for the users to express their information needs (Westbrook, 1993). Moreover, wants, demands, and needs for information are not the same and only the last is valuable (Nicholas & Herman, 2010).
The objectives of the artifact, a set of design principles, are: (1) to enable data providers, intermediaries, and users to develop and implement an efficient method that allows a defined group of users to express their information needs and (2) to enable them to reuse the output of the method to inform data providers and inspire intermediaries. The output should provide insights to the intermediaries to develop desirable information solutions for users and support data providers to identify datasets in relation to information needs. The design principles should be actionable methodological guidelines that help the actors to realign the OGD release and reuse on the information needs of users, in a given context, for relevant issues and user groups. They should contribute to the knowledge base by addressing the lack of impact of OGD with an original approach: the integration of users information needs in methodological design principles.
The next steps, design and development (3), demonstration (4), and evaluation (5) were developed in three successive iteration cycles, as summarized in Figure 1.
An instantiation is an implementation of the mechanisms, or methodological principles, developed based on our current knowledge. An artifact is the result: the abstraction of the instantiation (design principles addressed to a specific audience). Each iteration cycle was based on lessons learned from previous research, evaluation of the previous instantiations and artifact, and the problem domains of OGD and information systems.
The first iteration initiated with previous research. Collaborative workshops are more efficient than interviews to enable people to explore and identify their information needs as members of group or community (e.g., Barbosa Tavares et al., 2011). Therefore, we started by developing and implementing a preliminary version of the design principles (artifact V.1.) into a face-to-face workshop with users. The workshop was based on the idea of critical incident technique where participants, as citizens, were asked to reexamine a brief, but memorable information seeking episode in a given situation, called vignettes (e.g., socialize or find food in a new city) (Urquhart, 2001). The participants had to describe their possible actions in those circumstances, identify related information needs, existing solutions and their issues, following the intellectual process of Hörold et al. (2012). In the iteration 2, we refined the question scheme and wording for a better understanding of the participants and decided to define categories of settings or situations of information seeking behaviors to avoid dispersion of the participants that resulted in a limited reusability of the insights for a specific intermediary. In the last iteration (3), we improved the relevance and scalability of the design principles that were challenged in the evaluation of the artifact (V.2.). We instantiated the design principles in three digital workshops, with participants in a new role, future data providers, and adapted the activities as a new interpretation of the design principles. The participants were also invited to execute short sensitizing activities designed to raise their awareness of the topic and concepts (Visser et al., 2005).
We ran two demonstrations in face-to-face workshops with users, and three demonstrations in online workshops with data providers. For the first face-to-face workshops in iteration 1, we invited nine Belgian researchers as users and citizens of the city they work in (see Appendix A), and for the second, in iteration 2, eleven Belgian students as they belonged to a homogeneous group of users and citizens and perceived to share similar needs regarding information related to the city they study in (see Appendix C). All participants were volunteers and responded to an open call for participation. Both workshops lasted 1:30 hours.
We ran three demonstrations with future data providers in iteration 3, as initial session of a training program about OGD. The three digital workshops lasted 2:00 hours each, and involved respectively eight, fifteen, and seven civil servants of Belgian municipalities, association of municipalities, or regional administration (see Appendix F). The online format was a constraint by the governmental measures to control the COVID pandemic, but considered as an opportunity to test the scalability of the design principles in different settings.
In DSR, evaluation can be performed in many different ways, as it depends on the type of artifact and problem (Hevner, March, Park, & Ram, 2004; Peffers et al., 2007). Nevertheless, this step should ensure that the artifact is relevant and useful (Gregor & Hevner, 2013). We evaluated on two levels: the instantiation (implementation of the design principles as workshops) and the artifact (set of design principles, as abstraction of the instantiations). On the instantiation level, we evaluated whether the mechanisms reached the predefined objectives (Chandra et al., 2015; Peffers et al., 2007). For that purpose, we evaluated the smooth execution of the principles with observation (Hevner et al., 2004), analyzed the quality of the output produced by the participants, collected participants’ feedback (Peffers et al., 2007) (see Appendix B, Appendix D, and Appendix G), and finally, provided informed arguments based on previous knowledge (Hevner et al., 2004). On the artifact level, we evaluated the relevance and usefulness of the design principles for the audience. We reported the design principles as a popular science article and submitted it to potential users of the artifact. The first version of the design principles, a method under development, was evaluated in three semi-structured interviews with four representatives of future providers (two small municipalities) and three members of a team of experienced intermediaries (a Belgian digital start-up that developed an OGD based application for the group of users whose information needs were investigated in iteration 1 and 2) (see Appendix E). As the design principles were reaching their final form in the artifact V.3., as presented in this paper and the current popular science article, they were evaluated based on the design principles evaluation framework of Iivari et al. (2020). The popular science article, together with a questionnaire (online form) was sent to six experienced data providers, as a way to complement the feedback got from less experienced data providers (respondents of the first evaluation and participants of the iteration 3). They were offered to either fill the form at their own pace or suggest us a time at their convenience for an interview, based on the given questionnaire. Two respondents accepted the interview (the OGD managers of a large municipality in Belgium and a public traffic agency in Sweden), and one filled the questionnaire (a regional coordinator of OGD in Sweden) (see Appendix H).
This section presents the final design principles, a demonstration of the third iteration using these principles, and, finally, the evaluation of the principles.
We followed Gregor et al. (2020) for the structure of the final set of design principles (DP), which all share the same aim, actors and context, as presented in the first box. The following boxes present nine mechanisms and their rationales.
Set of Design Principles: Aim Actors and Context
For the data providers, intermediaries, or users (implementers) to capture the users’ information needs (user and aim) in an everyday, information-rich context, we recommend.
Design Principles 1: General principle
Mechanism: To employ a mix of data collection methods, activities, and tools to better grasp the complexity of information needs, involving users, but with little impact on them.
Rationale: Information needs are personal, context-related, and role-based: the involvement of the users is essential to visualize these needs. But, they are also complex and time-consuming to identify (Devadason & Lingam, 1997; Nicholas & Herman, 2010). We sought a balance between complexity and time invested (efficiency) in the implementation of the method and the interactions with all the stakeholders (especially, the users).
Design Principles 2: Main data collection method (1/2)
Mechanism: To employ a collective intelligence-based workshop as a key data collection method, using creative tools and activities (i.e., tools that encourage the ideation with the use of text, pictures, to present ideas) and idea-writing techniques (i.e., written structured brainstorming, that uses variation of work dynamics, personal and in small groups, with a final presentation of the ideas in a plenary session), with the guidance of a facilitator.
Rationale: Information needs are difficult to express spontaneously and in words (e.g., people have dormant information needs, unexpressed needs, misguided needs (Shenton, 2007). Collective workshop has been proven to be an efficient technique to identify people’s information needs (Barbosa Tavares et al., 2011; De Tuya et al., 2017; Hogan et al., 2017; Staron et al., 2019). Creative tools, e.g., generative tools, can help participants to express latent needs, thoughts and feelings (Sanders, 2002; Sanders & Stappers, 2008), and idea writing techniques ensure input from a diverse range of people and temperaments (Staron et al., 2019; Warfield, 1994).
Design Principles 3: Main data collection method (2/2)
Mechanism: To use preparatory material and activities for the participants that aim at developing their understanding of the information needs concept and issues of the context.
Rationale: An appropriate preparation can raise the awareness and understanding of the participants about the abstract concepts and questions used in the workshop, stimulate their memory about the plots given in the vignettes (critical incident)(Urquhart, 2001), improve the quality of their production during the session (Visser et al, 2005), and, finally, increase the social interactions, their enjoyment and sense of efficacy, as they are known as important motivational factors in innovation communities (Antikainen, Mäkipää, & Ahonen, 2010; Roberts, Hughes, & Kertbo, 2014).
Design Principles 4: Adaptation to the aim actors and context (1/3)
Mechanism: To employ short scenarios (vignettes) to focus the participants on clearly identified roles and contexts and the plots (critical incident technique) to trigger a knowledge gap (a brief but memorable information seeking episode based on their own experience but prompted by the problematic situation). The participants-short scenario fit implies a thoughtful recruitment.
Rationale: Information needs are context-related, personal, and appear when people realize a lack of knowledge (Al-Shboul & Abrizah, 2014; Belkin & Vickery, 1985; Nicholas & Herman, 2010), which can be prompted by the critical incident technique (Urquhart, 2001).
Design Principles 5: Adaptation to the aim, actors, and context (2/3)
Mechanism: To suggest predefined categories of information needs in regard with the roles and context clearly circumscribed.
Rationale: Information needs can be broad and poorly pre-selected information needs can lead to irrelevant information solutions (Warner Murray & Palmour 1973).
Design Principles 6: Adaptation to the aim, actors, and context (3/3)
Mechanism: To suit the vignettes with a relevant issue for the three actors (data providers, intermediaries, and users) and link them to the OGD already or likely available.
Rationale: The interest of the data provider is needed to release OGD the motivation. Knowledge of the intermediary is needed to invest time, skills, and resources in developing solutions. A fit with the need of the users is needed to increase the relevance, acceptance, and use of the solution. This also helps to identify actual needs.
Design Principles 7: Intellectual process
Mechanism: To use a sequential question scheme that guides the participants in the intellectual process of information needs exploration. It starts with the identification of primary needs in a given context and role, and the actions or tasks required to fulfill that need. It continues with identification of the information needs behind the tasks (by listing the needed and missed information and the weaknesses of existing solutions). It wraps up with ideas for new solutions.
Rationale: Information needs are secondary needs and can be identified by tracking first the tasks, then information, and finally categories of data (Hörold et al., 2012). People often need time to reflect and process information needs to express them properly. The use of activity, existing solutions, and future solutions help to give a frame to the need and make it more expressible.
Design Principles 8: Purpose of the result (1/2)
Mechanism: To ensure that the method results in the identification of information needs of groups of users.
Rationale: The development of information solutions should satisfy the needs for more than one person to be sustainable and economically viable (Nicholas & Herman, 2010; Westbrook, 1993; Wilson, 1994). It is important to study the result for similarities and quirks, but also for needs, wants, and demands (Nicholas & Herman, 2010). Results that are fragmented often indicate that the participants lacked experience with the information needed, as such it is important to ensure that the participants have a good understanding of the target group and discuss with them, or represent them.
Design Principles 9: Purpose of the result (2/2)
Mechanism: To ensure that the material produced by the method can be used by data providers for dataset prioritization and by intermediaries for inspiration to OGD innovation, considering the alternative information solutions that exist.
Rationale: Data providers tend to follow a supply logic (Susha, Grönlund, & Janssen, 2015), intermediaries tend to develop solutions with little impact (Carr & Lassiter, 2017; Hjalmarsson, Johannesson, Jüll-Skielse, & Rudmark, 2014), and the information needs are embedded in an environment (stakeholders, channels, habits in information use, other partial solutions available) that is important to evaluate when designing new information solutions (Devadason & Lingam, 1997; Hess et al., 2017; Staron et al., 2019; Timmerman et al., 2001).
For clarity purposes, Table 1 shows the way we combined the design principles in a mixed method, conducted with data providers in webinars (last instantiations). We summarize the objectives, activities, deliverables, and related design principles (DP + n°). The first DP was applied by developing a mixed method divided into three phases: (1) the preparation, (2) the collaborative workshop, and (3) complementary analysis and relevance check to complement the insights. The following DP were spread across the parts.
|Phase 1. Preparation||1. Define a context and issue (DP 5) based on previous knowledge (experience, strategic priorities, literature)
2. Define the profile, roles (DP 8), and categories of situations creating information needs (DP 5) of the users based on previous knowledge
3. Preparing the participants for the workshop (awareness of the topic and concepts) (DP 3, 6)
|1–2. Kit of preparatory activities with clear instructions for the participants to define a problem (trigger questions), the context (mapping exercise), and user group (persona)
3. Previous to the workshop, a short presentation of the concepts (information needs, sources, solutions) given by the implementer
|1. A precise context, problem and group of users with categories of situations creating information needs
2. A kit of preparatory activities for the participants
|Phase 2. Collaborative workshop (DP 2)||1. Enable the participants to identify progressively user’ information needs, existing solutions and weaknesses, and possible imagined solutions (DP 7) in a given role and context (vignettes) (DP 4)||1. Problem situation brought by the participants converted into vignettes, used as a start for a written brainstorming. Use of the participants’ field experience to limit categories of situations of information needs given the context. Brainstorming tool: question scheme written on a canvas, following the track: primary needs in the situation, related user tasks, related information needs, existing solutions and their issues.||1. New insights:
– Information needs (spontaneous and latent)
– Overview of the competing solutions
– Imagined solutions
|Phase 3. Analysis + Relevance check||1. Confirm the ideas and intuitions (imagined solutions) with users and knowledgeable experts (DP 8)
2. Connect the collected information needs with released or releasable OGD (DP 9)
3. Inspire possible developments and OGD intermediaries (DP 9)
|1. Examples of activities and task given to the participants to compare the brainstormed information needs with their user group and colleagues (question scheme in a workshop, interview, or focus group format)
2. Comparison of the categories of data with data owned in their administration
3. Presentation of the projects to developers (intermediaries) in a post-event
|1. A project
2. The identification of information needs and related OGD
3. The identification of future development avenues
Both face-to-face and digital workshops followed the same structure and objectives, involved the users at a moment, and resulted in the identification of information needs. However, the practical activities and tools were scaled to the participants. Given the focus of this paper, on the design principles as abstractions of the instantiations, the readers who want more extensive examples of tools and presentations of the instantiations can find them in the popular science article (accessible at http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-181126).
The evaluation focused on whether the objectives were achieved by the design principles, and the perceived relevance and usefulness of the artifact by the audience. The main insights of both are presented subsequently and are based on empirical material and previous literature (informed argument, Hevner et al. (2004)).
Objective (1) was to enable data providers, intermediaries, and users to develop and implement an efficient method that allows a defined group of users to express their information needs. To achieve that objective and overcome the intellectual challenge of information needs identification, the iterations and evaluations revealed the importance of (1) the group, (2) the preparation of the participants, and (3) the guidance in the intellectual process.
The group has proven its worth for two reasons. First, in the search for efficiency. An information seeking behavior is only one manifestation of a complex situation, influenced by individual’s roles, tasks, the environment, existing solutions. Exhaustive but time-consuming design principles (Devadason & Lingam, 1997) are likely to be rich in results but irrelevant to their audience. Hence, as a trade-off, we developed the three-phase method (DP 1), with a collaborative workshop as the main method (DP 2), since Barbosa Tavares et al. (2011); De Tuya et al. (2017); Hogan et al. (2017); Staron et al. (2019) have proven that collective intelligence improves the individual understanding of one’s information needs. The group support helped the participants to understand the concepts, questions, and generate more ideas. Second, the group contributed to the participants’ good experience. The data providers found a community of practice where to share field experience and ideas. The users had a good time. Roberts et al. (2014) found out that in value co-creation with community of users, the value of participation, for the participants, is in the process, not the output, and in the social interactions of like-minded people. The fun, enjoyment, and reward for participation are important motivational factors in innovation communities (Antikainen et al. 2010). This type of motivation was noticeable in the user workshop. In comparison with the face-to-face workshops, the online workshops brought difficulties in the work dynamics which can impact the efficiency and participants’ experience. More time was required to break the ice and engage the participants, simple participation rules were not always followed, and the first technical difficulties (access to the shared documents, broken links, dysfunctional microphone) can break the participants’ motivation and engagement down very quickly and for long, due to the enhanced difficulty to use the tools and the distance between the participants and implementer.
A need for a good preparation (DP 3) was also perceived from the first iteration of the users workshops. The participants tended to mix up the question (information need) with the answer (sources and solutions). For the users, we developed a sensitization kit inspired by Visser et al. (2005), and data providers were involved in narrowing down the problem, users profiles, roles (DP 5 and 8), and mapping their environment (Devadason & Lingam, 1997). As argued by Mulder and Stappers (2009), both time for reflection and a tool of expression are needed to help the participants become “experts of their experience”, which increases the breadth and depth of the results of participatory activities. The activities are designed to be an invitation to reflection, without conditioning the participants, and raise their awareness about habits and environment. The preparatory activities were tested on the data providers in the instantiation 3. Yet, they were first received by the participants as an extra and unnecessary workload, until they felt confused at the workshop with concepts they are not used to think about, and acknowledged their value in the post evaluation. In theory, the sensitization kits and similar are promising (Visser et al., 2005). In practice, the implementer has limited control over the engagement of the participants in off-site activities. To move a necessary preparation out of the facilitated workshop, to reduce the impact of the method on the participants’ agenda, still comes with the risk of reducing the participants’ awareness, their feeling of self-efficacy, and the quality of the insights produced. Self-efficacy is an important motivational factor for participants to join a workshop: they have to feel that they can contribute (Roberts et al., 2014). But, they might not foreseen the value of such activities and a lack of awareness can influence their motivation and production (Roberts et al., 2014).
The guidance in the intellectual process (DP 7) (Hörold et al., 2012) was ensured, first, by the presence of a skilled facilitator (method implementer), highly important to keep the focus of the participants and challenge their preconceptions, otherwise they can lose the track of their role and produce irrelevant output. Second, it was enabled by the use of the critical incident technique (Urquhart, 2001) that helped the participants to remember life episodes, and the wording of the questions was good enough to make the tools (brainstorming cards or canvas) self-sufficient. It increased substantially the quantity and reusability of the insights produced by the participants. We gained insights into how participants can be helped to express the hologram of information need and demand for an information solution (Nicholas & Herman, 2010; Westbrook, 1993).
Objective (2) was to enable the data providers, intermediaries, and users to reuse the output of the method to inform data providers and inspire intermediaries. The output should provide insights to the intermediaries to develop desirable information solutions for users and support data providers to identify datasets in relation to information needs. The guidance in the intellectual process allowed us to perceive (1) degrees of importance in the information needs expressed but also (2) a difficulty for the provider to come closer to the users.
We noticed degrees of importance. The participants start with, what we call, their top-of-mind information needs, for a role, in a context, given the existing solutions (“What information do I need for X task, that I already get from Y solution”). Then through the identification of issues with existing solutions, they can express a deeper level of needs, the latent and not spontaneously expressed needs (frustration and missing features help them to find out the must-have, what information they value most). The creative part, their imagined solutions, allowed them to express their wishes, the “nice-to-have” (what they would value and is not yet satisfied with present solutions, the difference between information needs and wants (Nicholas & Herman, 2010)). It can help the intermediary to identify potential paths of developments.
However, we also noticed the difficulty of getting the data providers and users closer, which is later reaffirmed by the evaluation of the design principles. Complementary methods were implemented by the researchers to check the relevance of the output of the user workshops (DP 8, 9). The same was suggested to the future data providers, participants of the last instantiation and users of their insights, as they were working on personal projects. They were suggested to meet users, use the participatory mechanisms and communication channels of their municipality to challenge their ideas with the target user groups. None of them did it. We identified several reasons for that reluctance: lack of time of the participants, lack of perceived relevance and unfamiliar with user engagement in the development of solutions, fear that citizen participation commits the municipality to deliver the solution, other projects and topics of citizen participation in the strategic plan, rigidity of the administration and its processes, exceptional circumstances (COVID and confinement). In conclusion, for the implementation of the DP 8 and 9, the participants needed more arguments and guidance (ideas of methods and tools relevant to their experience and context) to do it on their own, which opens for future research.
Following the evaluation framework of Iivari et al. (2020), the design principles were perceived by the experienced providers (OGD managers) as accessible (understandable with the provided examples and tools), important as they address a real problem in their practice (lack of reuse and understanding of users’ needs), novel in their approach and useful. However, they perceived their actionability differently. The first respondent challenged it, as it is usually difficult to get in contact with the right kind of users and stakeholders. The lack of budget, resources, and time to organize the implementation of the design principles and process their insights places the exploration of user’s information needs as a secondary task, which maintains the supply logic that current and previous research try to tackle (Susha et al., 2015). The second respondent felt ready to try the design principles and was only concerned about the difficulty to recruit the right participants, the strategies to adopt to motivate people for such activities (this corroborates the DP 4), and the adaptability of the questions to their level of expertise (field knowledge, technical, or user expertise). The last respondent solved it by thinking about trying them out with internal intermediaries: colleagues of his administration or other public services that are part of the data-sharing ecosystem. A group of professional intermediaries share a clear context, role and tasks, and is known and accessible by the data provider, which eases the preparation phase. This hesitation to implement the design principles was not noticeable when we presented the design principles in an early development stage to the start-up, as they were used to UX methods and user engagement in the development of their digital services.
The users’ insights were, according to the intermediaries (the experienced start-up), bringing up interesting perspectives. Questioning a problem by the information needs prevents the intermediary to fall into confirmation bias by validating their own ideas at an early stage of development instead of exploring user’s problems. Therefore, the implementation of the design principles revealed unexpected users’ information needs they never thought about before, which can generate new ideas. However, the risk is that the users stay too close to what they know and their individual needs (user-centred methods are known to generate incremental improvements (Verganti, 2008)), wish unrealistic solutions, or go beyond the core business of the intermediary. During the ideation of solutions in the workshops, both future data providers and users were sometimes going in every directions, coming up with solutions that would solve everything at once. They lacked realism in data availability (users wanted solutions based on privately owned and generated data, and data providers dreamt about Waze-like real-time traffic data produced and shared by the citizens). The data providers in the digital workshops were lacking knowledge regarding the data availability in their organization, the technical possibilities, and skipped the involvement of the users, focusing on their organizational priorities and projections. Therefore, for the data providers, participants of the digital workshop, the insights were useful to frame a project to address to intermediaries and to challenge their understanding of OGD. The impact of these insights on the data release remains limited, likely due to the context of the participants (civil servants with limited power of initiative).
The design principles presented in this paper originally combine previous research and approaches to capture information needs. The limited research about information needs within OGD research focuses on identifying needs in a specific context by researchers (e.g., Barbosa Tavares et al., 2011; De Tuya et al., 2017; Hogan et al., 2017; Paulus et al., 2018). This approach limits the generalizability, as information needs are determined by social roles, environments, previous knowledge, and experiences (Wilson, 1981). Our research overcomes this limitation by developing design principles that can be used by other researchers and practitioners to capture information needs. Similar to previous research, our design principles are based on collaborative workshops (e.g., Barbosa Tavares et al., 2011; De Tuya et al., 2017; Hogan et al., 2017; Staron et al., 2019). This approach fueled the creativity of the participants (Pallot et al., 2010; Sanders, 2002; Sanders & Stappers, 2008) and helped to overcome problems relating to expressing information needs (Westbrook, 1993) since the participants could discuss their need with peers (Kuhlthau, 1991). On the other hand, the design principles originally combine approaches to capture information needs with approaches from service design. In the information needs approaches, Barbosa Tavares et al. (2011) explain that it is important to unfold the information needs of participants following a structured mental framework (e.g., Hörold et al., 2012; Timmerman et al., 2001). Urquhart (2001) explains that critical incident technique helps participants remember information needs previously encountered. Westbrook (1993) explains that the approaches can focus on understanding (1) the need and the problem, (2) the information, and (3) the solution. In service design, Steen et al. (2007) explain that participants can imagine or envision a future product, which requires tools to help them express themselves (Sanders, 2002; Sanders & Stappers, 2008). The tools guide them in small steps to construct and express their experiences, tacit knowledge, and latent needs that are often difficult to express in words (Visser et al., 2005). Consequently, our design principles cover the three categories of Westbrook (1993) and their future possibilities. The combination of collaborative workshop, structured unfolding, and creative tools allows participants to identify present information needs, but also future and possible avenues of innovation. This extension is original for OGD research, as it takes a step away from the supply logic of OGD (Susha et al., 2015) towards an enrichment logic where OGD can supplement existing information solutions or find routes to surpass them. At the same time, it moves the responsibility of identifying information needs from researchers to practitioners where it can make the most value and impact.
The conceptual schema of Gregor et al. (2020) to write understandable and useful design principles helped us to frame our thoughts. Nevertheless, for the practitioners, the design principles became understandable with examples and illustrations. Stand-alone design principles might not be sufficient to be practically applicable, despite the coverage of the suggested conceptual schema.
The set of design principles is not either a recipe for success, but should be seen as a method that can sparks new ideas and practices in the intermediaries and data providers’ jobs. It should be continued with methods that value the involvement of the three actors: users, intermediaries, and data providers. Each actor has a limited perception of each other’s needs, objectives, and technical possibilities, that have to be mediated. Verganti (2008) suggests the role of the interpreter, who can help to connect user needs with the priorities and capabilities of the firms in the technological innovation process. The intermediary could be an interpreter to close the gap between a data provider’s fuzzy objectives and the user’s needs, just as the data provider can be interpreter between the intermediary’s market oriented solutions and the public needs, to move towards an OGD reuse that benefits to a broader group of users.
The objective of producing a maximum of insights with a low impact on the users implies limitations. The design principles could be developed to capture the information needs with the same scientific rigor as Devadason and Lingam (1997). However, we propose a dense set of methodological recommendations to approach information needs in the context of OGD and to get insightful results at short term. Yet, it requires considerable time and reflection for the implementer to recruit people, design, and run the required activities, and process the output, which could make the implementer to be selective or expeditious and lower the quality of the insights. Future work could enrich and develop design principles for the preparation phase, especially regarding the questions of user’s profile and recruitment based on their knowledge and role in context, and user involvement from a data provider’s perspective, to maximize the collective intelligence and the scalability of the insights. Finally, the design principles could be implemented in different contexts (e.g., administrations, companies) to develop further new principles regarding the integration of the information needs in the data provider’s and intermediary’s work and processes.
For practice, starting with the users’ information needs can open a broader range of solutions and potential paths of OGD reuses. For the data providers, it can instill a better understanding of the OGD potential. The insights can also inspire the intermediaries with new reuses of diverse formats, based on users problems and tasks. The investigation of information needs can enable the user’s involvement in the early stage of solutions development. A collaborative workshop enriches the method. However, a digital setting can impede the realization of collective intelligence benefits.
For research, the conceptual schema of Gregor et al. (2020) and evaluation framework of Iivari et al. (2020) offered a relevant structure to write and evaluate design principles. Moreover, more research of OGD from an information needs perspective is needed. It could allow the users to have a more active and influential role in data release and reuse. It highlights its importance, calls the researchers to go beyond the exclusive data provider – intermediary interaction abundantly discussed in the literature, and opens new research avenues to improve the realisation of the OGD benefits.
OGD is a resource allowing many opportunities of reuses. Most of the methods known in practice and research to stimulate reuses and develop solutions are technology-driven and attract skilled intermediaries in open innovation contests. The concept of users’ information needs in OGD is scarce and previous methods to capture information needs, developed in that context, are covering other objectives (e.g., the citizen empowerment, intermediaries engagement, solution development). To reduce the gap between current open data solutions and user needs, in an inclusive and novel approach, we developed a set of design principles to enable people to develop tailor-sized methods to capture users’ information needs. By taking users information needs as a starting point, we enabled people without specific skills to express their needs with the support of a mixed method.
The contribution of this research is an artifact developed with DSR that can be used by intermediaries and data providers, to take a user-centred approach in their work. The design principles provide prescriptive knowledge to develop mixed methods and inform the implementer about the main data collection method, how to adapt to its purpose and context, the intellectual process to follow, and how to suit purpose of the results with the actor’s needs and objectives. Despite it is possible to implement the design principles in face-to-face and digital settings, the first is easier for the implementer and was experienced as more enjoyable by the participants. The group, the guidance in the intellectual process and a good preparation of the participants enabled them to produce fine-grained insights, with perceptible degrees of importance, in a limited time. The imagined solutions, final results of a collective identification of information needs, are contextualized with a role, tasks, and information needs. They can enrich the intermediaries and data providers’ perception of the user needs and inform them about what information and services users value most. However, we learned from the evaluation that off-site activities, such as sensitization kits and self-organized user engagement activities come with the risk to not be executed by the participants as their value is perceived afterwards. The design principles challenge especially how the data providers define the scope of their role and relation with the users, which opens interesting research avenues.
In this paper, the data are the developed design principles and tools made accessible in the popular science article (at http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-181126). The data to produce them (e.g., evaluations and participants’ written notes specific to each iteration) are less relevant for the readers and, therefore, unpublished.
The additional files for this article can be found as follows:
Design of iteration 1. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1Appendix B
Evaluation of iteration 1 at the instantiation level. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1Appendix C
Design of iteration 2. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1Appendix D
Evaluation of iteration 2 at the instantiation level. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1Appendix E
Evaluation of iteration 2 at the artifact level. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1Appendix F
Design of iteration 3. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1Appendix G
Evaluation of iteration 3 at the instantiation level. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1Appendix H
Evaluation of iteration 3 at the artifact level. DOI: https://doi.org/https://doi.org/10.5334/dsj-2021-037.s1
All the participants were informed of the research purpose and the use of collected empirical material. They consented to participate in the research activities.
This research is part of the Wal-e-cities project funded by the European Union (FEDER H2020) and Walloon Region (Belgium) and part of a Belgian-Swedish research collaboration surrounding the utility of OGD. We would like to thank also the anonymous reviewers for their relevant feedback received at the 17th Scandinavian Workshop on E-Government (SWEG 2020) and at the EGOV-CeDEM-ePart 2020 conference (submitted as ongoing research).
The authors have no competing interests to declare.
Al-Shboul, MK and Abrizah, A. 2014. Information needs: Developing personas of humanities scholars. The Journal of Academic Librarianship, 40(5): 500–509. DOI: https://doi.org/10.1016/j.acalib.2014.05.016
Antikainen, M, Mäkipää, M and Ahonen, M. 2010. Motivating and supporting collaboration in open innovation. European Journal of Innovation Management, 13(1): 100–119. DOI: https://doi.org/10.1108/14601061011013258
Barbosa Tavares, R, Hepworth, M and De Souza Costa, SM. 2011. Investigating citizens’ information needs through participative research: a pilot study in candangolândia, brazil. Information Development, 27(2): 125–138. DOI: https://doi.org/10.1177/0266666911401167
Berends, J, Carrara, W, Engbers, W and Vollers, H. 2017. Re-using open data: a study on companies transforming open data into economic & societal value. Retrieved from https://www.europeandataportal.eu/sites/default/files/re-usingopendata.pdf ((accessed 2019-11-22).
Carr, SJ and Lassiter, A. 2017. Big data, small apps: premises and products of the civic hackathon. In Seeing cities through big data, 543–559. Springer. DOI: https://doi.org/10.1007/978-3-319-40902-3
Carrol, JM. 1999. Five reasons for scenario-based design. In Proceedings of the 32nd annual hawaii international conference on systems sciences. 1999. hicss-32. abstracts and cd-rom of full papers, 11–pp. DOI: https://doi.org/10.1109/HICSS.1999.772890
Chandra, L, Seidel, S and Gregor, S. 2015. Prescriptive knowledge in is research: Conceptualizing design principles in terms of materiality, action, and boundary conditions. In 2015 48th hawaii international conference on system sciences, 4039–4048. DOI: https://doi.org/10.1109/HICSS.2015.485
Charalabidis, Y, Zuiderwijk, A, Alexopoulos, C, Janssen, M, Höchtl, J and Ferro, E. 2018. The world of open data. Springer International Publishing. Retrieved from http://www.springer.com/gp/book/9783319908496. DOI: https://doi.org/10.1007/978-3-319-90850-2
Daft, RL and Lengel, RH. 1986. Organizational information requirements, media richness and structural design. Management science, 32(5): 554–571. DOI: https://doi.org/10.1287/mnsc.32.5.554
Davies, T. 2010. Open data, democracy and public sector reform. A look at open government data use from data.gov.uk, 1–47.
De Tuya, M, Cook, M, Sutherland, MK and Luna-Reyes, LF. 2017. Information requirements to create public value: sharing and opening data to address urban blight. Transforming Government: People, Process and Policy. DOI: https://doi.org/10.1108/TG-12-2015-0054
Devadason, FJ and Lingam, PP. 1997. A methodology for the identification of information needs of users. IFLA journal, 23(1): 41–51. DOI: https://doi.org/10.1177/034003529702300109
Dresch, A, Lacerda, DP and Antunes, JAV. 2015. Design science research. In Design science research, 67–102. Springer. DOI: https://doi.org/10.1007/978-3-319-07374-3
Erdelez, S. 1999. Information encountering: It’s more than just bumping into information. Bulletin of the American Society for Information Science and Technology, 25(3): 26–29. DOI: https://doi.org/10.1002/bult.118
Goldkuhl, G and Sjöström, J. 2018. Design science in the field: practice design research. In International conference on design science research in information systems and technology, 67–81. DOI: https://doi.org/10.1007/978-3-319-91800-6
Gregor, S and Hevner, AR. 2013. Positioning and presenting design science research for maximum impact. MIS quarterly, 337–355. Retrieved from https://www.jstor.org/stable/43825912. DOI: https://doi.org/10.25300/MISQ/2013/37.2.01
Gregor, S, Kruse, LC and Seidel, S. 2020. Research perspectives: The anatomy of a design principle. Journal of the Association for Information Systems, 21(6). DOI: https://doi.org/10.17705/1jais.00649
Hellberg, A-S and Hedström, K. 2015. The story of the sixth myth of open data and open government. Transforming Government: People, Process and Policy, 9(1): 35–51. DOI: https://doi.org/10.1108/TG-04-2014-0013
Hess, A, Diebold, P and Seyff, N. 2017. Towards requirements communication and documentation guidelines for agile teams. In 2017 ieee 25th international requirements engineering conference workshops(rew), 415–418. DOI: https://doi.org/10.1109/REW.2017.64
Hevner, A, March, ST, Park, J and Ram, S. 2004. Design science in information systems research. MIS quarterly, 28(1): 75–105. DOI: https://doi.org/10.2307/25148625
Hjalmarsson, A, Johannesson, P, Jüll-Skielse, G and Rudmark, D. 2014. Beyond innovation contests: A framework of barriers to open innovation of digital services. In Ecis 14 22nd european conference on information systems, tel aviv, 5–13 june 2014.
Hogan, M, Ojo, A, Harney, O, Ruijer, E, Meijer, A, Andriessen, J, … others. 2017. Governance, transparency and the collaborative design of open data collaboration platforms: understanding barriers, options, and needs. In Government 3.0-next generation government technology infrastructure and services, 299–332. Springer. DOI: https://doi.org/10.1007/978-3-319-63743-3
Huang, R, Lai, T and Zhou, L. 2017. Proposing a framework of barriers to opening government data in china. Library Hi Tech, 35(3): 421–438. DOI: https://doi.org/10.1108/LHT-01-2017-0014
Hunnius, S and Krieger, B. 2014. The social shaping of open data through administrative processes. In Proceedings of the international symposium on open collaboration, 1–5. DOI: https://doi.org/10.1145/2641580.2641601
Iivari, J, Hansen, MRP and Haj-Bolouri, A. 2020. A proposal for minimum reusability evaluation of design principles. Information Systems. DOI: https://doi.org/10.1080/0960085X.2020.1793697
Janssen, M, Charalabidis, Y and Zuiderwijk, A. 2012. Benefits, adoption barriers and myths of open data and open government. Information systems management, 29(4): 258–268. DOI: https://doi.org/10.1080/10580530.2012.716740
Janssen, M and Zuiderwijk, A. 2014. Infomediary business models for connecting open data providers and users. Social Science Computer Review, 32(5): 694–711. DOI: https://doi.org/10.1177/0894439314525902
Johnson, P and Robinson, P. 2014. Civic hackathons: Innovation, procurement, or civic engagement? Review of policy research, 31(4): 349–357. DOI: https://doi.org/10.1111/ropr.12074
Juell-Skielse, G, Hjalmarsson, A, Johannesson, P and Rudmark, D. 2014. Is the public motivated to engage in open data innovation? In International conference on electronic government, 277–288. DOI: https://doi.org/10.1111/ropr.12074
Kohler, T, Fueller, J, Matzler, K, Stieger, D and Füller, J. 2011. Co-creation in virtual worlds: The design of the user experience. MIS quarterly, 773–788. DOI: https://doi.org/10.2307/23042808
Komninos, N, Pallot, M and Schaffers, H. 2013. Special issue on smart cities and the future internet in europe. Journal of the Knowledge Economy, 4(2): 119–134. DOI: https://doi.org/10.1007/s13132-012-0083-x
Kuhlthau, CC. 1991. Inside the search process: Information seeking from the user’s perspective. Journal of the American society for information science, 42(5): 361–371. DOI: https://doi.org/10.1002/(SICI)1097-4571(199106)42:5<361::AID-ASI6>3.0.CO;2-#
Kujala, S. 2003. User involvement: a review of the benefits and challenges. Behaviour & information technology, 22(1): 1–16. DOI: https://doi.org/10.1080/01449290301782
March, ST and Smith, GF. 1995. Design and natural science research on information technology. Decision support systems, 15(4): 251–266. DOI: https://doi.org/10.1016/0167-9236(94)00041-2
Mulder, I and Stappers, PJ. 2009. Co-creating in practice: results and challenges. In 2009 ieee international technology management conference(ice), 1–8. DOI: https://doi.org/10.1109/ITMC.2009.7461369
Nicholas, D and Herman, E. 2010. Assessing information needs in the age of the digital consumer. Routledge. DOI: https://doi.org/10.4324/9780203855799
Open Data Handbook. 2015. The open data handbook. Retrieved 2021-04-29, from http://opendatahandbook.org/guide/en/.
Pallot, M, Trousse, B, Senach, B and Scapin, D. 2010. Living lab research landscape: From user centred design and user experience towards user cocreation. In First european summer school “living labs”. Retrieved from https://hal.inria.fr/inria-00612632.
Peffers, K, Tuunanen, T, Rothenberger, MA and Chatterjee, S. 2007. A design science research methodology for information systems research. Journal of management information systems, 24(3): 45–77. DOI: https://doi.org/10.2753/MIS0742-1222240302
Portal, ED. 2019. What is open data? Retrieved from https://www.europeandataportal.eu/elearning/en/module1/#/id/co-01 (Accessed on 2019-08-14)
Roberts, D, Hughes, M and Kertbo, K. 2014. Exploring consumers’ motivations to engage in innovation through co-creation activities. European Journal of Marketing, 48(1/2): 147–169. DOI: https://doi.org/10.1108/EJM-12-2010-0637
Sanders, EB-N. 2002. From user-centered to participatory design approaches. In Design and the social sciences, 18–25. CRC Press. DOI: https://doi.org/10.1201/9780203301302-8
Sanders, EB-N and Stappers, PJ. 2008. Co-creation and the new landscapes of design. Co-design, 4(1): 5–18. DOI: https://doi.org/10.1080/15710880701875068
Sein, MK, Henfridsson, O, Purao, S, Rossi, M and Lindgren, R. 2011. Action design research. MIS quarterly, 37–56. DOI: https://doi.org/10.2307/23043488
Shenton, AK. 2007. Viewing information needs through a johari window. Reference Services Review. DOI: https://doi.org/10.1108/00907320710774337
Staron, M, Meding, W and Baniasad, P. 2019. Information needs for safe teams and release train management: A design science research study. In Iwsmmensura, 55–70. Retrieved from http://ceur-ws.org/Vol-2476/paper4.pdf.
Susha, I, Grönlund, Å and Janssen, M. 2015. Organizational measures to stimulate user engagement with open data. Transforming Government: People, Process and Policy. DOI: https://doi.org/10.1108/TG-05-2014-0016
Timmerman, JG, Boer, J, Hisschemöller, M and Mulder, W. 2001. Specifying information needs: improving the working methodology. Regional Environmental Change, 2(2): 77–84. DOI: https://doi.org/10.1007/s101130100030
Urquhart, C. 2001. Bridging information requirements and information needs assessment: do scenarios and vignettes provide a link. Information Research, 6(2): 6–2. Retrieved from http://InformationR.net/ir/6-2/paper102.html.
van Schalkwyk, F, Willmers, M and McNaughton, M. 2016. Viscous open data: The roles of intermediaries in an open data ecosystem. Information Technology for Development, 22(November), 68–83. DOI: https://doi.org/10.1080/02681102.2015.1081868
Verganti, R. 2008. Design, meanings, and radical innovation: A metamodel and a research agenda. Journal of product innovation management, 25(5): 436–456. DOI: https://doi.org/10.1111/j.1540-5885.2008.00313.x
Visser, FS, Stappers, PJ, Van der Lugt, R and Sanders, EB. 2005. Contextmapping: experiences from practice. CoDesign, 1(2): 119–149. DOI: https://doi.org/10.1080/15710880500135987
Weerakkody, V, Irani, Z, Kapoor, K, Sivarajah, U and Dwivedi, YK. 2017. Open data and its usability: an empirical view from the citizen’s perspective. Information Systems Frontiers, 19(2): 285–300. DOI: https://doi.org/10.1007/s10796-016-9679-1
Westbrook, L. 1993. User needs: a synthesis and analysis of current theories for the practitioner. RQ, 541–549. Retrieved from http://www.jstor.org/stable/25829371.
Wilson, TD. 1981. On user studies and information needs. Journal of documentation, 37(1): 3–15. DOI: https://doi.org/10.1108/eb026702
Zuiderwijk, A, Susha, I, Charalabidis, Y, Parycek, P and Janssen, M. 2015. Open data disclosure and use: Critical factors from a case study. In Cedem15: Conference for e-democracy and open government (Vol. 197).