top of page
BWi Web Application
Period
Jan 2023 - Jun 2023
Master's Thesis with Together Tech
2 members
My Role
User Research | Workshopping | Prototyping | User Testing
Project Overview
As a consultancy firm, Together Tech utilises a method called the Better World Index (BWi) to evaluate the sustainability impact of each assignment. Currently, this is done using a basic Excel Sheet, which is influenced by personal bias and has substantial room for improvement in terms of user experience.
This thesis aims to transform this process into a web application, using the existing Excel Sheet as a foundation to enhance its user-friendliness and scalability. The objective is also to research and devise a solution to reduce the impact of personal bias and format it in a way that is suitable for both internal and external users.
Design Process
Understand
Together Tech has devised a method known as BWi, rooted in the global goals for sustainable development. Before business managers at Together Tech accept an assignment, they assess its sustainability using this method within an Excel Sheet framework. The Excel Sheet is currently structured into three aspects: social, economic, and environmental. It employs a measurement scale, where a small box is filled in with -1 (negative impact), 0 (no impact), or 1 (positive impact). The final score is determined by calculating the mean score of these three aspects.
Domain Research
We began the project by researching the 17 SDGs and existing sustainability assessment tools to inform our design approach. Our exploration revealed gaps in current methods, which often rely on quantitative metrics and offer post-project, reactive evaluations. We also studied potential biases in questionnaires.
17 SDG
A set of global goals established by the UN to address various social, economic, and environmental challenges and promote sustainable development worldwide.
Response bias
Response bias is a well-known issue in survey research that affect the validity, reliability, and data analysis in questionnaire results.
User Interview
We opted to engage in discussions with stakeholders, encompassing both the creators and users of the existing BWi Excel sheet. The aim was to gain insights into the creation process and collect feedback on user experiences related to the BWi Excel sheet.
9 Participants X 40 min
4 BWi Creators
-
What was the creation process like?
-
What was the main considerations when you created the Excel sheet?
5 Business Managers
-
How long have you and how frequently do you use the Excel sheet?
-
Could you tell us your first-time experience with the Excel sheet?
Define
Thematic Analysis
To enhance the analysis and visualisation of the data gathered from internal stakeholders at Together Tech, we applied thematic analysis. After transcribing the interview recordings and meticulously coding the transcripts, we conducted an initial grouping in Figjam. Numbers represented the frequency of the codes, and different colours of sticky notes indicated the source interview questions. This method enabled us to delve deeper into discussions and identify potential candidates for overarching themes.
Following this, key themes were extracted, encapsulating vital aspects of their experiences and opinions regarding the current Excel BWi spreadsheet. These encompassed encountered challenges, suggestions for website design, and human factors that could influence users' decision-making processes.
Decision Matrix
After identifying a set of 15 sub-themes during the thematic analysis, we created the Decision Matrix method to prioritise them. To gain a user's perspective, a business manager from Together Tech was consulted, helping us understand which issues were most crucial in achieving project goals.
The decision matrix utilised two criteria, impact and effort, guiding the prioritisation of issues and features in the development of the web-based tool.
MoSCoW Prioritisation
The results of the decision matrix were further refined using MoSCoW prioritisation. Based on the placement of each sub-theme within the decision matrix, the 15 sub-themes were divided into four categories. Sub-themes fell into 'must-haves' (low effort), 'should-haves' (high impact, medium/high effort), 'could-haves' (moderate/low impact, significant effort), and 'would-not-haves' (low impact, significant effort).
Additionally, interconnections between these sub-themes were identified, and related ones were linked with lines and arrows. This approach allowed for focused design and development goals, offering clear direction for the subsequent stages of the project development.
Sketch
Co-Creation Workshop
A co-creation workshop was planned and conducted, where co-creation refers to collective creativity. As a starting point of this phase, the results coming from the workshop could be seen as an "idea pool" to widen the design space.
Participants?
-
2 business managers
-
1 sustainability expert
-
1 technical consultant
Duration?
-
2 hours (3 activities)
Objective?
-
Explore possible ideas for rating scales, question formats, and result pages
Preliminary Analysis of the Workshop Results
The ideas and outcomes from each activity were organised alongside the corresponding photos. Yellow sticky notes were utilised to summarise insights generated during the workshop. These insights were then grouped based on relevance and connected to second-layer insights.
In the preliminary analysis, several initial findings have emerged. For instance...
-
Participants showed a preference for graphical elements over plain numbers when designing the assessment scale.
-
Regarding question format, participants favoured general questions accompanied by detailed explanations.
-
Participants found providing explanations significantly enhanced their understanding of the questions.
-
Participants expressed a desire for a visually appealing web page that could effectively present assessment results. This page should serve as a catalyst for sustainable development while being aesthetically engaging.
Decide
Assumption Map
To thoroughly assess the feasibility and significance of the ideas generated during the workshop, especially those from the first and third activities, a detailed evaluation was essential. The method of assumption mapping was utilised in the subsequent stage to delve deeper into the underlying reasoning behind the design choices, aiming for more profound insights.
The process resulted in three types of assumptions: "Desirable," "Feasible," and "Viable.", each represented by distinct coloured sticky notes.
Yellow - Desirable
-
Derived from the previous sticky notes collected during the preliminary analysis of the workshop sketches.
-
Represented users’ core desires.
Blue - Feasible
-
Evaluated the "Desirable" assumptions from a design and programming perspective.
Green - Viable
-
Assessed the "Desirable" assimptions from business and product standpoints.
Assumptions were then plotted on a matrix based on the "Known-Unknown" and "Important-Unimportant" dimensions.
Critical issues (important and known) were prioritised, unknown important aspects were earmarked for experimentation, and less important assumptions were deprioritised.
This structured approach aided in identifying, analysing, and prioritising assumptions for future design, facilitating strategies for validation or mitigation.
Moodboard
The moodboard served as a tool to gather and consolidate information, such as images, fonts, colours, and styles, representing the chosen keywords "simple, fun, informative" from the co-creation workshop.
Prototype
Sketch
Wireframe
.png)
Feedback on the Wireframe
Two feedback sessions on the wireframe were conducted with the university supervisor and mentors from the company. The feedback received included the following points:
-
The sidebar menu was chosen over the progress bar for the final design due to its flexibility and user-friendly interface.
-
The wireframe design was perceived as overly formal and lacking personality. Suggestions were made to incorporate playful elements to better align with the user needs identified during the co-creation workshop.
Initial High-Fidelity Prototype
Stakeholder Feedback Session
With Whom?
-
1 business manager
-
2 sustainability experts
What to Discuss?
-
Do you find this design appealing?
-
Does the information provided support the intended purpose of this page?
-
Is there anything that you feel is missing?
Landing Page
-
Style mismatched with company theme, potential distraction.
-
Align style with company website for consistency and user confidence.
Scale
-
Review layout, colours, and "N/A" buttons.
-
Clarify purpose of "note-taking" buttons.
-
Flower patterns added vibrancy.
Term Explanation
-
Hovering explanation inefficient.
-
Clicking for tooltip not intuitive
-
Persistent display after cursor move may confuse users.
Side Bar Menu
-
Convenient, but needs clearer indication of completed sections.
Result Page
-
Clear infographics.
-
SDG info motivating.
-
Caution needed in interpreting "Score of Each Section."
Design Solution
Landing on the Assessment
The landing page introduces the sustainability assessment, reminding users of its context and suggesting an objective evaluation process to reduce biases. Its low-saturation colours and clean design create a calming atmosphere, helping users focus and potentially reducing mood bias.
The assignment area selection section identifies the company's main business area for the project. This customization tailors the questionnaire, showing relevant examples, reducing information overload, and enhancing user efficiency.
Users select their project's area and click 'start' to begin the assessment.
Answering the Assessment
The assessment page presents questions aligned with the 17 Sustainable Development Goals, organised into six sections. This layout encourages focused responses, reduces scrolling, and maintains respondent interest.
Viewing the Assessment Result
After submitting the questionnaire, respondents are directed to a new tab displaying the result report. This report aids understanding of the business project's sustainability impact and informs future decisions. The questionnaire tab remains open for easy revisiting and alterations.
Two "Download Answers" buttons, located in the report's header and at the bottom, allow respondents to download their assessment responses in PDF format for accessibility.
Pressing the download button saves assessment responses in a PDF format. The PDF includes overall and section scores, categorized questions and responses, and section-specific notes. Colour-coded section titles aid understanding of sustainability impact and maintain visual consistency.
Validate
User Testing
This testing employed both qualitative and quantitative methods to evaluate the product's performance, concluding the design sprint with a validated product and identified areas for improvement.
6 Business Managers X 1 hour
Testing Procedure
.png)
Quantitative Method
(1-10 Likert scale)
-
Usability (System Usability Scale)
-
Understandability
Qualitative Method
-
Think Aloud Protocol
-
Post-Testing Interview
User Testing Result
Quantitative Analysis
Usability
The new website is 37.8% more usable than the Excel sheet, and the Wilcoxon signed-rank test shows a significant mean difference.
Understandability
The website is more understandable than the Excel sheet on average. However, the Wilcoxon signed-rank test showed an insignificant mean difference.
Qualitative Analysis
"I appreciate the examples. It is easy to identify the assignment's area and understand the questions."
"I already prefer this version at first glance! There are more options now, which is great."
"I like the division of areas into categories; it's now shorter and less overwhelming."
"The design is clear and clean, aligning perfectly with our company's style."
However, we also identified several areas for potential improvement...
-
The scale used may pose problems due to its colours, option labels, and order.
-
The question did not align well with the scale.
-
The "Possible Examples" text was not prominent enough and may require a re-evaluation of the interaction method.
-
The sidebar menu provided significant freedom to navigate but the content could be improved.
Reflection
-
Directly translating user desires doesn't always yield optimal solutions; it is essential to consider the business context. Strive for compromises between user needs and business requirements.
Users often desire 'fun' elements, but it's essential to evaluate their alignment with our business context and company's style. Maintaining open and transparent communication with internal stakeholders throughout the project's lifecycle is crucial to ensure the company's objectives remain clear and unwavering.
-
In internal applications, time and effort are crucial. If a product becomes too lengthy or complex, users might disengage, leading to potential inaccuracies in their responses.
Feasible approaches include breaking tasks into manageable segments and structuring the content to present easier, less sensitive tasks first, followed by more delicate inquiries.
You may also be interested in...
bottom of page