May 11–16, 2024 | Qian Wan, Xin Feng, Yining Bei, Zhiqi Gao, Zhicong Lu
Metamorpheus is an affective interface that engages users in the creation of metaphorical visual stories of emotional experiences during dreams. The system arranges a visual story based on users' recollection of their emotions in dream narratives, and facilitates self-discovery and self-reflection by enabling users to co-create metaphorical text depictions and images of visual metaphors of their emotions with generative AI models. Metamorpheus also allows users to alter the interface by relocating and resizing each emotional scene over a storyline visualisation, and applying colour filters extracted from created images based on the colour dominance, with the goal of encouraging them to further construct personalised meaning out of co-created text and images.
The design of Metamorpheus was informed by a review of affective expression in existing visual narratives. The creation of affective metaphors ended up being chosen as the main feature of Metamorpheus to develop a visual story. In this section we introduce our design considerations, and how they were implemented in practice.
The design of Metamorpheus attempts to engage users in the creation of a visual story to recount their emotions during a dream. For the purpose of affective reflection, we expect the system to be creative and open to interpretations for self-expression, and meanwhile remains accessible to common users with little art or design expertise. To this end, the authors, three HCI researchers plus two designers (one UX designer & one architecture designer), began by surveying papers on devices for affective expression in previous visual narratives such as films, cartoons, anime, comics, photography, etc.
A discussion session was later held among authors to review each paper and design approach we surveyed. We concurred to filter out approaches that are: I) too individualised and arbitrary for a visual storytelling system (e.g., pictorial runes in comics [31] ), II) too demanding that might seem distracting for storytelling (e.g., emoji or meme creation [29, 86] ), and III) not creative or open-ended enough for the purpose of self-expression and self-reflection (e.g., colours, shapes, filters, style transfer, etc).
In the end, we opted for the creation of affective metaphors to help develop visual stories in Metamorpheus. The creation of metaphors adapts better to affective visual storytelling because it is a cross-modal and emotionally arousing device for artistic expression that has been widely used across various mediums of communication and arts, even including HCI designs [74] . In clinical practice, metaphors also serve as an effective way of communicating lived experiences [84] . Additionally, we already have a well-defined design space [68] to scaffold the creation process, including both meaning and visual structures of metaphors. We also have corresponding creativity support tools [33, 46] , and text-to-image AI models [76Metamorpheus is an affective interface that engages users in the creation of metaphorical visual stories of emotional experiences during dreams. The system arranges a visual story based on users' recollection of their emotions in dream narratives, and facilitates self-discovery and self-reflection by enabling users to co-create metaphorical text depictions and images of visual metaphors of their emotions with generative AI models. Metamorpheus also allows users to alter the interface by relocating and resizing each emotional scene over a storyline visualisation, and applying colour filters extracted from created images based on the colour dominance, with the goal of encouraging them to further construct personalised meaning out of co-created text and images.
The design of Metamorpheus was informed by a review of affective expression in existing visual narratives. The creation of affective metaphors ended up being chosen as the main feature of Metamorpheus to develop a visual story. In this section we introduce our design considerations, and how they were implemented in practice.
The design of Metamorpheus attempts to engage users in the creation of a visual story to recount their emotions during a dream. For the purpose of affective reflection, we expect the system to be creative and open to interpretations for self-expression, and meanwhile remains accessible to common users with little art or design expertise. To this end, the authors, three HCI researchers plus two designers (one UX designer & one architecture designer), began by surveying papers on devices for affective expression in previous visual narratives such as films, cartoons, anime, comics, photography, etc.
A discussion session was later held among authors to review each paper and design approach we surveyed. We concurred to filter out approaches that are: I) too individualised and arbitrary for a visual storytelling system (e.g., pictorial runes in comics [31] ), II) too demanding that might seem distracting for storytelling (e.g., emoji or meme creation [29, 86] ), and III) not creative or open-ended enough for the purpose of self-expression and self-reflection (e.g., colours, shapes, filters, style transfer, etc).
In the end, we opted for the creation of affective metaphors to help develop visual stories in Metamorpheus. The creation of metaphors adapts better to affective visual storytelling because it is a cross-modal and emotionally arousing device for artistic expression that has been widely used across various mediums of communication and arts, even including HCI designs [74] . In clinical practice, metaphors also serve as an effective way of communicating lived experiences [84] . Additionally, we already have a well-defined design space [68] to scaffold the creation process, including both meaning and visual structures of metaphors. We also have corresponding creativity support tools [33, 46] , and text-to-image AI models [76