Vistoria: A Multimodal System to Support Fictional Story Writing through Instrumental Text‑Image Co‑Editing
Humans think visually—we remember in images, dream in pictures, and use visual metaphors to communicate. Yet, most creative writing tools remain text-centric, limiting how authors planning and translating ideas. We present Vistoria, a system for synchronized text–image co-editing in fictional story writing that treats visuals and text as co-equal narrative materials. A formative Wizard-of-Oz co-design study with 10 story writers revealed how sketches, images, and annotations serve as essential instruments for ideation and organization. Drawing on theories of Instrumental Interaction and Structural Mapping, Vistoria introduces multimodal operations-lasso, collage, filters, and perspective shifts that enable seamless narrative exploration across modalities. A controlled study with 12 participants shows that co-editing enhances expressiveness, immersion, and collaboration, enabling writers to explore divergent directions, embrace serendipitous randomness, and trace evolving storylines. While multimodality increased cognitive demand, participants reported stronger senses of authorship and agency. These findings demonstrate how multimodal co-editing expands creative potential by balancing abstraction and concreteness in narrative development.
Kexue Fu*, Jingfei Huang*, Long Ling*, Sumin Hong, Yihang Zuo, Ray LC, Toby Jia-Jun Li†.
