The ubiquity of innovative digital technologies has given rise to the developments of new narrative forms, and redefined what it means to access, engage and read texts (Hateley,2013); Lamb, 2011; Walsh, 2013). New technologies have not only redefined what it means to read, but also changed the agency of the reader during the narrative experience, facilitating unique, personal experiences between the reader and the text. Digital technologies enable the reader to choose the depth of engagement and exploration with the text through activation of metacommentary, images, audio, video and interactive graphics (Lue, McVerry, O’Byren, Kiili, Zawilinski, Everett-Cacopardo, Kennedy & Foirzani, 2011). The affordances of digital technologies also invite the reader to ‘… physically interact with the text through inserting, deleting, or replacing text; marking passages by highlighting, underlining or crossing out words… manipulating the page format, text size and screen layout. (Larson, 2009, p.255)’, altering the immersive reading experience. The definition of ‘text’ (Kress, 2003) and ‘meaning making’ (Unsworth, 2006) have expanded to include these multimodal, digital artefacts.
This paper critically reviews three texts across three multimedia platforms: Our Choice by Al Gore (app), My Mother’s lover by David Dobbs (Kindle) and The Watertower by Gary Crew, illustrated by Steven Woolman (online picture book). This review considers whether the digital platform has not only fully exploited the potential of the technology to enhance, extend or transform (Unsworth, 2006) the reading experience, but also complements the unique, narrative elements of each text. Why has the particular platform been used for a particular narrative (e.g. unsworth, 2006, p.117) ?
This review has adopted two frameworks to evaluate the use of each digital platform to enhance, extend or transform the reading experience. Firstly, the use of the SAMR model (Substitution, Augmentation, Modification, Redefinition) supports this evaluation to establish whether the digital platform has successfully transformed the text and narrative experience. Although the use of SAMR has been primarily to evaluate the use of technology in classrooms, it has been repurposed for this evaluation to rate whether the use of a particular digital platform has effectively enhanced, extended or transformed the narrative and reading experience. Specifically, has the platform merely substituted the print text with an electronic text format (Substitution) or has the platform substituted with some additional functionalities i.e. adjustable font size, screen size, dictionary feature (augmentation), redesigned the experience with options such as commenting, audio or video, agency of the reader (modification) or redefined the experience allowing for new agentive experiences and interactions not previously conceived of (redefinition) ?
Secondly, the evaluation of the key features of each platform has also been organized under three main headings: textual features and usability features features after David Lewis’s Ecology of the picturebook (cited in Meyers, Zaminpaima & Frederico, 2014, p.918), with the inclusion of the category multimodial features from Meyers et al. extended version of Lewis’s model.
My Mother’s Lover David Dobbs Atavist (Kindle)
My Mother’s Lover published by Atavist press is the poignant, true account of author David Dobbs’ investigation into his mother’s life after the unusual request on her deathbed for her ashes to be scattered over the Pacific Ocean. Due to the limitations of the platform features the rich, interactive features which were originally included as part of the narrative were not included. The Kindle version of the original media-rich text is disappointing.
The standard features of the Kindle platform present the narrative as text and images. The device allows for the scaling of text and column width, adjusting of font colour and provides the reader with some agentive opportunities such as bookmarking, highlighting, search function and a digital dictionary. These features are available via a side and top navigation menu.
Genette’s (1991) paratextual theory advocates that a book rarely appears in a ‘…naked state, without the reinforcement and accompaniment of a certain number of productions… like an authors’s name, a title, a preface, illustrations.’ The paratext is the means by which a ‘text makes a book of itself’ (Genette, 1991, 261). For Genette, the paratext is the ‘threshold’ the reader crosses to enter into the book. In My Mother’s Lover, paratext typically associated with printed text such as the front cover, index and contents page are included. In addition, the use of black and white photographs helps the reader to not only engage with the narrative, but also connects the reader with the main characters Evelyn Jane and Norman ‘Angus’ Hart.
Navigation is a key feature in the engagement of text in any format. Pressman (http://newhorizons.eliterature.org/essay.php@id=14.html ) notes that navigation is not just how we navigate through electronic literature, but offers the reader more agency to navigate text in a non-linear path. The navigation feature in the Kindle version of My Mother’s Lover allows the reader to move through the narrative via the hyperlinked contents page and navigations arrows either side of the page. However, this platform does not provide hypertext features whereby the reader could click or tap important facts and navigation within or outside the text. Since this text is rich with historical data linked to the biography of the two main characters, this limitation detracted from the reading experience and failed to meet the expectation of what an electronic text might afford the reader (James & de Kock, p.110). For example, interactive pdfs and eBooks created using Adobe Indesign support embedding of hyperlinks to external websites, video and audio.
My Mother’s Lover is also available from the boutique publisher Atavist as an app for iphone, iPad and iPod. This app provides access to the full in-line features which include character descriptions, audio, photo galleries, maps and official World war II documents and hyperlinks to the web and videos. Atavist’s version ensures the narrative is enhanced and extended by multimedia elements.
The SAMR rating for this text is an A (Augmented). While the Kindle platform provided features which extended the text beyond a printed version, it was unable to exploit the unique, multimodal opportunities which were originally included to enhance and extend the narrative.
Our Choice Al Gore (Rodale Inc, Melcher Media, PushPop Press app)
In contrast to My Mother’s Lover which had been originally developed and created with the intention of delivering a media-rich reading experience but was poorly delivered on the Kindle platform, the Our Choice app merges text and content into an interactive experience using an interface developed by PushPop Press. The app not only includes data-rich elements such as photos, audio, videos, maps, and infographics but also provides the reader with an easy to use, intuitive interface.
The interactive app transforms Gore’s original 2006 documentary An Inconvenient Truth filmed by director David Gugenheim which challenged audiences with realities of global warming. This app extends Gore’s campaign with multimedia features such as 250 full-screen images supporting key points, audio, video and hyperlinks to websites and interactive infographics.
Paratextual elements such as a front cover and identification of the author are not only retained as in a traditional print version but are extended using multimodal features such as a spinning earth on the front cover which when tapped activates an audio file of Al Gore quoting from the book of Deuteronomy.
Modern flat design icons such as video play arrows, hands, globes and audio buttons signal the reader to interact with video, audio, interactive graphics and infographics which extend the narrative.These multimodal features such as interactive graphics not only allow the reader to manipulate the media but also serve as a rich source of dynamic information. For example, in the chapter Designing for the Sun: Passive Solar Homes, taping the hand icon not only enlarges the image of a house but also provides the reader with a weather clock which can be turned to reveal how the house during winter or summer is able to use solar energy effectively. Graphic elements can be moved and folded up and dynamic, interactive graphics allow the reader to access data easily without leaving the app.
Narrative elements include not only the text, but also the author’s presence via audio and video, deepening the connection between the reader and the author with an emotive narrative (James & de Kock, 2013, p.110) of the author reading a poem he wrote.
The reader progress and interacts with the app by swiping, tapping and pinching the text and other multimedia elements. The interface allows the reader to smoothly scroll through the text, zoom into any page by tapping the thumbnail images at the bottom of the screen which serve as the contents pages. Tapping images and play buttons activate audio and video features.
Multimodal functions such as the touch interface, support the agentive experience of the reader allowing the reader to pick up and pop open any item from videos to photos, graphic images can moved and folder back to reveal text. Media items can be explored and interactive maps allow the reader to switch on their geo location and view themselves in relation to the various destinations.
Other multimodal features include documentary footage, over thirty original infographics, animations and original commentary which extend and enhance the narrative without interrupting the immersive reading experience. Rather, they seamlessly extend the textual narrative by elaborating on relevant segments of content. The reader is able to decide the degree to which they activate and interact with these elements and the interface allows the reader to quickly return back to the a page to continue reading, listening and watching. These features do not interrupt the immersive reading experience, rather they deepen the engagement with the often technical content and provide interactive demonstrations for the nontechnical reader.
The SAMR rating for this app is an R (Redefined). This app has enhanced and extended the text providing the reading with agentive experiences not previously conceived. These features authentically transform and enrich the readers experiences with the narrative while providing the reader with the choice of when and how to engage with these elements.
The Watertower Gary Crew, Illus. Steven Woolman Era (online in2Era)
While texts are available in various digital formats such as apps fir iPad, iPhone and iPod and ebooks delivered on device readers such as Kindle and Kobo, texts are also available online. Various sites on the internet and via public libraries link readers to online versions of narratives created either specifically for the online environment as well as traditional print texts redesigned as electronic texts.
The Watertower by Gary Crew and illustrated by the late Steven Woolman is an example of a picture book which has been enhanced but not extended by delivery via digital technology. Crew’s original text was enhanced by the illustrations of Steven Woolman and author Crew commented that Woolman’s illustrations made The Watertower (Crew, 2004, p. 18).
The narrative style is a blend of science fiction and fantasy and tells the story of two young boys, Spike and Bubba who go out to explore The Watertower.
The online version of The Watertower simulates the original, overall layout of page, and paratextual elements such as the original front cover at the top left corner as the entry point to the text. The name of the author and illustrator of the original print version are clearly identified and simulates the visual experience of a print picture book.Neither the placement of text nor the font have been altered for online reading, and the reader is able to view the picture book in full screen mode. The font is suitable for both print and online formats and is easy to read. While the reader can read the book in fullscreen mode, the text cannot be customized to adjust for font style, size or colour.
Woolman’s original illustrations have been used for this version and enhance the narrative through Woolman’s use of colour, line and angle to convey mood in Crew’s text in this online version. While The Watertower is a picture book, the narrative and science fiction theme is more suitable for young adult readers.
This online version has used the affordances of the online environment only to the extent of providing the reader with a navigation function to move forward and backward through the text via standard play, forward, backward and stop buttons.
The reader is able to activate narration which syncs with the text on each page. The narrative is also enhanced with sound effects such as the ‘swishing’ sound simulating page turns, and various sound effects which match the storyline such as the lonely sound of wind blowing against the watertower to convey isolation, birds singing as they move away from the town to convey the bush and the echoes of water dripping inside the dank, dark tank. While these features complement the eerie story, audio such as the narrator’s voice is intrusive and disrupts the immersive reading experience. The tone of voice is loud and does not match the almost ‘Twilight zone’ atmosphere created by Woolman’s illustrations. However, the reader can disable this function. While narration is an important function for accessibility, the feature has not been used to support the intention of Crew’s narrative.
The online text also does not use interactive functions such as clicking on images to enlarge them or narration to support characterisation. For example, using the voices of two boys to narrate dialogue or a narrator’s tone to complement the theme. The online format does not take advantage of digital technologies to extend the narrative by providing the reader with an opportunity to explore the text further or provide any added sense of agency that a print version would. For example, exploring words or providing an insight into character’s thoughts through the use audio buttons which the reader could click or tap to reveal the character’s thoughts. An example of this would be when Bubba is sitting by himself, waiting for Spike’s return.
Bubba climbed into the tank. “I’ll be all right,” he muttered, “I’ll be all right.”
The SAMR rating for this online version is an A. The online version has augmented the original print text by using multimodal functions such as navigation, narration and sound effects. However, the online version has not fully exploited the potential of multimodal experiences in this format.
This evaluation has considered the extent to which each digital platform used to deliver the electronic version of these three texts in this evaluation, enhanced, extended or transformed the readers agentive experience of each narrative. The Kindle version of My Mother’s Lover by David Dobbs, due to platform limitations, was not able to deliver the media-rich, interactive experience of the original Atavist app version. The app Our Choice by Al Gore on the other hand was able to deliver a transformative experience, fully exploiting the affordances of digital technology to deliver a media-rich, interactive experience which empowered the reader to interact and navigate text, images, audio, video and infographics. Finally, he picture book The Watertower by Gary Crew with illustrations by Steven Woolman, while delivering an online version of Crew and Woolman’s text, apart from the added functions of navigation and audio, did not fully exploit the potential offered by digital technology to enhance or transform the reader’s agentive experience with the text or imagery.
Crew, G. (1994). The Watertower (S.Woolman, Illus.). Era Publications. Retrieved from http://www.in2era.com.au/products/eerie-tales-online/items/the-watertower.
Crew, G. (2004). Vale: Steven Woolman – 1969-2004. Magpies, 18(4), 18.
Dobbs, D. (n.d.) My Mother’s lover [Kindle version]. Atavist.
Genette, G. (1991). Introduction to the paratext. New Literary History, 22(2), 262-272.
Gore, A. (2011) Our choice [App]. Rodale Inc., Melcher media, Push Pop Press. Available from http://pushpoppress.com/ourchoice/
Hateley, E. (2013). Reading: from Turning the page to Touching the Screen. In Y. Wu, K. Mallan and R. McGillis (eds.), (Re)imagining the world: Children’s literature’s response to Changing Times. Heidelberg: Springer-Verlag. Doi: 10.1007/978-3-642-36760-1_1
James, R. & de Kock. (2013). The Digital David and the Gutenberg Goliath: The Rise of the ‘Enhanced ‘ e-book. English Academy Review: South African Journal of English Studies, 30(1), 107-123.
Kress, G. (2003). Literacy in the new media age. London & New York: Routledge.
Lamb, A. (2011). Transmedia Universe. Learning & Leading with technology, November.
Larson, L. C. (2009). E-Reading and e-responding: New tools for the next generation of readers. Journal of Adolescent & Adult Literacy, 53(3). doi: 109.1598/JAAL 53.3.7
Leu, D.J., McVerry, J.G., O’Byrne, W.I., Kiili, C., Zawilinski, L., Everret-Cacopardo, H., Kennedy, C., & Forzani, E. (2011). The New literacies of online reading comprehension: Expanding the literacy and learning curriculum. Journal of Adolescent & Adult Literacy, 55(1), 5-15. Doi. 10.1598/JAAL 55.1.1
Meyers, E.M., Zaminpaima, E. & Frederico, A. (2014). The Futures of children’s texts: Evaluating books apps as multimodal reading experiences. In iConference 2014 Proceedings (p.916-920). Doi: 10.9776/14312) Retrieved https://www.ideals.illinois.edu/bitstream/handle/2142/47386/312_ready.pdf?sequence=2
Pressman, J. (n.d. ). Navigating electronic literature. New Horizons for the Literary:Essays. Retrieved http://newhorizons.eliterature.org/essay.php@id=14.html
Puentedura, R.R. (2014). SAMR. Retrieved http://www.hippasus.com/rrpweblog/archives/2014/06/29/LearningTechnologySAMRModel.pdf
Unsworth, L. (2006). E-literature for children: Enhancing digital literacy learning. London: Routledge.
Walsh, M. (2013). In L. McDonald (Ed.), A Literature companion for teachers (pp.181-194). Marrickville, NSW: Primary English teaching Association Australia (PETAA).