Design Proposal

Apr
29
2016
By pasqual6

Tuesday, April 20, 2016

 

PROTOTYPE DESIGN PROPOSAL
“Book of Fame" Project Team
University of Toronto

 

Background: The "Book of Fame" project team, a subset of the Old Books, New Science lab at the University of Toronto, is integrating the IIIF (International Image Interoperability Framework) application programming interface into a video game development engine with the intention of broadening and simplifying access to digital art repositories among the international community while presenting scholars with a new and innovative method of interacting with these otherwise inaccessible artefacts.

Prototype Proposal: The purpose of the project, henceforth referred to as the "prototype," is to create an edutainment tool that will enable users to interact with a fully rendered virtual book in three dimensional space. This virtual book will consist of two elements: (1) a single-piece model of a book (binding and pages) that opens to a predetermined place in the book, in which (2) an interactive gathering of individual, blank pages is embedded. The pages in the gathering will be organized according to the book's codicological formula and utilize the IIIF protocol to display the corresponding manuscript images. The prototype will also interact with metadata retrieved from the library server such as image area definitions, annotations, and text transcriptions.

Figure 1. Sketch distinguishing the two elements of the virtual book.

Project Goals:

  1. Expand the range of function and service for IIIF technologies
    By integrating the IIIF protocol into a video game development engine and rendering separated manuscript images as a virtual codex, the prototype positions and presents these artefacts within a controlled environment that is governed by conventions familiar to users of the information age. Creating such a virtual environment further overcomes logistical problems such as library access and copyright laws, offering a simple and fun means of interactive with digital repositories available to undergraduates and mature scholars alike.

  2. Recapturing the lost dimensions of digitized manuscripts
    The prototype aims to model the features of the physical book that have been ignored within the traditional web or e-publishing interface. Such features include (but are not limited to): the physical size of the object, binding, composition, texture, evidence of rebinding, damage sustained throughout its provenance, its codicological construction, a recto image’s direct relationship with its verso, paper composition, watermarks and other details only visible in certain circumstances (for example, underneath infrared lighting).

  3. Recreating artefacts that (can) no longer exist
    Because a book’s provenance witnesses the separation, redistribution, and rebinding of its leaves, medieval books in particular have taken several different forms throughout their lifetimes. One advantage of a virtual environment is to gather manuscript leaves from different libraries and reshuffle them into different orders, recreating the lost forms of a book or creating entirely new forms altogether. Using the prototype, all of this can be done and studied without ever disturbing the existing physical artefact.

  4. Open new ways in which glosses, annotations, and metadata can interact
    The prototype aims to place three levels of annotations in conversation with one another: (1) glosses inscribed on the physical text and present on the scanned digital image, (2) annotations by contemporary scholars that have been recorded on the library’s server and retrieved using the IIIF protocol, and (3) communal annotations hosted on a local server that do not alter the metadata on the library server.

  5. Make a contribution to the scholarly discussion of digital books
    By creating a virtual codex as a digital surrogate to the physical book, this prototype will serve as a point of discussion for the relationship between physical and digital books as well as raise questions regarding notions of authority that scholars traditionally invest in the physical artefact.

Target Audience: IIIF community, MoMA attendees, manuscript scholars (including, eventually,
undergraduate students)

Target Hardware: PC, Web

Engine: Unity (with Mirador integration)

Scripting Language: C#

Figure 2. Sketch of "Book Mode" displayed with generic toolset and user interface.

Interactive Features:

  1. Book Mode: IIIF Viewer
    Render a gathering of individual pages embedded within a single-piece model of a book.

    1. The single-piece model consists of a cover, its binding, and two non-interactive blocks of pages (one appearing on each side of the book when it is laid open). Pages within the interactive gathering should snap to the non-interactive blocks when they are not being turned or manipulated.
    2. Individual pages within the gathering will be represented by thin models that use the IIIF protocol to retrieve images from the library server for their textures.
      1. The engine will always load the two adjacent pages on each side of the open pages; hence only a maximum of six pages should be downloaded into the engine’s cache at a single moment in time.
      2. Pages should animate flexibly (as opposed to an automatic page flipping animation). The user should be able to point and drag pages with realistic physics, perhaps using Unity’s “cloth” modelling template.
      3. Pages can only be turned one at a time, though there will be an option to auto-turn the book to a specific page. This feature can use an automatic page flipping animation to speed things up.
    3. The prototype should track the user’s position within the gathering of pages and display it on some sort of tracking meter, if not on the single-piece model itself.
       
  2. Click and Drag
    When the book is closed, the user should be able to click and drag to rotate the book in three dimensional space. Double-clicking on the book will open it up to a predetermined page. While open, neither the user’s nor the book’s position in space can be changed. Instead, the user can click and drag pages in the interactive gathering. Once a page is dragged past a certain point (likely 90° relative to the base), the page is considered “turned” and will fall face-down if the click is released. The reader can close the book by double clicking on the edges of the cover, visible on either side of the book.
     
  3. Metadata: Hover and Identification
    The prototype will use the metadata to identify spaces of interest upon the page and label them with corresponding annotations. For example, the coordinates (x,y,w,h) may identify a region of a manuscript image containing an illumination and carry descriptive information regarding that illumination. When the user hovers over the region defined by the coordinates, the corresponding descriptive metadata will be displayed.
     
  4. Metadata: Selection
    Similarly, if the user activates the selection tool, he or she can click and drag a box around a specific region of the open page to highlight its contents. The user may then add his or her own annotation to the highlighted coordinates, which is then converted into the IIIF protocol and stored for retrieval upon the prototype’s local server.
     
  5. Metadata: Display Annotations
    The user will be able to click a button on the UI that will display three different levels of annotations: library, local, and global (i.e. library and local). These annotations appear as small windows with manicules pointing to the corresponding region. Library and local annotations will appear in different colours to signify that they are not from the same records, despite appearing within and in times occupying the same virtual space.
     
  6. Metadata: Transcription
    Using the IIIF retrieval protocol and the coordinate positioning system, the prototype will place a transcription of the open pages’ text on an invisible layer that hovers above the displayed image. When the user activates the magnifying glass tool, a fully modelled magnifying glass appears on screen and moves according to the mouse’s movements. Rather than distorting the image, the magnifying glass reveals the transcription text laid atop a generic manuscript page texture.
     
  7. Middle English Dictionary Integration
    If the user activates both the magnifying glass tool and the MED tool, double-clicking on a word will retrieve its corresponding entries from the MED web site and display them in a separate window.

User Interface

  1. Main Menu
    When the prototype application is first loaded, a splash screen will be displayed featuring a talking bird that will explain via dialogue boxes the purpose of the tool, its feature set, and the basic controls. This introduction can be skipped by pressing “SKIP” in the corner of the splash screen.
     
  2. Camera
    The camera represents the user’s scope of vision. If the book is positioned at chest
    height, the camera is positioned at the head. The user can zoom in and out on the book
    without his or her position within the virtual world.
     
  3. Controls
     

    1. Navigation Mode
      When the book is closed, the user can move around three dimensional space as
      well as move the book around.
       

      1. Arrow Keys
        The user can use the arrow keys to move his or her (and hence, the
        camera’s x and y) position in three dimensional space.
         
      2. Click
        The user can click on active buttons of the feature palette that appear at
        the bottom of the screen. However, the only buttons that are active during
        Navigation Mode are “Help,” “Reset,” and “Bird.”
         
      3. Click and Drag
        Clicking on the closed book and dragging the mouse (horizontal only) will
        move the book’s position in three dimensional space.
         
      4. Click and Drag while holding Shift
        Clicking on the closed book and dragging the mouse while holding shift
        will allow the user to rotate the book in three dimensional space so that
        he or she may have a better look at the book’s binding.
         
      5. Hover
        Hovering over the book will reveal its name and provide any descriptive
        details pertaining to the book as whole (as opposed to individual pages).
         
      6. Double-click
        Double-clicking on the closed book will reset the book to its default
        rotational orientation (horizontally parallel to the ground) and open the
        book up to a predetermined section. This will also fix the user in position
        before middle of the book and send the application into Book Mode.
         
    2. Book Mode (Default Tool)
       

      1. Arrow Keys
        Limited camera movement to simulate looking at the book from different
        angles without ever moving the user’s position in space.
         
      2. Click
        The user can click on active buttons in the feature palette to activate
        different point-and-click toolsets. All buttons are active during Book Mode.
         
      3. Click and Drag
        Clicking on a page and dragging the mouse will manipulate the page as if
        it were being pinched and pulled from that point. Dragging a page beyond
        90 degrees relative to the book’s horizontal axis will recognize the page
        as turned and cause it to fall face-down as opposed to face-up.
         
      4. Hover
        Hovering over a region of the page with its region coordinates specified in the metadata will cause any corresponding descriptive information to be displayed.
         
      5. Double-click
        Double-clicking on the protruding edge of the book cover on either side
        will cause the book to close in the corresponding direction and send the
        user back into Navigation Mode.
         
  4. Feature Palette
    There are a number of buttons on the menu bar at the bottom of the screen that can be clicked on to activate and deactivate. With the exception of assistance tools (Help, MED, Bird), clicking any tool will unclick all other tools.
     

    1. Hand Tool (Navigation and Book Mode)
      Activated by default, this tool allows the user to physically manipulate objects
       

      1. Click and Drag
        Navigation Mode: move book around in three dimensional space
        Book Mode: manipulate and turn pages
         
      2. Hover
        Navigation Mode: hover over the book to reveal a window containing
        descriptive information about the book
        Book Mode: hover over metadata-encoded regions to reveal a window
        containing descriptive information specific to that region
         
      3. Double-click
        Navigation Mode: double-click the book to open the book
        Book Mode: double-click the cover edge to close the book
         
    2. Annotation/Selection Tool (Book Mode only)
      When clicked, this tool allows the user to create his or her own annotations
       

      1. Click
        Book Mode: clicking without dragging clears any existing selection
         
      2. Click and Drag
        Book Mode: create a new set of region coordinates, which will then
        prompt the user to add corresponding descriptive metadata to be stored
        in the local server
         
      3. Double-click
        Book Mode: double-clicking on a previously created set of region
        coordinates will highlight that region and display the recorded descriptive
        metadata in a window
         
      4. Delete
        Book Mode: if the user has double-clicked and highlighted a previously
        created region, pressing delete will erase both the coordinates and the
        corresponding metadata from the local server
         
    3. Display Annotation Tool (Book Mode only)
      Clicking this button cycles through various levels of annotation display
       

      1. Click
        Book Mode: clicking multiple times on the button makes different layers of annotations appear in a predetermined order (library, local, global, off = cyclical)
         
    4. Lens of Authority (Book Mode only)
       

      1. Hover
        Book Mode: hovering with the magnifying glass tool reveals the invisible
        transcription layer within the confines of the glass
         
      2. Double-Click
        Book Mode: double-clicking on a word (i.e. the word at the precise center
        of the magnifying glass) will cause the dictionary window to open and
        retrieve its corresponding entries from the MED web site
         
    5. Dictionary/MED Tool (Book Mode only)
      Retrieves entries for a desired word from the Middle English Dictionary web site
       

      1. Click
        Book Mode: clicking on the dictionary button opens a window in which
        entries will be listed for words that the user double-clicks
         
      2. Double-Click
        Book Mode: double-clicking the region of a word on the image will search
        the transcribed word on the MED web site and display a list of entries in
        the previously revealed window
         
    6. Help Tool (Navigation and Book Mode)
      Opens a help window to assist the user
       

      1. Click
        Navigation and Book Mode: clicking on the Help button opens up a
        window listing the controls and other information
         
    7. Bird Tool (Navigation and Book Mode)
      Demonstrates twitter integration by displaying the most recent tweet
       

      1. Click
        Navigation and Book Mode: clicking on the bird in the bottom right corner
        will cause it to squawk and display the most recent tweet on the project’s
        twitter page
          
    8. Pencil Tool (Book Mode only)
      Allows users to mark up and deface the manuscript image on a layer that does
      not alter the library’s original image
       

      1. Click and Drag
        Book Mode: clicking and dragging allows the user to draw and
        markup/deface the page (perhaps with constraints), the data of which is
        then saved to the local server and can be displayed along with local
        annotations
         
    9. Reading Tracker (Book Mode only)
      Displayed at the center of the bottom menu in an arc-like shape, this clock would
      track the user’s position within the book.

Figure 3. Toolset icons for user interface.

Aesthetics:

  1. Artstyle
    A virtual, faux Fisher library with a grid-like digital floor and pixelated sky. There is no furniture in this place: the lone book hovers at chest height wherever the user places it.
     
  2. Art Assets
    1. Panoramic background of the Fisher library
    2. Single-piece model of book
    3. Individual pages
    4. Animations for opening the book and manipulating pages
    5. User interface buttons and windows
    6. Bird (possibly 2D) with simple animations (i.e. turning head, opening beak)
       
  3. Sound Assets
    1. Opening book
    2. Turning page
    3. Reveal annotation
    4. Open window
    5. Bird squawk

Project Scope: One gathering of 17 leaves embedded as a gathering within a single-piece model of a book

You will be redirected to the secure UTORid login page