Annotating music encodings

Annotation is an important affordance of musical score within a variety of use-cases including music rehearsal, performance, and scholarly analysis. Currently, mei-friend features rudimentary support for different kinds of annotation, and this is a feature we are planning to develop further in the future. Feature requests on this topic, as well as other aspects of mei-friend, are gladly received!

Inline vs. stand-off annotations

mei-friend supports two distinct annotation modalities: inline annotations, created within the MEI document, using the <annot> element; and, stand-off annotations, created using external Linked Data structures which target elements within the document.

The two modalities each have their benefits and drawbacks. Inline annotations are housed within the MEI document, which is useful where they are intended to be readily available to the encoding’s audience – they are tightly coupled (indeed, part of) the MEI document they are describing. However, they can only be created by those with write-access to the MEI encoding. Of course, a user might download an MEI file from a Web server and then add <annot> elements locally; but these annotations would not be available to others viewing the file at its original location, unless it were overwritten by the newly-annotated version.

In contrast, stand-off annotations use Linked Data structures to annotate target elements within Web-hosted MEI documents. As these annotations are not housed directly within the MEI document, they must be discovered in order to be read; however, as the annotations refer directly to the MEI document in order to identify elements within it (see Targetting stand-off annotations), the targetted document can be implicitly discovered from a given annotation.

The annotation panel

Annotations may be generated and listed using the annotation panel. This panel may be accessed via the menu by selecting View->Annotation panel, via keyboard shortcut CTRL-A(or CMD-Aon a Mac), or by clicking the corresponding ‘highlighter’ icon in the panel icons area at the top-right of the interface (Fig. 1).

Fig. 1: The 'annotation panel' icon.

The annotation panel has two tabs – Tools to generate annotations, and List to obtain a quick overview of the annotations currently present within your encoding.

Tools for making annotations

To generate inline annotations using mei-friend, you must first select one or more score elements that you wish to annotate. Note that each element must possess a valid @xml:id attribute in order to be annotated.

Choose among the tools available through the annotation panel to annotate your selection (Fig. 2). Currently implemented annotation types include Highlight (highlighter icon), Describe (pencil icon), and Link (chain-link icon).

Fig. 2: Annotation tools.

Highlight

The ‘Highlight’ tool (highlighter icon) creates an inline annotation that targets each selected element: an annot element with each selected element’s @xml:id in its <annot@plist>. The <annot> is placed within the encoding at the closest valid location relative to the first selected element.

If the Show annotations setting is not disabled, the selected elements will be highlighted using green shading within the notation panel (Fig. 3).

Fig. 3: A 'highlight' annotation.

Describe

The Describe tool (pencil icon) creates an inline annotation using the same mechanism as the Highlight tool. In addition, it pops up an alert window to prompt the user to enter a textual description. This description is then entered into the body of the generated annot element as textual content.

#

Fig. 4: A 'describe' annotation.

Link

The Link tool (chain-link icon) creates an inline annotation using the same mechanism as the Highlight tool. In addition, it pops up an alert window to prompt the user to enter a URI (Web address). This URI is then entered into the body of the generated annot element as a ptr element with its <ptr@target> pointing to the supplied URI.

If the Show annotations) setting is not disabled, the selected elements will be highlighted using red shading within the notation panel (Fig. 3). The mouse cursor will turn into a pointer indicator when hovering over the selected elements, and a tooltip will display the URI entered by the user (Fig. 5). Clicking on a selected element will open the URI in a new browser tab.

Fig. 5: A 'link' annotation.

The annotations list

Fig. 6: Annotations list.

The ‘List’ tab of the annotations panel (Fig. 6) provides a convenient overview of all annotations loaded for the current encoding – all inline <annot> alongside any stand-off annotations loaded from the Web, unless these exceed the maximum number of annotations setting).

Each annotation is represented by a ‘bubble’ (rounded rectangle), each split into an upper and lower region (see Fig. 6).

The upper region characterizes the annotation’s meaning, location, and extent:

• Type, represented by a corresponding icon (highlighter: highlight, pencil: describe, chain-link: link)
• Location, represented by a page number (or span of page numbers) corresponding to the (current) page location of the annotated element(s) in the rendered notation – updated dynamically to track layout changes
• Extent, representing the number of elements annotated
• Body, in the case of describe and link annotations, reflecting the textual description or link entered when creating the annotation

The lower region exposes four icons used to interact with the annotation:

• Flip to page (page icon with arrow), to jump to and select the <annot> (inline annotations) or the first annotated element (stand-off annotations) annotated element within the editor and notation panels
• Inline indicator (page icon with <> symbol), when not grayed out, indicates that the annotation is represented by an inline <annot> – click the icon to copy the corresponding <annot@xml:id>to the clipboard.
• Stand-off indicator (RDF icon, three connected circles – nodes – representing a graph), when not grayed out, indicates that the annotation is represented by a stand-off Web Annotation – click the icon to copy the corresponding URI to the clipboard.
• Delete annotation (square icon with -), to delete an annotation; this will remove inline <annot> elements, send an HTTP DELETE request to attempt to remove stand-off Web Annotations, and unload the annotation from the annotation list.

Note that we are planning functionality to convert between inline and stand-off annotations using the respective indicator icons in future development.

Loading stand-off annotations from the Web

When working with an encoding loaded from the Web via URL or through the GitHub integration, Web Annotations targetting element (fragment) URIs within the encoding may be loaded in by clicking on the Load Web Annotation(s) button with the RDF icon at the top of the annotations list. When prompted for a URI after clicking this button, you may either enter the URI of a Web Annotation to load it directly; or, you may enter the URI of a Linked Data Platform (LDP) container – such as those offered by Solid Pods – in which case, any Web Annotations targetting the currently loaded encoding within the container (or, recursively, within contained containers) will be loaded for you.

Background: Targetting stand-off annotations

The Linked Data paradigm involves the interconnection (linking) of resources stored on the Web by reference to their Web addresses or URIs. Any MEI document stored on the Web has its own URI; for instance, the following URI is the Web address of a sample encoding of an arrangement of Beethoven’s Hymn to Joy:

https://raw.githubusercontent.com/music-encoding/sample-encodings/main/MEI_4.0/Music/Complete_examples/Beethoven_Hymn_to_joy.mei

You can click here to open this file using mei-friend. If you do so, you should see a number of MEI elements selected for you: eight notes forming the first half of the opening phrase of the verse in the trombone part. Each of these notes has an @xml:id associated with it; in this case, d1e5417, d1e5433, and so on. Each @xml:id can be turned into a globally-unique URI referencing the specified note within this particular Web-hosted MEI file, using a concept called URI fragments. This works by taking the URI of the MEI file as provided above, and appending a # symbol, followed by the respective @xml:id, like so:

• https://raw.githubusercontent.com/music-encoding/sample-encodings/main/MEI_4.0/Music/Complete_examples/Beethoven_Hymn_to_joy.mei#d1e5417
• https://raw.githubusercontent.com/music-encoding/sample-encodings/main/MEI_4.0/Music/Complete_examples/Beethoven_Hymn_to_joy.mei#d1e5433
• and so on.

Different Linked Data structures of varying complexity and abstraction may be used to annotate MEI elements by reference to their (fragment) URIs. Currently, mei-friend supports annotations adhering to the Web Annotation Data Model, but support of more music-specific annotation models – see further reading – is under development.