Soundyarn is a new interactive way of experiencing sound while navigating your surroundings: it lets you design where sounds are located in space and how they interact with each other.
Scroops and Yarns
Each sound is placed in the real world through a scroop that covers an area at a particular location. Scroops can have different shapes, from simple circles to complicated polygons.
A collection of scroops that is connected with each other is called a yarn. A yarn starts unravelling when the user enters the area covered by one of its designated scroops: we call them "start-scroops". From this starting point, the creator determines which scroops can be activated next by entering their respective area: so-called "next-scroops".
Let's suppose the user has just entered the area of scroop #1 below and has activated it. The white arrows in the picture illustrate which scroops are next-scroops of each scroop, for example, #2 is a next-scroop of #1. The grey line is the path the user follows, so scroop #3 will be activated next. Scroop #4 will not be activated after that, as #4 is not a next-scroop of #3.
The shape of a scroop can be a circle or a polyline with some radius, or a polygon.
When the scroop is a circle or a polyline it has a radius.
The sound of a scroop can be taken from Soundcloud or recorded in-app, but a scroop can also be functional without a sound.
The volume of the scroop takes a value between 0 and 1. It sets the maximum possible volume for the scroop.
By setting a volume radius, which can differ from the scroop radius, the effective volume of the scroop can be tied to the position of the user relative to its center.
The volume function determines the effective volume of the scroop given the volume radius (as parameter) and the relative location of the user (as variable). The function can be "zero", "constant", "linear", "quadratic" or "exponential". When the function is set to "zero" the volume within the volume radius is 0, and when it is set to "constant" it is equal to the maximum volume. The "linear", "quadratic" and "exponential" functions describe how quickly the volume decreases with increasing distance of the user from the centre of the scroop. An inverted version also exists for these three functions, allowing the volume to increase as the distance of the user from the centre increases.
The outside volume can be switched on or off, setting the volume outside of the volume radius to 0 or the maximum volume, respectively.
Below, the linear and inverse linear functions are shown, both with the outside volume set to off.
A scroop that can be activated as the first in a yarn is a start-scroop. At least one of the scroops in a yarn must be a start-scroop.
A scroop with "activate once" set to ON can only be activated once.
An interruptible scroop can be interrupted while playing. Unless "interruptible" is set to ON, or the scroop has finished playing, no other scroop can be activated.
A repeat scroop repeats its sound. Avoid building scroops that repeat and are not interruptible!
End of cycle interruptible
Once an end of cycle interruptible scroop is activated, it repeats until another scroop can be activated. When the first scroop ends its cycle, the other one starts. This is useful when you want to sync the start of one scroop with the end of another scroop. An end of cycle interruptible scroop is therefore not the same as a scroop with "repeat" and "interruptible" set to ON! Please see the Examples section below for more info!
A scroop with "anytime" set to ON is a next-scroop of every other scroop. But it is not automatically a start-scroop!
A non-state scroop when activated does not become or change the active state of the yarn. There is no point in setting next-scroops for a non-state scroop!
An activated overlaid scroop plays its sound over that of other currently playing scroops. The overlaid volume takes a value between 0 and 1, multiplying the maximum volume of all currently playing sounds. For example, if an overlaid scroop with an overlaid volume of 0.5 is activated, the maximum volume of all currently playing sounds is halved.
The activation of a scroop with a delay is postponed by the specified time from the moment the user has entered its area and all other conditions for its activation are satisfied.
If "check delayed conditions" is set to ON, the conditions for the activation of the scroop are checked once again after the specified time has passed.
A set of external conditions can be attached to each scroop, restricting when it can be activated.
TimeA time window for the activation of a scroop can be set in increments of 15 minutes. For example, if the time window for a scroop is 23:00 to 2:00, the scroop can be activated only at night between these two times.
SpeedA range of speeds can be specified, so that the scroop can be activated only if the current speed of the user is within the given range.
OrientationA range of orientations can be set in degrees, such that a scroop can be activated only if the current orientation of the user from north is within the given range.
The colour of a scroop as shown on the map can be selected from a couple of different options. This lets you visually group scroops into different layers.
A scroop can also be "hidden", meaning that it will not show on the map in the soundyarn app. It will still show on the map in the soundthrowster app.
A scroop itself can be made to (repeatedly) move along a set path at a fixed speed.
Yarns are collections of interconnected scroops. Once a yarn is started, any "start-scroop" in the yarn can be activated. The last activated scroop determines the current state of the yarn. Any of the next-scroops of that state and any "anytime" scroop can be activated next.
Next-scroops of a given scroop are those scroops that can be activated after it.
Sync-scroops of a given scroop are those scroops whose sounds start playing at the same time as the sound of the given scroop. This is useful for syncing the sound of different scroops.
A scroop that is a stop-scroop of some other scroop stops playing when the latter is activated.
The soundThrowster app
The soundThrowster app lets throwsters create and manage yarns.
The throwster can see all their yarns on the front screen once the app is opened. To delete or duplicate a yarn, hold down on the image for a second until the menu appears.
Once a yarn has been opened there are two main modes of operation, the Editor mode and the Simulator mode. The Editor mode lets the throwster create and edit yarns, and the Simulator mode lets the throwster simulate their experience. Toggle between the two modes by pressing the respective button in the upper right corner of the screen.
To edit the different attributes of the yarn itself, tap on the yarn title on the top of the screen. By clicking on any of the scroops on the map, a menu of functions appears on the left of the screen. At the top of the map, select which colour layer of scroops to display.
Apart from the title, the description and the display image for a yarn, you can set a preview sound, mark the yarn as private or public, and attach one or more tags to it.
When no scroop is selected, the plus button can be used to add new scroops to the yarn. A dialog box then appears on the top of the screen, letting you set the radius of the scroop (for circles and polylines) as well as its colour. The menu on the left of the screen lets you select whether to add a circle, a polyline or a polygon. Touching the map sets the center of the circle, or the corners of the polyline or polygon, respectively.
To edit a scroop, select it and tap on "edit". The new view allows you to set a title, add a description, a sound, and set the different switches. This view also lets you delete the scroop.
Adding a sound
To set the sound for a scroop, select it, press "edit", and choose the sound pane. By tapping the title of the current sound, or the button to add a sound, the Sounds view appears. This view has 3 main tabs. The "Search" tab lets you search for sounds on Soundcloud, the "Your sounds" tab lets you select a sound from your Soundcloud profile, and the "Record" tab lets you record and use a new sound, and also upload it to your Soundcloud profile.
Copying and cutting scroops
To cut or paste a scroop, select it and tap on the respective action. You can select more than one scroop before tapping "next", when you choose where the scroop(s) will be pasted. You can also delete many scroops at once by cutting and not pasting them.
To declare next-scroops for a scroop, select it, and tap the first icon (shown above) located on the sidebar on the left of the screen. All scroops that are currently next-scroops are connected to the given scroop with a line. To add or remove next-scroops, tap on them on the map.
You can declare stop-scroops and sync-scroops in the same way, after pressing the respective icon on the sidebar.
Volume radius dependent scroops
To edit the volume radius of a scroop, select it and tap the concentric circle icon (shown above) on the sidebar that appears on the left of the screen. In the dialog box that appears there are controls for the (activation) radius, the volume radius, and the volume function for the scroop.
To let the scroop move along a path, select it and tap the single arrow icon (shown above) on the sidebar that appears on the left of the screen. Set its path by tapping on its endpoints on the map, and set its speed in the dialog box.
To adjust the location of a scroop, select it and tap on the four-arrow icon (shown above) on the sidebar that appears on the left of the screen. Use the arrows that are displayed to fine-tune the position of the scroop. To move a scroop over long distances, you should use the copy/paste functionality instead.
When selecting a scroop in Simulator mode, you can choose to play the sound associated with it. To simulate how the yarn is experienced, set the path of the avatar, and then start the simulator. A path can be saved and loaded to be used later. After setting a path, controls appear at the bottom of the screen, letting you start or stop the simulation, and adjust the speed of the avatar.
To create a simulation path, tap on the first icon (shown above) on the sidebar that appears on the left. In the dialog box that appears, you can load a saved path or choose to create a new one. In case a path is already active and not yet saved, a different dialog box appears with the option to save the active path. When creating a new path, its endpoints are set by tapping on the map.
Start the simulation by clicking "start" in the controls view that appears at the bottom of the screen. While the yarn is being simulated, the speed of the avatar can be adjusted with the controls at the bottom.
This is a simple example of a single scroop with a sound that is activated once the user enters the circle associated with the scroop.
This example shows how to build a simple tour. One scroop is set as the start-scroop, and then a succession of next-scroops is constructed. Each of these can be activated only after the previous scroop has been activated.
This example shows how to build a yarn using two scroops (a yellow and a blue one) with identical shapes, where the sound of the yellow scroop is heard only when the user is inside its area, while the sound of the blue scroop is heard only when the user is outside its area. Such a simple audio interface can guide the user on their path. By setting the two scroops to be hidden you can create a game on that premise!
In this example, a yarn is created to model a situation where a music track has different accoustics when the user is inside a building than when the user is outside that building.
Initially, two polygon-shaped scroops are created, one for each of the two versions of the music track. The two music tracks are of the same length, and one is simply an edited version of the other, emulating the indoors accoustics (the effect has been applied using some other audio software). Furthermore, the volume function of both polygon scroops is set to be constant while the user is inside the polygons and zero when outside. This way, when the user goes from being inside the outer polygon to being inside the inner polygon, the music changes as if the user moved from outside a building, to the inside of that building.
Finally, a scroop of circular shape is added, being the start scroop with no associated sound, whose function is primarily to sync the two sounds associated with the two versions of the music track.
Need For Jogging Speed
This example shows a yarn for joggers, with music only playing when the jogger is running. The yarn contains 3 simple circle scroops, a blue, a yellow and a red one. The blue and the yellow scroop can only be activated when the user's speed is at least 3m/s. The red one can be activated when the user's speed is lower than 3m/s.
When the red scroop is activated, the volume of all other playing sounds is dropped to 0, as the red scroop has the overlaid property set to 0.
The blue scroop is a start-scroop and has music associated with it. It has only one next-scroop, the yellow one. The only next-scroop for the yellow scroop is the red scroop, and vice versa. The red scroop also acts as a stop-scroop for the yellow one.
When the yellow scroop is activated, the red scroop stops and all playing sounds go back to full volume. The yarn toggles between the yellow scroop and the red one, whenever the speed of the jogger increases or decreases. In the simulator the slow speed is represented by 'walking' and the faster one by 'bike'.
This example shows the creation of a sonic fence in a park. The sonic fence emits an alarm whenever the user steps outside the green scroop. It can be deactivated by locating the hidden switch (the blue scroop). To find the switch, the user follows the ambient soundscape (the red scroop) that increases in volume as the user approaches the switch.
Driving with soundyarn
Watch the video to see how the yarn depicted in the figure below would be experienced while driving through the location.
The final yarn for the video above: