1 tplarson 1.1 The Sound of the Sun:
2 An Introduction to the Sonification of Solar Harmonics (SoSH) Project
3
4 a more extensive discussion of helioseismology with added graphics can
5 be found at http://solar-center.stanford.edu/SoSH/ . short instructions
6 can be found in quickstart_audio.txt .
7
8 Section 1: Background
9 motivation for the project
10
11 Section 2: Helioseismology and Sonificiation
12 an introduction for beginners
13
14 Section 3: Instruments and Data
15 the data you will need and how to get it
16
17 Section 4: Software (Pure Data)
18 a detailed description of the software
19
20 Section 5: Building Your Own Patches
21 examples of applications
22 tplarson 1.1
23 Section 6: Extensions
24 combining timeseries
25
26 Section 7: Conclusion
27 prospects for the future and contact info
28
29
30 Section 1: Background
31
32 the sun is a resonant cavity for very low frequency acoustic waves, and
33 just like a musical instrument, it supports a number of oscillation
34 modes, also commonly known has harmonics. we are able to measure the
35 frequencies of these various harmonics by looking at how the sun's
36 surface oscillates in response to them. then, just as the frequency of
37 a plucked guitar string gets higher with more tension and lower with
38 greater thickness, we are able to infer properties of the solar interior
39 such as its pressure and density.
40
41 this study of acoustic waves inside the sun is called helioseismology;
42 an overview can be found at http://soi.stanford.edu/results/heliowhat.html .
43 tplarson 1.1 although it has allowed us to make measurements of unprecedented
44 precision, it remains largely unknown to the general public. of course,
45 when one learns of it for the first time, a natural question arises:
46 "what does the sun sound like?". and unfortunately even
47 helioseismologists rarely have much experiential knowledge of it. that
48 is, we analyze solar data scientifically, but we never listen to it. it
49 is the goal of this project to make such listening widely available.
50
51 the first widely recognized effort toward the sonification of
52 helioseismic data was undertaken by Alexander Kosovichev, based on
53 earlier work by Douglas Gough (see http://soi.stanford.edu/results/
54 sounds.html). although only a small amount of data was sonified,
55 physical effects such as scattering off the solar core were still
56 audible. with the sonification of a vastly larger dataset, one would
57 also be able to hear solar rotation, or perhaps even the effect of the
58 solar cycle. not only does this bring helioseismology and solar physics
59 into the realm of everyday experience for the nonscientist, but it might
60 even allow for new scientific discoveries. the fact is that we simply
61 don't know what might be audible in the data because we have never
62 listened to it.
63
64 tplarson 1.1 finally, we said above that the sun is like a musical instrument, but
65 one could also say that the sun *is* a musical instrument, in that it
66 has its own distinct set of harmonics. of course we can't play the sun
67 in the sense of sounding individual notes; rather all of the solar notes
68 are playing all the time simultaneously. with a little analysis,
69 however, the various solar tones can be separated from each other and
70 then used in musical composition.
71
72
73 Section 2: Helioseismology and Sonification
74
75 the strongest of the sun's harmonics have periods of about 5 minutes,
76 corresponding to frequencies of only about 0.003 hertz. unfortunately,
77 this is far below the range of human hearing, which is typically taken
78 to be 20 - 20,000 hertz, although most people are only sensitive to a
79 smaller range. hence, if we would like to experience the sound of the
80 sun with our ears, these very low sounds must be scaled to the range we
81 can hear.
82
83 but first, we need some exposition of what a mode on the sun looks like.
84 to begin with, it is a mathematical theorem that any arbitrary shape of
85 tplarson 1.1 the sun's surface can be expressed as a sum over its harmonics (this is
86 also true for a guitar string). in the case of the sun, we call them
87 spherical harmonics, and each of them are labelled by two integers: the
88 spherical harmonic degree l (ell) and the azimuthal order m. The degree
89 l is equal to or greater than zero, and for each l, there are 2*l+1 values
90 of m, ranging from -l to l.
91
92 the different spherical harmonics also sample different regions of the
93 sun. low values of the degree l penetrate almost all the way to the
94 core, whereas higher values are trapped closer to the surface.
95 similarly, modes with high values of m have their maximum amplitude at
96 low latitudes, whereas lower values sample higher latitudes. it is
97 because the different modes sample different regions that we are able to
98 use their frequencies to determine solar properties as a function of
99 both depth and latitude.
100
101 to determine the frequencies of the sun's harmonics, we might take an
102 image once a minute for 72 days. for each image, we decompose it into
103 its various spherical harmonic components. for each of these
104 components, we form a timeseries of its amplitude. from the timeseries
105 we are able to construct the power spectrum (acoustic power as a
106 tplarson 1.1 function of frequency). the location of peaks in the power spectrum
107 correspond to the frequencies of the modes (harmonics). the height of
108 the peak tells us the mode amplitude, and the width of the peak tells us
109 how much the oscillation is damped.
110
111 an easy way to understand spherical harmonics is in terms of their node
112 lines, which are the places on the sphere where the spherical harmonics
113 are zero. the degree l tells how many of these node lines there are in
114 total, and the order m gives the number in longitude, so the number of
115 node lines in latitude is l-m. so a spherical harmonic with m=0 has
116 only latitudinal bands, while one with m=l has only sections like an
117 orange. a third integer, the radial order n, tells how many nodes the
118 oscillation has along the sun's radius. since only the surface of the
119 sun is visible to us, all the values of n are present in each spherical
120 harmonic labelled by l and m, although only some of them will be excited
121 to any appreciable amplitude. The total mode, then, is represented as a
122 product of a spherical harmonic, which is a function of latitude and
123 longitude, and another function of radius. each n will have its own peak
124 in the power spectrum.
125
126 now, in a spherically symmetric sun, all values of m for a given l and n
127 tplarson 1.1 would have the same frequency. a break in spherical symmetry causes the
128 frequency to vary with m. the most significant deviation from spherical
129 symmetry in the sun is rotation about its axis. the spherical harmonic
130 decomposition, however, is only sensitive to the absolute value of m.
131 therefore the positive and negative values of m must be separated in the
132 power spectrum. we say that the positive frequency part of the spectrum
133 corresponds to negative m, and that the negative frequency part
134 corresponds to positive m. note that this particular convention for the
135 sign of m is completely arbitrary. if you want to understand what is
136 meant by "the negative frequency part of the power spectrum", you will
137 need to study the fourier transform, but such understanding is not
138 strictly necessary.
139
140 let us now return to the issue of sonification, the conversion of data
141 into audible sound. the most straightforward way to do so would be to
142 use the spherical harmonic timeseries we already have in hand and speed
143 them up. but by how much? the answer of course is arbitrary and will
144 depend on your preference, but as long as this choice is applied
145 consistently you will still be able to hear the real relationship
146 between different solar tones.
147
148 tplarson 1.1 let us suppose that we want to transpose a mode in the peak power range
149 at about 0.003 hertz up to 300 hertz; this amounts to speeding up the
150 timeseries by a factor of 100,000. if we have 72 days of data taken
151 once a minute, this amounts to 103,680 data points, or samples. it's
152 easy to see that the sped-up timeseries would now play in just over a
153 minute. one must also consider the sample rate, however, or the rate at
154 which audio is played back. speeding up the original sample rate of
155 (1/60) hertz by a factor of 100,000 yields a new sample rate of 1666.67
156 hertz, and one unfortunately finds very few audio players that will play
157 any sample rate less than 8000 hertz. assuming this sample rate, our
158 0.003 hertz mode on the sun will now be transposed up to 1440 hertz and
159 the timeseries will play in about 13 seconds.
160
161 but suppose you want to play it in a shorter time; 13 seconds is a long
162 time to sound a single note, although you might want to do so in some
163 circumstances. you could increase the sample rate further still, but at
164 some point the mode will be transposed to an uncomfortably high
165 frequency. to understand the solution to this problem, we must explore
166 the process by which we shall isolate the modes.
167
168 at this point in our processing, playing an unfiltered timeseries would
169 tplarson 1.1 sound just like static, or noise. this is because very many modes are
170 sounding simultaneously in any given timeseries, not to mention the
171 background noise involved in our observation of the modes. therefore,
172 if we want to isolate a single mode, we have to do some filtering.
173 luckily, as mentioned above, we have already measured the frequency,
174 amplitude, and width of many modes. we can use these fitted mode
175 parameters to pick out the particular part of the power spectrum
176 corresponding to a single mode, and set the rest of the power spectrum
177 artificially to zero. we then transform back into a function of time so
178 that we can play the filtered data back as a timeseries.
179
180 since we are selecting only a narrow range of frequencies, we have the
181 freedom to shift the entire power spectrum down in frequency before we
182 transform back to timeseries. this timeseries will play in the same
183 amount of time as before, but the frequencies in it will be transposed
184 down by the same factor that we shifted the power spectrum. for every
185 power of 2 shifted down, the tone will drop by one octave.
186
187 one approach might be to decide how long you want to sound each tone
188 (keeping in mind that looping is also an option). this will determine
189 the sample rate at which you will play the timeseries. then you can
190 tplarson 1.1 choose a downshift factor to suite yourself. as long as you use the
191 same sample rate and downshift factor when you sonify every mode, the
192 frequency relationships between them will be preserved.
193
194
195 Section 3: Data
196
197 in order to use the Sonification of Solar Harmonics (SoSH) tool, you
198 will first need to download some data. two types of data are required:
199 text files containing ascii tables of fitted mode parameters and wav
200 files containing the raw acoustic data. furthermore, these data can
201 originate from two different instruments: the Michelson Dopper Imager
202 (MDI) and the Helioseismic and Magnetic Imager (HMI). MDI is the older
203 instrument, and took data from may 1996 to april 2011. it was
204 superceded by HMI, which began taking data in april 2010 and remains in
205 operation today. the two instruments are quite similar; the most
206 important difference between them for our purpose is that MDI produced
207 an image once a minute, while HMI produces an image every 45 seconds.
208 for both instruments, however, we analyze the data using 72 day long
209 timeseries.
210
211 tplarson 1.1 if you have also downloaded the SoshPy Visualization Package, then you
212 may use its included python module to interactively retrieve whatever
213 data you need. instructions for this module are included in the
214 package. however, these data may also be downloaded from the web, and
215 instructions for doing so follow.
216
217 first, pick a single directory for storing data on your computer; for
218 your convenience we have included a data directory in the zip archive
219 you have unpacked. if you elect to use a different directory (such as
220 your browser's download directory), you will simply need to enter it the
221 first time you run the SoSH tool. if you have downloaded the version of
222 the SoSH tool that includes demo data files, these will already be found
223 in the unpacked data directory. the quickstart guide includes
224 instructions for using those specific files.
225
226 in any case, the data are available at
227 http://sun.stanford.edu/~tplarson/audio/ , where you will find separate
228 directories for MDI and HMI. within each, you will find a series of
229 subdirectories that are day numbers suffixed with 'd'. the day number
230 corresponds to the first day of the 72 day timeseries and is actually
231 the number of days since 1 january 1993. day number 1216 was 1 may
232 tplarson 1.1 1996. a full table converting day numbers to dates can be found at the
233 above url as well.
234
235 clicking on a directory will show two ascii files containing the mode
236 parameters; download both of these to your data directory. the file
237 without "msplit" at the end contains one line for every degree l and
238 (radial) order n for which the fitting succeeded. the first five
239 numbers of each line are all that we will use here; they are degree l,
240 order n, mean frequency, amplitude, and width. these numbers are the
241 same for all m, the mean frequency being the average over m. the file
242 with "mpslit" at the end tells us only how the frequency varies with m.
243 make sure that however you download them, the file names stay intact;
244 some browsers like to add or delete filename extensions. this is good
245 to check if you get "file not found" errors later.
246
247 you need not view these files, but keep in mind that in general the
248 fitting does not succeed for every value of n. put another way, every
249 file will have l values ranging exactly from 0 to 300, the upper limit
250 being a somewhat arbitrary choice. for every value of l, exactly 2*l+1
251 values of m will appear in the msplit file. what may vary widely,
252 however, is which values of n appear for different values of l. we
253 tplarson 1.1 typically only find modes with n=0 in timeseries with l>100. modes with
254 n=28 (higher values are rare) are only likely to be found for l=5-15.
255 looked at from the other direction, for l=0 we typically find modes with
256 n=10-25. for l=100, n=10 is the highest value we might find. above
257 l=200, we have only n=0. in all cases, one may expect to find holes in
258 the coverage of n for values of l close to the edge of the range.
259
260 one way to get much higher mode coverage, at the cost of time
261 resolution, is to perform an average. an example of such an average can
262 be found in the files "mdi.average.modes" and "hmi.average.modes" in the
263 corresponding directories at http://sun.stanford.edu/~tplarson/audio/,
264 along with the corresponding msplit files. the stand alone patch
265 described in the next section is set to use these averaged mode
266 parameters by default, which means those averages will be used for all
267 day numbers. in this case, no further mode parameter files would need
268 to be downloaded.
269
270 next you will need to download some actual audio files. to do so click
271 on the wavfiles subdirectory, where you will find a selection of modes,
272 labelled by l and m. except for m=0, each mode has both a real and an
273 imaginary part, labelled by "datar" and "datai" respectively; make sure
274 tplarson 1.1 you always get both. these files contain exactly the same data as we
275 use for scientific analysis. the file formats have simply been changed
276 from fits (flexible image transport system) to wav. pick an assortment
277 of modes and download them to your data directory. of course you may
278 play these sounds files just as they are if you want to hear the
279 unfiltered data.
280
281
282 Section 4: Software (Pure Data)
283
284 now we are ready to dive into a description of the software by which the
285 scheme laid out in section 2 can be accomplished. one freely available
286 option is pure data, available from http://puredata.info . pure data
287 provides a graphical interface for audio processing. programs in pure
288 data are called "patches". once you have it installed, run the program
289 and the main Pd console will open. first you will want to test that it
290 is communicating with your soundcard. to do so, click on "Media" and
291 then "Test Audio and MIDI". this will open a patch called testtone.pd .
292 if you see changing numbers under audio input, you are connected to your
293 computer's microphone, although that is unneeded for this project. more
294 important is the audio out, which you can test by clicking in the boxes
295 tplarson 1.1 under "test tones".
296
297 once this is working, you are ready to use the SoSH tool. unzip the
298 archive and open the patch modefilter_standalone.pd . if you've never
299 looked at a pure data patch before, this will probably look rather
300 confusing, so i will provide an extremely brief introduction. there are
301 three types of boxes in pure data: messages, numbers, and objects.
302 messages are the boxes with an indentation along the right side, perhaps
303 to make the box look like a flag. messages are basically the equivalent
304 of strings, but they can also be automatically converted to numbers.
305 number boxes have a little notch out of the upper right corner. the
306 internal storage for numbers is floating point, but you can also cast to
307 int. an important difference between number boxes and message boxes is
308 that the contents of the latter can be saved. for instance, if one
309 wants to initialize a number, a common way is with a message. also,
310 numbers may be entered while the patch is running, whereas messages
311 cannot. the remaining rectangular boxes are objects, which are like
312 functions. the first element in an object box is the name of the
313 object, which often corresponds to a patch file (extension .pd) of the
314 same name. this is optionally followed by the object's creation
315 arguments. all three types of boxes have inlets on the top and outlets
316 tplarson 1.1 on the bottom.
317
318 another important concept in pure data is its own unique data type
319 called bang, which can be thought of like a mouse click. the message
320 "bang" will also automatically convert to a bang. bangs are used
321 throughout pure data as triggers for various events, or they can be used
322 to signal event detection as well. in the graphical interface, bangs
323 are represented by clickable circles, which we have enlarged and colored
324 green or light blue in our patch. you have probably noticed an object
325 called [loadbang]; its sole purpose is to output a single bang when the
326 patch loads. this is typically used for initialization: sending a bang
327 to a number or message box causes its contents to be output. you may
328 also notice a toggle, represented as an empty green square in our patch,
329 also clickable. this functions like a normal boolean, but it is not a
330 separate data type; it is simply 0 or 1. finally, arrays in pure data,
331 also called tables, come with their own graphical representation.
332 examples visible on the front of our patch are gain, input-r, input-i,
333 and output. (the $0 preceding these names resolves to a unique integer
334 when the patch loads; this becomes necessary when this patch is used as
335 a subpatch to avoid conflicting names. other dollar sign substitutions
336 are more like one would expect: they resolve to some element of the
337 tplarson 1.1 input, depending on the context.)
338
339 now, to use the patch, the first thing you have to do is make sure the
340 data directory is set properly. if you are using the data directory
341 that was unpacked along with the zip archive, you don't have to do
342 anything, since the patch is already to set to look in "../data".
343 otherwise, set the data directory by clicking the light blue bang at
344 lower left. a dialog box will open; just select any file in your data
345 directory and the object [set-directory] will strip the file name and
346 output the path. you should now see your path show up in the message
347 box at right. if you now save the patch file, this will be saved as
348 your default data directory and you won't need to set it any more.
349
350 by default, the patch is set to use MDI data. in particular, this means
351 that it assumes the data was taken at a sample rate of (1/60) hertz,
352 which in turn means that 72 days of data contain 103680 data points.
353 the patch will also use the string "mdi" as the stem for input file
354 names. if you are using HMI data, you may click the message box with
355 "hmi" at the lower left of the patch, and this will be used as the stem
356 for file names. the HMI sample rate of (1/45) hertz will also be used,
357 which means that 72 days of data would contain 138240 data points.
358 tplarson 1.1
359 the next step is to click on the message box with "pd dsp 1", which will
360 turn on digital signal processing. this doesn't need to be on to load
361 files or access arrays, but it does to do any fourier transforms or
362 playback. finally, the inputs you must provide are the day number
363 corresponding to the 72 day timeseries, the spherical harmonic degree l,
364 the radial order n, and the azimuthal order m. note that even if you
365 want to leave one of these at its default value of zero, you must still
366 click on the number box and enter "0". now, to search for this mode,
367 click the green bang at the upper left. the object [triggerlogic] will
368 then decide what to do. typically, the object [loadaudio] is triggered,
369 as you will see when the bang connected to it flashes. we have left
370 these large bangs throughout the patch to make the triggering visible,
371 but you may also use them to manually run the different parts
372 separately.
373
374 by default, the patch will look for an averaged mode parameter file. if
375 you have downloaded the mode parameters for a particular day number and
376 wish to use them, you must click on the message box with "%dd.modes" at
377 the bottom left of the patch before clicking the bang to start
378 processing. the "%d" will be replaced with the day number you have
379 tplarson 1.1 entered.
380
381 the object [loadaudio] searches for audio files such as the ones you
382 have just downloaded. note that the needed input files will depend only
383 on l and the absolute value of m, and that the modes have a real and an
384 imaginary part. the exception is m=0, which has a real part only. if
385 [loadaudio] is successfully able to load the necessary audio files into
386 the arrays input-r and input-i, it will automatically trigger the object
387 [fft-analyze]. this object will perform a fast fourier transform (fft)
388 of the input arrays, storing the result in two arrays called fft-r1 and
389 fft-i1. if you want to see these arrays, simply click on the object [pd
390 fft-arrays]. if you do so you will see an option to write them out to
391 wav files, in case you want to compare pure data's fft output to your
392 expectations. you will also see two unused arrays; these could be used
393 to store amplitude or phase information if such were desired.
394
395 by this time, if you are inquisitive enough, you might have noticed that
396 (for MDI) all of the arrays so far have a length of 131072, which is
397 2^17, rather than the actual number of samples, which is 103680. this
398 is because pure data requires the block size to be a power of 2 for its
399 fft algorithm. the result is effectively to zero pad the end of the
400 tplarson 1.1 timeseries. this has no effect on the frequency content of the sound,
401 and we truncate the output array at the original number of samples, so
402 it will play in the same amount of time as the original. (if you to set
403 block size to less than the number of samples, only this many are
404 output.) if using HMI data, the number of samples is 138240, and so a
405 block size of 2^18=262144 will be used.
406
407 (note: pure data in windows doesn't work with block sizes above 65536.
408 if you are running windows, you may have already seen pure data crash.
409 to avoid this, click on the message box containing "65536" before
410 turning on the dsp. the patch will function normally, but only the
411 first 65536 samples of each file will be used. you may also avoid the
412 crash by altering the source code and recompiling. a discussion of this
413 topic can be found at https://github.com/pure-data/pure-data/issues/68 .)
414
415 at this point, if the fft has been performed successfully, the object
416 [text-filer-reader] is triggered. this searches the text files you
417 downloaded earlier to find the mode parameters corresponding to the
418 numbers you entered. if it finds the mode, it ouputs, after its status
419 code, the mode's amplitude, width, and a measure of background noise.
420 if no mode is found, the status code triggers the message "no data
421 tplarson 1.1 found", and you should try another value of n. (also check the Pd
422 console for error messages.) the amplitude is wired to a number box for
423 your information. the width will be converted into units of bins and
424 then used as input to the object [makegain]. the noise parameter is
425 unused here. these three parameters will depend on l and n, but not m.
426 finally, the last outlet from [text-file-reader] gives the mode's
427 frequency, which does vary with m. the frequency is also converted to
428 bin number, and the [makegain] object is triggered. this function
429 creates the gain array, which is 1.0 in a frequency interval centered on
430 the mode frequency and of length 2 times the width, and 0.0 elsewhere.
431 if so desired, you may enter the parameter "width factor" to
432 multiplicatively increase the width of this interval. notice how a
433 message was used to initialize this number to one.
434
435 once the gain array is generated, then one of the [fft-resynth-???m]
436 objects will be triggered, depending on the sign of m. as mentioned
437 above, the two signs of m are extracted differently from the fourier
438 transform, but in both cases the fft is multiplied by the gain array and
439 then inverse transformed, the result being written into the output array
440 ($0-output). if you have entered a value for the downshift factor, the
441 fft will be shifted down by this amount before the inverse transform.
442 tplarson 1.1 note that we treat a value of zero as meaning no shift.
443
444 next, the output array is played back in a loop. the default sample
445 rate is 8000 hertz, but you may go up to 44100 hertz for the 103680
446 samples to play in only 2.4 seconds (in that case, if you haven't
447 applied any downshift factor, the mode will probably sound quite high).
448 you can change the sample rate by clicking on one of the nearby message
449 boxes, or by entering one manually. to hear the output, you will need
450 to enter the output level (volume). note that each loop is multiplied
451 by a window function, which consists of a 50 ms fade in/out at the
452 beginning/end of the loop. the length of the fade ramp may be adjusted
453 on the front of the patch in the lower left corner. the object
454 [window-gen] then calculates the window array. in section 6 we discuss
455 how and when you might turn windowing off.
456
457 at this point you may adjust the downshift factor, which will retrigger
458 the resynthesis, and the result should play immediately. you can turn
459 off playback by clicking the toggle. you may also elect to save the
460 output as a wav file file by clicking the light blue bang at lower
461 right. the instrument, day number, l, m, and n will be encoded in the
462 output file name.
463 tplarson 1.1
464 now, should you like to listen to another mode, you may enter its
465 "quantum" numbers l, n, and m, and then click on the green bang again.
466 if only n or the sign of m has changed, no new audio needs to be loaded,
467 and the object [text-file-reader] is triggered directly. the rest of
468 the processing chain follows as before. note that the names of output
469 wav files do not encode the sample rate, downshift factor, or width
470 factor. hence, if you want to save the same mode with different values
471 for these parameters, you will have to rename the output file or
472 manually edit the patch.
473
474
475 Section 5: Building Your Own Patches
476
477 once you have played with the patch for a while, you may become
478 interested in creating a pure data patch yourself. in what follows we
479 describe a short sequence of patches that we have created to illustrate
480 how this is done. the first of these is example1.pd . open this file
481 and click on the object [modefilter0] to see how we have converted the
482 modefilter_standalone.pd patch from above for use as a preliminary
483 subpatch in example1.pd . first, you will see that all of the
484 tplarson 1.1 initialization that we had in modefilter_standalone.pd has been moved to
485 the outer patch, including the object [window-gen]. don't forget to
486 reset the data directory if needed. second, you will see that we have
487 added inlets and outlets. the order in which these objects appear from
488 left to right in the patch determine the order the inlets and outlets
489 have in the outer patch. this is the reason you see [inlet] and
490 [outlet] objects sometimes placed far away from what they are connected
491 to. for inlets, we have chosen, from left to right, the following: a
492 bang to start the processing, the day number, degree l, radial order n,
493 azimuthal order m, width multiplication factor, and a toggle to turn
494 playback on and off. for outlets, we have chosen, from left to right,
495 the following: the output audio stream, the amplitude of the mode
496 determined by the fit, and the name of the output array for this
497 particular instance of [modefilter0]. by this time you have probably
498 noticed that some of the connections between objects are drawn as thin
499 lines while others are drawn bold. the difference is that thin lines
500 carry control information, while the thick lines carry signal data,
501 which is always processed at 44100 samples per second. furthermore, it
502 is conventional for objects that handle signal data to have names ending
503 in '~'.
504
505 tplarson 1.1 note here a distinction in how pure data uses subpatches. often, as you
506 have seen here, the subpatch is loaded from a file of the same name with
507 the .pd suffix. this type of subpatch is also called an abstraction.
508 however, subpatches may also be defined as part of the parent patch,
509 using the [pd ] object. here we have used this type of subpatch to hold
510 arrays that needn't be visible on the front of the patch, or to make a
511 patch more readable.
512
513 the inputs to [modefilter0] which must be given are the first five. in
514 examle1.pd we have left the width factor to take on its default value.
515 we have connected a toggle to the final inlet, and we have connected the
516 starting bang to this toggle so that everything will run with a single
517 click. you may want to delete this last connection if you will be
518 sonifying multiple modes, since the next time you click the bang it will
519 turn off the toggle. the right two outlets are now connected for
520 information only, but one might imagine using the amplitude to set the
521 volume the mode is played at, for example. the amplitude units are
522 arbitrary, but the values do accurately reflect the amplitude ratios
523 between the modes as measured on the sun. there are two parameters that
524 should be the same for all instances of [modefilter0]: the playback
525 sample rate and the frequency downshift factor. these two parameters
526 tplarson 1.1 are therefore set in the outer patch and broadcast using a [send] object
527 (abbreviated to [s ] in practice). inside [modefilter0] the broadcast
528 is received by the [receive] object (abbreviated [r ]). to hear the
529 output coming out of the left outlet, we must connect to the
530 digital-to-analog converter, represented by the object [dac~], just as
531 we previously did in modefilter_standalone.pd . (the object
532 [audio_safety~] is one provided with this project; its purpose is to
533 filter out corrupted data.)
534
535 you will also see that we have also connected the output audio stream to
536 the inlet of a [fiddle~] object. the documentation for this built-in
537 object can be viewed by right clicking on it and selecting "Help". in
538 short, it measures the pitch and amplitude of the stream on its inlet.
539 here, it tells us what tone we are actually generating at a particular
540 playback sample rate and after shifting down in frequency.
541
542 finally, the next step is to make a copy of the [modefilter0] object and
543 everything connected to it, which you can do in Edit mode by
544 highlighting the relevant boxes and selecting "Duplicate" from the edit
545 menu. move the new copy to wherever you would like to put it, and now
546 you can listen to two modes at once, turning them on and off with the
547 tplarson 1.1 toggles connected to the right inlets. it works to have two copies of
548 [audio-safety~] and [dac~], but common practice would be to have only
549 one, and connect all the left outlets of the [modefilter0] objects to
550 the same inlet of [audio-safety~]. it is one of the features of pure
551 data that it automatically sums audio signals at inlets. of course, one
552 should also adjust the respective volumes of the modes to avoid
553 clipping. you should end up with something like example2.pd, which has
554 two [modefilter0] objects, but you may add as many as you like.
555
556 aside from the second copy of [modefilter0], in example2.pd we have also
557 added a calculation of the total transposition factor. this illustrates
558 another important consideration to bear in mind, which is that a visual
559 programming language does not explicitly specify the order in which
560 operations are carried out. furthermore, the default behavior for
561 objects in pure data is for only their leftmost inlet to trigger the
562 output. we call this the hot inlet and the other inlets cold. the
563 canonical way to deal with this situation is with the [trigger] object
564 (abbreviated [t ]). as before, you can view its documentation by right
565 clicking on it and selecting "Help". basically this object distributes
566 its inlet to its outlets in right to left order, converting as specified
567 by its creations arguments. in the example shown in example2.pd, the
568 tplarson 1.1 downshift factor is sent first as a float to the cold inlet of the [/ ]
569 (division) object, and then a bang is sent to the hot inlet of the
570 [float] object (abbreviated [f ]). here, the built-in object [select]
571 (abbreviated [sel ]) is used to replace 0 with 1 and pass all other
572 numbers unchanged. the [float] object serves to store numbers and
573 output them when it receives a bang on its left inlet. the result is
574 that the division will be performed regardless of the order in which the
575 sample rate and downshift factor are specified. for more examples of
576 using the [trigger] object, see modefilter0.pd . note that the [float]
577 object is often not needed because most of pure data's mathematical
578 objects store the value of their left inlet automatically and reuse it
579 when a bang is received. on the other hand, if space allows, explicitly
580 using the [float] object can make code more readable.
581
582 (note: if you open example1.pd and example2.pd at the same time, you
583 will get error messages in the pure data console about the array window
584 being multiply defined. also, the first time you run any of the
585 example*.pd patches, you will have to set the data directory and save,
586 unless you are using the default data directory.)
587
588 keep in mind that if you want to see what is happening inside the
589 tplarson 1.1 [modefilter0] subpatch, you can click on it and interact with it
590 directly. you can view all the arrays involved, for instance, or change
591 the value of the width factor. you can also still save the output array
592 to a wav file as before. each copy of [modefilter0] in your patch
593 corresponds to a separate instance, and you can interact with each
594 instance separately, although this can be confusing if you have many
595 instances open at once.
596
597 if we want to have greater control over the synchronization of playback
598 between various modes, we will need to use the output arrays generated
599 by the various instances of [modefilter0]. as an example, we have
600 created example3.pd, wherein we use the array names with the [tabread]
601 object, which simply reads elements of an array. the other new object
602 here is [until], which outputs a given number of bangs, here equal to
603 the block size. (see the help for both these objects.)
604
605 once the two output arrays have been created by the two instances of
606 [modefilter0], you can simply click the new bang to create the sum of
607 each multiplied by its amplitude, which will be displayed in the array
608 sumtest. click the message box with "sumtest normalize" to scale the
609 new array so that its absolute value never exceeds one (necessary to
610 tplarson 1.1 avoid clipping). you may click the message box with "sumtest const 0"
611 to reset this array to zero.
612
613 perhaps you have noticed the absence of [trigger] objects in the new
614 code. this was done to make the code more readable, but this practice
615 should be avoided. pure data actually sends data along the connections
616 in the same order that you made them in time, although this is
617 invisible. as it happens, we made the connections in the correct order
618 for the code to work, but there is no way for one to tell the order by
619 looking at the patch.
620
621 we now move from "teaching" patches to real finalized patches. in
622 example_addition.pd we have moved the mode addition logic into a new
623 object [modeaddition] and correctly implemented the triggering. this
624 new object takes as a creation argument the name of the array for the
625 result. this array must be created on the parent patch first. we also
626 added the ability to write the result out as wav file. to listen to the
627 new array (be sure to normalize first!) we have put an [arbitrarySR]
628 object on the parent patch, previously used only inside [modefilter0].
629
630 you will also notice that we have replaced [modefilter0] with a new
631 tplarson 1.1 finalized version of the base patch, [modefilter], which has subsumed
632 the [fiddle~] object. we added two new outlets: one for the frequency
633 resulting from the mode parameter file (in units of microhertz) and one
634 for the frequency measured by the [fiddle~] object (in hertz). this
635 allows us to use the calculated transposition factor to compare the
636 input frequency (measured by fitting) to the output frequency (generated
637 by the patch). [modefilter] also sends a bang to outdone once the
638 resynthesis completes; this will be used in the next section.
639
640 now let us suppose that we would like to add an arbitrary number of
641 modes. one way to do so would be to modify [modeaddition] so that
642 rather than take two arrays and two amplitudes as input, it would take
643 only one of each and add the corresponding array to a running sum. we
644 could execute this object once for each array we want to add and then
645 normalize the result at the end. this is implemented in the object
646 [modesum], illustrated in example_sum.pd . because we are reading and
647 writing from the same array, we need an additional [trigger] object to
648 make sure this is done in the proper order. further, we need to
649 condition the array name somewhat, due to subtleties in the way pure
650 data handles symbols (essentially repeating what is done inside
651 [modefilter]).
652 tplarson 1.1
653 although there are too many overlapping connections to follow by eye, in
654 example_sum.pd we now have five instances of the [modefilter] subpatch,
655 and we have arranged them so that most of the inputs fall along the
656 right edge of the screen (the toggles are especially enlarged for ease
657 of use with a touchscreen). once you have the modes you want loaded,
658 you may add them up by clicking the corresponding bangs. the result is
659 placed in the array sumhold, which is now placed in the subpatch [pd
660 sumarray]. to listen to it, first normalize the array by clicking the
661 corresponding message (top middle of the patch) and then play it with
662 [arbitrarySR]. to create a new sum, first reset sumhold to zero. you
663 may also save to file in either the [modesum] or [pd sumarray]
664 subpatches, but notice the file name is a constant "modesum.wav", so
665 this file must be renamed if you want to save multiple sums.
666
667
668 Section 6: Extensions
669
670 various properties of the sun are known to change with an 11 year
671 period; this variation is known as the solar cycle. since we have 15
672 years of MDI data and 9 years of HMI data so far, we now have the
673 tplarson 1.1 opportunity to discover whether or not the effects of the solar cycle
674 might be audible. in order to do so, we would like to concatenate our
675 various output arrays together. for MDI, the available data span 76
676 72-day time intervals with 74 72-day timeseries (144 days of data are
677 missing). as of this writing, HMI has been operating for almost 9
678 years, or 44 contiguous 72-day timeseries so far. this is a significant
679 extension of our coverage of the solar cycle. further, one might
680 inquire as to whether there might be systematic differences between the
681 two instruments during the time of their overlap.
682
683 an example of how one might concatenate timeseries is shown in
684 example_concat.pd. here you will also notice two new objects: [modecat]
685 and [arbitrarySRmulti]. the first of these is quite simple: it takes
686 the array named on its right inlet and copies it into the array named as
687 a creation argument for the object, starting at the index given on its
688 middle inlet. for this index to be calculated properly, you must first
689 enter the day number of the first timeseries you will process. inside
690 [modecat] it is assumed that the target array is large enough, but we
691 have already ensured this in the outer patch. as usual, you may write
692 the resulting array to a file. to listen to it, the new object
693 [arbitrarySRmulti] takes as a middle inlet the total number of
694 tplarson 1.1 timeseries that have been concatenated. the same window array will be
695 applied to each segment. more complicated crossfading may be desired,
696 especially in the case of MDI, which has one of its timeseries offset
697 from the others. this is dealt with in the next example below.
698
699 we have also introduced the number of samples per day, sperday, which is
700 1440 for MDI and 1920 for HMI. in this connection, we also note that
701 for a given sample rate, HMI will require a different downshift factor
702 to achieve the same total transposition factor as we use for MDI, namely
703 only 3/4 as much. alternatively, if you want for the HMI timeseries to
704 play in the same amount of time as the MDI timeseries, use a different
705 sample rate and the same downshift factor.
706
707 another change that we have made is that by default this patch will look
708 for a different mode parameter file for each day number; this has both
709 advantages and disadvantages. an advantage of using the averaged mode
710 parameters is that the filtering of each timeseries will use exactly the
711 same frequency interval. the peak frequency may shift within this
712 interval, but the output sound would not be contaminated by the loss or
713 addition of frequency bins at the edge of the interval. the
714 disadvantage of using the averaged mode parameters is that we lose all
715 tplarson 1.1 information about how the amplitudes change with time. in any case, we
716 now give the width multiplication factor a default value of 1.2 to
717 somewhat mitigate the effect of varying widths.
718
719 of course if one is ambitious, then they could create new mode parameter
720 files where some parameters are averaged and some are not. or, by
721 interpolating mode parameters, one could attempt to use 36 day
722 timeseries.
723
724 finally, in example_concat.pd we have included an example of how to use
725 the built-in object [qlist]. its purpose is to act as a sequencer,
726 sending messages to specified [recieve] objects at certain times. it
727 reads this information from a text file. we have included two such
728 files, qlist.mdi and qlist.mdi, which list the available day numbers for
729 the two instruments. for testing, you may wish to use a subset of one
730 of these. for example, suppose we take the first 5 lines of qlist.hmi
731 and place them in a new file, qlist.hmitest. to use it, we would then
732 click the bang to set it as the sequence file. then we would click the
733 "hmi" message box and enter "6328" for the day number of the first
734 timeseries. then we can enter the desired mode's l, n, and m as usual,
735 and click the bang to start the sequence. before listening to the
736 tplarson 1.1 result, we would also enter "5" at the top of the patch for the number
737 of timeseries.
738
739 the text file read by [qlist] contains lines of the form "time
740 send_object message;". for exampe, the first line of qlist.hmi is "0
741 daynumber 6328;" which means at time zero send the message "6328" to
742 daynumber. the next line is "14000 daynumber 6400;" which means to wait
743 14 seconds and then send the message "6400" to daynumber. the 14
744 seconds is the time it takes for the fft's to run: 262144/44100 ~ 6
745 seconds to run through the block, which must happen twice, and we add an
746 additional 2 seconds for file loads and array processing. the remaining
747 lines of qlist.hmi are of this same form. as each new timeseries is
748 generated, the concatenation is triggered automatically when the
749 resynthesis completes, with the [s outdone] inside [modefilter]. once
750 the sequence is finished running, you may listen to the result using
751 [arbitrarySRmulti].
752
753 you may also use this patch to do the same thing with MDI. the file
754 qlist.mdi contains the sequence for all 74 MDI timeseries, but for the
755 number of timeseries you should enter "76". this will leave 144 days
756 worth of the target array at zero, which is as it should be for missing
757 tplarson 1.1 data. as before, make sure that the "mdi" message box has been clicked
758 and enter "1216" for the day number of the first timeseries. of course
759 you may edit qlist.mdi to make a shorter sequence; in that case adjust
760 the day number of the first timeseries and the total number
761 accordinglingly.
762
763 a close inspection of qlist.mdi will reveal the the timeseries beginning
764 on day number 2116 is offset from the others by 36 days. the 108 days
765 preceding and the 36 days following this timeseries are missing. hence,
766 if we use [arbitrarySRmulti] to listen to the full mission, the window
767 will not be applied properly to this one segment. often this is not a
768 problem, but it could be. further, we have as yet no mechanism to apply
769 any windowing to the files we write. if you listen to the concatenated
770 timeseries without windowing, you will hear clicks between some of them.
771
772 these concerns are addressed in example_concat2.pd. we have added a new
773 object, [applywindow], which simply multiplies the array specified on
774 its right inlet with the window and puts the result in the array named
775 as a creation argument. we have also generalized the previous patch so
776 that now one may add two modes together before concatenating, and each
777 sum will be multiplied by the window beforehand. we now count the
778 tplarson 1.1 number of bangs sent to outdone and trigger [modeaddition] every two of
779 them. [modeaddition] will trigger [applywindow] which will then trigger
780 [modecat]. at the end of the sequence the final array will be in
781 sumtest as before, which can be viewed by clicking [pd catarrary]. to
782 listen to the result, you will need to manually normalize sumtest.
783 also, make sure the toggle connected to [s window-on] is set to zero,
784 which it is when the patch loads. you may want to turn it on to listen
785 to individual modes.
786
787 a somewhat more sophisticated implementation of the same idea can be
788 found in example_sequencer.pd. as before, we have replaced
789 [modeaddition] with [modecat], but we now need only a single instance of
790 modefilter. another notable change from the previous example is that we
791 now use [qlist] in its other mode, where each line of the input file is
792 retrieved with a "next" message. the first "next" message will
793 immediately send every line that is not preceded by a number.
794 subsequent "next" messages will step through the rest of the lines. in
795 our implementation the numbers at the beginning of these lines are
796 unused, so we simply use "0".
797
798 we have connected new receive objects to specify the sample rate,
799 tplarson 1.1 downshift factor, and length of the output array. hence, the first
800 three lines of the qlist file could be
801
802 samprate 8000;
803 downshift 4;
804 numseries 10;
805
806 but one may still set these manually instead. the remaining global
807 parameters, such as the instrument, whether or not to use averaged mode
808 parameters, and the length of the ramp used to generate the window,
809 still must be set manually if values other than the default are
810 desired.
811
812 the remaining lines of the qlist file will take one of two forms.
813 first, one provides a line for every mode they wish to add together.
814 these lines define a combination of day number, l, n, and m. the whole
815 list is sent as daylnm, which is then parsed by the new object
816 [parsedaylnm]. for example, one such line could be
817
818 0 daylnm day 1216 l 1 n 18 m 1;
819
820 tplarson 1.1 which means to send the entire message "day 1216 l 1 n 18 m 1" to
821 daylnm. the elements of this message will be picked out two at a time
822 and used to send the 4 inputs needed. the first such line must specify
823 all 4 numbers, but subsequent lines need only specify changing values,
824 just as if you were using [modefilter] directly. the order is
825 unimportant. of course you may also manually enter any values that you
826 wish to remain constant before starting the sequence. when
827 [parsedaylnm] gets to the end of the message it received, it sends a
828 bang to startbang.
829
830 next comes a single line telling where to put this combination of modes
831 in the output array. for instance, the first such line would typically
832 be "0 dayindex 0;". then after a number of daylnm lines, the second
833 would typically be "0 dayindex 72;" and so on. however, one may specify
834 whatever positions they want, for example leaving silence in between the
835 segments, or even partially overwriting previously written segments.
836
837 when the sequence is finished, the [qlist] object sends a bang to
838 normalize the output array. you may listen to it using
839 [arbitrarySRmulti] as before. you may also write it to file inside the
840 [pd catarray] subpatch.
841 tplarson 1.1
842
843 Section 7: Conclusion
844
845 we hope that the level of detail presented here has allowed you to
846 effectively use the SoSH tool, and perhaps even given you some ideas
847 about new directions the project could take. contributions are welcome
848 from everyone, even if it is only the idea; do not hesitate to contact
849 us if you need help implementing it or simply think it would be a
850 valuable addition to the tool. new applications are likely to be added
851 to the project as time goes on, so check back with us if you want the
852 most recent version. if you have discovered combinations of solar tones
853 that you find pleasing, either aesthetically or scientifically, feel
854 free to share them with us, especially if you would like them to appear
855 in our online gallery.
856
857 contact the developers tim larson at tplarson@sun.stanford.edu or Seth
858 Shafer at sethshafer.com. the website for the Sonification of Solar
859 Harmonics Project is http://solar-center.stanford.edu/SoSH/ .
|
Return to
instructions_audio.txt
CVS log
Up to [Development] / JSOC / proj / globalhs / sosh