(file) Return to instructions_audio.txt CVS log (file) (dir) Up to [Development] / JSOC / proj / globalhs / sosh

  1 tplarson 1.1 The Sound of the Sun:
  2              An Introduction to the Sonification of Solar Harmonics (SoSH) Project
  4              a more extensive discussion of helioseismology with added graphics can 
  5              be found at http://solar-center.stanford.edu/SoSH/ .  short instructions 
  6              can be found in quickstart_audio.txt .
  8              Section 1: Background
  9                motivation for the project
 11              Section 2: Helioseismology and Sonificiation
 12                an introduction for beginners
 14              Section 3: Instruments and Data
 15                the data you will need and how to get it
 17              Section 4: Software (Pure Data)
 18                a detailed description of the software
 20              Section 5: Building Your Own Patches
 21                examples of applications 
 22 tplarson 1.1 
 23              Section 6: Extensions
 24                combining timeseries
 26              Section 7: Conclusion
 27                prospects for the future and contact info
 30              Section 1: Background
 32              the sun is a resonant cavity for very low frequency acoustic waves, and 
 33              just like a musical instrument, it supports a number of oscillation 
 34              modes, also commonly known has harmonics.  we are able to measure the 
 35              frequencies of these various harmonics by looking at how the sun's 
 36              surface oscillates in response to them.  then, just as the frequency of 
 37              a plucked guitar string gets higher with more tension and lower with 
 38              greater thickness, we are able to infer properties of the solar interior 
 39              such as its pressure and density.
 41              this study of acoustic waves inside the sun is called helioseismology; 
 42              an overview can be found at http://soi.stanford.edu/results/heliowhat.html .  
 43 tplarson 1.1 although it has allowed us to make measurements of unprecedented 
 44              precision, it remains largely unknown to the general public.  of course, 
 45              when one learns of it for the first time, a natural question arises: 
 46              "what does the sun sound like?".  and unfortunately even 
 47              helioseismologists rarely have much experiential knowledge of it.  that 
 48              is, we analyze solar data scientifically, but we never listen to it.  it 
 49              is the goal of this project to make such listening widely available.
 51              the first widely recognized effort toward the sonification of 
 52              helioseismic data was undertaken by Alexander Kosovichev, based on 
 53              earlier work by Douglas Gough (see http://soi.stanford.edu/results/ 
 54              sounds.html).  although only a small amount of data was sonified, 
 55              physical effects such as scattering off the solar core were still 
 56              audible.  with the sonification of a vastly larger dataset, one would 
 57              also be able to hear solar rotation, or perhaps even the effect of the 
 58              solar cycle.  not only does this bring helioseismology and solar physics 
 59              into the realm of everyday experience for the nonscientist, but it might 
 60              even allow for new scientific discoveries. the fact is that we simply 
 61              don't know what might be audible in the data because we have never 
 62              listened to it.
 64 tplarson 1.1 finally, we said above that the sun is like a musical instrument, but 
 65              one could also say that the sun *is* a musical instrument, in that it 
 66              has its own distinct set of harmonics.  of course we can't play the sun 
 67              in the sense of sounding individual notes; rather all of the solar notes 
 68              are playing all the time simultaneously.  with a little analysis, 
 69              however, the various solar tones can be separated from each other and 
 70              then used in musical composition.
 73              Section 2: Helioseismology and Sonification
 75              the strongest of the sun's harmonics have periods of about 5 minutes, 
 76              corresponding to frequencies of only about 0.003 hertz.  unfortunately, 
 77              this is far below the range of human hearing, which is typically taken 
 78              to be 20 - 20,000 hertz, although most people are only sensitive to a 
 79              smaller range.  hence, if we would like to experience the sound of the 
 80              sun with our ears, these very low sounds must be scaled to the range we 
 81              can hear.
 83              but first, we need some exposition of what a mode on the sun looks like.  
 84              to begin with, it is a mathematical theorem that any arbitrary shape of 
 85 tplarson 1.1 the sun's surface can be expressed as a sum over its harmonics (this is 
 86              also true for a guitar string).  in the case of the sun, we call them 
 87              spherical harmonics, and each of them are labelled by two integers: the 
 88              spherical harmonic degree l (ell) and the azimuthal order m.  The degree 
 89              l is equal to or greater than zero, and for each l, there are 2*l+1 values 
 90              of m, ranging from -l to l.
 92              the different spherical harmonics also sample different regions of the 
 93              sun.  low values of the degree l penetrate almost all the way to the 
 94              core, whereas higher values are trapped closer to the surface.  
 95              similarly, modes with high values of m have their maximum amplitude at 
 96              low latitudes, whereas lower values sample higher latitudes.  it is 
 97              because the different modes sample different regions that we are able to 
 98              use their frequencies to determine solar properties as a function of 
 99              both depth and latitude.
101              to determine the frequencies of the sun's harmonics, we might take an 
102              image once a minute for 72 days.  for each image, we decompose it into 
103              its various spherical harmonic components.  for each of these 
104              components, we form a timeseries of its amplitude.  from the timeseries 
105              we are able to construct the power spectrum (acoustic power as a 
106 tplarson 1.1 function of frequency).  the location of peaks in the power spectrum 
107              correspond to the frequencies of the modes (harmonics).  the height of 
108              the peak tells us the mode amplitude, and the width of the peak tells us 
109              how much the oscillation is damped.
111              an easy way to understand spherical harmonics is in terms of their node 
112              lines, which are the places on the sphere where the spherical harmonics 
113              are zero.  the degree l tells how many of these node lines there are in 
114              total, and the order m gives the number in longitude, so the number of 
115              node lines in latitude is l-m.  so a spherical harmonic with m=0 has 
116              only latitudinal bands, while one with m=l has only sections like an 
117              orange. a third integer, the radial order n, tells how many nodes the 
118              oscillation has along the sun's radius.  since only the surface of the 
119              sun is visible to us, all the values of n are present in each spherical 
120              harmonic labelled by l and m, although only some of them will be excited 
121              to any appreciable amplitude. The total mode, then, is represented as a 
122              product of a spherical harmonic, which is a function of latitude and 
123              longitude, and another function of radius. each n will have its own peak 
124              in the power spectrum.
126              now, in a spherically symmetric sun, all values of m for a given l and n 
127 tplarson 1.1 would have the same frequency.  a break in spherical symmetry causes the 
128              frequency to vary with m.  the most significant deviation from spherical 
129              symmetry in the sun is rotation about its axis.  the spherical harmonic 
130              decomposition, however, is only sensitive to the absolute value of m.  
131              therefore the positive and negative values of m must be separated in the 
132              power spectrum.  we say that the positive frequency part of the spectrum 
133              corresponds to negative m, and that the negative frequency part 
134              corresponds to positive m.  note that this particular convention for the 
135              sign of m is completely arbitrary.  if you want to understand what is 
136              meant by "the negative frequency part of the power spectrum", you will 
137              need to study the fourier transform, but such understanding is not 
138              strictly necessary.
140              let us now return to the issue of sonification, the conversion of data 
141              into audible sound.  the most straightforward way to do so would be to 
142              use the spherical harmonic timeseries we already have in hand and speed 
143              them up.  but by how much?  the answer of course is arbitrary and will 
144              depend on your preference, but as long as this choice is applied 
145              consistently you will still be able to hear the real relationship 
146              between different solar tones.
148 tplarson 1.1 let us suppose that we want to transpose a mode in the peak power range 
149              at about 0.003 hertz up to 300 hertz; this amounts to speeding up the 
150              timeseries by a factor of 100,000.  if we have 72 days of data taken 
151              once a minute, this amounts to 103,680 data points, or samples.  it's 
152              easy to see that the sped-up timeseries would now play in just over a 
153              minute.  one must also consider the sample rate, however, or the rate at 
154              which audio is played back.  speeding up the original sample rate of 
155              (1/60) hertz by a factor of 100,000 yields a new sample rate of 1666.67 
156              hertz, and one unfortunately finds very few audio players that will play 
157              any sample rate less than 8000 hertz.  assuming this sample rate, our 
158              0.003 hertz mode on the sun will now be transposed up to 1440 hertz and 
159              the timeseries will play in about 13 seconds.
161              but suppose you want to play it in a shorter time; 13 seconds is a long 
162              time to sound a single note, although you might want to do so in some 
163              circumstances.  you could increase the sample rate further still, but at 
164              some point the mode will be transposed to an uncomfortably high 
165              frequency.  to understand the solution to this problem, we must explore 
166              the process by which we shall isolate the modes.
168              at this point in our processing, playing an unfiltered timeseries would 
169 tplarson 1.1 sound just like static, or noise.  this is because very many modes are 
170              sounding simultaneously in any given timeseries, not to mention the 
171              background noise involved in our observation of the modes.  therefore, 
172              if we want to isolate a single mode, we have to do some filtering.  
173              luckily, as mentioned above, we have already measured the frequency, 
174              amplitude, and width of many modes.  we can use these fitted mode 
175              parameters to pick out the particular part of the power spectrum 
176              corresponding to a single mode, and set the rest of the power spectrum 
177              artificially to zero.  we then transform back into a function of time so 
178              that we can play the filtered data back as a timeseries.
180              since we are selecting only a narrow range of frequencies, we have the 
181              freedom to shift the entire power spectrum down in frequency before we 
182              transform back to timeseries.  this timeseries will play in the same 
183              amount of time as before, but the frequencies in it will be transposed 
184              down by the same factor that we shifted the power spectrum.  for every 
185              power of 2 shifted down, the tone will drop by one octave.
187              one approach might be to decide how long you want to sound each tone 
188              (keeping in mind that looping is also an option).  this will determine 
189              the sample rate at which you will play the timeseries.  then you can 
190 tplarson 1.1 choose a downshift factor to suite yourself.  as long as you use the 
191              same sample rate and downshift factor when you sonify every mode, the 
192              frequency relationships between them will be preserved.
195              Section 3: Data
197              in order to use the Sonification of Solar Harmonics (SoSH) tool, you 
198              will first need to download some data.  two types of data are required: 
199              text files containing ascii tables of fitted mode parameters and wav 
200              files containing the raw acoustic data.  furthermore, these data can 
201              originate from two different instruments: the Michelson Dopper Imager 
202              (MDI) and the Helioseismic and Magnetic Imager (HMI).  MDI is the older 
203              instrument, and took data from may 1996 to april 2011.  it was 
204              superceded by HMI, which began taking data in april 2010 and remains in 
205              operation today.  the two instruments are quite similar; the most 
206              important difference between them for our purpose is that MDI produced 
207              an image once a minute, while HMI produces an image every 45 seconds.  
208              for both instruments, however, we analyze the data using 72 day long 
209              timeseries.
211 tplarson 1.1 if you have also downloaded the SoshPy Visualization Package, then you 
212              may use its included python module to interactively retrieve whatever 
213              data you need.  instructions for this module are included in the 
214              package.  however, these data may also be downloaded from the web, and 
215              instructions for doing so follow.
217              first, pick a single directory for storing data on your computer; for 
218              your convenience we have included a data directory in the zip archive 
219              you have unpacked.  if you elect to use a different directory (such as 
220              your browser's download directory), you will simply need to enter it the 
221              first time you run the SoSH tool.  if you have downloaded the version of 
222              the SoSH tool that includes demo data files, these will already be found 
223              in the unpacked data directory.  the quickstart guide includes 
224              instructions for using those specific files.
226              in any case, the data are available at 
227              http://sun.stanford.edu/~tplarson/audio/ , where you will find separate 
228              directories for MDI and HMI.  within each, you will find a series of 
229              subdirectories that are day numbers suffixed with 'd'.  the day number 
230              corresponds to the first day of the 72 day timeseries and is actually 
231              the number of days since 1 january 1993.  day number 1216 was 1 may 
232 tplarson 1.1 1996.  a full table converting day numbers to dates can be found at the 
233              above url as well.
235              clicking on a directory will show two ascii files containing the mode 
236              parameters; download both of these to your data directory.  the file 
237              without "msplit" at the end contains one line for every degree l and 
238              (radial) order n for which the fitting succeeded.  the first five 
239              numbers of each line are all that we will use here; they are degree l, 
240              order n, mean frequency, amplitude, and width.  these numbers are the 
241              same for all m, the mean frequency being the average over m.  the file 
242              with "mpslit" at the end tells us only how the frequency varies with m.  
243              make sure that however you download them, the file names stay intact; 
244              some browsers like to add or delete filename extensions.  this is good 
245              to check if you get "file not found" errors later.
247 tplarson 1.2 UPDATE: the "msplit" file is no longer required by default.
249 tplarson 1.1 you need not view these files, but keep in mind that in general the 
250              fitting does not succeed for every value of n.  put another way, every 
251              file will have l values ranging exactly from 0 to 300, the upper limit 
252              being a somewhat arbitrary choice.  for every value of l, exactly 2*l+1 
253              values of m will appear in the msplit file.  what may vary widely, 
254              however, is which values of n appear for different values of l.  we 
255              typically only find modes with n=0 in timeseries with l>100.  modes with 
256              n=28 (higher values are rare) are only likely to be found for l=5-15.  
257              looked at from the other direction, for l=0 we typically find modes with 
258              n=10-25. for l=100, n=10 is the highest value we might find.  above 
259              l=200, we have only n=0.  in all cases, one may expect to find holes in 
260              the coverage of n for values of l close to the edge of the range.
262              one way to get much higher mode coverage, at the cost of time 
263              resolution, is to perform an average.  an example of such an average can 
264              be found in the files "mdi.average.modes" and "hmi.average.modes" in the 
265              corresponding directories at http://sun.stanford.edu/~tplarson/audio/, 
266              along with the corresponding msplit files.  the stand alone patch 
267              described in the next section is set to use these averaged mode 
268              parameters by default, which means those averages will be used for all 
269              day numbers.  in this case, no further mode parameter files would need 
270 tplarson 1.1 to be downloaded.
272              next you will need to download some actual audio files.  to do so click 
273              on the wavfiles subdirectory, where you will find a selection of modes, 
274              labelled by l and m.  except for m=0, each mode has both a real and an 
275              imaginary part, labelled by "datar" and "datai" respectively; make sure 
276              you always get both.  these files contain exactly the same data as we 
277              use for scientific analysis.  the file formats have simply been changed 
278              from fits (flexible image transport system) to wav.  pick an assortment 
279              of modes and download them to your data directory.  of course you may 
280              play these sounds files just as they are if you want to hear the 
281              unfiltered data.
284              Section 4: Software (Pure Data)
286              now we are ready to dive into a description of the software by which the 
287              scheme laid out in section 2 can be accomplished.  one freely available 
288              option is pure data, available from http://puredata.info .  pure data 
289              provides a graphical interface for audio processing.  programs in pure 
290              data are called "patches".  once you have it installed, run the program 
291 tplarson 1.1 and the main Pd console will open.  first you will want to test that it 
292              is communicating with your soundcard.  to do so, click on "Media" and 
293              then "Test Audio and MIDI".  this will open a patch called testtone.pd .  
294              if you see changing numbers under audio input, you are connected to your 
295              computer's microphone, although that is unneeded for this project.  more 
296              important is the audio out, which you can test by clicking in the boxes 
297              under "test tones".
299              once this is working, you are ready to use the SoSH tool. unzip the 
300              archive and open the patch modefilter_standalone.pd .  if you've never 
301              looked at a pure data patch before, this will probably look rather 
302              confusing, so i will provide an extremely brief introduction.  there are 
303              three types of boxes in pure data: messages, numbers, and objects.  
304              messages are the boxes with an indentation along the right side, perhaps 
305              to make the box look like a flag.  messages are basically the equivalent 
306              of strings, but they can also be automatically converted to numbers.  
307              number boxes have a little notch out of the upper right corner.  the 
308              internal storage for numbers is floating point, but you can also cast to 
309              int.  an important difference between number boxes and message boxes is 
310              that the contents of the latter can be saved.  for instance, if one 
311              wants to initialize a number, a common way is with a message.  also, 
312 tplarson 1.1 numbers may be entered while the patch is running, whereas messages 
313              cannot.  the remaining rectangular boxes are objects, which are like 
314              functions.  the first element in an object box is the name of the 
315              object, which often corresponds to a patch file (extension .pd) of the 
316              same name.  this is optionally followed by the object's creation 
317              arguments.  all three types of boxes have inlets on the top and outlets 
318              on the bottom.
320              another important concept in pure data is its own unique data type 
321              called bang, which can be thought of like a mouse click.  the message 
322              "bang" will also automatically convert to a bang.  bangs are used 
323              throughout pure data as triggers for various events, or they can be used 
324              to signal event detection as well.  in the graphical interface, bangs 
325              are represented by clickable circles, which we have enlarged and colored 
326              green or light blue in our patch.  you have probably noticed an object 
327              called [loadbang]; its sole purpose is to output a single bang when the 
328              patch loads.  this is typically used for initialization: sending a bang 
329              to a number or message box causes its contents to be output.  you may 
330              also notice a toggle, represented as an empty green square in our patch, 
331              also clickable.  this functions like a normal boolean, but it is not a 
332              separate data type; it is simply 0 or 1.  finally, arrays in pure data, 
333 tplarson 1.1 also called tables, come with their own graphical representation.  
334              examples visible on the front of our patch are gain, input-r, input-i, 
335              and output.  (the $0 preceding these names resolves to a unique integer 
336              when the patch loads; this becomes necessary when this patch is used as 
337              a subpatch to avoid conflicting names.  other dollar sign substitutions 
338              are more like one would expect: they resolve to some element of the 
339              input, depending on the context.)
341              now, to use the patch, the first thing you have to do is make sure the 
342              data directory is set properly.  if you are using the data directory 
343              that was unpacked along with the zip archive, you don't have to do 
344              anything, since the patch is already to set to look in "../data". 
345              otherwise, set the data directory by clicking the light blue bang at 
346              lower left.  a dialog box will open; just select any file in your data 
347              directory and the object [set-directory] will strip the file name and 
348              output the path.  you should now see your path show up in the message 
349              box at right.  if you now save the patch file, this will be saved as 
350              your default data directory and you won't need to set it any more.
352              by default, the patch is set to use MDI data.  in particular, this means 
353              that it assumes the data was taken at a sample rate of (1/60) hertz, 
354 tplarson 1.1 which in turn means that 72 days of data contain 103680 data points.  
355              the patch will also use the string "mdi" as the stem for input file 
356              names.  if you are using HMI data, you may click the message box with 
357              "hmi" at the lower left of the patch, and this will be used as the stem 
358              for file names.  the HMI sample rate of (1/45) hertz will also be used, 
359              which means that 72 days of data would contain 138240 data points.
361              the next step is to click on the message box with "pd dsp 1", which will 
362              turn on digital signal processing.  this doesn't need to be on to load 
363              files or access arrays, but it does to do any fourier transforms or 
364              playback.  finally, the inputs you must provide are the day number 
365              corresponding to the 72 day timeseries, the spherical harmonic degree l, 
366              the radial order n, and the azimuthal order m.  note that even if you 
367              want to leave one of these at its default value of zero, you must still 
368              click on the number box and enter "0".  now, to search for this mode, 
369              click the green bang at the upper left.  the object [triggerlogic] will 
370              then decide what to do.  typically, the object [loadaudio] is triggered, 
371              as you will see when the bang connected to it flashes.  we have left 
372              these large bangs throughout the patch to make the triggering visible, 
373              but you may also use them to manually run the different parts 
374              separately.
375 tplarson 1.1 
376              by default, the patch will look for an averaged mode parameter file.  if 
377              you have downloaded the mode parameters for a particular day number and 
378              wish to use them, you must click on the message box with "%dd.modes" at 
379              the bottom left of the patch before clicking the bang to start 
380              processing.  the "%d" will be replaced with the day number you have 
381              entered.
383              the object [loadaudio] searches for audio files such as the ones you 
384              have just downloaded.  note that the needed input files will depend only 
385              on l and the absolute value of m, and that the modes have a real and an 
386              imaginary part.  the exception is m=0, which has a real part only.  if 
387              [loadaudio] is successfully able to load the necessary audio files into 
388              the arrays input-r and input-i, it will automatically trigger the object 
389              [fft-analyze].  this object will perform a fast fourier transform (fft) 
390              of the input arrays, storing the result in two arrays called fft-r1 and 
391              fft-i1.  if you want to see these arrays, simply click on the object [pd 
392              fft-arrays].  if you do so you will see an option to write them out to 
393              wav files, in case you want to compare pure data's fft output to your 
394              expectations.  you will also see two unused arrays; these could be used 
395              to store amplitude or phase information if such were desired.
396 tplarson 1.1 
397              by this time, if you are inquisitive enough, you might have noticed that 
398              (for MDI) all of the arrays so far have a length of 131072, which is 
399              2^17, rather than the actual number of samples, which is 103680.  this 
400              is because pure data requires the block size to be a power of 2 for its 
401              fft algorithm.  the result is effectively to zero pad the end of the 
402              timeseries.  this has no effect on the frequency content of the sound, 
403              and we truncate the output array at the original number of samples, so 
404              it will play in the same amount of time as the original.  (if you to set 
405              block size to less than the number of samples, only this many are 
406              output.)  if using HMI data, the number of samples is 138240, and so a 
407              block size of 2^18=262144 will be used.
409              (note: pure data in windows doesn't work with block sizes above 65536.  
410              if you are running windows, you may have already seen pure data crash.  
411              to avoid this, click on the message box containing "65536" before 
412              turning on the dsp.  the patch will function normally, but only the 
413              first 65536 samples of each file will be used.  you may also avoid the 
414              crash by altering the source code and recompiling.  a discussion of this 
415              topic can be found at https://github.com/pure-data/pure-data/issues/68 .)
417 tplarson 1.1 at this point, if the fft has been performed successfully, the object 
418              [text-filer-reader] is triggered.  this searches the text files you 
419              downloaded earlier to find the mode parameters corresponding to the 
420              numbers you entered.  if it finds the mode, it ouputs, after its status 
421              code, the mode's amplitude, width, and a measure of background noise.  
422              if no mode is found, the status code triggers the message "no data 
423              found", and you should try another value of n.  (also check the Pd 
424              console for error messages.)  the amplitude is wired to a number box for 
425              your information.  the width will be converted into units of bins and 
426              then used as input to the object [makegain].  the noise parameter is 
427              unused here.  these three parameters will depend on l and n, but not m.  
428              finally, the last outlet from [text-file-reader] gives the mode's 
429              frequency, which does vary with m.  the frequency is also converted to 
430              bin number, and the [makegain] object is triggered.  this function 
431              creates the gain array, which is 1.0 in a frequency interval centered on 
432              the mode frequency and of length 2 times the width, and 0.0 elsewhere.  
433              if so desired, you may enter the parameter "width factor" to 
434              multiplicatively increase the width of this interval.  notice how a 
435              message was used to initialize this number to one.
437              once the gain array is generated, then one of the [fft-resynth-???m] 
438 tplarson 1.1 objects will be triggered, depending on the sign of m.  as mentioned 
439              above, the two signs of m are extracted differently from the fourier 
440              transform, but in both cases the fft is multiplied by the gain array and 
441              then inverse transformed, the result being written into the output array 
442              ($0-output).  if you have entered a value for the downshift factor, the 
443              fft will be shifted down by this amount before the inverse transform.  
444              note that we treat a value of zero as meaning no shift.
446              next, the output array is played back in a loop.  the default sample 
447              rate is 8000 hertz, but you may go up to 44100 hertz for the 103680 
448              samples to play in only 2.4 seconds (in that case, if you haven't 
449              applied any downshift factor, the mode will probably sound quite high).  
450              you can change the sample rate by clicking on one of the nearby message 
451              boxes, or by entering one manually.  to hear the output, you will need 
452              to enter the output level (volume).  note that each loop is multiplied 
453              by a window function, which consists of a 50 ms fade in/out at the 
454              beginning/end of the loop.  the length of the fade ramp may be adjusted 
455              on the front of the patch in the lower left corner.  the object 
456              [window-gen] then calculates the window array.  in section 6 we discuss 
457              how and when you might turn windowing off.
459 tplarson 1.1 at this point you may adjust the downshift factor, which will retrigger 
460              the resynthesis, and the result should play immediately.  you can turn 
461              off playback by clicking the toggle.  you may also elect to save the 
462              output as a wav file file by clicking the light blue bang at lower 
463              right.  the instrument, day number, l, m, and n will be encoded in the 
464              output file name.
466              now, should you like to listen to another mode, you may enter its 
467              "quantum" numbers l, n, and m, and then click on the green bang again.  
468              if only n or the sign of m has changed, no new audio needs to be loaded, 
469              and the object [text-file-reader] is triggered directly.  the rest of 
470              the processing chain follows as before.  note that the names of output 
471              wav files do not encode the sample rate, downshift factor, or width 
472              factor. hence, if you want to save the same mode with different values 
473              for these parameters, you will have to rename the output file or 
474              manually edit the patch.
477              Section 5: Building Your Own Patches
479              once you have played with the patch for a while, you may become 
480 tplarson 1.1 interested in creating a pure data patch yourself.  in what follows we 
481              describe a short sequence of patches that we have created to illustrate 
482              how this is done.  the first of these is example1.pd .  open this file 
483              and click on the object [modefilter0] to see how we have converted the 
484              modefilter_standalone.pd patch from above for use as a preliminary 
485              subpatch in example1.pd .  first, you will see that all of the 
486              initialization that we had in modefilter_standalone.pd has been moved to 
487              the outer patch, including the object [window-gen].  don't forget to 
488              reset the data directory if needed.  second, you will see that we have 
489              added inlets and outlets.  the order in which these objects appear from 
490              left to right in the patch determine the order the inlets and outlets 
491              have in the outer patch.  this is the reason you see [inlet] and 
492              [outlet] objects sometimes placed far away from what they are connected 
493              to.  for inlets, we have chosen, from left to right, the following: a 
494              bang to start the processing, the day number, degree l, radial order n, 
495              azimuthal order m, width multiplication factor, and a toggle to turn 
496              playback on and off.  for outlets, we have chosen, from left to right, 
497              the following: the output audio stream, the amplitude of the mode 
498              determined by the fit, and the name of the output array for this 
499              particular instance of [modefilter0].  by this time you have probably 
500              noticed that some of the connections between objects are drawn as thin 
501 tplarson 1.1 lines while others are drawn bold.  the difference is that thin lines 
502              carry control information, while the thick lines carry signal data, 
503              which is always processed at 44100 samples per second.  furthermore, it 
504              is conventional for objects that handle signal data to have names ending 
505              in '~'.
507              note here a distinction in how pure data uses subpatches.  often, as you 
508              have seen here, the subpatch is loaded from a file of the same name with 
509              the .pd suffix.  this type of subpatch is also called an abstraction.  
510              however, subpatches may also be defined as part of the parent patch, 
511              using the [pd ] object.  here we have used this type of subpatch to hold 
512              arrays that needn't be visible on the front of the patch, or to make a 
513              patch more readable.
515              the inputs to [modefilter0] which must be given are the first five.  in 
516              examle1.pd we have left the width factor to take on its default value.  
517              we have connected a toggle to the final inlet, and we have connected the 
518              starting bang to this toggle so that everything will run with a single 
519              click.  you may want to delete this last connection if you will be 
520              sonifying multiple modes, since the next time you click the bang it will 
521              turn off the toggle.  the right two outlets are now connected for 
522 tplarson 1.1 information only, but one might imagine using the amplitude to set the 
523              volume the mode is played at, for example.  the amplitude units are 
524              arbitrary, but the values do accurately reflect the amplitude ratios 
525              between the modes as measured on the sun.  there are two parameters that 
526              should be the same for all instances of [modefilter0]: the playback 
527              sample rate and the frequency downshift factor.  these two parameters 
528              are therefore set in the outer patch and broadcast using a [send] object 
529              (abbreviated to [s ] in practice).  inside [modefilter0] the broadcast 
530              is received by the [receive] object (abbreviated [r ]).  to hear the 
531              output coming out of the left outlet, we must connect to the 
532              digital-to-analog converter, represented by the object [dac~], just as 
533              we previously did in modefilter_standalone.pd .  (the object 
534              [audio_safety~] is one provided with this project; its purpose is to 
535              filter out corrupted data.)
537              you will also see that we have also connected the output audio stream to 
538              the inlet of a [fiddle~] object.  the documentation for this built-in 
539              object can be viewed by right clicking on it and selecting "Help".  in 
540              short, it measures the pitch and amplitude of the stream on its inlet.  
541              here, it tells us what tone we are actually generating at a particular 
542              playback sample rate and after shifting down in frequency.
543 tplarson 1.1 
544              finally, the next step is to make a copy of the [modefilter0] object and 
545              everything connected to it, which you can do in Edit mode by 
546              highlighting the relevant boxes and selecting "Duplicate" from the edit 
547              menu.  move the new copy to wherever you would like to put it, and now 
548              you can listen to two modes at once, turning them on and off with the 
549              toggles connected to the right inlets.  it works to have two copies of 
550              [audio-safety~] and [dac~], but common practice would be to have only 
551              one, and connect all the left outlets of the [modefilter0] objects to 
552              the same inlet of [audio-safety~].  it is one of the features of pure 
553              data that it automatically sums audio signals at inlets.  of course, one 
554              should also adjust the respective volumes of the modes to avoid 
555              clipping.  you should end up with something like example2.pd, which has 
556              two [modefilter0] objects, but you may add as many as you like.
558              aside from the second copy of [modefilter0], in example2.pd we have also 
559              added a calculation of the total transposition factor.  this illustrates 
560              another important consideration to bear in mind, which is that a visual 
561              programming language does not explicitly specify the order in which 
562              operations are carried out.  furthermore, the default behavior for 
563              objects in pure data is for only their leftmost inlet to trigger the 
564 tplarson 1.1 output.  we call this the hot inlet and the other inlets cold.  the 
565              canonical way to deal with this situation is with the [trigger] object 
566              (abbreviated [t ]).  as before, you can view its documentation by right 
567              clicking on it and selecting "Help".  basically this object distributes 
568              its inlet to its outlets in right to left order, converting as specified 
569              by its creations arguments.  in the example shown in example2.pd, the 
570              downshift factor is sent first as a float to the cold inlet of the [/ ] 
571              (division) object, and then a bang is sent to the hot inlet of the 
572              [float] object (abbreviated [f ]).  here, the built-in object [select] 
573              (abbreviated [sel ]) is used to replace 0 with 1 and pass all other 
574              numbers unchanged.  the [float] object serves to store numbers and 
575              output them when it receives a bang on its left inlet.  the result is 
576              that the division will be performed regardless of the order in which the 
577              sample rate and downshift factor are specified.  for more examples of 
578              using the [trigger] object, see modefilter0.pd .  note that the [float] 
579              object is often not needed because most of pure data's mathematical 
580              objects store the value of their left inlet automatically and reuse it 
581              when a bang is received.  on the other hand, if space allows, explicitly 
582              using the [float] object can make code more readable.
584              (note: if you open example1.pd and example2.pd at the same time, you 
585 tplarson 1.1 will get error messages in the pure data console about the array window 
586              being multiply defined.  also, the first time you run any of the 
587              example*.pd patches, you will have to set the data directory and save, 
588              unless you are using the default data directory.)
590              keep in mind that if you want to see what is happening inside the 
591              [modefilter0] subpatch, you can click on it and interact with it 
592              directly.  you can view all the arrays involved, for instance, or change 
593              the value of the width factor.  you can also still save the output array 
594              to a wav file as before.  each copy of [modefilter0] in your patch 
595              corresponds to a separate instance, and you can interact with each 
596              instance separately, although this can be confusing if you have many 
597              instances open at once.
599              if we want to have greater control over the synchronization of playback 
600              between various modes, we will need to use the output arrays generated 
601              by the various instances of [modefilter0].  as an example, we have 
602              created example3.pd, wherein we use the array names with the [tabread] 
603              object, which simply reads elements of an array.  the other new object 
604              here is [until], which outputs a given number of bangs, here equal to 
605              the block size. (see the help for both these objects.)
606 tplarson 1.1 
607              once the two output arrays have been created by the two instances of 
608              [modefilter0], you can simply click the new bang to create the sum of 
609              each multiplied by its amplitude, which will be displayed in the array 
610              sumtest.  click the message box with "sumtest normalize" to scale the 
611              new array so that its absolute value never exceeds one (necessary to 
612              avoid clipping).  you may click the message box with "sumtest const 0" 
613              to reset this array to zero.
615              perhaps you have noticed the absence of [trigger] objects in the new 
616              code.  this was done to make the code more readable, but this practice 
617              should be avoided.  pure data actually sends data along the connections 
618              in the same order that you made them in time, although this is 
619              invisible.  as it happens, we made the connections in the correct order 
620              for the code to work, but there is no way for one to tell the order by 
621              looking at the patch.
623              we now move from "teaching" patches to real finalized patches. in 
624              example_addition.pd we have moved the mode addition logic into a new 
625              object [modeaddition] and correctly implemented the triggering.  this 
626              new object takes as a creation argument the name of the array for the 
627 tplarson 1.1 result. this array must be created on the parent patch first.  we also 
628              added the ability to write the result out as wav file.  to listen to the 
629              new array (be sure to normalize first!) we have put an [arbitrarySR] 
630              object on the parent patch, previously used only inside [modefilter0].
632              you will also notice that we have replaced [modefilter0] with a new 
633              finalized version of the base patch, [modefilter], which has subsumed 
634              the [fiddle~] object.  we added two new outlets: one for the frequency 
635              resulting from the mode parameter file (in units of microhertz) and one 
636              for the frequency measured by the [fiddle~] object (in hertz).  this 
637              allows us to use the calculated transposition factor to compare the 
638              input frequency (measured by fitting) to the output frequency (generated 
639              by the patch).  [modefilter] also sends a bang to outdone once the 
640              resynthesis completes; this will be used in the next section.
642              now let us suppose that we would like to add an arbitrary number of 
643              modes.  one way to do so would be to modify [modeaddition] so that 
644              rather than take two arrays and two amplitudes as input, it would take 
645              only one of each and add the corresponding array to a running sum.  we 
646              could execute this object once for each array we want to add and then 
647              normalize the result at the end.  this is implemented in the object 
648 tplarson 1.1 [modesum], illustrated in example_sum.pd .  because we are reading and 
649              writing from the same array, we need an additional [trigger] object to 
650              make sure this is done in the proper order.  further, we need to 
651              condition the array name somewhat, due to subtleties in the way pure 
652              data handles symbols (essentially repeating what is done inside 
653              [modefilter]).
655              although there are too many overlapping connections to follow by eye, in 
656              example_sum.pd we now have five instances of the [modefilter] subpatch, 
657              and we have arranged them so that most of the inputs fall along the 
658              right edge of the screen (the toggles are especially enlarged for ease 
659              of use with a touchscreen).  once you have the modes you want loaded, 
660              you may add them up by clicking the corresponding bangs.  the result is 
661              placed in the array sumhold, which is now placed in the subpatch [pd 
662              sumarray].  to listen to it, first normalize the array by clicking the 
663              corresponding message (top middle of the patch) and then play it with 
664              [arbitrarySR].  to create a new sum, first reset sumhold to zero.  you 
665              may also save to file in either the [modesum] or [pd sumarray] 
666              subpatches, but notice the file name is a constant "modesum.wav", so 
667              this file must be renamed if you want to save multiple sums.
669 tplarson 1.1 
670              Section 6: Extensions
672              various properties of the sun are known to change with an 11 year 
673              period; this variation is known as the solar cycle. since we have 15 
674              years of MDI data and 9 years of HMI data so far, we now have the 
675              opportunity to discover whether or not the effects of the solar cycle 
676              might be audible.  in order to do so, we would like to concatenate our 
677              various output arrays together.  for MDI, the available data span 76 
678              72-day time intervals with 74 72-day timeseries (144 days of data are 
679              missing).  as of this writing, HMI has been operating for almost 9 
680              years, or 44 contiguous 72-day timeseries so far.  this is a significant 
681              extension of our coverage of the solar cycle.  further, one might 
682              inquire as to whether there might be systematic differences between the 
683              two instruments during the time of their overlap.
685              an example of how one might concatenate timeseries is shown in 
686              example_concat.pd.  here you will also notice two new objects: [modecat] 
687              and [arbitrarySRmulti].  the first of these is quite simple: it takes 
688              the array named on its right inlet and copies it into the array named as 
689              a creation argument for the object, starting at the index given on its 
690 tplarson 1.1 middle inlet.  for this index to be calculated properly, you must first 
691              enter the day number of the first timeseries you will process.  inside 
692              [modecat] it is assumed that the target array is large enough, but we 
693              have already ensured this in the outer patch.  as usual, you may write 
694              the resulting array to a file.  to listen to it, the new object 
695              [arbitrarySRmulti] takes as a middle inlet the total number of 
696              timeseries that have been concatenated.  the same window array will be 
697              applied to each segment.  more complicated crossfading may be desired, 
698              especially in the case of MDI, which has one of its timeseries offset 
699              from the others.  this is dealt with in the next example below.
701              we have also introduced the number of samples per day, sperday, which is 
702              1440 for MDI and 1920 for HMI.  in this connection, we also note that 
703              for a given sample rate, HMI will require a different downshift factor 
704              to achieve the same total transposition factor as we use for MDI, namely 
705              only 3/4 as much.  alternatively, if you want for the HMI timeseries to 
706              play in the same amount of time as the MDI timeseries, use a different 
707              sample rate and the same downshift factor.
709              another change that we have made is that by default this patch will look 
710              for a different mode parameter file for each day number; this has both 
711 tplarson 1.1 advantages and disadvantages.  an advantage of using the averaged mode 
712              parameters is that the filtering of each timeseries will use exactly the 
713              same frequency interval.  the peak frequency may shift within this 
714              interval, but the output sound would not be contaminated by the loss or 
715              addition of frequency bins at the edge of the interval.  the 
716              disadvantage of using the averaged mode parameters is that we lose all 
717              information about how the amplitudes change with time.  in any case, we 
718              now give the width multiplication factor a default value of 1.2 to 
719              somewhat mitigate the effect of varying widths.
721              of course if one is ambitious, then they could create new mode parameter 
722              files where some parameters are averaged and some are not.  or, by 
723              interpolating mode parameters, one could attempt to use 36 day 
724              timeseries.
726              finally, in example_concat.pd we have included an example of how to use 
727              the built-in object [qlist].  its purpose is to act as a sequencer, 
728              sending messages to specified [recieve] objects at certain times.  it 
729              reads this information from a text file.  we have included two such 
730              files, qlist.mdi and qlist.mdi, which list the available day numbers for 
731              the two instruments.  for testing, you may wish to use a subset of one 
732 tplarson 1.1 of these.  for example, suppose we take the first 5 lines of qlist.hmi 
733              and place them in a new file, qlist.hmitest.  to use it, we would then 
734              click the bang to set it as the sequence file.  then we would click the 
735              "hmi" message box and enter "6328" for the day number of the first 
736              timeseries.  then we can enter the desired mode's l, n, and m as usual, 
737              and click the bang to start the sequence.  before listening to the 
738              result, we would also enter "5" at the top of the patch for the number 
739              of timeseries.
741              the text file read by [qlist] contains lines of the form "time 
742              send_object message;". for exampe, the first line of qlist.hmi is "0 
743              daynumber 6328;" which means at time zero send the message "6328" to 
744              daynumber.  the next line is "14000 daynumber 6400;" which means to wait 
745              14 seconds and then send the message "6400" to daynumber.  the 14 
746              seconds is the time it takes for the fft's to run: 262144/44100 ~ 6 
747              seconds to run through the block, which must happen twice, and we add an 
748              additional 2 seconds for file loads and array processing.  the remaining 
749              lines of qlist.hmi are of this same form.  as each new timeseries is 
750              generated, the concatenation is triggered automatically when the 
751              resynthesis completes, with the [s outdone] inside [modefilter]. once 
752              the sequence is finished running, you may listen to the result using 
753 tplarson 1.1 [arbitrarySRmulti].
755              you may also use this patch to do the same thing with MDI.  the file 
756              qlist.mdi contains the sequence for all 74 MDI timeseries, but for the 
757              number of timeseries you should enter "76".  this will leave 144 days 
758              worth of the target array at zero, which is as it should be for missing 
759              data.  as before, make sure that the "mdi" message box has been clicked 
760              and enter "1216" for the day number of the first timeseries.  of course 
761              you may edit qlist.mdi to make a shorter sequence; in that case adjust 
762              the day number of the first timeseries and the total number 
763              accordinglingly.
765              a close inspection of qlist.mdi will reveal the the timeseries beginning 
766              on day number 2116 is offset from the others by 36 days.  the 108 days 
767              preceding and the 36 days following this timeseries are missing.  hence, 
768              if we use [arbitrarySRmulti] to listen to the full mission, the window 
769              will not be applied properly to this one segment.  often this is not a 
770              problem, but it could be.  further, we have as yet no mechanism to apply 
771              any windowing to the files we write.  if you listen to the concatenated 
772              timeseries without windowing, you will hear clicks between some of them.
774 tplarson 1.1 these concerns are addressed in example_concat2.pd.  we have added a new 
775              object, [applywindow], which simply multiplies the array specified on 
776              its right inlet with the window and puts the result in the array named 
777              as a creation argument.  we have also generalized the previous patch so 
778              that now one may add two modes together before concatenating, and each 
779              sum will be multiplied by the window beforehand.  we now count the 
780              number of bangs sent to outdone and trigger [modeaddition] every two of 
781              them. [modeaddition] will trigger [applywindow] which will then trigger 
782              [modecat].  at the end of the sequence the final array will be in 
783              sumtest as before, which can be viewed by clicking [pd catarrary].  to 
784              listen to the result, you will need to manually normalize sumtest.  
785              also, make sure the toggle connected to [s window-on] is set to zero, 
786              which it is when the patch loads.  you may want to turn it on to listen 
787              to individual modes.
789              a somewhat more sophisticated implementation of the same idea can be 
790              found in example_sequencer.pd.  as before, we have replaced 
791              [modeaddition] with [modecat], but we now need only a single instance of 
792              modefilter.  another notable change from the previous example is that we 
793              now use [qlist] in its other mode, where each line of the input file is 
794              retrieved with a "next" message.  the first "next" message will 
795 tplarson 1.1 immediately send every line that is not preceded by a number.  
796              subsequent "next" messages will step through the rest of the lines.  in 
797              our implementation the numbers at the beginning of these lines are 
798              unused, so we simply use "0".
800 tplarson 1.2 we also now make use of the capability of modefilter.pd to use an 
801              absolute number of frequency bins for downshifting, rather than 
802              downshifting by a multiplicative factor.  this will no longer preserve 
803              the musical intervals between modes, but it will preserve the absolute 
804              frequency difference between them, making certain small differences 
805              audible.
807 tplarson 1.1 we have connected new receive objects to specify the sample rate, 
808 tplarson 1.2 downshift, and length of the output array.  hence, the first 
809 tplarson 1.1 three lines of the qlist file could be
811              samprate 8000;
812              downshift 4;
813              numseries 10;
815              but one may still set these manually instead.  the remaining global 
816              parameters, such as the instrument, whether or not to use averaged mode 
817 tplarson 1.2 parameters, and the length of the ramp used to generate the window, can 
818              also be set in this way if values other than the default are desired.
819 tplarson 1.1 
820              the remaining lines of the qlist file will take one of two forms.  
821              first, one provides a line for every mode they wish to add together.  
822 tplarson 1.2 these lines define a combination of day number, l, n, m and width 
823              multiplication factor.  the whole list is sent as daylnm, which is then 
824              parsed by the new object [parsedaylnm].  for example, one such line 
825              could be
827              0 daylnm day 1216 l 1 n 18 m 1 width 10;
829              which means to send the entire message "day 1216 l 1 n 18 m 1 width 10" 
830              to daylnm.  the elements of this message will be picked out two at a 
831              time and used to send the 5 inputs needed.  the first such line must 
832              specify at least the 4 required numbers, but subsequent lines need only 
833              specify changing values, just as if you were using [modefilter] 
834              directly.  the order is unimportant.  of course you may also manually 
835              enter any values that you wish to remain constant before starting the 
836              sequence.  when [parsedaylnm] gets to the end of the message it 
837              received, it sends a bang to startbang.
838 tplarson 1.1 
839              next comes a single line telling where to put this combination of modes 
840              in the output array.  for instance, the first such line would typically 
841              be "0 dayindex 0;".  then after a number of daylnm lines, the second 
842              would typically be "0 dayindex 72;" and so on.  however, one may specify 
843              whatever positions they want, for example leaving silence in between the 
844              segments, or even partially overwriting previously written segments.
846              when the sequence is finished, the [qlist] object sends a bang to 
847              normalize the output array.  you may listen to it using 
848              [arbitrarySRmulti] as before.  you may also write it to file inside the 
849              [pd catarray] subpatch.
852              Section 7: Conclusion
854              we hope that the level of detail presented here has allowed you to 
855              effectively use the SoSH tool, and perhaps even given you some ideas 
856              about new directions the project could take.  contributions are welcome 
857              from everyone, even if it is only the idea; do not hesitate to contact 
858              us if you need help implementing it or simply think it would be a 
859 tplarson 1.1 valuable addition to the tool.  new applications are likely to be added 
860              to the project as time goes on, so check back with us if you want the 
861              most recent version.  if you have discovered combinations of solar tones 
862              that you find pleasing, either aesthetically or scientifically, feel 
863              free to share them with us, especially if you would like them to appear 
864              in our online gallery.
866              contact the developers tim larson at tplarson@sun.stanford.edu or Seth 
867              Shafer at sethshafer.com.  the website for the Sonification of Solar 
868              Harmonics Project is http://solar-center.stanford.edu/SoSH/ .

Karen Tian
Powered by
ViewCVS 0.9.4