Circular buffer with AudioIn

[dhowe]

Trying to implement a circular buffer that holds (and displays the waveform for) the last k seconds of input from audioIn. I started on a version with java's ArrayDeque, but I'm having trouble getting the data into a continuously scrolling waveform (events in the waveform would move from right to left as they move into the past). Any suggestions or examples I might check would be great.. thanks!

[kevinstadler]

An interesting problem! You don't even need to create your own circular buffer, you can just use the Waveform class to create one that's large enough for the number of seconds you want to display, and then selectively read from that large array. Below is some working code for you, note how it might look like there are artefacts (samples jumping up and down as they're scrolling along) but that is simply an effect of being unable to draw every sample all the time -- you would need to draw 44 thousands pixels for every second of audio. If you want to smooth out the graphics you might have to do something like averaging over all the adjacent samples that go into a pixel...

import processing.sound.*;


AudioIn input;
Waveform waveform;

void setup() {
  size(640, 480);
  noStroke();
  input = new AudioIn(this, 0);

  int numberOfSecondsToView = 3;
  int numberOfSamplesToView = numberOfSecondsToView * new Sound(this).sampleRate();

  waveform = new Waveform(this, numberOfSamplesToView);
  waveform.input(input);  
}

void draw() {
  background(125, 255, 125);
  fill(255, 0, 150);

  // get the most recent samples
  float[] samples = waveform.analyze();

  // only draw a subset of the samples
  for (int i = 0; i < width; i++) {
    // map the pixel position in [0,width - 1] to samples array
    // indices in [0,samples.length - 1]
    int index = (int) map(i, 0, width-1, 0, samples.length - 1);
    // for every pixeldraw a rectangle up or down from the mid-point (the height can be 'negative'!)
    rect(i, height / 2, 1, map(samples[index], -1, 1, -height/2, height/2));
  }
}

[dhowe]

Nice! I tried your averaging suggestion, dividing the width into buckets and averaging the sample values for each, but found the negative and positive values cancel each other out, so below uses the max absolute value of the samples in a bucket:

  background(245);

  float[] samples = waveform.analyze();
  for (int i = 0; i < width; i++) {
    values[i] = 0;
    for (int j = 0; j < samplesPerPixel; j++) {
      float val = abs(samples[i * samplesPerPixel + j]);
      if (val > values[i]) values[i] = val;
    }
  }
  
  for (int i = 0; i < width; i++) {
    float h = values[i] * height;
    rect(i, height / 2 - h / 2, 1, h);
  }

[dhowe]

This full version uses the same method, but attempts to only update the new pixels. Still doesn't seem quite right to me though. Any thoughts?

import processing.sound.*;

AudioIn input;
Waveform waveform;
float[] values;
int lastFrameMs;
int samplesPerPixel;

void setup() {
  size(800, 600);
  input = new AudioIn(this);
  int targetBufferSz = new Sound(this).sampleRate() * 3; // secs
  samplesPerPixel = floor(targetBufferSz / (float) width);
  waveform = new Waveform(this, samplesPerPixel * width);
  values = new float[width];
  waveform.input(input);
  lastFrameMs = millis();
}

void draw() {
  float[] samples = waveform.analyze();

  // calculate the # of new samples
  int elapsed = millis() - lastFrameMs;
  int shift = (int)((elapsed / 1000f) * width);

  // shift the array to the left 
  values = shiftLeft(values, shift);

  // fill in the new values on the right
  for (int i = width - shift; i < width; i++) {
    values[i] = 0;
    for (int j = 0; j < samplesPerPixel; j++) {
      float val = abs(samples[i * samplesPerPixel + j]);
      if (val > values[i]) values[i] = val;
    }
  }

  // draw the values as lines
  background(245);
  for (int i = 0; i < width; i++) {
    float h = values[i] * height;
    rect(i, height / 2 - h / 2, 1, h);
  }

  lastFrameMs = millis();
}

float[] shiftLeft(float[] arr, int shift) {
  float[] tmp = new float[arr.length];
  System.arraycopy(arr, shift, tmp, 0, arr.length - shift);
  return tmp;
}

[kevinstadler]

The problem is that the Processing millis() might not be perfectly in sync with the synthesis engine, so that's not a reliable way to determine how many of the samples in the waveform buffer are new in comparison to the last call. This information (number of new frames) is of course available in the underlying synthesis engine, there's just no way to access it from the current Sound library release.

I added some new functions to the library and uploaded a test build here, just extract the zip in your Documents/Arduino/libraries folder and you should be good to go: sound.zip

The main method of interest for you is:

• int waveform.getLastAnalysisOffset() -- after calling analyze(), the int returned by this function (in the range [0, nsamples-1]) will tell you the offset into the underlying circular buffer from which the first element of the float[] array returned by analyze() is taken. The absolute value of this is not relevant for you, but you can calculate exactly how many new samples have been added since the last call by taking the difference between the offsets after each call!

I've also added two more methods that allow you to view the current state of the underlying circular buffer directly, namely:

• float[] waveform.analyzeCircular()
• float[] waveform.analyzeCircular(float[] target)

The Waveform class will continuously overwrite data in the array, so if you want to find out where the newest data has just been written to you can use the new waveform.getLastAnalysisOffset() method from above, and actually use its absolute return value to access the most recent data. (You don't have to use this information, just replace analyze() with analyzeCircular() in the code I posted above and you should already get good results!)

Hope that helps, let me know if you find anything wrong with the test build!

[dhowe]

So running the following code in draw (with a 1-sec buffer) , I get the following number of (what I think should be) new samples for each of the first 100 frames (sometimes 0, occasionally negative) -- does this look correct? I guess I would need to handle the wraparounds, but the zeros are still perplexing.

  public void draw()
  {
    float[] samples = waveform.analyze();
    int offset = waveform.getLastAnalysisOffset();
    int newSamples = offset - lastOffset;
    println(frameCount, newSamples);
    ...
    lastOffset = offset;
  }

1 0
2 0
3 0
4 0
5 0
6 0
7 0
8 0
9 0
10 0
11 0
12 0
13 0
14 0
15 0
16 0
17 0
18 0
19 0
20 0
21 0
22 0
23 580
24 2148
25 336
26 1152
27 8
28 1784
29 1024
30 512
31 1024
32 512
33 1024
34 824
35 712
36 512
37 512
38 1024
39 512
40 1024
41 512
42 576
43 960
44 512
45 1024
46 512
47 1024
48 512
49 512
50 1024
51 512
52 1024
53 512
54 1024
55 1024
56 512
57 512
58 1024
59 512
60 1024
61 512
62 1024
63 512
64 1024
65 512
66 512
67 1024
68 512
69 512
70 1024
71 512
72 1024
73 1024
74 0
75 1024
76 1024
77 512
78 1024
79 512
80 -42976
81 0
82 1536
83 512
84 752
85 272
86 1024
87 1024
88 512
89 1024
90 0
91 1536
92 0
93 1024
94 1024
95 512
96 512
97 1024
98 1024
99 512

[dhowe]

Getting closer...

import processing.sound.*;

AudioIn input;
Waveform waveform;
float[] values, rms;
int lastOffset, samplesPerPixel;

void setup()
{
  size(800, 600);

  input = new AudioIn(this);
  int targetBufferSz = new Sound(this).sampleRate() * 3;
  samplesPerPixel = floor(targetBufferSz / (float) width);
  waveform = new Waveform(this, samplesPerPixel * width);
  values = new float[width];
  rms = new float[width];
  waveform.input(input);
  lastOffset = 0;
}

void draw()
{
  float[] samples = waveform.analyze();

  // calculate the # of new samples
  int offset = waveform.getLastAnalysisOffset();
  int newSamples = (offset - lastOffset);
  if (newSamples < 0) newSamples += samples.length;
  int shift = ceil((newSamples / (float)samples.length) * values.length);
  lastOffset = offset;

  // shift the array to the left
  values = shiftLeft(values, shift);
  rms = shiftLeft(rms, shift);

  // fill in the new values on the right
  for (int i = width - shift; i < width; i++) {
    values[i] = absAvg(samples, i);
    rms[i] = rmsAvg(samples, i);
  }

  // draw the values as rects
  background(245);
  noStroke();
  for (int i = 0; i < width; i++) {
    fill(49, 48, 205);
    float h = values[i] * height;
    rect(i, height / 2 - h / 2, 1, h);
    fill(98, 101, 222);
    h = rms[i] * height;
    rect(i, height / 2 - h / 2, 1, h);
  }
}

float rmsAvg(float[] samples, int startIdx)
{
  float sumSq = 0;
  for (int j = 0; j < samplesPerPixel; j++) {
    float val = samples[startIdx * samplesPerPixel + j];
    sumSq += (val * val);
  }
  float mean = sumSq / samplesPerPixel;
  return (float) Math.sqrt(mean);
}

float absAvg(float[] samples, int startIdx)
{
  float sum = 0;
  for (int j = 0; j < samplesPerPixel; j++) {
    sum += Math.abs(samples[startIdx * samplesPerPixel + j]);
  }
  return (sum * 2) / (float) samplesPerPixel;
}

float[] shiftLeft(float[] arr, int shift)
{
  float[] tmp = new float[arr.length];
  System.arraycopy(arr, shift, tmp, 0, arr.length - shift);
  return tmp;
}

[kevinstadler]

This will be addressed as a special case of #55