Add support for Android 5.1 (API Level 22) and earlier (NoClassDefFoundError)

[daniel-tran]

Suppose I have a sketch that simply plays a sound when the screen is pressed:

import processing.sound.*;
SoundFile file;

void setup() {
  file = new SoundFile(this, "barrel_hop.wav");
}

void mousePressed() {
  file.play();
}

void draw() {

}

While this works as expected on my Windows 10 PC, running the same sketch on my Android 5.1 Moto E device shows the following error in the console log and does not play any audio when the screen is pressed:

java.lang.NoClassDefFoundError: Failed resolution of: Landroid/media/AudioRecord$Builder;
	at processing.sound.JSynAndroidAudioDeviceManager$AndroidAudioInputStream.start(Unknown Source)
	at com.jsyn.engine.SynthesisEngine$EngineThread.run(Unknown Source)
Caused by: java.lang.ClassNotFoundException: Didn't find class "android.media.AudioRecord$Builder" on path: DexPathList[[zip file "/data/app/processing.test.soundexample-1/base.apk"],nativeLibraryDirectories=[/vendor/lib, /system/lib]]
	at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
	... 2 more
	Suppressed: java.lang.ClassNotFoundException: android.media.AudioRecord$Builder
		at java.lang.Class.classForName(Native Method)
		at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
		at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
		... 3 more
	Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack available

This is also using Processing 3.5.4 with Android Mode 4.2.1 installed in the Contribution Manager.

I've found an existing question on the Processing Discourse which very closely matches the error I'm seeing here, but it doesn't have any replies.

So my questions are:

1. Is this an Android version issue? From what I understand, Android 5.1 is very old, so it could be that using a more modern version of Android might turn out to be a valid fix.
2. If this issue has been observed before, is there a workaround? I haven't found anything related to this particular error in the current set of opened & closed issues, so a fix may have been reported elsewhere.

[kevinstadler]

Thanks for the detailed report!

I just had a look at the Android SDK release notes and it really looks like Android 5.1 only supports API Level 22, while the AudioRecord.Builder class used by this library was only added in API level 23 (Android 6.0).

If upgrading your phone to Android 6 or higher is not an option, I think it should be possible to add support for Android 5 by avoiding the Builder class used here:

processing-sound/src/processing/sound/JSynAndroidAudioDeviceManager.java

Lines 74 to 86 in 8bc3339

	this.audioTrack = new AudioTrack.Builder()
	.setAudioAttributes(new AudioAttributes.Builder()
	.setUsage(AudioAttributes.USAGE_MEDIA)
	.setContentType(AudioAttributes.CONTENT_TYPE_MUSIC)
	.build())
	.setAudioFormat(new AudioFormat.Builder()
	.setChannelMask(AudioFormat.CHANNEL_OUT_STEREO)
	.setEncoding(AudioFormat.ENCODING_PCM_16BIT)
	.setSampleRate(this.frameRate)
	.build())
	.setBufferSizeInBytes(this.bufferSize)
	.setTransferMode(AudioTrack.MODE_STREAM)
	.build();

in favour of a direct instantiation of AudioTrack using this constructor.

If you'd be happy to try out some test builds of the library I could give it a go, but not for another week or so. (It might be a pretty trivial fix, so if you're somewhat familiar with ant you could try building the library yourself!)

[kevinstadler]

This one should/could work:

import android.media.AudioManager;

...

this.audioTrack = new AudioTrack(new AudioAttributes.Builder() 
 				.setUsage(AudioAttributes.USAGE_MEDIA) 
 				.setContentType(AudioAttributes.CONTENT_TYPE_MUSIC) 
 				.build(), new AudioFormat.Builder() 
 				.setChannelMask(AudioFormat.CHANNEL_OUT_STEREO) 
 				.setEncoding(AudioFormat.ENCODING_PCM_16BIT) 
 				.setSampleRate(this.frameRate) 
 				.build(), this.bufferSize, AudioTrack.MODE_STREAM, AudioManager.AUDIO_SESSION_ID_GENERATE);

[daniel-tran]

I think using a more modern Android version is probably the easiest fix, but the Processing for Android documentation seems to imply that the minimum supported Android version is 4.2 (for running sketches), and I'm not sure if Processing libraries are required to adhere to this.

In any case, I've built the library locally with the suggested change, following the steps on the README for building and deploying it. However, the problem is still occurring. Looking at JSynAndroidAudioDeviceManager.java and the console error more closely, I'm led to believe this is the real source of the issue:

processing-sound/src/processing/sound/JSynAndroidAudioDeviceManager.java

Lines 144 to 153 in 5e57e77

	this.audioRecord = new AudioRecord.Builder()
	.setAudioSource(this.deviceID)
	.setAudioFormat(new AudioFormat.Builder()
	.setChannelMask(AudioFormat.CHANNEL_IN_MONO)
	.setEncoding(AudioFormat.ENCODING_PCM_16BIT)
	.setSampleRate(this.frameRate)
	.build())
	.setBufferSizeInBytes(this.bufferSize)
	.build();
	this.audioRecord.startRecording();

If I comment out this section, rebuild and redeploy the library locally, the sketch code does play audio when the screen is pressed on my Android device, and the console error no longer occurs.

[kevinstadler]

Ah yes you are right, I mixed up AudioTrack and AudioRecord there! Glad to hear that it's working, although I'm surprised that the AudioTrack part doesn't throw an exception as well, since both Builder classes were only added in Android 6? Either way your fix of commenting out the audioRecord instantiation probably means that you won't be able to capture any audio from the microphone with your build of the library, but I'll look into publishing a permanent fix. Thanks again for reporting!

[daniel-tran]

I'm surprised that the AudioTrack part doesn't throw an exception as well

This might be because I happen to be using both the suggested AudioTrack change and the commented-out AudioRecord section. I suspect reverting the former back to its original code will give me a similar looking console error, though I haven't tested it.