Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Converter synapse not applied to neurons-to-TensorNode connections #214

Open
xchoo opened this issue Apr 22, 2021 · 1 comment
Open

Converter synapse not applied to neurons-to-TensorNode connections #214

xchoo opened this issue Apr 22, 2021 · 1 comment

Comments

@xchoo
Copy link
Member

xchoo commented Apr 22, 2021

If you convert a Keras model where there is a connection from a neurons layer (e.g., dense, conv2D, etc.) to a layer that cannot be natively converted to a Nengo object (i.e., has to be replaced by a TensorNode), the connection to the TensorNode does not have the converter's synapse value applied to it.
Note: The NengoDL converter docs state:

Synaptic filter to be applied on the output of all neurons.

Minimal code to reproduce issue:

inp = tf.keras.Input(shape=(28, 28, 1))
conv0 = tf.keras.layers.Conv2D(
    filters=32,
    kernel_size=3,
    activation=tf.nn.relu,
)(inp)
max_pool = tf.keras.layers.MaxPool2D()(conv0)
# max_pool = tf.keras.layers.AvgPool2D()(conv0)  # Uncomment to see expected output
model = tf.keras.Model(inputs=inp, outputs=max_pool)

converter = nengo_dl.Converter(model, synapse=0.1)

# Find connection wehere pre is conv0
for conn in converter.net.all_connections:
    if conn.pre_obj == converter.layers[conv0]:
        break
print(conn, conn.synapse)

Expected output: <Connection from <Neurons of <Ensemble 'conv2d.0'>> to <TensorNode 'max_pooling2d'>> Lowpass(tau=0.1)
Observed output: <Connection from <Neurons of <Ensemble 'conv2d.0'>> to <TensorNode 'max_pooling2d'>> None

To see the difference between a TensorNode and native Nengo object, change MaxPool2D layer to AvgPool2D.

Probable fix:
The ConvertFallback function needs to be made aware of a non-default synapse value if the input_obj object is an Ensemble:

    def convert(self, node_id):
        input_obj = self.get_input_obj(node_id)

        # Identify the correct synapse value to use
        synapse = (
            self.converter.synapse
            if isinstance(input_obj, nengo.ensemble.Neurons)
            else None
        )

        output = self.tensor_layer(
            input_obj,
            shape_in=self.input_shape(node_id),
            label=self.layer.name,
            synapse=synapse,  # Pass synapse value to TensorNode
        )
@R-Gaurav
Copy link

Just wondering if I should keep an eye for a future release of NengoDL where this would be fixed? For now I am explicitly synapsing the connections :) .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants