Skip to content

Commit

Permalink
Fix bad links, typos in docs
Browse files Browse the repository at this point in the history
  • Loading branch information
raghakot committed Jul 6, 2017
1 parent 41d503b commit 72bb8df
Show file tree
Hide file tree
Showing 10 changed files with 42 additions and 45 deletions.
20 changes: 11 additions & 9 deletions docs/templates/visualizations/activation_maximization.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ output activations. i.e., we compute

$$\frac{\partial ActivationMaximizationLoss}{\partial input}$$

and use that estimate to update the input. [ActivationMaximization](../vis.losses#ActivationMaximization) loss simply
and use that estimate to update the input. [ActivationMaximization](../vis.losses#activationmaximization) loss simply
outputs small values for large filter activations (we are minimizing losses during gradient descent iterations).
This allows us to understand what sort of input patterns activate a particular filter. For example, there could be
an eye filter that activates for the presence of eye within the input image.
Expand All @@ -25,12 +25,14 @@ activations.
2. [visualize_activation_with_losses](../vis.visualization#visualize_activation_with_losses): This is intended for
research use-cases where some custom weighted losses can be minimized.

See [examples/](https://github.com/raghakot/keras-vis/tree/master/examples) for code examples.

### Scenarios

The API is very general purpose and can be used in a wide variety of scenarios. We will list the most common use-cases
below:

#### Categorical Output `Dense` layer visualization
#### Categorical Output Dense layer visualization

How can we assess whether a network is over/under fitting or generalizing well? Given an input image, a CNN can
classify whether it is a cat, bird etc. How can we be sure that it is capturing the correct notion of what it means
Expand All @@ -46,7 +48,7 @@ category.
multiple categories to see what a cat-fish might look like, as an example.
- For multi-label classifier, simply set the appropriate `filter_indices`.

#### Regression Output `Dense` layer visualization
#### Regression Output Dense layer visualization

Unlike class activation visualizations, for regression outputs, we could visualize input that

Expand All @@ -61,25 +63,25 @@ with respect to inputs. By default, `ActivationMaximization` loss is used to inc
[gradient_modifiers](../vis.grad_modifiers) are very powerful and show up in other visualization APIs as well.


#### `Conv` filter visualization
#### Conv filter visualization

By pointing `layer_idx` to `Conv` layer, you can visualize what pattern activates a filter. This might help you discover
what a filter might be computing. Here, `filter_indices` refers to the index of the `Conv` filter within the layer.

### Advanced usage

`backprop_modifier` allows you to modify the backpropagation behavior. For examples, you could tweak backprop to only
propagate positive gradients by using `backprop_modifier='relu'`. This parameter also accepts a function and can be
used to implement your crazy research idea :)
[backprop_modifiers](../vis.backprop_modifiers) allow you to modify the backpropagation behavior. For examples,
you could tweak backprop to only propagate positive gradients by using `backprop_modifier='relu'`. This parameter also
accepts a function and can be used to implement your crazy research idea :)

## Tips and tricks

- If you get garbage visualization, try setting `verbose=True` to see various losses during gradient descent iterations.
By default, `visualize_activation` uses `TotalVariation` and `LpNorm` regularization to enforce natural image prior. It
is very likely that you would see `ActivationMax Loss` bounce back and forth as they are dominated by regularization
is very likely that you would see `ActivationMaximization Loss` bounce back and forth as they are dominated by regularization
loss weights. Try setting all weights to zero and gradually try increasing values of total variation weight.

- To get sharper looking images, use [Jitter](../vis.input_modifiers#Jitter) input modifier.
- To get sharper looking images, use [Jitter](../vis.input_modifiers#jitter) input modifier.

- Regression models usually do not provide enough gradient information to generate meaningful input images. Try seeding
the input using `seed_input` and see if the modifications to the input make sense.
Expand Down
2 changes: 1 addition & 1 deletion docs/templates/visualizations/class_activation_maps.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ whose output gradients are used. By default, keras-vis will search for the neare

### Scenarios

See [saliency scenarios](saliency#Scenarios). Everything is identical expect the added `penultimate_layer_idx` param.
See [saliency scenarios](saliency#scenarios). Everything is identical expect the added `penultimate_layer_idx` param.

## Gotchas

Expand Down
21 changes: 11 additions & 10 deletions docs/templates/visualizations/saliency.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,7 @@ $$\frac{\partial output}{\partial input}$$

This should tell us how the output value changes with respect to a small change in inputs. We can use these gradients
to highlight input regions that cause the most change in the output. Intuitively this should highlight salient image
regions that most contribute towards the output.

Refer to [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://arxiv.org/pdf/1312.6034v2.pdf)
for details.
regions that most contribute towards the output.

## Usage

Expand All @@ -34,20 +31,22 @@ saliency.
2. [visualize_saliency_with_losses](../vis.visualization#visualize_saliency_with_losses): This is intended for
research use-cases where some custom weighted loss can be used.

See [examples/](https://github.com/raghakot/keras-vis/tree/master/examples) for code examples.

### Scenarios

The API is very general purpose and can be used in a wide variety of scenarios. We will list the most common use-cases
below:

#### Categorical `Dense` layer visualization
#### Categorical Dense layer visualization

By setting `layer_idx` to final `Dense` layer, and `filter_indices` to the desired output category, we can visualize
parts of the `seed_input` that contribute most towards activating the corresponding output nodes,

- For multi-class classification, `filter_indices` can point to a single class.
- For multi-label classifier, simply set the appropriate `filter_indices`.

#### Regression `Dense` layer visualization
#### Regression Dense layer visualization

For regression outputs, we could visualize attention over input that

Expand All @@ -69,9 +68,11 @@ gradients (which indicate the decrease) as positive and therefore visualize decr

- To visualize what contributed to the predicted output, we want to consider gradients that have very low positive
or negative values. This can be achieved by performing `grads = abs(1 / grads)` to magnifies small gradients. Equivalently,
you can use `grad_modifier='small_values'`, which does the same thing.
you can use `grad_modifier='small_values'`, which does the same thing. [gradient_modifiers](../vis.grad_modifiers)
are very powerful and show up in other visualization APIs as well.

[gradient_modifiers](../vis.grad_modifiers) are very powerful and show up in other visualization APIs as well.
You can see a practical application for this in the
[self diving car](https://github.com/raghakot/keras-vis/tree/master/applications/self_driving) example.

#### Guided / rectified saliency

Expand All @@ -83,9 +84,9 @@ In guided saliency, the backprop step is modified to only propagate positive gra
For details see the paper: [String For Simplicity: The All Convolutional Net](https://arxiv.org/pdf/1412.6806.pdf).

For both these cases, we can use `backprop_modifier='relu'` and `backprop_modifier='guided'` respectively. You
can also implement your own `backprop_modifier` to try your crazy research idea :)
can also implement your own [backprop_modifier](../vis.backprop_modifiers) to try your crazy research idea :)

#### `Conv` filter saliency
#### Conv filter saliency

By pointing `layer_idx` to `Conv` layer, you can visualize parts of the image that influence the filter. This might
help you discover what a filter cares about. Here, `filter_indices` refers to the index of the `Conv` filter within
Expand Down
2 changes: 1 addition & 1 deletion vis/callbacks.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def _check_imageio():


class OptimizerCallback(object):
"""Abstract class for defining callbacks for use with [optimizer.minimize](vis.optimizer.md#minimize).
"""Abstract class for defining callbacks for use with [Optimizer.minimize](vis.optimizer#optimizerminimize).
"""

def callback(self, i, named_losses, overall_loss, grads, wrt_value):
Expand Down
2 changes: 1 addition & 1 deletion vis/input_modifiers.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

class InputModifier(object):
"""Abstract class for defining an input modifier. An input modifier can be used with the
[Optimizer.minimize](vis.optimizer/#optimizerminimize) to make `pre` and `post` changes to the optimized input
[Optimizer.minimize](vis.optimizer#optimizerminimize) to make `pre` and `post` changes to the optimized input
during the optimization process.
```python
Expand Down
3 changes: 1 addition & 2 deletions vis/losses.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ def build_loss(self):
conv_layer[utils.slicer[:, filter_idx, ...]]
```
[utils.get_img_shape](vis.utils.utils.md#get_img_shape) and
[utils.get_img_indices](vis.utils.utils.md#get_img_indices) are other optional utilities that make this easier.
[utils.get_img_shape](vis.utils.utils.md#get_img_shape) is another optional utility that make this easier.
Returns:
The loss expression.
Expand Down
5 changes: 2 additions & 3 deletions vis/optimizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,14 +106,13 @@ def minimize(self, seed_input=None, max_iter=200,
channels_first` or `(samples, image_dims..., channels)` if `image_data_format=channels_last`.
Seeded with random noise if set to None. (Default value = None)
max_iter: The maximum number of gradient descent iterations. (Default value = 200)
input_modifiers: A list of [../vis/input_modifier/#InputModifier](InputModifier) instances specifying
input_modifiers: A list of [InputModifier](vis.input_modifiers#inputmodifier) instances specifying
how to make `pre` and `post` changes to the optimized input during the optimization process.
`pre` is applied in list order while `post` is applied in reverse order. For example,
`input_modifiers = [f, g]` means that `pre_input = g(f(inp))` and `post_input = f(g(inp))`
grad_modifier: gradient modifier to use. See [grad_modifiers](vis.grad_modifiers.md). If you don't
specify anything, gradients are unchanged. (Default value = None)
callbacks: A list of [vis.callbacks#optimizercallback](OptimizerCallback) instances to trigger.
callbacks: A list of [OptimizerCallback](vis.callbacks#optimizercallback) instances to trigger.
verbose: Logs individual losses at the end of every gradient descent iteration.
Very useful to estimate loss weight factor(s). (Default value = True)
Expand Down
5 changes: 4 additions & 1 deletion vis/visualization/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,10 @@


def get_num_filters(layer):
"""Determines the number of filters within the give `layer`.
"""Determines the number of filters within the given `layer`.
Args:
layer: The keras layer to use.
Returns:
Total number of filters within `layer`.
Expand Down
12 changes: 4 additions & 8 deletions vis/visualization/activation_maximization.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ def visualize_activation_with_losses(input_tensor, losses,
(Default value = None)
input_range: Specifies the input range as a `(min, max)` tuple. This is used to rescale the
final optimized input to the given range. (Default value=(0, 255))
optimizer_params: The **kwargs for optimizer [params](vis.optimizer.md##optimizerminimize). Will default to
optimizer_params: The **kwargs for optimizer [params](vis.optimizer#optimizerminimize). Will default to
reasonable values when required keys are not found.
Returns:
Expand Down Expand Up @@ -63,13 +63,9 @@ def visualize_activation(model, layer_idx, filter_indices=None,
layer_idx: The layer index within `model.layers` whose filters needs to be visualized.
filter_indices: filter indices within the layer to be maximized.
If None, all filters are visualized. (Default value = None)
For `keras.layers.Dense` layer, `filter_idx` is interpreted as the output index.
If you are visualizing final `keras.layers.Dense` layer, you tend to get
better results with 'linear' activation as opposed to 'softmax'. This is because 'softmax'
output can be maximized by minimizing scores for other classes.
If you are visualizing final `keras.layers.Dense` layer, consider switching 'softmax' activation for
'linear' using [utils.apply_modifications](vis.utils.utils#apply_modifications) for better results.
seed_input: Seeds the optimization with a starting input. Initialized with a random value when set to None.
(Default value = None)
input_range: Specifies the input range as a `(min, max)` tuple. This is used to rescale the
Expand All @@ -81,7 +77,7 @@ def visualize_activation(model, layer_idx, filter_indices=None,
act_max_weight: The weight param for `ActivationMaximization` loss. Not used if 0 or None. (Default value = 1)
lp_norm_weight: The weight param for `LPNorm` regularization loss. Not used if 0 or None. (Default value = 10)
tv_weight: The weight param for `TotalVariation` regularization loss. Not used if 0 or None. (Default value = 10)
optimizer_params: The **kwargs for optimizer [params](vis.optimizer.md##optimizerminimize). Will default to
optimizer_params: The **kwargs for optimizer [params](vis.optimizer#optimizerminimize). Will default to
reasonable values when required keys are not found.
Example:
Expand Down
15 changes: 6 additions & 9 deletions vis/visualization/saliency.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,11 +91,10 @@ def visualize_saliency(model, layer_idx, filter_indices, seed_input,
`image_data_format=channels_last`.
layer_idx: The layer index within `model.layers` whose filters needs to be visualized.
filter_indices: filter indices within the layer to be maximized.
If None, all filters are visualized. (Default value = None)
For `keras.layers.Dense` layer, `filter_idx` is interpreted as the output index.
If you are visualizing final `keras.layers.Dense` layer, you tend to get
better results with 'linear' activation as opposed to 'softmax'. This is because 'softmax'
output can be maximized by minimizing scores for other classes.
If you are visualizing final `keras.layers.Dense` layer, consider switching 'softmax' activation for
'linear' using [utils.apply_modifications](vis.utils.utils#apply_modifications) for better results.
seed_input: The model input for which activation map needs to be visualized.
backprop_modifier: backprop modifier to use. See [backprop_modifiers](vis.backprop_modifiers.md). If you don't
specify anything, no backprop modification is applied. (Default value = None)
Expand Down Expand Up @@ -203,12 +202,10 @@ def visualize_cam(model, layer_idx, filter_indices,
`image_data_format=channels_last`.
layer_idx: The layer index within `model.layers` whose filters needs to be visualized.
filter_indices: filter indices within the layer to be maximized.
If None, all filters are visualized. (Default value = None)
For `keras.layers.Dense` layer, `filter_idx` is interpreted as the output index.
If you are visualizing final `keras.layers.Dense` layer, you tend to get
better results with 'linear' activation as opposed to 'softmax'. This is because 'softmax'
output can be maximized by minimizing scores for other classes.
If you are visualizing final `keras.layers.Dense` layer, consider switching 'softmax' activation for
'linear' using [utils.apply_modifications](vis.utils.utils#apply_modifications) for better results.
seed_input: The input image for which activation map needs to be visualized.
penultimate_layer_idx: The pre-layer to `layer_idx` whose feature maps should be used to compute gradients
wrt filter output. If not provided, it is set to the nearest penultimate `Conv` or `Pooling` layer.
Expand Down

0 comments on commit 72bb8df

Please sign in to comment.