-
Notifications
You must be signed in to change notification settings - Fork 87
Quick API Guide
Misa Ogura edited this page Sep 10, 2019
·
7 revisions
Below are currently available modules.
-
flashtorch.utils: some useful utilility functions for data handling & transformation -
flashtorch.utils.imagenet:ImageNetIndexclass for easy-ish retrieval of class index -
flashtorch.saliency.backprop:Backpropclass for calculating gradients -
flashtorch.activmax.gradient_ascent:GradientAscentclass for activation maximization
You can inspect available functions within each module with built-in Python
function help.
>>> import flashtorch
>>> help(flashtorch.utils)
Help on package flashtorch.utils in flashtorch:
NAME
flashtorch.utils - flashtorch.utils
DESCRIPTION
This module provides utility functions for image handling and tensor
transformation.
PACKAGE CONTENTS
imagenet
resources (package)
FUNCTIONS
apply_transforms(image, size=224)
Transforms a PIL image to torch.Tensor.
Applies a series of tranformations on PIL image including a conversion
to a tensor. The returned tensor has a shape of :math:`(N, C, H, W)` and
is ready to be used as an input to neural networks.
First the image is resized to 256, then cropped to 224. The `means` and
`stds` for normalisation are taken from numbers used in ImageNet, as
currently developing the package for visualizing pre-trained models.
The plan is to to expand this to handle custom size/mean/std.
Args:
image (PIL.Image.Image or numpy array)
size (int, optional, default=224): Desired size (width/height) of the
output tensor
Shape:
Input: :math:`(C, H, W)` for numpy array
Output: :math:`(N, C, H, W)`
Returns:
torch.Tensor (torch.float32): Transformed image tensor
Note:
Symbols used to describe dimensions:
- N: number of images in a batch
- C: number of channels
- H: height of the image
- W: width of the image
denormalize(tensor)
Reverses the normalisation on a tensor.
Performs a reverse operation on a tensor, so the pixel value range is
between 0 and 1. Useful for when plotting a tensor into an image.
Normalisation: (image - mean) / std
Denormalisation: image * std + mean
Args:
tensor (torch.Tensor, dtype=torch.float32): Normalized image tensor
Shape:
Input: :math:`(N, C, H, W)`
Output: :math:`(N, C, H, W)` (same shape as input)
Return:
torch.Tensor (torch.float32): Demornalised image tensor with pixel
values between [0, 1]
Note:
Symbols used to describe dimensions:
- N: number of images in a batch
- C: number of channels
- H: height of the image
- W: width of the image
format_for_plotting(tensor)
Formats the shape of tensor for plotting.
Tensors typically have a shape of :math:`(N, C, H, W)` or :math:`(C, H, W)`
which is not suitable for plotting as images. This function formats an
input tensor :math:`(H, W, C)` for RGB and :math:`(H, W)` for mono-channel
data.
Args:
tensor (torch.Tensor, torch.float32): Image tensor
Shape:
Input: :math:`(N, C, H, W)` or :math:`(C, H, W)`
Output: :math:`(H, W, C)` or :math:`(H, W)`, respectively
Return:
torch.Tensor (torch.float32): Formatted image tensor (detached)
Note:
Symbols used to describe dimensions:
- N: number of images in a batch
- C: number of channels
- H: height of the image
- W: width of the image
load_image(image_path)
Loads image as a PIL RGB image.
Args:
image_path (str): A path to the image
Returns:
An instance of PIL.Image.Image in RGB
standardize_and_clip(tensor, min_value=0.0, max_value=1.0, saturation=0.1, brightness=0.5)
Standardizes and clips input tensor.
Standardizes the input tensor (mean = 0.0, std = 1.0). The color saturation
and brightness are adjusted, before tensor values are clipped to min/max
(default: 0.0/1.0).
Args:
tensor (torch.Tensor):
min_value (float, optional, default=0.0)
max_value (float, optional, default=1.0)
saturation (float, optional, default=0.1)
brightness (float, optional, default=0.5)
Shape:
Input: :math:`(C, H, W)`
Output: Same as the input
Return:
torch.Tensor (torch.float32): Normalised tensor with values between
[min_value, max_value]
FILE
/usr/local/lib/python3.6/dist-packages/flashtorch/utils/__init__.py
>>> import flashtorch
>>> help(flashtorch.utils.imagenet)
Help on module flashtorch.utils.imagenet in flashtorch.utils:
NAME
flashtorch.utils.imagenet - # -*- coding: utf-8 -*-
CLASSES
collections.abc.Mapping(collections.abc.Collection)
ImageNetIndex
class ImageNetIndex(collections.abc.Mapping)
| Interface to retrieve ImageNet class indeces from class names.
|
| This class implements a dictionary like object, aiming to provide an
| easy-to-use look-up table for finding a target class index from an ImageNet
| class name.
|
| Reference:
| - ImageNet class index: https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json
| - Synsets: http://image-net.org/challenges/LSVRC/2015/browse-synsets
|
| Note:
| Class names in `imagenet_class_index.json` has been slightly modified
| from the source due to duplicated class names (e.g. crane). This helps
| make the use of this tool simpler.
|
| Method resolution order:
| ImageNetIndex
| collections.abc.Mapping
| collections.abc.Collection
| collections.abc.Sized
| collections.abc.Iterable
| collections.abc.Container
| builtins.object
|
| Methods defined here:
|
| __contains__(self, key)
|
| __getitem__(self, phrase)
|
| __init__(self)
| Initialize self. See help(type(self)) for accurate signature.
|
| __iter__(self)
|
| __len__(self)
|
| items(self)
| D.items() -> a set-like object providing a view on D's items
|
| keys(self)
| D.keys() -> a set-like object providing a view on D's keys
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| __abstractmethods__ = frozenset()
|
| ----------------------------------------------------------------------
| Methods inherited from collections.abc.Mapping:
|
| __eq__(self, other)
| Return self==value.
|
| get(self, key, default=None)
| D.get(k[,d]) -> D[k] if k in D, else d. d defaults to None.
|
| values(self)
| D.values() -> an object providing a view on D's values
|
| ----------------------------------------------------------------------
| Data and other attributes inherited from collections.abc.Mapping:
|
| __hash__ = None
|
| __reversed__ = None
|
| ----------------------------------------------------------------------
| Class methods inherited from collections.abc.Collection:
|
| __subclasshook__(C) from abc.ABCMeta
| Abstract classes can override this to customize issubclass().
|
| This is invoked early on by abc.ABCMeta.__subclasscheck__().
| It should return True, False or NotImplemented. If it returns
| NotImplemented, the normal algorithm is used. Otherwise, it
| overrides the normal algorithm (and the outcome is cached).
FILE
/Users/misao/Projects/personal/flashtorch/flashtorch/utils/imagenet.py
>>> import flashtorch
>>> help(flashtorch.saliency.backprop)
Help on module flashtorch.saliency.backprop in flashtorch.saliency:
NAME
flashtorch.saliency.backprop
CLASSES
builtins.object
Backprop
class Backprop(builtins.object)
| Provides an interface to perform backpropagation.
|
| This class provids a way to calculate the gradients of a target class
| output w.r.t. an input image, by performing a single backprobagation.
|
| The gradients obtained can be used to visualise an image-specific class
| saliency map, which can gives some intuition on regions within the input
| image that contribute the most (and least) to the corresponding output.
|
| More details on saliency maps: `Deep Inside Convolutional Networks:
| Visualising Image Classification Models and Saliency Maps
| <https://arxiv.org/pdf/1312.6034.pdf>`_.
|
| Args:
| model: A neural network model from `torchvision.models
| <https://pytorch.org/docs/stable/torchvision/models.html>`
|
| Methods defined here:
|
| __init__(self, model)
| Initialize self. See help(type(self)) for accurate signature.
|
| calculate_gradients(self, input_, target_class=None, take_max=False, guided=False, use_gpu=False)
| Calculates gradients of the target_class output w.r.t. an input_.
|
| The gradients is calculated for each colour channel. Then, the maximum
| gradients across colour channels is returned.
|
| Args:
| input_ (torch.Tensor): With shape :math:`(N, C, H, W)`.
| target_class (int, optional, default=None)
| take_max (bool, optional, default=False): If True, take the maximum
| gradients across colour channels for each pixel.
| guided (bool, optional, default=Fakse): If True, perform guided
| backpropagation. See `Striving for Simplicity: The All
| Convolutional Net <https://arxiv.org/pdf/1412.6806.pdf>`_.
| use_gpu (bool, optional, default=False): Use GPU if set to True and
| `torch.cuda.is_available()`.
|
| Returns:
| gradients (torch.Tensor): With shape :math:`(C, H, W)`.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
FILE
/Users/misao/Projects/personal/flashtorch/flashtorch/saliency/backprop.py
>>> import flashtorch
>>> help(flashtorch.activmax.gradient_ascent)
Help on module flashtorch.activmax.gradient_ascent in flashtorch.activmax:
NAME
flashtorch.activmax.gradient_ascent
CLASSES
builtins.object
GradientAscent
class GradientAscent(builtins.object)
| Provides an interface for activation maximization via gradient descent.
|
| This class implements the gradient ascent algorithm in order to perform
| activation maximization with convolutional neural networks (CNN).
|
| `Activation maximization <https://pdfs.semanticscholar.org/65d9/94fb778a8d9e0f632659fb33a082949a50d3.pdf>`_
| is one form of feature visualization that allows us to visualize what CNN
| filters are "looking for", by applying each filter to an input image and
| updating the input image so as to maximize the activation of the filter of
| interest (i.e. treating it as a gradient ascent task with activation as the
| loss). The implementation is inspired by `this demo <https://blog.keras.io/category/demo.html>`_
| by Francois Chollet.
|
| Args:
| model: A neural network model from `torchvision.models
| <https://pytorch.org/docs/stable/torchvision/models.html>`_,
| typically without the fully-connected part of the network.
| e.g. torchvisions.alexnet(pretrained=True).features
| img_size (int, optional, default=224): The size of an input image to be
| optimized.
| lr (float, optional, default=1.): The step size (or learning rate) of
| the gradient ascent.
| use_gpu (bool, optional, default=False): Use GPU if set to True and
| `torch.cuda.is_available()`.
|
| Methods defined here:
|
| __init__(self, model, img_size=224, lr=1.0, use_gpu=False)
| Initialize self. See help(type(self)) for accurate signature.
|
| deepdream(self, img_path, layer, filter_idx, lr=0.1, num_iter=20, figsize=(4, 4), title='DeepDream', return_output=False)
| Creates DeepDream.
|
| It applies the optimization on the image provided. The image is loaded
| and made into a torch.Tensor that is compatible as the input to the
| network.
|
| Read the original blog post by Google for more information on
| `DeepDream <https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html>`_.
|
| Args:
| img_path (str): A path to the image you want to apply DeepDream on
| layer (torch.nn.modules.conv.Conv2d): The target Conv2d layer from
| which the filter to be chosen, based on `filter_idx`.
| filter_idx (int): The index of the target filter.
| lr (float, optional, default=.1): The step size of optimization.
| num_iter (int, optional, default=30): The number of iteration for
| the gradient ascent operation.
| figsize (tuple, optional, default=(4, 4)): The size of the plot.
| Relevant in case 1 above.
| title (str, optional default='Conv2d'): The title of the plot.
| return_output (bool, optional, default=False): Returns the
| output(s) of optimization if set to True.
|
| Returns:
| output (list of torch.Tensor): With dimentions
| :math:`(num_iter, C, H, W)`. The size of the image is
| determined by `img_size` attribute which defaults to 224.
|
| optimize(self, layer, filter_idx, input_=None, num_iter=30)
| Generates an image that maximally activates the target filter.
|
| Args:
| layer (torch.nn.modules.conv.Conv2d): The target Conv2d layer from
| which the filter to be chosen, based on `filter_idx`.
| filter_idx (int): The index of the target filter.
| num_iter (int, optional, default=30): The number of iteration for
| the gradient ascent operation.
|
| Returns:
| output (list of torch.Tensor): With dimentions
| :math:`(num_iter, C, H, W)`. The size of the image is
| determined by `img_size` attribute which defaults to 224.
|
| visualize(self, layer, filter_idxs=None, lr=1.0, num_iter=30, num_subplots=4, figsize=(4, 4), title='Conv2d', return_output=False)
| Optimizes for the target layer/filter and visualizes the output.
|
| A method that combines optimization and visualization. There are
| mainly 3 types of operations, given a target layer:
|
| 1. If `filter_idxs` is provided as an integer, it optimizes for the
| filter specified and plots the output.
| 2. If `filter_idxs` is provided as a list of integers, it optimizes for
| all the filters specified and plots the output.
| 3. if `filter_idx` is not provided, i.e. None, it randomly chooses
| `num_subplots` number of filters from the layer provided and
| plots the output.
|
| It also returns the output of the optimization, if specified with
| `return_output=True`.
|
| Args:
| layer (torch.nn.modules.conv.Conv2d): The target Conv2d layer from
| which the filter to be chosen, based on `filter_idx`.
| filter_idxs (int or list of int, optional, default=None): The index
| or indecies of the target filter(s).
| lr (float, optional, default=.1): The step size of optimization.
| num_iter (int, optional, default=30): The number of iteration for
| the gradient ascent operation.
| num_subplots (int, optional, default=4): The number of filters to
| optimize for and visualize. Relevant in case 3 above.
| figsize (tuple, optional, default=(4, 4)): The size of the plot.
| Relevant in case 1 above.
| title (str, optional default='Conv2d'): The title of the plot.
| return_output (bool, optional, default=False): Returns the
| output(s) of optimization if set to True.
|
|
| Returns:
| For a single optimization (i.e. case 1 above):
| output (list of torch.Tensor): With dimentions
| :math:`(num_iter, C, H, W)`. The size of the image is
| determined by `img_size` attribute which defaults to 224.
| For multiple optimization (i.e. case 2 or 3 above):
| output (list of list of torch.Tensor): With dimentions
| :math:`(num_subplots, num_iter, C, H, W)`. The size of the
| image is determined by `img_size` attribute which defaults
| to 224.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
|
| img_size
|
| lr
|
| use_gpu
FILE
/Users/misao/Projects/personal/flashtorch/flashtorch/activmax/gradient_ascent.py