luna.featurevis¶

luna.featurevis.featurevis¶

The main file for the feature vis process

class luna.featurevis.featurevis.OptimizationParameters(iterations: int, learning_rate: Optional[int], optimizer: Optional[object])¶

Bases: object

object for generalizing optimization parameters.

iterations: int¶

learning_rate: Optional[int]¶

optimizer: Optional[object]¶

luna.featurevis.featurevis.compute_activation(input_image, model, filter_index, regularization)¶

Computes the loss for the feature visualization process.

Parameters

input_image (array) – the image that is used to compute the loss.
model (object) – the model on which to compute the loss.
filter_index (int) – for which filter to compute the loss.
False. (Defaults to) –
regularization (function) – a function defining the regularizations to be perfromed.

Returns

the activation for the specified setting

Return type

number

luna.featurevis.featurevis.get_feature_extractor(model, layer_name)¶

Builds a model that that returns the activation of the specified layer.

Parameters

model (object) – the model used as a basis for the feature extractor.
layer (string) – the layer at which to cap the original model.

luna.featurevis.featurevis.gradient_ascent_step(img, model, filter_index, regularization, optimization_parameters, minimize)¶

Performing one step of gradient ascend.

Parameters

img (array) – the image to be changed by the gradiend ascend.
model (object) – the model with which to perform the image change.
filter_index (number) – which filter to optimize for.
regularization (function) – a function defining the regularizations to be perfromed.
optimization_parameters (OptimizationParameters) – optimizer (only Adam is supported)
minimize (bool) – whether or not to apply minimize as opposed to calling apply_gradient() for adam optimizer.

Returns

the activation and the modified image

Return type

tuple

luna.featurevis.featurevis.threshold_view(image)¶

Intermediate visualizer.

Parameters: image (list) – Image.

luna.featurevis.featurevis.visualize_filter(image, model, layer, filter_index, optimization_parameters, transformation=None, regularization=None, threshold=None, minimize=False)¶

Create a feature visualization for a filter in a layer of the model.

Parameters

image (array) – the image to be modified by the feature vis process.
model (object) – the model to be used for the feature visualization.
layer (string) – the name of the layer to be used in the visualization.
filter_index (number) – the index of the filter to be visualized.
optimization_parameters (OptimizationParameters) – the optimizer class to be applied.
transformations (function) – a function defining the transformations to be perfromed.
regularization (function) – customized regularizers to be applied. Defaults to None.
threshold (list) – Intermediate steps for visualization. Defaults to None.
minimize (bool) – whether or not to apply minimize as opposed to calling apply_gradient() for adam optimizer.

Returns

activation and result image for the process.

Return type

tuple

luna.featurevis.image_reader¶

A utility file that takes all numpy arrays in the input folder and saves them as pngs in the output folder

luna.featurevis.image_reader.save_npy_as_png(input_path, output_path)¶

Saves a numpy image as a png given an image path.

Parameters: path (string) – the path to the numpy file containing the image data.

luna.featurevis.images¶

The utility functions for creating and processing the images for the feature visualisation process.

luna.featurevis.images.deprocess_image(img)¶

Takes the values of an image array and normalizes them to be in the standard 0-255 RGB range.

Parameters: img (List[float]) – The generated image array.
Returns: A rescaled version of the image.

luna.featurevis.images.fft_image(img, std=None, decay_power=1)¶

Image parameterization using 2D Fourier coefficients.

Parameters

shape (list[int]) – Image shape.
sd (float) – standard deviation as noise.
decay_power (int) – Defaults to 1.

Returns

New image in spatial domain.

luna.featurevis.images.initialize_image(width, height, val_range_top=1.0, val_range_bottom=- 1.0)¶

Creates an initial randomized image to start feature vis process. This could be subject to optimization in the future.

Parameters

width (int) – The width of the image.
height (int) – The height of the image.

Returns

A randomly generated image.

luna.featurevis.images.initialize_image_ref(width, height, std=None, fft=True, decorrelate=True, channels=None, seed=None)¶

Creates an initial randomized image to start feature vis process. This could be subject to optimization in the future.

Parameters

width (int) – The width of the image.
height (int) – The height of the image.
sd (float) – standard deviation for noise initialization.
fft (boolean) – Image parameterization with fast fourier transformation.
deccorelate (boolean) – the color interpretation of the image tensor’s color.
channels (boolean) – True for gray images.
seed (int) – Random seed for the image initialization for reproducibility.

Returns

A randomly generated image.

luna.featurevis.images.rfft2d_freqs(height, width)¶

computation of 2D frequencies of spectrum.

Parameters

h (int) – image height.
w (int) – image width.

Returns

2D spectrum frequencies.

Return type

spectrum frequency

luna.featurevis.images.save_image(img, name=None)¶

Saves a generated image array as a numpy array in a file.

Parameters

img (pil.Image) – The generated image.
name (str) – A possible name, if none given it is auto generated.

luna.featurevis.images.to_valid_rgb(image, decorrelate=False, sigmoid=True)¶

Transformation of input tensor to valid rgb colors.

Parameters

image (tf.Tensor) – Input image.
decorrelate (bool) – Color interpretation from whitened space if it is True.
sigmoid (bool) – Color constrained if it is True.

Returns

Transfomed image with the innermost dimension.

luna.featurevis.relu_grad¶

Adopted from [2017] [https://github.com/tensorflow/lucid]

luna.featurevis.relu_grad.gradient_override_map(override_dict)¶

Convenience wrapper for graph.gradient_override_map(). This functions provides two conveniences over normal tensorflow gradient overrides: it auomatically uses the default graph instead of you needing to find the graph, and it automatically .. rubric:: Example

def _foo_grad_alt(op, grad): … with gradient_override({“Foo”: _foo_grad_alt}):

Parameters

override_dict – A dictionary describing how to override the gradient.
keys – strings correponding to the op type that should have their gradient overriden.
values – functions or strings registered to gradient functions

luna.featurevis.relu_grad.redirected_relu6_grad(operation, grad)¶

Compute ReLu6 gradients

Parameters

operation (str) – the type of operation to convert, needs to be Relu6.
grad (tf.Tensor) – the gradient tensor.

Returns

Redirected gradients bigger than 6.

luna.featurevis.relu_grad.redirected_relu_grad(operation, grad)¶

Compute ReLu gradient.

Parameters

operation (str) – the type of operation to convert, needs to be Relu.
grad (tf.Tensor) – the gradient tensor.

Returns

Redirected gradients smaller than 0.

luna.featurevis.relu_grad.register_to_random_name(grad_f)¶

Register a gradient function to a random string. In order to use a custom gradient in TensorFlow, it must be registered to a string. This is both a hassle, and – because only one function can every be registered to a string – annoying to iterate on in an interactive environemnt. This function registers a function to a unique random string of the form: {FUNCTION_NAME}_{RANDOM_SALT} and then returns the random string. This is a helper in creating more convenient gradient overrides.

Parameters: grad_f – gradient function to register. Should map (op, grad) -> grad(s)
Returns: String that gradient function was registered to.

luna.featurevis.transformations¶

The transformations for feature vis process

luna.featurevis.transformations.angle_conversion(angle, units)¶

Converting the angle to the desirable format

Parameters

angle (list) – list of random angle values
units (str) – conversion unit

Returns

converted angle values to the desirable form

Return type

list

luna.featurevis.transformations.bilinear_rescale(img, *args, seed=None)¶

rescaling image by bilinear interpolation

Parameters

img (list) – the current state of the feature vis image.
*args (tuple) – list of arguments i.e. rescale values

Returns

the rescaled image.

Return type

list

luna.featurevis.transformations.blur(img)¶

Gaussian blur the image to be modified.

Parameters: img (list) – the image to be blurred.
Returns: the blurred image.
Return type: list

luna.featurevis.transformations.color_augmentation(img)¶

Augmenting the color.

Parameters: image (list) – Image.
Returns: Augmented image.
Return type: list

luna.featurevis.transformations.crop_or_pad(img)¶

Randomly crop or pad the image.

Parameters: image (list) – Image.
Returns: the cropped/padded image.
Return type: list

luna.featurevis.transformations.flip(img)¶

Fliping the image up, down, left, right.

Parameters: img (list) – image.
Returns: the flipped image.
Return type: list

luna.featurevis.transformations.gaussian_blur(img, kernel_size=3, sigma=5)¶

Calculates the gaussian convolution kernel for the blurring process.

Parameters

img – the current state of the image.
kernel_size – size of the convolution kernel used for blurring.
sigma – gaussian blurring constant.

Returns

The altered image.

luna.featurevis.transformations.jitter(img, jitter_dist, seed=None)¶

Jitter the image & add an artistic effect to the image.

Parameters

img (list) – the image data to be jittered.
jitter_dist (int) – the distance of effective neighborhoods.
seed (int, optional) – Defaults to None.

Returns

the jittered image.

Return type

list

luna.featurevis.transformations.ninety_degree_rotation(img)¶

Rotating the image by ninety degree.

Parameters: img (list) – image.
Returns: rotated image.
Return type: img (list)

luna.featurevis.transformations.noise(img)¶

Adds noise to the image to be manipulated.

Parameters: img (list) – the image data to which noise should be added.
Returns: the modified image.
Return type: list

luna.featurevis.transformations.pad(img, pad_size, pad_mode='REFLECT', constant_value=0.5)¶

Pad the image

Parameters

img (list) – the image data to be padded.
pad_size (int) – size of pad.
mode (str, optional) – mode for padding; can be “CONSTANT”, “REFLECT” or “SYMMETRIC”.
constant_value (float, optional) – The scalar pad value. Defaults to 0.5.

Returns

the padded image.

Return type

list

luna.featurevis.transformations.random_select(rand_values, seed=None)¶

Generate the upper bound on the range of random value.

Parameters

rand_scale (list) – random selection for upper bound
seed (int, optional) – for reproducibility. Defaults to None.

Returns

list of random values

Return type

list

luna.featurevis.transformations.rotation(img, *args, units='degrees', seed=None)¶

Rotating the image with random angle

Parameters

img (list) – the image data
*args (tuple) – list of arguments i.e. angles of rotatation
units (str, optional) – the unit of rotation move. Defaults to “degrees”.
seed ([type], optional) – for reproducibility. Defaults to None.

Returns

rotated image with selected values

Return type

list

luna.featurevis.transformations.scale_values(img)¶

Rescales the image.

Parameters: img (list) – the current state of the feature vis image.
Returns: the scaled image.
Return type: list

luna.featurevis.transformations.standard_transformation(img)¶

Standard transformations if no transformations were chosen by user (Suggested by Lucid)

Parameters: img (list) – image.
Returns: transformed imaged with standard modes.
Return type: list