Internal API¶
Bricks¶
-
class
blocks.bricks.base.
Application
(application_function)[source]¶ Bases:
object
An application method belonging to a particular type of brick.
The application methods of each
Brick
class are automatically replaced by an instance ofApplication
. This allows us to store metadata about particular application methods (such as their in- and outputs) easily.-
delegate_function
¶ callable – A function that takes a
Brick
instance as an argument and returns aBoundApplication
object to which attribute requests should be routed.
-
properties
¶ dict
(str
,callable
) – A dictionary of property getters that should be called when an attribute with the given name is requested.
-
instances
¶ dict
(Brick
,BoundApplication
) – A record of bound application instances created by the descriptor protocol.
-
call_stack
¶ list
ofBrick
– The call stack of brick application methods. Used to check whether the current call was made by a parent brick.
-
brick
¶ type – The brick class to which this instance belongs.
Raises: ValueError
– If a brick’s application method is applied by another brick which does not list the former as a child.ValueError
– If the application method’s inputs and/or outputs don’t match with the function signature or the values returned (respectively).
Notes
When a
Brick
is instantiated and its application method (i.e. an instance of this class) requested, the descriptor protocol (through the__get__()
method) automatically instantiates aBoundApplication
class and returns this. This bound application class can be used to store application information particular to a brick instance. Any attributes unknown to the bounded application are automatically routed to the application that instantiated it.-
application_function
¶
-
call_stack
= []
-
delegate
(f)[source]¶ Decorator to assign a delegate application.
An application method can assign a delegate application. Whenever an attribute is not available, it will be requested from the delegate instead.
Examples
>>> class Foo(Brick): ... @application(outputs=['baz']) ... def apply(self, x): ... return x + 1 ... ... @apply.property('inputs') ... def apply_inputs(self): ... return ['foo', 'bar'] >>> class Bar(Brick): ... def __init__(self, foo): ... self.foo = foo ... ... @application(outputs=['foo']) ... def apply(self, x): ... return x + 1 ... ... @apply.delegate ... def apply_delegate(self): ... return self.foo.apply >>> foo = Foo() >>> bar = Bar(foo) >>> bar.apply.outputs ['foo'] >>> bar.apply.inputs ['foo', 'bar']
-
inputs
¶
-
name
¶
-
property
(name)[source]¶ Decorator to make application properties.
Parameters: name (str) – The name the property should take. Examples
>>> class Foo(Brick): ... @application ... def apply(self, x): ... return x + 1 ... ... @apply.property('inputs') ... def apply_inputs(self): ... return ['foo', 'bar'] >>> foo = Foo() >>> foo.apply.inputs ['foo', 'bar']
-
-
class
blocks.bricks.base.
ApplicationCall
(application)[source]¶ Bases:
blocks.graph.annotations.Annotation
A link between the variable tags and bricks.
The application call can be used to attach to an apply call auxiliary variables (e.g. monitors or regularizers) that do not form part of the main computation graph.
The application call object is created before the call to the application method and can be accessed by specifying an application_call argument.
Also see
Annotation
.Parameters: application ( BoundApplication
instance) – The bound application (i.e. belong to a brick instance) object being calledExamples
>>> class Foo(Brick): ... @application ... def apply(self, x, application_call): ... application_call.add_auxiliary_variable(x.mean()) ... return x + 1 >>> x = tensor.vector() >>> y = Foo().apply(x) >>> from blocks.filter import get_application_call >>> get_application_call(y) <blocks.bricks.base.ApplicationCall object at ...>
-
add_auxiliary_variable
(variable, roles=None, name=None)[source]¶ Attach an auxiliary variable to the graph.
Auxiliary variables are Theano variables that are not part of a brick’s output, but can be useful nonetheless e.g. as a regularizer or to monitor during training progress.
Parameters: - variable (
TensorVariable
) – The variable you want to add. - roles (list of
VariableRole
instances, optional) – The roles of this variable. TheAUXILIARY
role will automatically be added. Other options areCOST
,WEIGHT
, etc. - name (str, optional) – Name to give to the variable. If the variable already has a name it will be overwritten.
Examples
>>> from blocks.bricks.base import application, Brick >>> from blocks.roles import COST >>> from blocks.utils import shared_floatx_nans >>> class Foo(Brick): ... def _allocate(self): ... W = shared_floatx_nans((10, 10)) ... self.add_auxiliary_variable(W.mean(), name='mean_W') ... @application ... def apply(self, x, application_call): ... application_call.add_auxiliary_variable( ... x - 1, name='x_minus_1') ... application_call.add_auxiliary_variable( ... x.mean(), roles=[COST], name='mean_x') ... return x + 1 >>> from theano import tensor >>> x = tensor.vector() >>> y = Foo().apply(x) >>> from blocks.graph import ComputationGraph >>> cg = ComputationGraph([y]) >>> from blocks.filter import VariableFilter >>> var_filter = VariableFilter(roles=[AUXILIARY]) >>> var_filter(cg.variables) {x_minus_1, mean_W, mean_x} >>> var_filter = VariableFilter(roles=[COST]) >>> var_filter(cg.variables) {mean_x}
- variable (
-
-
class
blocks.bricks.base.
BoundApplication
(application, brick)[source]¶ Bases:
object
An application method bound to a
Brick
instance.-
name
¶
-
-
class
blocks.bricks.base.
Brick
(name=None, children=None)[source]¶ Bases:
blocks.graph.annotations.Annotation
A brick encapsulates Theano operations with parameters.
A brick goes through the following stages:
- Construction: The call to
__init__()
constructs aBrick
instance with a name and creates any child bricks as well. - Allocation of parameters:
- Allocation configuration of children: The
push_allocation_config()
method configures any children of this block. - Allocation: The
allocate()
method allocates the shared Theano variables required for the parameters. Also allocates parameters for all children.
- Allocation configuration of children: The
- The following can be done in either order:
- Application: By applying the brick to a set of Theano variables a part of the computational graph of the final model is constructed.
- The initialization of parameters:
- Initialization configuration of children: The
push_initialization_config()
method configures any children of this block. - Initialization: This sets the initial values of the
parameters by a call to
initialize()
, which is needed to call the final compiled Theano function. Also initializes all children.
- Initialization configuration of children: The
Not all stages need to be called explicitly. Step 3(a) will automatically allocate the parameters if needed. Similarly, step 3(b.2) and 2(b) will automatically perform steps 3(b.1) and 2(a) if needed. They only need to be called separately if greater control is required. The only two methods which always need to be called are an application method to construct the computational graph, and the
initialize()
method in order to initialize the parameters.At each different stage, a brick might need a certain set of configuration settings. All of these settings can be passed to the
__init__()
constructor. However, by default many bricks support lazy initialization. This means that the configuration settings can be set later.Note
Some arguments to
__init__()
are always required, even when lazy initialization is enabled. Other arguments must be given before callingallocate()
, while others yet only need to be given in order to callinitialize()
. Always read the documentation of each brick carefully.Lazy initialization can be turned off by setting
Brick.lazy = False
. In this case, there is no need to callinitialize()
manually anymore, but all the configuration must be passed to the__init__()
method.Parameters: name (str, optional) – The name of this brick. This can be used to filter the application of certain modifications by brick names. By default, the brick receives the name of its class (lowercased). -
name
¶ str – The name of this brick.
-
print_shapes
¶ bool –
False
by default. IfTrue
it logs the shapes of all the input and output variables, which can be useful for debugging.
-
parameters
¶ list of
TensorSharedVariable
andNone
– After calling theallocate()
method this attribute will be populated with the shared variables storing this brick’s parameters. Allows forNone
so that parameters can always be accessed at the same index, even if some parameters are only defined given a particular configuration.
-
children
¶ list of bricks – The children of this brick.
-
allocated
¶ bool –
False
ifallocate()
has not been called yet.True
otherwise.
-
initialized
¶ bool –
False
ifallocate()
has not been called yet.True
otherwise.
-
allocation_config_pushed
¶ bool –
False
ifallocate()
orpush_allocation_config()
hasn’t been called yet.True
otherwise.
-
initialization_config_pushed
¶ bool –
False
ifinitialize()
orpush_initialization_config()
hasn’t been called yet.True
otherwise.
Notes
To provide support for lazy initialization, apply the
lazy()
decorator to the__init__()
method.Brick implementations must call the
__init__()
constructor of their parent using super(BlockImplementation, self).__init__(**kwargs) at the beginning of the overriding __init__.The methods
_allocate()
and_initialize()
need to be overridden if the brick needs to allocate shared variables and initialize their values in order to function.A brick can have any number of methods which apply the brick on Theano variables. These methods should be decorated with the
application()
decorator.If a brick has children, they must be listed in the
children
attribute. Moreover, if the brick wants to control the configuration of its children, the_push_allocation_config()
and_push_initialization_config()
methods need to be overridden.Examples
Most bricks have lazy initialization enabled.
>>> import theano >>> from blocks.initialization import IsotropicGaussian, Constant >>> from blocks.bricks import Linear >>> linear = Linear(input_dim=5, output_dim=3, ... weights_init=IsotropicGaussian(), ... biases_init=Constant(0)) >>> x = theano.tensor.vector() >>> linear.apply(x) # Calls linear.allocate() automatically linear_apply_output >>> linear.initialize() # Initializes the weight matrix
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 34¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
_allocate
()[source]¶ Brick implementation of parameter initialization.
Implement this if your brick needs to allocate its parameters.
Warning
This method should never be called directly. Call
initialize()
instead.
-
_initialize
()[source]¶ Brick implementation of parameter initialization.
Implement this if your brick needs to initialize its parameters.
Warning
This method should never be called directly. Call
initialize()
instead.
-
_push_allocation_config
()[source]¶ Brick implementation of configuring child before allocation.
Implement this if your brick needs to set the configuration of its children before allocation.
Warning
This method should never be called directly. Call
push_allocation_config()
instead.
-
_push_initialization_config
()[source]¶ Brick implementation of configuring child before initialization.
Implement this if your brick needs to set the configuration of its children before initialization.
Warning
This method should never be called directly. Call
push_initialization_config()
instead.
-
allocate
()[source]¶ Allocate shared variables for parameters.
Based on the current configuration of this
Brick
create Theano shared variables to store the parameters. After allocation, parameters are accessible through theparameters
attribute.This method calls the
allocate()
method of all children first, allowing the_allocate()
method to override the parameters of the children if needed.Raises: ValueError
– If the configuration of this brick is insufficient to determine the number of parameters or their dimensionality to be initialized.Notes
This method sets the
parameters
attribute to an empty list. This is in order to ensure that calls to this method completely reset the parameters.
-
children
-
get_dim
(name)[source]¶ Get dimension of an input/output variable of a brick.
Parameters: name (str) – The name of the variable.
-
get_dims
(names)[source]¶ Get list of dimensions for a set of input/output variables.
Parameters: names (list) – The variable names. Returns: dims – The dimensions of the sources. Return type: list
-
get_hierarchical_name
(parameter, delimiter='/')[source]¶ Return hierarhical name for a parameter.
Returns a path of the form
brick1/brick2/brick3.parameter1
. The delimiter is configurable.Parameters: delimiter (str) – The delimiter used to separate brick names in the path.
-
initialize
()[source]¶ Initialize parameters.
Intialize parameters, such as weight matrices and biases.
Notes
If the brick has not allocated its parameters yet, this method will call the
allocate()
method in order to do so.
-
parameters
-
print_shapes
= False
-
push_allocation_config
()[source]¶ Push the configuration for allocation to child bricks.
Bricks can configure their children, based on their own current configuration. This will be automatically done by a call to
allocate()
, but if you want to override the configuration of child bricks manually, then you can call this function manually.
-
push_initialization_config
()[source]¶ Push the configuration for initialization to child bricks.
Bricks can configure their children, based on their own current configuration. This will be automatically done by a call to
initialize()
, but if you want to override the configuration of child bricks manually, then you can call this function manually.
- Construction: The call to
-
class
blocks.bricks.base.
Children
(brick, *args, **kwargs)[source]¶ Bases:
blocks.utils.containers.AnnotatingList
Adds the brick to the list of parents of its children.
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 34¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
blocks.bricks.base.
Parameters
(brick, *args, **kwargs)[source]¶ Bases:
blocks.utils.containers.AnnotatingList
Adds the PARAMETER role to parameters automatically.
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 34¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
blocks.bricks.base.
_Brick
[source]¶ Bases:
abc.ABCMeta
Metaclass which attaches brick instances to the applications.
In addition picklability of
Application
objects is ensured. This means thatApplication
objects can not be added to a brick class after it is created. To allow adding application methods programatically, the following hook is supported: the class namespace is searched for decorators attribute, which can contain a list of functions to be applied to the namespace of the class being created. These functions can arbitratily modify this namespace.
-
blocks.bricks.base.
application
(*args, **kwargs)[source]¶ Decorator for methods that apply a brick to inputs.
Parameters: - optional (**kwargs,) – The application method to wrap.
- optional – Attributes to attach to this application.
Notes
This decorator replaces application methods with
Application
instances. It also sets the attributes given as keyword arguments to the decorator.Note that this decorator purposely does not wrap the original method using e.g.
wraps()
orupdate_wrapper()
, since that would make the class impossible to pickle (see notes atApplication
).Examples
>>> class Foo(Brick): ... @application(inputs=['x'], outputs=['y']) ... def apply(self, x): ... return x + 1 ... @application ... def other_apply(self, x): ... return x - 1 >>> foo = Foo() >>> Foo.apply.inputs ['x'] >>> foo.apply.outputs ['y'] >>> Foo.other_apply <blocks.bricks.base.Application object at ...>
-
blocks.bricks.base.
copy_and_tag
(variable, brick, call, role, application_name, name)[source]¶ Helper method to copy a variable and annotate it.
-
blocks.bricks.base.
create_unbound_method
(func, cls)[source]¶ Create an unbounded method from a function and a class.
Notes
-
blocks.bricks.base.
lazy
(allocation=None, initialization=None)[source]¶ Makes the initialization lazy.
This decorator allows the user to define positional arguments which will not be needed until the allocation or initialization stage of the brick. If these arguments are not passed, it will automatically replace them with a custom
None
object. It is assumed that the missing arguments can be set after initialization by setting attributes with the same name.Parameters: Examples
>>> class SomeBrick(Brick): ... @lazy(allocation=['a'], initialization=['b']) ... def __init__(self, a, b, c='c', d=None): ... print(a, b, c, d) >>> brick = SomeBrick('a') a NoneInitialization c None >>> brick = SomeBrick(d='d', b='b') NoneAllocation b c d
-
class
blocks.bricks.
Activation
(name=None, children=None)[source]¶ Bases:
blocks.bricks.base.Brick
Elementwise application of activation function.
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 34¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
blocks.bricks.interfaces.
ActivationDocumentation
[source]¶ Bases:
blocks.bricks.base._Brick
Dynamically adds documentation to activations.
Notes
Extensions¶
-
class
blocks.extensions.predicates.
OnLogRecord
(record_name)[source]¶ Bases:
object
Trigger a callback when a certain log record is found.
Parameters: record_name (str) – The record name to check.
-
class
blocks.monitoring.evaluators.
AggregationBuffer
(variables, use_take_last=False)[source]¶ Bases:
object
Intermediate results of aggregating values of Theano variables.
Encapsulates aggregators for a list of Theano variables. Collects the respective updates and provides initialization and readout routines.
Parameters: - variables (list of
TensorVariable
) – The variable names are used as record names in the logs. Hence, all the variable names must be unique. - use_take_last (bool) – When
True
, theTakeLast
aggregation scheme is used instead of_DataIndependent
for those variables that do not require data to be computed.
-
initialization_updates
¶ list of tuples – Initialization updates of the aggregators.
-
accumulation_updates
¶ list of tuples – Accumulation updates of the aggregators.
-
readout_variables
¶ dict – A dictionary of record names to
TensorVariable
representing the aggregated values.
-
inputs
¶ list of
TensorVariable
– The list of inputs needed for accumulation.
- variables (list of
-
class
blocks.monitoring.evaluators.
DatasetEvaluator
(variables, updates=None)[source]¶ Bases:
object
A DatasetEvaluator evaluates many Theano variables or other quantities.
The DatasetEvaluator provides a do-it-all method,
evaluate()
, which computes values ofvariables
on a dataset.Alternatively, methods
initialize_aggregators()
,process_batch()
,get_aggregated_values()
can be used with a custom loop over data.The values computed on subsets of the given dataset are aggregated using the
AggregationScheme`s provided in the `aggregation_scheme
tags. If no tag is given, the value is averaged over minibatches. However, care is taken to ensure that variables which do not depend on data are not unnecessarily recomputed.Parameters: - variables (list of
TensorVariable
and) –MonitoredQuantity
The variable names are used as record names in the logs. Hence, all the names must be unique.Each variable can be tagged with an
AggregationScheme
that specifies how the value can be computed for a data set by aggregating minibatches. - updates (list of tuples or
OrderedDict
or None) –TensorSharedVariable
updates to be performed during evaluation. This parameter is only for Theano variables. Be careful not to update any model parameters as this is not intended to alter your model in any meaningfullway. A typical use case of this option arises when the theano function used for evaluation contains a call to:function:~theano.scan which might have returned shared variable updates.
-
_compile
()[source]¶ Compiles Theano functions.
Todo
The current compilation method does not account for updates attached to ComputationGraph elements. Compiling should be out-sourced to ComputationGraph to deal with it.
- variables (list of
-
class
blocks.monitoring.evaluators.
MonitoredQuantityBuffer
(quantities)[source]¶ Bases:
object
Intermediate results of aggregating values of monitored-quantity.
Aggregate results for a list of monitored-quantity for every single batch. Provides initialization and readout routines to initialize each quantity and capture its aggregated results.
Parameters: quantities (list of MonitoredQuantity
) – The quantity names are used as record names in the logs. Hence, all the quantity names must be unique.-
requires
¶ list of
TensorVariable
– Needed to calculate monitored-quantities.
-
quantity_names
¶ list of str – Names of quantities.
-
inputs
¶ list of
TensorVariable
– The list of inputs needed for variables in requires.
-
Utils¶
-
class
blocks.utils.containers.
AnnotatingList
(items=None)[source]¶ Bases:
_abcoll.MutableSequence
Mutable sequence performing operations on inserted/removed items.
Parameters: items (iterable, optional) – An iterable of items to initialize the sequence with. -
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 34¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
blocks.utils.profile.
Profile
[source]¶ Bases:
object
A profile of hierarchical timers.
Keeps track of timings performed with
Timer
. It also keeps track of the way these timings were nested and makes use of this information when reporting.
-
class
blocks.utils.profile.
Timer
(name, profile)[source]¶ Bases:
object
A context manager to time the execution time of code within it.
This timer is attached to a
Profile
object that it reports timings to. TheProfile
object accumulates the timings. Timers can be nested, which theProfile
will automatically keep track of and use in its reporting.Parameters: Notes
Timings are reported using
timeit.default_timer()
.