Accumulator Module

Source code

Entry in Module Selection	Module Appearance

What Does The Accumulator Do?

The accumulator module reconstructs frames from events by accumulating the changes to obtain a corresponding intensity information. The computation of the intensity is done with a configurable decay function.

Use cases:

• Create high-speed event frames (more details here)

• Frame reconstruction (more details here or here)

• Long term noise accumulation

• Creating representations to feed into a convolutional neural network

Accumulation can be used whenever there is a need to convert events into frames

Note: More info about accumulator implementation can be found in the dv-processing documentation.

Accumulator Settings Overview

Accumulation Time

Accumulation time sets the time window to accumulate events before sending out a new frame in milliseconds. This effectively sets the frame rate of the accumulator. The accumulator does not reset the frame at generation time, the contents of the old frame are carried over to the next frame. To achieve an effect where the frame gets cleared out after each frame, set the decay function to Step and make sure the Decay param is set to the same value as the accumulation time. Note that this value is in milliseconds, while the other is in microseconds.

Decay Function / Decay Parameter

One of None, Linear, Exponential, Step. Defines the data degradation function that should be applied to the image. For each function, the Decay param setting assumes a different function:

Function	Decay param function	Explanation
None	No function	Does not apply any decay
Linear	The slope a of the linear function, in intensity per microsecond	Assume the intensity of a pixel is I0 at time 0, this function applies a linear decay of the form of I0 - (t * decayparam) or I0 + (t * decayparam)until the value hits the value specified in neutral potential
Exponential	The time constant tau of the exponential function in microseconds	Assume the intensity of a pixel is I0 at time 0, this function applies an exponential decay of the form I0 * exp(-(t/decayparam)). The decay approaches a value of neutralPotential over time.
Step	No function	Set all pixel values to neutral potential after a frame is extracted.

Event Contribution

The contribution an event has to the image. If an event arrives at a position x, y, the pixel value in the frame at x, y gets increased / decreased by the value of Event contribution, based on the events polarity.

Except:

• The resulting pixel value would be higher than Max potential, the value gets set to Max potential instead

• The resulting pixel value would be lower than Min potential, the value gets set to Min potential instead

• The event polarity is negative, and Rectify polarity is enabled, then the event is counted positively

Min Potential / Max Potential

Sets the minimum and maximum values a pixel can achieve. If the value of the pixel would reach higher or lower, it is capped at these values. These values are also used for normalization at the output. The frame the module generates is an unsigned 8-bit grayscale image, normalized between Min potential and Max potential. A pixel with the value Min potential corresponds to a pixel with the value 0 in the output frame. A pixel with the value Max potential corresponds to a pixel with the value 255 in the output frame.

Neutral Potential

Sets the neutral potential. This has different effects, depending on the decay function:

Function	Neutral potential function
None	No function
Linear	The value the linear function tends to over time
Exponential	No function
Step	The value the frame gets reset to after getting a frame

Rectify Polarity

If this value is set, all events act as if they had positive polarity. In this case, Event contribution is always taken positively.

Synchronous Decay

If this value is set, decay happens continuously for all pixels. In every frame, each pixel will be eagerly decayed to the time the image gets generated. If this value is not set, decay at the individual pixel only happens when the pixel receives an event. Decay is lazily evaluated at the pixel.

Note that both decay regimes yield the same overall decay over time, but the time it is applied changes. This parameter does not affect Step decay. Step decay is always synchronous at generation time.

Color Demosaicing

Adds a Bayer color demosaicing filter onto the generated frame. Note that this setting only makes sense when a color dvs camera is connected.

Example Configurations

High FPS Event Frames

For a use case with a very fast frame-based processing engine, one can get high-speed HDR images of the moving edges. The following settings give 1000 fps with frames that are exposed over 1 ms integration time.

Setting	Value
Accumulation time	1 (ms)
Decay function	Step
Decay param	Ignored
Min potential	0
Max potential	1
Neutral potential	0.5
Event contribution	0.5
Rectify polarity	No
Synchronous decay	Ignored

Exponentially Decayed Frame Reconstruction

The following configuration gives a good frame reconstruction if the scene has a lot of activity. The exponential decay is strong, which leads to strong shadows when a movement changes direction. The strong decay suppresses noise quite effectively.

Setting	Value
Accumulation time	33 (ms)
Decay function	Exponential
Decay param	1000000 (us)
Min potential	0
Max potential	0.3
Neutral potential	0
Event contribution	0.04
Rectify polarity	No
Synchronous decay	Yes

The fps can be arbitrarily set. In this example, Accumulation time is set to 33ms to achieve 30 fps. The quality is independent of the fps, if you require 1000 fps, you can set the Accumulation time to 1 ms.

Linear Decayed Frame Reconstruction

A reconstruction can also be achieved with a linear decay. Generally, this tends to preserve less detail but suffers less from shadows.

Setting	Value
Accumulation time	33 (ms)
Decay function	Linear
Decay param	0.000001 (intensity / us)
Min potential	0
Max potential	1
Neutral potential	0.5
Event contribution	0.15
Rectify polarity	No
Synchronous decay	Yes

Analogous to the exponential case, one can set the fps by changing the Accumulation time parameter.

Further Configurations

It is advised to experiment with the accumulator settings until the desired effect is achieved.