Data stream interface for tape data. Encapsulates data that is written e.g. for each statement or argument. More...

#include <dataInterface.hpp>

Inheritance diagram for codi::DataInterface< T_NestedData, T_InternalPosHandle >:

[legend]

Public Types
using	InternalPosHandle = T_InternalPosHandle
	See DataInterface.

using	NestedData = T_NestedData
	See DataInterface.

using	Position = EmptyPosition
	Contains position data for this DataInterface and all nested interfaces.

Public Member Functions
Adding items
void	addDataSize (size_t const &size)
	Add this many items to the data stream, after the data has been manipulated via pointers obtained from getDataPointers().

template<typename... Data>
void	getDataPointers (Data *&... pointers)
	Get pointers to the data from the storage implementation. The method can only be called after a call to reserveItems() and data can only be accessed from 0 to the number given by reserveItems (excluding). Afterwards, addDataSize() needs to be called with the actual number of elements that have been written.

template<typename... Data>
void	pushData (Data const &... data)
	Add data to the storage allocated by the implementation. The method can only be called after a call to reserveItems and only as often as the number of reserved items.

InternalPosHandle	reserveItems (size_t const &items)
	Reserve this many items on the data stream. See pushData for details.

Size management
void	resize (size_t const &totalSize)

void	reset ()

void	resetHard ()

void	resetTo (Position const &pos)

void	erase (Position const &start, Position const &end, bool recursive=true)

Position functions
size_t	getDataSize () const

Position	getPosition () const

size_t	getPushedDataCount (InternalPosHandle const &startPos)

Position	getZeroPosition () const

Misc functions
void	addToTapeValues (TapeValues &values) const
	Add amount of stored data to the TapeValues object. Not called on the nested vector.

template<typename TargetPosition >
TargetPosition	extractPosition (Position const &pos) const
	Extract the position of a nested DataInterface from the global position object provide by this interface.

void	setNested (NestedData *v)

void	swap (DataInterface &other)

Iterator functions
template<int selectedDepth = -1, typename FunctionObject , typename... Args>
void	evaluateForward (Position const &start, Position const &end, FunctionObject function, Args &&... args)
	Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.

template<int selectedDepth = -1, typename FunctionObject , typename... Args>
void	evaluateReverse (Position const &start, Position const &end, FunctionObject function, Args &&... args)
	Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.

template<typename FunctionObject , typename... Args>
void	forEachChunk (FunctionObject &function, bool recursive, Args &&... args)
	Calls the function object for each continuous segment of data.

template<typename FunctionObject , typename... Args>
void	forEachForward (Position const &start, Position const &end, FunctionObject function, Args &&... args)
	Calls the function object for each item in the data stream. This call is not recursive.

template<typename FunctionObject , typename... Args>
void	forEachReverse (Position const &start, Position const &end, FunctionObject function, Args &&... args)
	Calls the function object for each item in the data stream. This call is not recursive.

Detailed Description

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>
struct codi::DataInterface< T_NestedData, T_InternalPosHandle >

Data stream interface for tape data. Encapsulates data that is written e.g. for each statement or argument.

This interface defines the basic abstraction mechanism of how data is stored in an AD tape. During the recording of an AD tape, different types of data with a varying amount of items need to be stored and associated with each other. There is e.g. data for each statement and data for each argument. Each DataInterface covers one type of data. The management of multiple data streams can become quite cumbersome, therefore the DataInterface is designed in a recursive fashion. Each data stream can be nested with another data stream such that they can exchange position information and synchronize themselves.

Note that the recursive implementation is only that - an implementation. The data itself is not recursive. Think about is as multiple streams of associated data that grow alongside each other in varying speeds.

current state                  new batch of associated data
 
stream 1 ========== * <------ |====
stream 2 =============== * <- |=====
stream 3 === * <------------- |===
stream 4 = * <--------------- |=

The * mark the current joint position of the streams (end of the last pushed batch of data). A new batch of associated data is now appended to the streams.

A data item on the data stream can consist of multiple entries, e.g., a data entry can be an int or it can be an int and a double. The underlying implementation defines how this data is stored, e.g. as an array of objects or as an object of arrays. For counting the number of items, each call to pushData() counts as one item regardless of how many entries each item has.

The getPosition() function produces a position for this DataInterface and all nested DataInterfaces. All methods that have the Position type as an argument or modify the position of the DataInterface work recursively on all nested DataInterfaces.

How the data is stored and allocated is determined by the actual implementation of this interface.

See Example tape implementation with CoDiPack for a usage of this interface.

Example usage:

ArgDataInterface argVector = ...;
StmtDataInterface<ArgDataInterface> stmtVector = ...;  // argVector is nested into the stmtVector.
 
stmtVector.setNested(&argVector);  // Set the pointer to the nested vector.
 
// Record some data.
// 1. Request space, from child to parent.
argVector.reserveItems(2);
stmtVector.reserveItems(1);
 
// 2. Add the data, any order.
argVector.pushData(1.0);
stmtVector.pushData(100);
argVector.pushData(2.0);
 
// Perform some operations on the data.
// How data is provided to func depends on the interface implementations, let's assume start, end, and a data
// pointer.
auto func = (startStmt, endStmt, stmt, startArg, endArg, arg) {
    for (int i = startStmt; i < endStmt; i += 1) { std::cout << stmt[i] << ", ";}
    for (int i = startArg; i < endArg; i += 1) { std::cout << arg[i] << ", ";}
};
stmtVector.evaluateForward(stmtVector.getZeroPosition(), stmtVector.getPosition(), func);
 
// Clear all data vectors.
stmtVector.reset();

For an example on how to use getDataPointers please see the method documentation.

The interface defines methods for adding data to the data stream, getting positional information, resetting the data and iterating over the data.

Adding data:

reserveItems(): Needs to be called before a pushData or getDataPointers() call to ensure that enough space is left.
pushData(): Add the actual data to the data stream.
getDataPointers(): Get pointers to the internally allocated data.
addDataSize(): After using pointers obtained by getDataPointers and adding data via these pointers, inform about the number of data items added.

Positional information:

getPosition(), getZeroPosition(): Global position of the all nested data interfaces.

Reset:

reset(): Clear data, but do not delete allocated memory.
resetHard(): Clear data and free allocated memory.
resetTo(): Clear all data up to the specified position.
erase(): Erase a specific range of data. Whether memory is freed is implementation dependent.

Iterating:

evaluateForward() / evaluateReverse(): Evaluate from start to end and call the function object for each valid data range in the nested data streams.
forEachChunk(): Call the function for each data chunk stored by this DataInterface.
forEachForward() / forEachReverse(): Call a function for each data item in this DataInterface.

Template Parameters

T_NestedData	Nested data vector needs to implement this DataInterface.
T_InternalPosHandle	Position handle of the the implementing class for internal size computations.

Member Function Documentation

◆ addDataSize()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::addDataSize ( size_t const & size )

inline

Add this many items to the data stream, after the data has been manipulated via pointers obtained from getDataPointers().

See getDataPointers for details.

Parameters

[in] size Number of data items that have been written.

◆ addToTapeValues()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::addToTapeValues ( TapeValues & values ) const

Add amount of stored data to the TapeValues object. Not called on the nested vector.

Parameters

[in,out] values Will only create new data entries and no new section.

◆ erase()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::erase	(	Position const &	start,
		Position const &	end,
		bool	recursive = `true`
	)

Erase the given range of data. Implementations may choose to free allocated memory. The parameter recursive controls whether erase is also called on nested interfaces.

◆ evaluateForward()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<int selectedDepth = -1, typename FunctionObject , typename... Args>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::evaluateForward	(	Position const &	start,
		Position const &	end,
		FunctionObject	function,
		Args &&...	args
	)

inline

Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.

func is called for each region of continuous data that is valid for all nested DataInterfaces. func is then called as

function(args...,
         start, end, dataEntry1*, dataEntry2*,
         startNested, endNested, dataEntry1Nested*, dataEntry2Nested*,
         startNestedNested, endNestedNested, dataEntry1NestedNested*,
         ...);

What kind of data is appended by each DataInterface is implementation dependent. The default is the set (start, end, dataEntry1, dataEntry2, etc.) which is appended.

It has to hold start <= end.

Positions count up in this call.

Parameters

[in]	start	Starting position.
[in]	end	Ending position.
[in]	function	Function object called.
[in,out]	args	Additional arguments for the function object.

Template Parameters

selectedDepth	Selected depth for the recursion into nested data interfaces. 0 means that only the data from this interface is used. A value of 1 means that the data from the first nested interface is used, etc.. A value of -1 can be used to select the innermost interface, regardless of the number of nested interfaces.
FunctionObject	Function object which is called.
Args	Arguments for the function object.

◆ evaluateReverse()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<int selectedDepth = -1, typename FunctionObject , typename... Args>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::evaluateReverse	(	Position const &	start,
		Position const &	end,
		FunctionObject	function,
		Args &&...	args
	)

inline

Evaluates the function object with segments of continuous and valid data for all nested DataInterfaces.

Same as evaluateForward. It has to hold start >= end. Positions count down in this call.

Parameters

[in]	start	Starting position.
[in]	end	Ending position.
[in]	function	Function object called.
[in,out]	args	Additional arguments for the function object.

Template Parameters

selectedDepth	Selected depth for the recursion into nested data interfaces. 0 means that only the data from this interface is used. A value of 1 means that the data from the first nested interface is used, etc.. A value of -1 can be used to select the innermost interface, regardless of the number of nested interfaces.
FunctionObject	Function object which is called.
Args	Arguments for the function object.

◆ extractPosition()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<typename TargetPosition >

TargetPosition codi::DataInterface< T_NestedData, T_InternalPosHandle >::extractPosition ( Position const & pos ) const

inline

Extract the position of a nested DataInterface from the global position object provide by this interface.

Parameters

[in] pos Position of the DataInterface.

Template Parameters

TargetPosition Position definition of a nested DataInterface.

◆ forEachChunk()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<typename FunctionObject , typename... Args>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::forEachChunk	(	FunctionObject &	function,
		bool	recursive,
		Args &&...	args
	)

inline

Calls the function object for each continuous segment of data.

The call is

function(chunk, args...);

Chunk is of the type ChunkBase.

Parameters

[in]	function	Function object called.
[in]	recursive	True if same call should be performed for all nested DataInterfaces.
[in,out]	args	Additional arguments for the function object.

Template Parameters

FunctionObject	Function object which is called.
Args	Arguments for the function object.

◆ forEachForward()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<typename FunctionObject , typename... Args>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::forEachForward	(	Position const &	start,
		Position const &	end,
		FunctionObject	function,
		Args &&...	args
	)

inline

Calls the function object for each item in the data stream. This call is not recursive.

The call to function is

function(args..., entry1*, entry2*, ...);

It has to hold start <= end.

Parameters

[in]	start	Starting position.
[in]	end	Ending position
[in]	function	Function object called.
[in,out]	args	Additional arguments for the function object.

Template Parameters

FunctionObject	Function object which is called.
Args	Arguments for the function object.

◆ forEachReverse()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<typename FunctionObject , typename... Args>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::forEachReverse	(	Position const &	start,
		Position const &	end,
		FunctionObject	function,
		Args &&...	args
	)

inline

Calls the function object for each item in the data stream. This call is not recursive.

See forEachForward.

It has to hold start >= end.

Parameters

[in]	start	Starting position.
[in]	end	Ending position
[in]	function	Function object called.
[in,out]	args	Additional arguments for the function object.

Template Parameters

FunctionObject	Function object which is called.
Args	Arguments for the function object.

◆ getDataPointers()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<typename... Data>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::getDataPointers ( Data *&... pointers )

inline

Get pointers to the data from the storage implementation. The method can only be called after a call to reserveItems() and data can only be accessed from 0 to the number given by reserveItems (excluding). Afterwards, addDataSize() needs to be called with the actual number of elements that have been written.

The call to reserveItems only represents the maximum number of data items that can be accessed safely. It is fine if less data items are accessed.

After all elements have been written to the arrays, addDataSize needs to be called with the final written number of entries.

Example usage:

DataInterface<int, double> argVector = ...;
 
// 1. Request space.
argVector.reserveItems(10);
 
// 2. Add the data.
int* dataInt;
double* dataDouble;
argVector.getDataPointers(dataInt, dataDouble);
for(int i = 0; i < 5; i += 1) {
  dataInt[i] = i + 1;
  dataDouble[i] = i + 11;
}
 
// 3. Add the number of items.
argVector.addDataSize(5);

Parameters

[in] pointers The pointers that are populated with the data from the internal representation.

Template Parameters

Data	Types of the pointers.

◆ getDataSize()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

size_t codi::DataInterface< T_NestedData, T_InternalPosHandle >::getDataSize ( ) const

inline

Returns: Total number of data items stored.

◆ getPosition()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

Position codi::DataInterface< T_NestedData, T_InternalPosHandle >::getPosition ( ) const

inline

Returns: The current global position of this DataInterface and all nested interfaces.

◆ getPushedDataCount()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

size_t codi::DataInterface< T_NestedData, T_InternalPosHandle >::getPushedDataCount ( InternalPosHandle const & startPos )

inline

Compute the number of data items stored after a call to reserveItems.

◆ getZeroPosition()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

Position codi::DataInterface< T_NestedData, T_InternalPosHandle >::getZeroPosition ( ) const

inline

Returns: The start position of the DataInterface and all nested interfaces.

◆ pushData()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

template<typename... Data>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::pushData ( Data const &... data )

inline

Add data to the storage allocated by the implementation. The method can only be called after a call to reserveItems and only as often as the number of reserved items.

pushData() can be called less often than indicated with reserveItems(). The call to reserveItems only represents the maximum number of data items that can be pushed safely.

After a new call to reserveItems(), only this many number of data items can be pushed, leftovers will not accumulate.

For an example of how to use pushData, please see the DataInterface documentation.

Parameters

[in] data The number of arguments has to match the number of data stores of the implementation.

Template Parameters

Data	Types of the pushed data.

◆ reserveItems()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

InternalPosHandle codi::DataInterface< T_NestedData, T_InternalPosHandle >::reserveItems ( size_t const & items )

inline

Reserve this many items on the data stream. See pushData for details.

Parameters

[in] items Number of data items to reserve.

Returns: Can be used in getPushedDataCount(). Only the newest handle is valid.

◆ reset()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::reset ( )

Reset to the zero position. Data is not deallocated. Also called on nested interfaces.

◆ resetHard()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::resetHard ( )

Reset to the zero position. Data is deallocated and the default size is allocated again. Also called on nested interfaces.

◆ resetTo()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::resetTo ( Position const & pos )

Reset to the given position. Data is not deallocated. Also called on the nested interfaces.

◆ resize()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::resize ( size_t const & totalSize )

Allocate the requested number of data items.

◆ setNested()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::setNested ( NestedData * v )

Set the pointer to the nested vector. Needs to be done before any other action and only once.

◆ swap()

template<typename T_NestedData = void, typename T_InternalPosHandle = size_t>

void codi::DataInterface< T_NestedData, T_InternalPosHandle >::swap ( DataInterface< T_NestedData, T_InternalPosHandle > & other )

Swap with other DataInterface of the same type.

The documentation for this struct was generated from the following file:

include/codi/tapes/data/dataInterface.hpp

Public Types

Public Member Functions

Detailed Description

Member Function Documentation

◆ addDataSize()

◆ addToTapeValues()

◆ erase()

◆ evaluateForward()

◆ evaluateReverse()

◆ extractPosition()

◆ forEachChunk()

◆ forEachForward()

◆ forEachReverse()

◆ getDataPointers()

◆ getDataSize()

◆ getPosition()

◆ getPushedDataCount()

◆ getZeroPosition()

◆ pushData()

◆ reserveItems()

◆ reset()

◆ resetHard()

◆ resetTo()

◆ resize()

◆ setNested()

◆ swap()