Commit 9bcdcdb8 authored by Sebastian Wolf's avatar Sebastian Wolf

Merge branch 'v3' of into v3

parents 82865e49 fdf94b53
Pipeline #737 passed with stages
in 8 minutes and 37 seconds
......@@ -16,9 +16,14 @@
# ...even if they are in subdirectories
......@@ -2,13 +2,16 @@
Potentially breaking changes are marked with an exclamation point '!' at the begin of their description.
* 2016-??-?? v3.0.0
* 2017-??-?? v3.0.0
* Python wrapper now stable.
* ! REQUIRE macro now logs as error instead of fatal error.
* ! All macros and preprocessor defines now use the XERUS_ prefix. The file changed accordingly.
* ! TT::find_largest_entry and TT::dyadic_product left the TT scope.
* ! Tensor::modify_diag_elements renamed to Tensor::modify_diagonal_entries for naming consistency.
* Some minor bugfixes.
* Much faster solve of matrix equations Ax=b by exploiting symmetry and definiteness where possible. This directly speeds up the ALS as well.
* Added a highly optimized minimal version of the ALS algorithm as xALS.
* Added Tensor.one_norm() and one_norm(Tensor) to calculate the one norm of a Tensor.
* Some minor bugfixes and performance improvements.
* 2016-06-23 v2.4.0
* Introduced nomeclature 'mode'. Marked all functions that will be renamed / removed in v3.0.0 as deprecated.
......@@ -26,7 +29,7 @@ Potentially breaking changes are marked with an exclamation point '!' at the beg
* Bug fix in the handling of fixed indices in TensorNetworks.
* Several static member function now warn if their return type is not used.
* Initial support for compilation with the intel ICC.
* 2016-03-11 v2.2.1
* Added support for 32bit systems.
......@@ -107,7 +107,7 @@ define \n
PYTHON_FLAGS = $(strip $(WARNINGS) $(LOGGING) $(DEBUG) $(ADDITIONAL_INCLUDE) $(OTHER) -fno-var-tracking-assignments)
MINIMAL_DEPS = Makefile makeIncludes/ makeIncludes/ makeIncludes/
......@@ -73,6 +73,7 @@ COMPILE_THREADS = 8 # Number of threads to use during link time optimizatio
# DEBUG += -fsanitize=undefined # GCC only
# DEBUG += -fsanitize=memory # Clang only
# DEBUG += -fsanitize=address # find out of bounds access
# DEBUG += -pg # adds profiling code for the 'gprof' analyzer
# Xerus has a buildin logging system to provide runtime information. Here you can adjust the logging level used by the library.
# Notes for Advanced Users
## Multi-Threading
Please note, that `xerus` is only thread-safe up to 1024 threads at the time of this writing. This number is due to the internal handling of indices. To ensure that indices can
be compared and identified uniquely, they store a unique id of which the first 10 bits are reserved to denote the number of the current thread. With more than 1024 threads (when these
10 bits overflow) it can thus lead to collisions and indices that were shared between threads and were meant to be seperate suddenly evaluate to be equal to all algorithms.
......@@ -44,7 +44,7 @@ The output of this executable should then list a number of passed tests and end
Note in particular, that all tests were passed. Should this not be the case please file a bug report with as many details as you
can in our [issuetracker](
can in our [issuetracker]( or write us an [email](
## Building the Library
# Indices and Equations
## Blockwise Construction of Tensors
......@@ -24,7 +24,9 @@ The current development version is also available in the same git repository (br
There are a number of tutorials to get you started using the `xerus` library.
* [Building xerus](@ref md_building_xerus) - instruction on how to build the library iteself and your first own program using it.
* [Quick-Start guide](_quick-_start-example.html) - a short introduction into the basic `xerus` functionality.
* [Tensor Class](@ref md_tensor) - an introduction into the most important functions relating to the `xerus::Tensor` class.
* [TT Tensors](_t_t-_tensors_01_07_m_p_s_08-example.html) - using the MPS or Tensor-Train decomposition.
* [Optimization](@ref md_optimization) - some hints how to achieve the fastest possible numerical code using `xerus`.
* [Debugging](@ref md_tut_debugging) - using `xerus`'s capabilities to debug your own application.
## Issues
# Jekyll configuration
name: Xerus Documentation
description: Documentation for the xerus library.
# baseurl will often be '', but for a project page on gh-pages, it needs to
# be the project name.
# *** IMPORTANT: If your local "jekyll serve" throws errors change this to '' or
# run it like so: jekyll serve --baseurl=''
baseurl: ''
permalink: /:title
# paginate: 3
highlighter: rouge
# highlighter: coderay
markdown: kramdown
# gems: ['TabsConverter']
# exclude: ['', 'LICENSE']
#include <xerus.h>
using namespace xerus;
class InternalSolver {
const size_t d;
std::vector<Tensor> leftAStack;
std::vector<Tensor> rightAStack;
std::vector<Tensor> leftBStack;
std::vector<Tensor> rightBStack;
TTTensor& x;
const TTOperator& A;
const TTTensor& b;
const double solutionsNorm;
size_t maxIterations;
InternalSolver(const TTOperator& _A, TTTensor& _x, const TTTensor& _b)
: d(, x(_x), A(_A), b(_b), solutionsNorm(frob_norm(_b)), maxIterations(1000)
void push_left_stack(const size_t _position) {
Index i1, i2, i3, j1 , j2, j3, k1, k2;
const Tensor &xi = x.get_component(_position);
const Tensor &Ai = A.get_component(_position);
const Tensor &bi = b.get_component(_position);
Tensor tmpA, tmpB;
tmpA(i1, i2, i3) = leftAStack.back()(j1, j2, j3)
*xi(j1, k1, i1)*Ai(j2, k1, k2, i2)*xi(j3, k2, i3);
tmpB(i1, i2) = leftBStack.back()(j1, j2)
*xi(j1, k1, i1)*bi(j2, k1, i2);
void push_right_stack(const size_t _position) {
Index i1, i2, i3, j1 , j2, j3, k1, k2;
const Tensor &xi = x.get_component(_position);
const Tensor &Ai = A.get_component(_position);
const Tensor &bi = b.get_component(_position);
Tensor tmpA, tmpB;
tmpA(i1, i2, i3) = xi(i1, k1, j1)*Ai(i2, k1, k2, j2)*xi(i3, k2, j3)
*rightAStack.back()(j1, j2, j3);
tmpB(i1, i2) = xi(i1, k1, j1)*bi(i2, k1, j2)
*rightBStack.back()(j1, j2);
double calc_residual_norm() {
Index i,j;
return frob_norm(A(i/2, j/2)*x(j&0) - b(i&0)) / solutionsNorm;
void solve() {
// Build right stack
x.move_core(0, true);
for (size_t pos = d-1; pos > 0; --pos) {
Index i1, i2, i3, j1 , j2, j3, k1, k2;
std::vector<double> residuals(10, 1000.0);
for (size_t itr = 0; itr < maxIterations; ++itr) {
// Calculate residual and check end condition
if (residuals.back()/residuals[residuals.size()-10] > 0.99) {
XERUS_LOG(simpleALS, "Done! Residual decrease from " << std::scientific << residuals[10] << " to " << std::scientific << residuals.back() << " in " << residuals.size()-10 << " iterations.");
return; // We are done!
XERUS_LOG(simpleALS, "Iteration: " << itr << " Residual: " << residuals.back());
// Sweep Left -> Right
for (size_t corePosition = 0; corePosition < d; ++corePosition) {
Tensor op, rhs;
const Tensor &Ai = A.get_component(corePosition);
const Tensor &bi = b.get_component(corePosition);
op(i1, i2, i3, j1, j2, j3) = leftAStack.back()(i1, k1, j1)*Ai(k1, i2, j2, k2)*rightAStack.back()(i3, k2, j3);
rhs(i1, i2, i3) = leftBStack.back()(i1, k1) * bi(k1, i2, k2) * rightBStack.back()(i3, k2);
xerus::solve(x.component(corePosition), op, rhs);
if (corePosition+1 < d) {
x.move_core(corePosition+1, true);
// Sweep Right -> Left : only move core and update stacks
x.move_core(0, true);
for (size_t corePosition = d-1; corePosition > 0; --corePosition) {
void simpleALS(const TTOperator& _A, TTTensor& _x, const TTTensor& _b) {
InternalSolver solver(_A, _x, _b);
return solver.solve();
int main() {
Index i,j,k;
auto A = TTOperator::random(std::vector<size_t>(16, 4), std::vector<size_t>(7,2));
A(i/2,j/2) = A(i/2, k/2) * A(j/2, k/2);
auto solution = TTTensor::random(std::vector<size_t>(8, 4), std::vector<size_t>(7,3));
TTTensor b;
b(i&0) = A(i/2, j/2) * solution(j&0);
auto x = TTTensor::random(std::vector<size_t>(8, 4), std::vector<size_t>(7,3));
simpleALS(A, x, b);
XERUS_LOG(info, "Residual: " << frob_norm(A(i/2, j/2) * x(j&0) - b(i&0))/frob_norm(b));
XERUS_LOG(info, "Error: " << frob_norm(solution-x)/frob_norm(x));
import xerus as xe
class InternalSolver :
def __init__(self, A, x, b):
self.A = A
self.b = b
self.x = x
self.d =
self.solutionsNorm = b.frob_norm()
self.leftAStack = [ xe.Tensor.ones([1,1,1]) ]
self.leftBStack = [ xe.Tensor.ones([1,1]) ]
self.rightAStack = [ xe.Tensor.ones([1,1,1]) ]
self.rightBStack = [ xe.Tensor.ones([1,1]) ]
self.maxIterations = 1000
def push_left_stack(self, pos) :
i1,i2,i3, j1,j2,j3, k1,k2 = xe.indices(8)
Ai = self.A.get_component(pos)
xi = self.x.get_component(pos)
bi = self.b.get_component(pos)
tmpA = xe.Tensor()
tmpB = xe.Tensor()
tmpA(i1, i2, i3) << self.leftAStack[-1](j1, j2, j3)\
*xi(j1, k1, i1)*Ai(j2, k1, k2, i2)*xi(j3, k2, i3)
tmpB(i1, i2) << self.leftBStack[-1](j1, j2)\
*xi(j1, k1, i1)*bi(j2, k1, i2)
def push_right_stack(self, pos) :
i1,i2,i3, j1,j2,j3, k1,k2 = xe.indices(8)
Ai = self.A.get_component(pos)
xi = self.x.get_component(pos)
bi = self.b.get_component(pos)
tmpA = xe.Tensor()
tmpB = xe.Tensor()
tmpA(j1, j2, j3) << xi(j1, k1, i1)*Ai(j2, k1, k2, i2)*xi(j3, k2, i3) \
* self.rightAStack[-1](i1, i2, i3)
tmpB(j1, j2) << xi(j1, k1, i1)*bi(j2, k1, i2) \
* self.rightBStack[-1](i1, i2)
def calc_residual_norm(self) :
i,j = xe.indices(2)
return xe.frob_norm(self.A(i/2, j/2)*self.x(j&0) - self.b(i&0)) / self.solutionsNorm
def solve(self) :
# build right stack
self.x.move_core(0, True)
for pos in reversed(xrange(1, self.d)) :
i1,i2,i3, j1,j2,j3, k1,k2 = xe.indices(8)
residuals = [1000]*10
for itr in xrange(self.maxIterations) :
if residuals[-1]/residuals[-10] > 0.99 :
print("Done! Residual decreased from:", residuals[10], "to", residuals[-1], "in", len(residuals)-10, "sweeps")
print("Iteration:",itr, "Residual:", residuals[-1])
# sweep left -> right
for pos in xrange(self.d):
op = xe.Tensor()
rhs = xe.Tensor()
Ai = self.A.get_component(pos)
bi = self.b.get_component(pos)
op(i1, i2, i3, j1, j2, j3) << self.leftAStack[-1](i1, k1, j1)*Ai(k1, i2, j2, k2)*self.rightAStack[-1](i3, k2, j3)
rhs(i1, i2, i3) << self.leftBStack[-1](i1, k1) * bi(k1, i2, k2) * self.rightBStack[-1](i3, k2)
tmp = xe.Tensor()
tmp(i1&0) << rhs(j1&0) / op(j1/2, i1/2)
self.x.set_component(pos, tmp)
if pos+1 < self.d :
self.x.move_core(pos+1, True)
# right -> left, only move core and update stack
self.x.move_core(0, True)
for pos in reversed(xrange(1,self.d)) :
def simpleALS(A, x, b) :
solver = InternalSolver(A, x, b)
if __name__ == "__main__":
i,j,k = xe.indices(3)
A = xe.TTOperator.random([4]*16, [2]*7)
A(i/2,j/2) << A(i/2, k/2) * A(j/2, k/2)
solution = xe.TTTensor.random([4]*8, [3]*7)
b = xe.TTTensor()
b(i&0) << A(i/2, j/2) * solution(j&0)
x = xe.TTTensor.random([4]*8, [3]*7)
simpleALS(A, x, b)
print("Residual:", xe.frob_norm(A(i/2, j/2) * x(j&0) - b(i&0))/xe.frob_norm(b))
print("Error:", xe.frob_norm(solution-x)/xe.frob_norm(x))
* @file
* @short the source code to the "Quick-Start" guide
#include <xerus.h>
int main() {
......@@ -30,9 +25,7 @@ int main() {
std::cout << "ttA ranks: " << ttA.ranks() << std::endl;
// the right hand side of the equation both as Tensor and in (Q)TT format
xerus::Tensor b({512}, []() {
return 1.0;
auto b = xerus::Tensor::ones({512});
b.reinterpret_dimensions(std::vector<size_t>(9, 2));
xerus::TTTensor ttb(b);
......@@ -56,8 +49,5 @@ int main() {
x(j^9) = b(i^9) / A(i^9, j^9);
// and calculate the Frobenius norm of the difference
// here i&0 denotes a multiindex large enough to fully index the respective tensors
// the subtraction of different formats will default to Tensor subtraction such that
// the TTTensor ttx will be evaluated to a Tensor prior to subtraction.
std::cout << "error: " << frob_norm(x(i&0) - ttx(i&0)) << std::endl;
std::cout << "error: " << frob_norm(x - xerus::Tensor(ttx)) << std::endl;
import xerus as xe
# construct the stiffness matrix A using a fill function
def A_fill(idx):
if idx[0] == idx[1] :
return 2.0
elif idx[1] == idx[0]+1 or idx[1]+1 == idx[0] :
return -1.0
return 0.0
A = xe.Tensor.from_function([512,512], A_fill)
# and dividing it by h^2 = multiplying it with N^2
A *= 512*512
# reinterpret the 512x512 tensor as a 2^18 tensor
# and create (Q)TT decomposition of it
ttA = xe.TTOperator(A)
# and verify its rank
print("ttA ranks:", ttA.ranks())
# the right hand side of the equation both as Tensor and in (Q)TT format
b = xe.Tensor.ones([512])
ttb = xe.TTTensor(b)
# construct a random initial guess of rank 3 for the ALS algorithm
ttx = xe.TTTensor.random([2,]*9, [3,]*8)
# and solve the system with the default ALS algorithm for symmetric positive operators
xe.ALS_SPD(ttA, ttx, ttb)
# to perform arithmetic operations we need to define some indices
i,j,k = xe.indices(3)
# calculate the residual of the just solved system to evaluate its accuracy
# here i^9 denotes a multiindex named i of dimension 9 (ie. spanning 9 indices of the respective tensors)
residual = xe.frob_norm( ttA(i^9,j^9)*ttx(j^9) - ttb(i^9) )
print("residual:", residual)
# as an comparison solve the system exactly using the Tensor / operator
x = xe.Tensor()
x(j^9) << b(i^9) / A(i^9, j^9)
# and calculate the Frobenius norm of the difference
print("error:", xe.frob_norm(x - xe.Tensor(ttx)))
module Jekyll
class TabsConverter < Converter
safe true
priority :low
def matches(ext)
ext =~ /^\.md$/i
def output_ext(ext)
def convert(content)
content.gsub('<p>__tabsInit</p>', "<input id=\"tab1\" type=\"radio\" name=\"tabs\" checked><input id=\"tab2\" type=\"radio\" name=\"tabs\">")
.gsub('<p>__tabsStart</p>', "<div id=\"tabs\"><label for=\"tab1\">C++</label><label for=\"tab2\">Python</label><div id=\"content\"><section id=\"content1\">")
.gsub('<p>__tabsMid</p>', "</section><section id=\"content2\">")
.gsub('<p>__tabsEnd</p>', "</section></div></div>")
.gsub('<p>__dangerStart</p>', "<div class=\"alert alert-danger\">")
.gsub('<p>__dangerEnd</p>', "</div>")
.gsub('<p>__warnStart</p>', "<div class=\"alert alert-warning\">")
.gsub('<p>__warnEnd</p>', "</div>")
This diff is collapsed.
layout: post
title: "QTT Decomposition"
date: 1000-12-10
topic: "Examples"
section: "Examples"
# QTT Decomposition
This guide can be used as a quick start into `xerus`. It will introduce some basic functionality of the library,
demonstrate the general layout and is enough for very basic applications. It is recommended to also have a look
at the more detailed guides for all classes one wishes to use though - or even have a look at the doxygen class documentation for details on all functions.
It is assumed that you have already obtained and compiled the library itself as well as know how to link against it.
If this is not the case, please refer to the [building xerus](building_xerus) page.
In the following we will solve a FEM equation arising from the heat equation using a QTT decomposition and the ALS algorithm.
To start we create the stiffness matrix as a dense (ie. not sparse or decomposed) tensor.
To define the entries we pass a function to the constructor of the `Tensor` object that will be
called for every entry with a vector of size_t integers defining the indices of the current entry.
As it is simpler to think of the stiffness matrix in its original form rather than the QTT form we will
set the dimensions to 512x512.
~~~ cpp
xerus::Tensor A({512,512}, [](const std::vector<size_t> &idx){
if (idx[0] == idx[1]) {
return 2.0;
} else if (idx[1] == idx[0]+1 || idx[1]+1 == idx[0]) {
return -1.0;
} else {
return 0.0;
~~~ python
def A_fill(idx):
if idx[0] == idx[1] :
return 2
elif idx[1] == idx[0]+1 or idx[1]+1 == idx[0] :
return -1
else :
return 0
A = xerus.Tensor([512,512], A_fill)
To account for the $ h^2 $ factor that we have ignored so far we simply multipy the operator by $ N^2 $.
~~~ cpp
A *= 512*512;
~~~ python
A *= 512*512
By reinterpreting the dimension and thus effectively treating the tensor as a $ 2^{18} $ instead of a $ 512^2 $ tensor,
the decomposition into a `TTTensor` will give us the stiffness matrix in a QTT format.
~~~ cpp
A.reinterpret_dimensions(std::vector<size_t>(18, 2));
xerus::TTOperator ttA(A);
~~~ python
ttA = xerus.TTOperator(A)
As the Laplace operator is representable exactly in a low-rank QTT format, the rank of this `ttA` should be low after this construction.
We can verify this by printing the ranks:
~~~ cpp
using xerus::misc::operator<<; // to be able to pipe vectors
std::cout << "ttA ranks: " << ttA.ranks() << std::endl;
~~~ python
print("ttA ranks:", ttA.ranks())
For the right-hand-side we perform similar operations to obtain a QTT decomposed vector $ b_i = 1 \forall i $.
As the generating function needs no index information, we create a `[]()->double` lambda function:
~~~ cpp
auto b = xerus::Tensor::ones({512});
b.reinterpret_dimensions(std::vector<size_t>(9, 2));
xerus::TTTensor ttb(b);
~~~ python
b = xerus.Tensor.ones([512])
ttb = xerus.TTTensor(b)
To have an initial vector for the ALS algorithm we create a random TTTensor of the desired rank
(3 in this case - note, that this is the exact rank of the solution).
~~~ cpp
xerus::TTTensor ttx = xerus::TTTensor::random(std::vector<size_t>(9, 2), std::vector<size_t>(8, 3));
~~~ python
ttx = xerus.TTTensor.random([2,]*9, [3,]*8)
With these three tensors (the operator `ttA`, the right-hand-side `ttb` and the initial guess `ttx`)
we can now perform the ALS algorithm to solve for `ttx` (note here, that the _SPD suffix chooses the variant of the ALS
that assumes that the given operator is symmetric positive definite)
~~~ cpp
xerus::ALS_SPD(ttA, ttx, ttb);
~~~ python
xerus.ALS_SPD(ttA, ttx, ttb)
To verify the calculation performed by the ALS we will need to perform some arithmetic operations.
As these require the definition of (relative) index orderings in the tensors, we define some indices
~~~ cpp
xerus::Index i,j,k;
~~~ python
i,j,k = xerus.indices(3)
and use these in calculations like `A(i,j)*x(j) - b(i)`. Note though, that our tensors are of a higher
degree due to the QTT decomposition and we thus need to specify the corresponding dimension of the
multiindices i,j, and k with eg. `i^9` to denote a multiindex of dimension 9.
~~~ cpp
double residual = frob_norm( ttA(i^9,j^9)*ttx(j^9) - ttb(i^9) );
std::cout << "residual: " << residual << std::endl;
~~~ python
residual = xerus.frob_norm( ttA(i^9,j^9)*ttx(j^9) - ttb(i^9) )