Subtracting a best rank-1 approximation may increase tensor rank

Is has been shown that a best rank-R approximation of an order-k tensor may not exist when R is at most 2 and k is at most 3. This poses a serious problem to data analysts using Candecomp/Parafac and related models. It has been observed numerically that, generally, this issue cannot be solved by consecutively computing and substracting best rank-1 approximations. The reason for this is that subtracting a best rank-1 approximation generally does not decrease tensor rank. In this paper, we provide a mathematical treatment of this property for real-valued 2x2x2 tensors, with symmetric tensors as a special case. Regardless of the symmetry, we show that for generic 2x2x2 tensors (which have rank 2 or 3), subtracting a best rank-1 approximation will result in a tensor that has rank 3 and lies on the boundary between the rank-2 and rank-3 sets. Hence, for a typical tensor of rank 2, subtracting a best rank-1 approximation has increased the tensor rank.