What is a Tensor?

If you're new to machine learning, you've almost certainly seen the word "tensor." Tensors are common data structures in machine learning and deep learning (Google's open-source software library for machine learning is even called TensorFlow). But what is a tensor, exactly?

In simple terms, a tensor is a dimensional data structure. Vectors are one-dimensional data structures and matrices are two-dimensional data structures. Tensors are superficially similar to these other data structures, but the difference is that they can exist in dimensions ranging from zero to n (referred to as the tensor's rank, as in a first-rank tensor which is one-dimensional).

This surface similarity is often what makes tensors difficult for people to grasp at first. For instance, we can represent second-rank tensors as matrices. This stress on "can be" is important because tensors have properties that not all matrices will have. Using the logic of "all squares are rectangles, but not all rectangles are squares," Steven Steinke explains that "any rank-2 tensor can be represented as a matrix, but not every matrix is a rank-2 tensor."

Tensor vs Matrix - What is the Difference?

The critical difference that sets tensors apart from matrices is that tensors are dynamic. This mathematical entity means that tensors obey specific transformation rules as part of the structures they inhabit. If other mathematical entities in the structure transform in a regular way, the tensor will transform as well, according to the rules established for the system. Not all matrices have this property, hence the fact that not all matrices are second-rank tensors.

Why are Tensors Used in Machine Learning?

Now that we have a working definition for tensors, why are they so popular in machine learning? Well, computers need data to learn, and tensors are a more natural, intuitive way of processing many kinds of data, especially big and complex data sets. Here's an example:

Video is a series of images correlated over time. We can use tensors to represent that correlation better and more intuitively than trying to convert it down to two-dimensional matrices. A third-rank tensor can encode all the aspects of each image (height, width, and color), while a rank-4 tensor could also hold information about time or order for the images.

Thus, tensors allow powerful computers to solve big data problems more quickly and allow deep learning devices and neural networks (which usually require hundreds or thousands of dimensions of data) to process the data more intuitively.

Related news articles

Latest News

Sorry, your filter selection returned no results.

We've updated our privacy policy. Please take a moment to review these changes. By clicking I Agree to Arrow Electronics Terms Of Use  and have read and understand the Privacy Policy and Cookie Policy.

Our website places cookies on your device to improve your experience and to improve our site. Read more about the cookies we use and how to disable them here. Cookies and tracking technologies may be used for marketing purposes.
By clicking “Accept”, you are consenting to placement of cookies on your device and to our use of tracking technologies. Click “Read More” below for more information and instructions on how to disable cookies and tracking technologies. While acceptance of cookies and tracking technologies is voluntary, disabling them may result in the website not working properly, and certain advertisements may be less relevant to you.
We respect your privacy. Read our privacy policy here