Pytorch Tutorial-Tensors

这是我学习Pytorch时记录的一些笔记 ，希望能对你有所帮助😊

  • You will sometimes see a 1-dimensional tensor called a vector.
  • Likewise, a 2-dimensional tensor is often referred to as a matrix.
  • Anything with more than two dimensions is generally just called a tensor

Creating Tensors

  • using factory methods
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    x = torch.empty(3, 4)
    print(type(x)) # <class 'torch.Tensor'>

    zeros = torch.zeros(2, 3)
    print(zeros)

    ones = torch.ones(2, 3)
    print(ones)

    torch.manual_seed(1729)
    random = torch.rand(2, 3)
    print(random)

    # The output
    tensor([[0., 0., 0.],
            [0., 0., 0.]])
    tensor([[1., 1., 1.],
            [1., 1., 1.]])
    tensor([[0.3126, 0.3791, 0.3087],
            [0.0736, 0.4216, 0.0691]])

Random Tensors and Seeding

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
torch.manual_seed(1729)
random1 = torch.rand(2, 3)
print(random1)

random2 = torch.rand(2, 3)
print(random2)

torch.manual_seed(1729)
random3 = torch.rand(2, 3)
print(random3)

random4 = torch.rand(2, 3)
print(random4)

tensor([[0.3126, 0.3791, 0.3087],
        [0.0736, 0.4216, 0.0691]])
tensor([[0.2332, 0.4047, 0.2162],
        [0.9927, 0.4128, 0.5938]])
tensor([[0.3126, 0.3791, 0.3087],
        [0.0736, 0.4216, 0.0691]])
tensor([[0.2332, 0.4047, 0.2162],
        [0.9927, 0.4128, 0.5938]])
  • 可以发现random1,2 与 3,4完全一致，随机数生成器的种子都是1729，这样可以保证“结果可复现”
  • r1与r2的值不同，因为生成r1后随机数生成器的状态更新了，而r3相当于重置了，类似于execve

Tensor Shapes

  •  when you’re performing operations on two or more tensors, they will need to be of the same shape - that is, having the same number of dimensions and the same number of cells in each dimension. For that, we have the torch.*_like() methods:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
x = torch.empty(2, 2, 3)
print(x.shape)
print(x)

empty_like_x = torch.empty_like(x)
print(empty_like_x.shape)
print(empty_like_x)

zeros_like_x = torch.zeros_like(x)
print(zeros_like_x.shape)
print(zeros_like_x)

rand_like_x = torch.rand_like(x)
print(rand_like_x.shape)
print(rand_like_x)

torch.Size([2, 2, 3])
tensor([[[ 0.0000e+00,  1.0842e-19,  4.9628e-26],
         [-2.5250e-29,  9.8091e-45,  0.0000e+00]],

        [[ 0.0000e+00,  0.0000e+00,  0.0000e+00],
         [ 0.0000e+00,  0.0000e+00,  0.0000e+00]]])
torch.Size([2, 2, 3])
tensor([[[ 2.5776e-33,  1.4013e-45,         nan],
         [ 0.0000e+00,  1.4013e-45,  0.0000e+00]],

        [[ 4.9477e-26, -3.6902e+19,  2.6082e-33],
         [ 1.4013e-45,  4.9633e-26, -8.5920e+09]]])
torch.Size([2, 2, 3])
tensor([[[0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.]]])
torch.Size([2, 2, 3])
tensor([[[0.6128, 0.1519, 0.0453],
         [0.5035, 0.9978, 0.3884]],

        [[0.6929, 0.1703, 0.1384],
         [0.4759, 0.7481, 0.0361]]])
  • Below that, we call the .empty_like(), .zeros_like(),  and .rand_like() methods. Using the .shape property, we can verify that each of these methods returns a tensor of identical dimensionality and extent.
1
2
3
4
5
6
7
8
9
some_integers = torch.tensor((2, 3, 5, 7, 11, 13, 17, 19))
print(some_integers)

more_integers = torch.tensor(((2, 4, 6), [3, 6, 9]))
print(more_integers)

tensor([ 2,  3,  5,  7, 11, 13, 17, 19])
tensor([[2, 4, 6],
        [3, 6, 9]])
  • Using torch.tensor() is the most straightforward way to create a tensor if you already have data in a Python tuple or list. As shown above, nesting the collections will result in a multi-dimensional tensor.(嵌套集合将产生多维张量)
    Note: torch.tensor() creates a copy of the data.

Tensor Data Types

默认是float32, Available data types include:

  • torch.bool
  • torch.int8
  • torch.uint8
  • torch.int16
  • torch.int32
  • torch.int64
  • torch.half
  • torch.float
  • torch.double
  • torch.bfloat

设置数据类型有两种方式，一是在创建时设置，二是使用.to()函数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
a = torch.ones((2, 3), dtype=torch.int16)
print(a)

b = torch.rand((2, 3), dtype=torch.float64) * 20.
print(b)

c = b.to(torch.int32)
print(c)

tensor([[1, 1, 1],
        [1, 1, 1]], dtype=torch.int16) # 这里如果不指定数据类型会显示 1.
tensor([[ 0.9956,  1.4148,  5.8364],
        [11.2406, 11.2083, 11.6692]], dtype=torch.float64)
tensor([[ 0,  1,  5],
        [11, 11, 11]], dtype=torch.int32)

Math & Logic with Tensors

  • 大部分运算都和你的直觉差不多，而且比较“聪明”
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    ones = torch.zeros(2, 2) + 1
    twos = torch.ones(2, 2) * 2
    threes = (torch.ones(2, 2) * 7 - 1) / 2
    fours = twos ** 2
    sqrt2s = twos ** 0.5

    tensor([[1., 1.],
            [1., 1.]])
    tensor([[2., 2.],
            [2., 2.]])
    tensor([[3., 3.],
            [3., 3.]])
    tensor([[4., 4.],
            [4., 4.]])
    tensor([[1.4142, 1.4142],
            [1.4142, 1.4142]])
            
    powers2 = twos ** torch.tensor([[1, 2], [3, 4]])
    print(powers2)

    fives = ones + fours
    print(fives)

    dozens = threes * fours
    print(dozens)
    tensor([[ 2.,  4.],
            [ 8., 16.]])
    tensor([[5., 5.],
            [5., 5.]])
    tensor([[12., 12.],
            [12., 12.]])

In Brief: Tensor Broadcasting

1
2
3
4
5
6
7
8
9
10
rand = torch.rand(2, 4)
doubled = rand * (torch.ones(1, 4) * 2)

print(rand)
print(doubled)

tensor([[0.2024, 0.5731, 0.7191, 0.4067],
        [0.7301, 0.6276, 0.7357, 0.0381]])
tensor([[0.4049, 1.1461, 1.4382, 0.8134],
        [1.4602, 1.2551, 1.4715, 0.0762]])

The rules for broadcasting are:

  • Each tensor must have at least one dimension - no empty tensors.
  • Comparing the dimension sizes of the two tensors, going from last to first 从后往前:
    • Each dimension must be equal, or
    • One of the dimensions must be of size 1, or
    • The dimension does not exist in one of the tensors
  • some correct examples:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    a =     torch.ones(4, 3, 2)

    b = a * torch.rand(   3, 2) # 3rd & 2nd dims identical to a, dim 1 absent
    print(b)

    c = a * torch.rand(   3, 1) # 3rd dim = 1, 2nd dim identical to a
    print(c)

    d = a * torch.rand(   1, 2) # 3rd dim identical to a, 2nd dim = 1
    print(d)
    For more information on broadcasting, see the PyTorch documentation on the topic.

More Math with Tensors

This is a small sample of For more details and the full inventory of math functions, have a look at the documentation.
给出一些经典示例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
# common functions
a = torch.rand(2, 4) * 2 - 1
print('Common functions:')
print(a)
print(torch.abs(a))
print(torch.ceil(a))
print(torch.floor(a))
print(torch.clamp(a, -0.5, 0.5)) # `torch.clamp()` 是一个张量裁剪函数，用于限制张量 `a` 中的元素值在指定的范围内 `[min, max]`

# trigonometric functions and their inverses
angles = torch.tensor([0, math.pi / 4, math.pi / 2, 3 * math.pi / 4])
sines = torch.sin(angles)
inverses = torch.asin(sines)
print('\nSine and arcsine:')
print(angles)
print(sines)
print(inverses)

# bitwise operations
print('\nBitwise XOR:')
b = torch.tensor([1, 5, 11])
c = torch.tensor([2, 7, 10])
print(torch.bitwise_xor(b, c))

# comparisons:
print('\nBroadcasted, element-wise equality comparison:')
d = torch.tensor([[1., 2.], [3., 4.]])
e = torch.ones(1, 2)  # many comparison ops support broadcasting!
print(torch.eq(d, e)) # returns a tensor of type bool

# reductions:
print('\nReduction ops:')
print(torch.max(d))        # returns a single-element tensor
print(torch.max(d).item()) # extracts the value from the returned tensor
print(torch.mean(d))       # average
print(torch.std(d))        # standard deviation 标准差
print(torch.prod(d))       # product of all numbers 所有元素相乘
print(torch.unique(torch.tensor([1, 2, 1, 2, 1, 2]))) # filter unique elements

# vector and linear algebra operations
v1 = torch.tensor([1., 0., 0.])         # x unit vector
v2 = torch.tensor([0., 1., 0.])         # y unit vector
m1 = torch.rand(2, 2)                   # random matrix
m2 = torch.tensor([[3., 0.], [0., 3.]]) # three times identity matrix

print('\nVectors & Matrices:')
print(torch.cross(v2, v1)) # negative of z unit vector 叉乘
print(m1)
m3 = torch.matmul(m1, m2)  # mm: matrix multiplication
print(m3)                  # 3 times m1
print(torch.svd(m3))       # singular value decomposition,奇异值分解


Common functions:
tensor([[ 0.2719,  0.1760,  0.8550,  0.0404],
        [-0.5709,  0.4227, -0.9504, -0.8692]])
tensor([[0.2719, 0.1760, 0.8550, 0.0404],
        [0.5709, 0.4227, 0.9504, 0.8692]])
tensor([[1., 1., 1., 1.],
        [-0., 1., -0., -0.]])
tensor([[ 0.,  0.,  0.,  0.],
        [-1.,  0., -1., -1.]])
tensor([[ 0.2719,  0.1760,  0.5000,  0.0404],
        [-0.5000,  0.4227, -0.5000, -0.5000]])

Sine and arcsine:
tensor([0.0000, 0.7854, 1.5708, 2.3562])
tensor([0.0000, 0.7071, 1.0000, 0.7071]) # x比较小的时候和sinx差不多
tensor([0.0000, 0.7854, 1.5708, 0.7854])

Bitwise XOR:
tensor([3, 2, 1])

Broadcasted, element-wise equality comparison:
tensor([[ True, False],
        [False, False]])

Reduction ops:
tensor(4.)
4.0
tensor(2.5000)
tensor(1.2910)
tensor(24.)
tensor([1, 2])

Vectors & Matrices:
tensor([ 0.,  0., -1.])
tensor([[0.8629, 0.6615],
        [0.8881, 0.5365]])
tensor([[2.5888, 1.9844],
        [2.6643, 1.6095]])
torch.return_types.svd(
U=tensor([[-0.7236, -0.6903],
        [-0.6903,  0.7236]]),
S=tensor([4.5019, 0.2488]),
V=tensor([[-0.8246,  0.5657],
        [-0.5657, -0.8246]]))

Copying Tensors

  • clone()
    1
    2
    3
    4
    5
    6
    7
    8
    a = torch.ones(2, 2)
    b = a.clone()

    assert b is not a      # different objects in memory...
    print(torch.eq(a, b))  # ...but still with the same contents!

    a[0][1] = 561          # a changes...
    print(b)               # ...but b is still all ones
  • There is an important thing to be aware of when using clone(). If your source tensor has autograd, enabled then so will the clone. 
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    a = torch.rand(2, 2, requires_grad=True) # turn on autograd
    print(a)

    b = a.clone()
    print(b)

    c = a.detach().clone()
    print(c)

    print(a)

    tensor([[0.5461, 0.5396],
            [0.3053, 0.1973]], requires_grad=True)
    tensor([[0.5461, 0.5396],
            [0.3053, 0.1973]], grad_fn=<CloneBackward>)
    tensor([[0.5461, 0.5396],
            [0.3053, 0.1973]])
    tensor([[0.5461, 0.5396],
            [0.3053, 0.1973]], requires_grad=True)
  • The detach() method detaches the tensor from its computation history. It says, “do whatever comes next as if autograd was off.” It does this without changing a - you can see that when we print a again at the end, it retains its requires_grad=True property.

Moving to GPU

引入：One of the major advantages of PyTorch is its robust acceleration on CUDA-compatible Nvidia GPUs. (“CUDA” stands for Compute Unified Device Architecture, which is Nvidia’s platform for parallel computing.) So far, everything we’ve done has been on CPU. How do we move to the faster hardware?

1
2
3
4
if torch.cuda.is_available():
    print('We have a GPU!')
else:
    print('Sorry, CPU only.')

CPU在RAM中进行计算，而GPU提供了附加内存，用GPU计算时需要将数据迁移到GPU

创建时迁移

  • 张量默认创建在CPU上，你可以使用device来制定设备

    1
    2
    3
    4
    5
    if torch.cuda.is_available():
        gpu_rand = torch.rand(2, 2, device='cuda')
        print(gpu_rand)
    else:
        print('Sorry, CPU only.')

  • 一个好的习惯是使用my_device来表征自己的设备，而不直接使用字符串

    1
    2
    3
    4
    5
    if torch.cuda.is_available():
        my_device = torch.device('cuda')
    else:
        my_device = torch.device('cpu')
    print('Device: {}'.format(my_device))

to()迁移

1
2
y = torch.rand(2, 2)
y = y.to(my_device)

It is important to know that in order to do computation involving two or more tensors, all of the tensors must be on the same device.

Manipulating Tensor Shapes

PyTorch models generally expect batches of input.For example, imagine having a model that works on 3 x 226 x 226 images - a 226-pixel square with 3 color channels. When you load and transform it, you’ll get a tensor of shape (3, 226, 226). Your model, though, is expecting input of shape (N, 3, 226, 226), where N is the number of images in the batch. So how do you make a batch of one?

1
2
3
4
5
6
7
8
a = torch.rand(3, 226, 226)
b = a.unsqueeze(0)

print(a.shape)
print(b.shape)

torch.Size([3, 226, 226])
torch.Size([1, 3, 226, 226])
  • The unsqueeze() method adds a dimension of extent 1. unsqueeze(0) adds it as a new zeroth dimension - now you have a batch of one!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
a = torch.rand(1, 20)
print(a.shape)
print(a)

b = a.squeeze(0)
print(b.shape)
print(b)

c = torch.rand(2, 2)
print(c.shape)

d = c.squeeze(0)
print(d.shape)

torch.Size([1, 20])
tensor([[0.4887, 0.8625, 0.6191, 0.9935, 0.1844, 0.6138, 0.6854, 0.0438, 0.0636,
         0.2884, 0.4362, 0.2368, 0.1394, 0.1721, 0.1751, 0.3851, 0.0732, 0.3118,
         0.9180, 0.7293]])
torch.Size([20])
tensor([0.4887, 0.8625, 0.6191, 0.9935, 0.1844, 0.6138, 0.6854, 0.0438, 0.0636,
        0.2884, 0.4362, 0.2368, 0.1394, 0.1721, 0.1751, 0.3851, 0.0732, 0.3118,
        0.9180, 0.7293])
torch.Size([2, 2])
torch.Size([2, 2])
  • 其实增加或减少一个范围为1的维度，其实对数据本身并没有影响，这也能提示我们： Calls to squeeze() and unsqueeze() can only act on dimensions of extent 1 because to do otherwise would change the number of elements in the tensor.
1
2
3
4
5
a = torch.ones(4, 3, 2)
b = torch.rand(   3)     # trying to multiply a * b will give a runtime error
c = b.unsqueeze(1)       # change to a 2-dimensional tensor, adding new dim at the end
print(c.shape)
print(a * c)             # broadcasting works again!

Sometimes you’ll want to change the shape of a tensor more radically, while still preserving the number of elements and their contents. One case where this happens is at the interface between a convolutional layer of a model and a linear layer of the model - this is common in image classification models. A convolution kernel will yield an output tensor of shape features x width x height, but the following linear layer expects a 1-dimensional input. reshape() will do this for you, provided that the dimensions you request yield the same number of elements as the input tensor has:

1
2
3
4
5
6
7
8
9
10
11
12
output3d = torch.rand(6, 20, 20)
print(output3d.shape)

input1d = output3d.reshape(6 * 20 * 20)
print(input1d.shape)

# can also call it as a method on the torch module:
print(torch.reshape(output3d, (6 * 20 * 20,)).shape)

torch.Size([6, 20, 20])
torch.Size([2400])
torch.Size([2400])

NumPy Bridge

NumPy 有和张量比较想通的地方，比如Broadcast，所以在二者之间转换很容易

1
2
3
4
5
6
7
8
9
10
11
12
import numpy as np

numpy_array = np.ones((2, 3))
print(numpy_array)

pytorch_tensor = torch.from_numpy(numpy_array)
print(pytorch_tensor)

[[1. 1. 1.]
 [1. 1. 1.]]
tensor([[1., 1., 1.],
        [1., 1., 1.]], dtype=torch.float64)
  • PyTorch 创建了一个与 NumPy 数组具有相同形状并包含相同数据的张量，甚至保留了 NumPy 默认的 64 位浮点数据类型。
1
2
3
4
5
pytorch_rand = torch.rand(2, 3)
print(pytorch_rand)

numpy_rand = pytorch_rand.numpy()
print(numpy_rand)

这些都不重要，重要的是要知道这些转换后的对象使用与其源对象有相同的底层内存_ ，这意味着对一个对象的更改会反映在另一个对象中：

1
2
3
4
5
6
7
8
9
10
numpy_array[1, 1] = 23
print(pytorch_tensor)

pytorch_rand[1, 1] = 17
print(numpy_rand)

tensor([[ 1.,  1.,  1.],
        [ 1., 23.,  1.]], dtype=torch.float64)
[[ 0.5646949   0.91600937  0.77828014]
 [ 0.82769746 17.          0.6381657 ]]
Pytorch
#Pytorch #Python #ML #Tensor
Pytorch Tutorial-Tensors
http://pzhwuhu.github.io/2025/08/03/Tensors/
本文作者
pzhwuhu
发布于
2025年8月3日
更新于
2025年8月16日
许可协议