PyTorch Cheatsheet

Tensors

Use this PyTorch reference while you build software engineering projects, review code for technical interview prep, or polish examples for a software engineer resume.

Creation

From data

import torch

t = torch.tensor([1, 2, 3])                    # from Python list (int64)
t = torch.tensor([[1.0, 2.0], [3.0, 4.0]])     # 2-D float32
t = torch.tensor(data, dtype=torch.float32)    # explicit dtype
t = torch.as_tensor(numpy_arr)                 # zero-copy if possible
t = torch.from_numpy(numpy_arr)                # shares memory with ndarray

Factory functions

torch.zeros(3, 4)                   # shape (3,4) filled with 0.0
torch.ones(2, 3, dtype=torch.int8)  # shape (2,3) filled with 1
torch.full((2, 3), 7.0)             # filled with 7.0
torch.empty(4, 4)                   # uninitialized (garbage values)
torch.eye(3)                        # 3×3 identity matrix
torch.arange(0, 10, 2)              # [0, 2, 4, 6, 8]
torch.linspace(0, 1, 5)             # [0.00, 0.25, 0.50, 0.75, 1.00]
torch.logspace(0, 2, 3)             # [1, 10, 100]
torch.randint(low=0, high=10, size=(3, 3))
torch.rand(3, 3)                    # uniform [0, 1)
torch.randn(3, 3)                   # standard normal
torch.normal(mean=0.0, std=1.0, size=(3,))

Like-existing-tensor factories

torch.zeros_like(t)    # same shape/dtype/device as t, filled 0
torch.ones_like(t)
torch.rand_like(t)     # requires float dtype
torch.empty_like(t)
torch.full_like(t, 3.14)

Data Types (dtype)

dtypeDescriptionBytes
torch.float32 / torch.floatsingle-precision float4
torch.float64 / torch.doubledouble-precision float8
torch.float16 / torch.halfhalf-precision float2
torch.bfloat16brain float 162
torch.int8signed 8-bit int1
torch.int16 / torch.shortsigned 16-bit int2
torch.int32 / torch.intsigned 32-bit int4
torch.int64 / torch.longsigned 64-bit int8
torch.uint8unsigned 8-bit int1
torch.boolboolean1
torch.complex64complex (2×float32)8
torch.complex128complex (2×float64)16
t = t.float()    # cast to float32
t = t.long()     # cast to int64
t = t.to(torch.float16)
t = t.type(torch.DoubleTensor)
print(t.dtype)

Default float dtype is float32. Call torch.set_default_dtype(torch.float64) to change it globally.

Shape and Layout

t.shape          # torch.Size([2, 3])  — same as t.size()
t.size(0)        # size of dim 0
t.ndim           # number of dimensions
t.numel()        # total number of elements
t.element_size() # bytes per element
t.is_contiguous()
t.contiguous()   # returns a contiguous copy if not already

Reshaping

t.reshape(6)             # returns view or copy as needed
t.view(2, 3)             # view (tensor must be contiguous)
t.view(-1)               # flatten
t.view(-1, 3)            # infer one dimension
t.flatten()              # always returns 1-D
t.flatten(start_dim=1)   # flatten from dim 1 onward
t.squeeze()              # remove all size-1 dims
t.squeeze(0)             # remove size-1 at dim 0
t.unsqueeze(0)           # insert size-1 at dim 0
t.expand(2, 3)           # broadcast without copying data
t.repeat(2, 1)           # tile data

Transposing

t.T                          # transpose 2-D tensor
t.transpose(0, 1)            # swap dim 0 and 1
t.permute(2, 0, 1)           # reorder all dims
t.movedim(0, -1)             # move specific dim
t.swapaxes(0, 1)             # alias of transpose

Indexing and Slicing

t[0]          # first row
t[-1]         # last row
t[1, 2]       # element at row 1, col 2
t[:, 1]       # all rows, col 1
t[0:2, :]     # rows 0-1, all cols
t[..., -1]    # last element along last dim (ellipsis)

# Boolean indexing
mask = t > 0
t[mask]                     # 1-D tensor of matching values
t[t > 0] = 0               # in-place masked fill

# Fancy (advanced) indexing
idx = torch.tensor([0, 2])
t[idx]                      # rows 0 and 2

# torch.where
torch.where(t > 0, t, torch.zeros_like(t))  # element-wise conditional

Memory and Device

t.device                   # device('cpu') or device('cuda:0')
t.is_cuda                  # bool
t.cpu()                    # move to CPU
t.cuda()                   # move to default GPU
t.to('cuda:0')             # explicit GPU
t.to(device)               # move to variable device
t.pin_memory()             # pin for faster host→device transfer

t.data_ptr()               # raw memory address (int)
t.untyped_storage()        # underlying storage (t.storage() is deprecated)
t.is_shared()              # in shared memory (for multiprocessing)

Copying and Cloning

t.clone()                   # deep copy, keeps grad_fn
t.detach()                  # no-copy view, severs grad graph
t.detach().clone()          # safe copy outside autograd
t.copy_(other)              # in-place copy from other tensor

Converting Out of PyTorch

t.item()                    # Python scalar (single-element tensors only)
t.tolist()                  # nested Python list
t.numpy()                   # NumPy array (CPU, no grad, shares memory)
t.detach().cpu().numpy()    # safe pattern for GPU / grad tensors

Attributes Quick Reference

AttributeReturns
t.shapetorch.Size
t.dtypetorch.float32 etc.
t.devicedevice('cpu') etc.
t.requires_gradbool
t.gradgradient tensor or None
t.grad_fngrad function or None
t.is_leafbool — user-created or detached
t.layouttorch.strided (default), torch.sparse_coo, etc.

Gotchas

t.view() requires the tensor to be contiguous — call .contiguous() first or use .reshape() which handles it automatically.

t.numpy() and torch.from_numpy() share memory. Mutating one mutates the other.

Indexing with a Python int returns a tensor with one fewer dimension; indexing with a slice keeps the dimension. t[0] vs t[0:1].

In-place operations (e.g. t.add_(1)) on a leaf tensor that has requires_grad=True will raise an error if called after a backward pass has started.