PyTorch Cheatsheet
Tensors
Use this PyTorch reference while you build software engineering projects, review code for technical interview prep, or polish examples for a software engineer resume.
Creation
From data
import torch t = torch.tensor([1, 2, 3]) # from Python list (int64) t = torch.tensor([[1.0, 2.0], [3.0, 4.0]]) # 2-D float32 t = torch.tensor(data, dtype=torch.float32) # explicit dtype t = torch.as_tensor(numpy_arr) # zero-copy if possible t = torch.from_numpy(numpy_arr) # shares memory with ndarray
Factory functions
torch.zeros(3, 4) # shape (3,4) filled with 0.0 torch.ones(2, 3, dtype=torch.int8) # shape (2,3) filled with 1 torch.full((2, 3), 7.0) # filled with 7.0 torch.empty(4, 4) # uninitialized (garbage values) torch.eye(3) # 3×3 identity matrix torch.arange(0, 10, 2) # [0, 2, 4, 6, 8] torch.linspace(0, 1, 5) # [0.00, 0.25, 0.50, 0.75, 1.00] torch.logspace(0, 2, 3) # [1, 10, 100] torch.randint(low=0, high=10, size=(3, 3)) torch.rand(3, 3) # uniform [0, 1) torch.randn(3, 3) # standard normal torch.normal(mean=0.0, std=1.0, size=(3,))
Like-existing-tensor factories
torch.zeros_like(t) # same shape/dtype/device as t, filled 0 torch.ones_like(t) torch.rand_like(t) # requires float dtype torch.empty_like(t) torch.full_like(t, 3.14)
Data Types (dtype)
dtype | Description | Bytes |
|---|---|---|
torch.float32 / torch.float | single-precision float | 4 |
torch.float64 / torch.double | double-precision float | 8 |
torch.float16 / torch.half | half-precision float | 2 |
torch.bfloat16 | brain float 16 | 2 |
torch.int8 | signed 8-bit int | 1 |
torch.int16 / torch.short | signed 16-bit int | 2 |
torch.int32 / torch.int | signed 32-bit int | 4 |
torch.int64 / torch.long | signed 64-bit int | 8 |
torch.uint8 | unsigned 8-bit int | 1 |
torch.bool | boolean | 1 |
torch.complex64 | complex (2×float32) | 8 |
torch.complex128 | complex (2×float64) | 16 |
t = t.float() # cast to float32 t = t.long() # cast to int64 t = t.to(torch.float16) t = t.type(torch.DoubleTensor) print(t.dtype)
Default float dtype is
float32. Calltorch.set_default_dtype(torch.float64)to change it globally.
Shape and Layout
t.shape # torch.Size([2, 3]) — same as t.size() t.size(0) # size of dim 0 t.ndim # number of dimensions t.numel() # total number of elements t.element_size() # bytes per element t.is_contiguous() t.contiguous() # returns a contiguous copy if not already
Reshaping
t.reshape(6) # returns view or copy as needed t.view(2, 3) # view (tensor must be contiguous) t.view(-1) # flatten t.view(-1, 3) # infer one dimension t.flatten() # always returns 1-D t.flatten(start_dim=1) # flatten from dim 1 onward t.squeeze() # remove all size-1 dims t.squeeze(0) # remove size-1 at dim 0 t.unsqueeze(0) # insert size-1 at dim 0 t.expand(2, 3) # broadcast without copying data t.repeat(2, 1) # tile data
Transposing
t.T # transpose 2-D tensor t.transpose(0, 1) # swap dim 0 and 1 t.permute(2, 0, 1) # reorder all dims t.movedim(0, -1) # move specific dim t.swapaxes(0, 1) # alias of transpose
Indexing and Slicing
t[0] # first row t[-1] # last row t[1, 2] # element at row 1, col 2 t[:, 1] # all rows, col 1 t[0:2, :] # rows 0-1, all cols t[..., -1] # last element along last dim (ellipsis) # Boolean indexing mask = t > 0 t[mask] # 1-D tensor of matching values t[t > 0] = 0 # in-place masked fill # Fancy (advanced) indexing idx = torch.tensor([0, 2]) t[idx] # rows 0 and 2 # torch.where torch.where(t > 0, t, torch.zeros_like(t)) # element-wise conditional
Memory and Device
t.device # device('cpu') or device('cuda:0') t.is_cuda # bool t.cpu() # move to CPU t.cuda() # move to default GPU t.to('cuda:0') # explicit GPU t.to(device) # move to variable device t.pin_memory() # pin for faster host→device transfer t.data_ptr() # raw memory address (int) t.untyped_storage() # underlying storage (t.storage() is deprecated) t.is_shared() # in shared memory (for multiprocessing)
Copying and Cloning
t.clone() # deep copy, keeps grad_fn t.detach() # no-copy view, severs grad graph t.detach().clone() # safe copy outside autograd t.copy_(other) # in-place copy from other tensor
Converting Out of PyTorch
t.item() # Python scalar (single-element tensors only) t.tolist() # nested Python list t.numpy() # NumPy array (CPU, no grad, shares memory) t.detach().cpu().numpy() # safe pattern for GPU / grad tensors
Attributes Quick Reference
| Attribute | Returns |
|---|---|
t.shape | torch.Size |
t.dtype | torch.float32 etc. |
t.device | device('cpu') etc. |
t.requires_grad | bool |
t.grad | gradient tensor or None |
t.grad_fn | grad function or None |
t.is_leaf | bool — user-created or detached |
t.layout | torch.strided (default), torch.sparse_coo, etc. |
Gotchas
t.view()requires the tensor to be contiguous — call.contiguous()first or use.reshape()which handles it automatically.
t.numpy()andtorch.from_numpy()share memory. Mutating one mutates the other.
Indexing with a Python int returns a tensor with one fewer dimension; indexing with a slice keeps the dimension.
t[0]vst[0:1].
In-place operations (e.g.
t.add_(1)) on a leaf tensor that hasrequires_grad=Truewill raise an error if called after a backward pass has started.