Python ImmutaInsights: Understanding Memory and Immutable Variables

We all know that Python has two types of objects: mutable and immutable. The names are also self-explanatory. Values stored in mutable objects can be changed, but the value of an immutable object cannot be changed. For example, a string containing my first name, 'Ritwik,' cannot be changed directly; you have to assign a new string to that variable if necessary. In the case of a list, you can change one of the elements. You can also increase, decrease, or change all elements of that list.

Although Python doesn't support pointers directly, you must have heard about them. Pointers store the memory address of a value. Languages like C, C++, Go, Rust, and many more heavily depend on pointers for compilation and execution speed. On the other hand, pointers are hard to understand and maintain. They also promote implicit code, which is against the Zen of Python.

The following example shows how Go uses pointers. Here, the num_pointer variable points to the memory address of the num variable. You cannot pass a variable by reference without using pointers in Go.

package main

import "fmt"

func main() {
    var num int8 = 2
    var num_pointer *int8 = &num // use of pointer
    fmt.Println(num) // Output: 2
    *num_pointer = 3
    fmt.Println(num) // Output: 3
}

Now see how python automatically passes a variable by reference.

my_list = [2]

pp = my_list
print(my_list) #  Output: [2]
pp[0] = 100
print(my_list) #  Output: [100]

So, the question is how does python manages variables without pointers?

The answer lies within the fundamental concept of mutable and immutable objects. It goes deeper than value cannot be changed. Whenever, you are assigning a value to variable, python always prefer pass by reference. You can assign integer 2 to a variable named num. Then assign 2 to variable num2. Then create list containing an element with value 2. If you assert that num, num2 and list element holding value 2 has same address, it will pass the test.

num = 2
num2 = 2
num_list = [2]

assert num is num2 is num_list[0] # Test passed ✅

As soon as you assign value 3 to num, the test will fail. Because instead of storing 3 in same memory address, python changed num's reference.

num = 3
num2 = 2
num_list = [2]

assert num is num2 is num_list[0] # Test failed ❌

This is because an integer is an immutable object, and Python stores integer 3 into a new memory block. Then it changes the variable num's reference to the new memory address. So far, what we have understood is Python will use pass-by-reference even if you are assigning the values directly to different variables. (This is not always true; for large objects, Python does not use pass-by-reference if you are only assigning the values directly.)

But Python does not use pointers for memory referencing. Yes, Python uses an object named PyObject for this. PyObject mainly holds three essential data:

• type

• value

• reference count

When you assign a value to a variable, Python binds that variable to a PyObject (exclusive to CPython). Actually, all data types are PyObject. In the case of lists and other complex data types, checking the value becomes hard and time-consuming, so Python creates a separate PyObject. But for small integer values or strings, even if you assign raw values, Python will refer to the existing PyObject. For integers, strings, booleans, and other atomic data types, the entire value remains in a single memory block. If the integer value in a memory block could be changed, it would collapse Python. Python uses PyObject heavily for internal processes. At random times, the reference count of an integer 2 can be 1000000064.

This is also a major reason why even if you assign a simple list like [2] to two separate variables, it will create two PyObjects. Because lists are mutable, change in a existing list used by python internal processes can create uncontrollable side-effects.

list_one = [2]
list_two = [2]
assert list_one is not list_two # Test passed ✅

You may wonder then why tuple is immutable? Frankly I do not know. But my guess would be it is a design choice. You can also create your own complex data types that are immutable.