Scientific Computing

1 Getting to Know Scientific Computing

In artificial intelligence research and development, the essence is turning every problem into a mathematical problem, so mathematical operations are very important. Many mathematical operations use the numpy library, because it provides a great many scientific computing methods that make our work very convenient. In this chapter I’ll start with the basic usage of numpy and gradually solve those mathematical problems, so that Python and mathematics can be combined more closely.

2 Getting to Know numpy

At its core, numpy is really a multidimensional array. Although the array objects we learned about before (list or tuple in Python) look similar to numpy data, those arrays can’t directly take part in numerical operations, whereas numpy objects can.

3 Creating Arrays

import numpy as np

arr1 = np.array([1, 2, 3, 4, 5, 6])
arr2 = np.array([[1, 2, 3, 4, ], [5, 6, 7, 8, ]])
print(arr1, arr1.shape, arr1.dtype)
print(arr2, arr2.shape, arr1.dtype)  # shape gets the shape of the array (2 rows, 4 columns), dtype gets the type of the elements in the array

If the elements have different types when we create an array, numpy automatically converts them to the same type.

arr3 = np.array([1, 2.5, 3])  # As soon as a float appears among the array elements, everything is converted to float
print(arr3, arr3.dtype)
arr4 = np.array(['4', 5, 5.6])  # As soon as a str appears among the array elements, everything is converted to str
print(arr4, arr4.dtype)

4 Accessing Arrays

Accessing numpy arrays works on the same principle as accessing a list or tuple in Python, and the methods are very similar; the only addition is the concept of dimensions.

# First, define a two-dimensional array
arr5 = np.array([[1, 2, 3],
                 [4, 5, 6],
                 [7, 8, 9],
                 [10, 11, 12]])

print(arr5[1])  # Row 1 [4 5 6]
print(arr5[1][0])  # Row 1, element 0: 4
print(arr5[:, 2])  # Element 2 of every row [ 3  6  9 12]
print(arr5[:2])  # The first 2 rows [[1 2 3] [4 5 6]]
print(arr5[1, :])  # All columns of row 1 [4 5 6]
print(arr5[:, 1:2])  # Columns 2 to 3 of all rows [[ 2] [ 5] [ 8] [11]]

For two dimensions, if there is a comma, the part before the comma selects rows and the part after selects columns. For n dimensions, the part before the first comma is the first dimension, followed in turn by the second dimension, the third dimension, and so on.

5 Shape Manipulation

1 Previewing Changes vs. Actually Modifying

numpy objects have a shape attribute. In basic Python we don’t pay much attention to shape, but in scientific computing shape is very important, and we’ll use it very frequently in the algorithm and model computations later on.

# Define a numpy array object with 6 rows and 3 columns
arr6 = np.array([[1, 2, 3],
                 [4, 5, 6],
                 [7, 8, 9],
                 [10, 11, 12],
                 [13, 14, 15],
                 [16, 17, 18]])

print(arr6.shape)  # Pay attention to the result printed each time
print(arr6.reshape(2, 9))  # Get a new shape; the original object is unchanged
print(arr6.shape, arr6)
print(arr6.resize(2, 9))  # No return value; actually modifies the array
print(arr6.shape, arr6)

In this code we used two methods, reshape and resize, to change the shape of the array. reshape only returns a preview of the reshaped array; in other words, if we want to use this result, we have to assign it to a separate variable and then use it. The resize method returns nothing, but it actually changes the shape of the array. Look carefully at the printed results and you’ll see the difference between these two shape operations.

2 Dimensionality Reduction

Dimensionality reduction is a very common and important operation in AI algorithms. The reason is that when we describe the features of something, there can be a great many dimensions, and too many dimensions cause trouble for our computations. In that case we need to reduce the number of dimensions before computing.

arr7 = np.array([[1, 10, 100], [2, 20, 200], [3, 30, 300]])

# Flatten to one dimension in row order
print(arr7)
print(arr7.ravel())
print(arr7.reshape(-1))
print(arr7.flatten())

# Flatten to one dimension in column order
print(arr7.ravel(order="F"))
print(arr7.reshape(-1, order="F"))
print(arr7.flatten(order="F"))

Modifying after flattening

print('ravel：{}'.format(arr7.ravel()))
arr7.ravel()[1] = 1000
print(arr7)

print('reshape: {}'.format(arr7.reshape(-1)))
arr7.reshape(-1)[2] = 3000
print(arr7)

print('flatten: {}'.format(arr7.flatten()))
arr7.flatten()[0] = 2000
print(arr7)

From the results we can see that flattening with the flatten method returns a copy; if you want to use it, you have to assign the result to another variable, and it doesn’t affect the original array. The ravel and reshape methods, on the other hand, return views, which means modifying the view directly affects the values in the original array.

3 Stacking Arrays

arr8 = np.array([[1, 10, 100], [2, 20, 200], [3, 30, 300]])
arr9 = np.array([1, 2, 3])
arr10 = np.array([[5], [6], [7]])

# Vertical stacking
print(np.vstack([arr8, arr9]))
print(np.row_stack([arr8, arr9]))

# Horizontal merging
print(np.hstack([arr8, arr10]))
print(np.column_stack([arr8, arr10]))

Note that when stacking multiple arrays horizontally, they must have the same number of rows; when merging vertically, they must have the same number of columns.

6 Basic Operations

1 Arithmetic Operations

In the past, if we wanted to perform element-wise arithmetic on two arrays of the same shape, we had to loop over both arrays, which took quite a bit of code and was error-prone. With NumPy, these operations become very simple.

import numpy as np

arr1 = np.array([11, 12, 13])
arr2 = np.array([21, 22, 23])
arr3 = np.array([31, 32, 33])

print(arr1 + arr2)
print(arr1 + arr2 + arr3)
print(arr1 - arr2)
print(arr1 - arr2 - arr3)
print(arr1 * arr2)
print(arr1 * arr2 * arr3)
print(arr1 / arr2)
print(arr1 / arr2 / arr3)
print(arr1 // arr2)
print(arr1 % arr2)
print(arr1 ** arr2)

print(np.add(arr1, arr2))
print(np.add(np.add(arr1, arr2), arr3))
print(np.subtract(arr1, arr2))
print(np.multiply(arr1, arr2))
print(np.divide(arr1, arr2))

From the output of the code we can see that when we use operators for arithmetic, we can chain the operations. When we use the methods for arithmetic, we can’t chain them, because the method only accepts two arguments. If we want to operate on several array objects, we have to nest the method calls. Besides arithmetic, the remainder, floor division, exponentiation and other operations we learned in basic Python are all supported.

2 Comparison Operations

print(arr1 <= arr2)
print(arr1 == arr2)
print(arr1 != arr2)

print(np.greater(arr1, arr2))
print(np.greater_equal(arr1, arr2))
print(np.less(arr1, arr2))
print(np.less_equal(arr1, arr2))
print(np.equal(arr1, arr2))
print(np.not_equal(arr1, arr2))

From the results, comparison operators return boolean values, i.e. True and False. So when would we use such operations? The first case is querying the elements in an array that satisfy a condition; the second case is performing different operations depending on the result of a comparison. Example code:

arr3 = np.array([23, 12, 25])
arr4 = np.array([21, 15, 23])

print(arr3[arr3 > arr4])  # Take the elements of arr3 that are greater than those of arr4
print(arr3[arr3 > 24])  # Take the elements of arr3 that are greater than 24
print(np.where(arr3 > 24, 0, arr3))  # Like a ternary expression: change values greater than 24 to 0, leave the rest unchanged
print(list(0 if x > 24 else x for x in arr3))
print(np.where(arr4 > 16, 0, arr4))

3 Broadcasting

All the operations above were based on arrays of the same shape. So can arrays of different shapes be operated on together? The answer is yes, but there are rules; you can’t compute arbitrarily. This kind of computation is called broadcasting.

# 1 Broadcasting: the trailing dimensions match, so the values are added
arr3 = np.arange(60).reshape(5, 4, 3)
arr4 = np.arange(12).reshape(4, 3)
print(arr3)
print(arr4)
print(arr3 + arr4)

# 2 One of the dimension values is 1
arr5 = np.arange(60).reshape(5, 4, 3)
arr6 = np.arange(4).reshape(4, 1)
print(arr5)
print(arr6)
print(arr5 + arr6)

# 3 arr7 is padded automatically, similar to the case above where one dimension value is 1
arr7 = np.arange(12).reshape(4, 3)
arr8 = np.array([1, 2, 3])
print(arr7)
print(arr8)
print(arr7 + arr8)

arr9 = np.arange(60).reshape(5, 4, 3)
arr10 = np.arange(8).reshape(4, 2)
print(arr9)
print(arr10)
# print(arr9 + arr10)  # Not one of the three cases discussed above, so it can't be computed

In fact, the “broadcast” in broadcasting means one-to-many. The rule is that if the two arrays have similar parts that line up, they can be operated on, and the missing parts are automatically filled in with the matching parts.

7 Mathematical Functions

numpy provides some common functions. Besides constants like np.pi or np.e, numpy also provides many mathematical functions for us to call directly.

import numpy as np

arr1 = np.array([1.3, 1.5, -1.8, 2.4, 3.2])
arr2 = np.array([1, 2, 3, 4, 5])

print(np.fabs(arr1))  # Absolute value
print(np.ceil(arr1))  # Round up
print(np.floor(arr1))  # Round down
print(np.round(arr1))  # Round to nearest
print(np.fmod(arr2, arr1))  # Remainder
print(np.modf(arr1))  # Get the fractional and integer parts
print(np.sqrt(arr2))  # Arithmetic square root
print(np.square(arr2))  # Square
print(np.exp(arr2))  # Exponential with base e
print(np.power(arr2, 3))  # Each element to the power of 3
print(np.log2(arr2))  # Logarithm base 2
print(np.log10(arr2))  # Logarithm base 10
print(np.log(arr2))  # Logarithm base e

8 Axis Direction

An array object has as many axes as it has dimensions. For the common two-dimensional array, axis 0 is the vertical direction and axis 1 is the horizontal direction.

import numpy as np

arr1 = np.array([[3, 7, 25, 8, 15, 20],
                 [4, 5, 6, 9, 14, 21]])

print(arr1)
print(np.max(arr1))  # Maximum of all array elements
print(np.max(arr1, axis=1))  # Maximum along axis 1
print(np.max(arr1, axis=0))  # Maximum along axis 0

numpy provides many functions that can compute along an axis.

print(np.min(arr1, axis=0))  # Minimum
print(np.min(arr1, axis=1))
print(np.mean(arr1, axis=0))  # Mean
print(np.median(arr1, axis=0))  # Median
print(np.sum(arr1, axis=1))  # Sum