Python Libraries Numpy and Scipy for Beginers

Introduction to Numpy

NumPy, which stands for Numerical Python, is a library consisting of multidimensional array objects and a collection of routines for processing those arrays. Using NumPy, mathematical and logical operations on arrays can be performed. This tutorial explains the basics of NumPy. It is prepared for those who want to learn about various functions of NumPy. It is specifically useful for algorithm developers.


In [1]:
import numpy as np
      In [2]:
a = np.arange(10) 
print (type (a))
<class 'numpy.ndarray'>
In [3]:
b=a[np.newaxis, :]
In [4]:
print (b.shape)
(1, 10)
In [5]:
A=np.array([[1,2,3], [1,3,2],[1,1,2]], dtype=float) #array creation
print(A)
print(A[...,1])#slicing
print (A[A > 1]) #boolean indexing
A.reshape(1,9)
a = np.array([np.nan, 1,2,np.nan,3,4,5]) 
print (a[~np.isnan(a)]) # removing nan
[[1. 2. 3.]
 [1. 3. 2.]
 [1. 1. 2.]]
[2. 3. 1.]
[2. 3. 3. 2. 2.]
[1. 2. 3. 4. 5.]
In [6]:
A.reshape(1,9)
Out[6]:
array([[1., 2., 3., 1., 3., 2., 1., 1., 2.]])

Numpy Broadcasting

Broadcasting refers to the ability of NumPy to treat arrays of different shapes during arithmetic operations.
In [7]:
a = np.array([1,2,3,4]) 
b = np.array([[10,20,30,40], [1,2,3,4]]) 
c = a * b 
print (c) 
[[ 10  40  90 160]
 [  1   4   9  16]]

Numpy iterarting

NumPy package contains an iterator object numpy.nditer
In [8]:
a = np.arange(0,60,5)
a = a.reshape(3,4)

print ('Original array is:')
print(a)
print ("\n")
a = a.T  #if we want iter transpose
print ('Modified array is:')
i=0
for x in np.nditer(a): #it will take elementwise
#for x in a:  ##it will take row wise 
    i=i+1
    print(x)
print("total count is ",i) #Best way to print anything
Original array is:
[[ 0  5 10 15]
 [20 25 30 35]
 [40 45 50 55]]


Modified array is:
0
5
10
15
20
25
30
35
40
45
50
55
total count is  12
In [9]:
##If the same elements are stored using F-style order, t
#efficient way of iterating over an array.
#c-style --> row wise iteration
#f-style --> column wise iteration

a = np.arange(0,60,5)
a = a.reshape(3,4)
print ('Original array is:')
print (a)
print ('\n')

print ('Transpose of the original array is:')
b = a.T
print( b)
print ('\n')

print ('Sorted in C-style order:')
c = b.copy(order = 'C')
print (c)
for x in np.nditer(c):
    print (x)

print ('\n')

print ('Sorted in F-style order:')
c = b.copy(order = 'F')
print (c)
for x in np.nditer(c):
    print (x)
Original array is:
[[ 0  5 10 15]
 [20 25 30 35]
 [40 45 50 55]]


Transpose of the original array is:
[[ 0 20 40]
 [ 5 25 45]
 [10 30 50]
 [15 35 55]]


Sorted in C-style order:
[[ 0 20 40]
 [ 5 25 45]
 [10 30 50]
 [15 35 55]]
0
20
40
5
25
45
10
30
50
15
35
55


Sorted in F-style order:
[[ 0 20 40]
 [ 5 25 45]
 [10 30 50]
 [15 35 55]]
0
5
10
15
20
25
30
35
40
45
50
55
In [10]:
print ('Sorted in C-style order:' )
for x in np.nditer(a, order = 'C'): 
    print (x,)  
print ('\n' )

print ('Sorted in F-style order:' )
for x in np.nditer(a, order = 'F'): 
    print (x,)
Sorted in C-style order:
0
5
10
15
20
25
30
35
40
45
50
55


Sorted in F-style order:
0
20
40
5
25
45
10
30
50
15
35
55
In [11]:
## Modifying Array Values

print ('Original array is:')
print( a)
print ('\n')

for x in np.nditer(a, op_flags = ['readwrite']): ##op_flag default value 'read-only'
    x[...] = 2*x
print ('Modified array is:')
print (a)
### other flags
##x_index, f_index,multi-ndex,external_loop
for x in np.nditer(a, flags = ['external_loop'], order = 'F'):
    print (x,)
Original array is:
[[ 0  5 10 15]
 [20 25 30 35]
 [40 45 50 55]]


Modified array is:
[[  0  10  20  30]
 [ 40  50  60  70]
 [ 80  90 100 110]]
[ 0 40 80]
[10 50 90]
[ 20  60 100]
[ 30  70 110]
In [12]:
## Broadcasting iteration
a = np.arange(0,60,5) 
a = a.reshape(3,4) 

print ('First array is:' )
print (a )
print ('\n'  )

print ('Second array is:' )
b = np.array([1, 2, 3, 4], dtype = int) 
print (b  )
print ('\n' )
print ('Modified array is:' )
for x,y in np.nditer([a,b]): 
    print ("%d:%d" % (x,y),)
First array is:
[[ 0  5 10 15]
 [20 25 30 35]
 [40 45 50 55]]


Second array is:
[1 2 3 4]


Modified array is:
0:1
5:2
10:3
15:4
20:1
25:2
30:3
35:4
40:1
45:2
50:3
55:4

NumPy - Array Manipulation

Changing shape: reshape, flat, flatten, ravel
Transpose Operations: transpose, ndarray.T, rollaxis, swapaxes
Changing Dimensions: broadcast, broadcast_to,expand_dims, squeeze
joining arrays: concatenate, stack, hstack,vstack
Splitting Arrays: split,hsplit, vsplit
Adding / Removing Elements: append, resize,insert, unit, delete
NumPy - Binary Operators: bitwise_and, bitwise_or, invert,left_shift,right_shift
NumPy - String Functions: add(), multiply(),center(),capitalize(), title(), lower(), upper(), split(), splitness() strip(), join(), replace(), decode(), encode()
Functions for Rounding/floor: around(), floor(),ceil()-->ceil of the scalar x is the smallest integer i, where i >= x.
In [13]:
a = np.arange(0,60,5) 
a=a.reshape(3,4)
print(a)
print(a.ravel)
[[ 0  5 10 15]
 [20 25 30 35]
 [40 45 50 55]]
<built-in method ravel of numpy.ndarray object at 0x7fbfda4f9300>

Trignometric functions

In [14]:
a = np.array([0,30,45,60,90]) 

print ('Array containing sine values:' )
sin = np.sin(a*np.pi/180) 
print (sin )
print ('\n' )
print ('Compute sine inverse of angles. Returned values are in radians.' )
inv = np.arcsin(sin) 
print (inv )
print ('\n'  
)
print ('Check result by converting to degrees:' )
print (np.degrees(inv) )
print ('\n')  

print ('arccos and arctan functions behave similarly:' )
cos = np.cos(a*np.pi/180) 
print (cos )
print ('\n'  )

print ('Inverse of cos:' )
inv = np.arccos(cos) 
print (inv )
print ('\n'  )

print ('In degrees:' )
print (np.degrees(inv) )
print ('\n'  )

print ('Tan function:' )
tan = np.tan(a*np.pi/180) 
print (tan)
print ('\n')  

print ('Inverse of tan:' )
inv = np.arctan(tan) 
print (inv )
print ('\n' ) 

print ('In degrees:' )
print (np.degrees(inv))
Array containing sine values:
[0.         0.5        0.70710678 0.8660254  1.        ]


Compute sine inverse of angles. Returned values are in radians.
[0.         0.52359878 0.78539816 1.04719755 1.57079633]


Check result by converting to degrees:
[ 0. 30. 45. 60. 90.]


arccos and arctan functions behave similarly:
[1.00000000e+00 8.66025404e-01 7.07106781e-01 5.00000000e-01
 6.12323400e-17]


Inverse of cos:
[0.         0.52359878 0.78539816 1.04719755 1.57079633]


In degrees:
[ 0. 30. 45. 60. 90.]


Tan function:
[0.00000000e+00 5.77350269e-01 1.00000000e+00 1.73205081e+00
 1.63312394e+16]


Inverse of tan:
[0.         0.52359878 0.78539816 1.04719755 1.57079633]


In degrees:
[ 0. 30. 45. 60. 90.]

NumPy - Arithmetic Operations

add(), subtract(),multiply(), divide(),reciprocal(), power() mod(), remainder()
In [15]:
a = np.array([10,20,30]) 
b = np.array([3,5,7]) 

print ('Our array is:' )
print (a )
print("\n power 2:",np.power(a,2)) 
print("\n reciprocal 2:",np.reciprocal(a))
print("\n mod a/b:",np.mod(a,b)) 
print("\n remainder a/b:",np.remainder(a,b)) 

a = np.array([-5.6j, 0.2j, 11. , 1+1j]) 
print("\n imaginry", np.imag(a))
print("\n real", np.real(a))
print("\n complex conjugate", np.conj(a)) # complex conjugate
print("\n angle in rad", np.angle(a))
print("\n angle in degrees", np.angle(a, deg=True))
Our array is:
[10 20 30]

 power 2: [100 400 900]

 reciprocal 2: [0 0 0]

 mod a/b: [1 0 2]

 remainder a/b: [1 0 2]

 imaginry [-5.6  0.2  0.   1. ]

 real [-0.  0. 11.  1.]

 complex conjugate [-0.+5.6j  0.-0.2j 11.-0.j   1.-1.j ]

 angle in rad [-1.57079633  1.57079633  0.          0.78539816]

 angle in degrees [-90.  90.   0.  45.]
In [16]:
##NumPy - Statistical Functions 
## amin, amax
### more functions mean(), median(), percentile()
a = np.array([[3,7,5],[8,4,3],[2,4,9]]) 
print("\n original array\n", a)
print("\n row wise min :", np.amin(a,1))
print("\n column wise min :", np.amin(a,0))
print("\n row wise max :", np.amax(a,1))
print("\n overall max :", np.amax(a))
print("\n overall min :", np.amin(a))

##The numpy.ptp() function returns the range (maximum-minimum) of values along an axis
print ('\n Applying ptp() function:' )
print (np.ptp(a) )


print ('\n Applying ptp() function along axis 1:' )
print (np.ptp(a, axis = 1) )


print ('\n Applying ptp() function along axis 0:')
print( np.ptp(a, axis = 0) )

b = np.array([1,2,3,4]) 
wts = np.array([4,3,2,1]) 

print ('Applying average() function again:' )
print (np.average(b,weights = wts) ) # weighted average

print("\n standard deviation of a ", np.std(a)) # std = sqrt(mean(abs(x - x.mean())**2))
print("\n variance of a ", np.var(a)) # var= mean(abs(x - x.mean())**2).
 original array
 [[3 7 5]
 [8 4 3]
 [2 4 9]]

 row wise min : [3 3 2]

 column wise min : [2 4 3]

 row wise max : [7 8 9]

 overall max : 9

 overall min : 2

 Applying ptp() function:
7

 Applying ptp() function along axis 1:
[4 5 7]

 Applying ptp() function along axis 0:
[6 3 6]
Applying average() function again:
2.0

 standard deviation of a  2.309401076758503

 variance of a  5.333333333333333

NumPy - Sort, Search & Counting Functions

$ numpy.sort(a, axis, kind, order)
In [17]:
#dt = np.dtype([('name', 'str'),('age', int)]) 
dt = np.dtype([('name', 'S10'),('age', int)])  # S10 starts each string with b for bytes
a = np.array([("raju",21),("anil",25),("ravi", 17), ("amar",27)], dtype = dt) 

print ('Our array is:' )
print( a )
print ('\n'  )

print( 'Order by name:' )

print (np.sort(a,  order = 'name', kind='heapsort'))
#print (np.sort(a, kind='mergesort', order = 'name')) ## default kind=quicksort()
Our array is:
[(b'raju', 21) (b'anil', 25) (b'ravi', 17) (b'amar', 27)]


Order by name:
[(b'amar', 27) (b'anil', 25) (b'raju', 21) (b'ravi', 17)]
In [18]:
##argsort gives indices of sorting
#numpy.lexsort() left for free time

x = np.array([3, 1, 2]) 

print ('Our array is:' )
print (x )
print ('\n'  )

print ('Applying argsort() to x:' )
y = np.argsort(x) 
print (y )
print ('\n'  )

print ('Reconstruct original array in sorted order:' )
print (x[y]) 
print ('\n'  )

print ('Reconstruct the original array using loop:' )
for i in y: 
    print( x[i])
    
print ("\n max value index is ", np.argmax(x) ) #gives index of max value
print ("\n min value index is ", np.argmin(x) ) #gives index of min value

a = np.array([[30,40,0],[0,20,10],[50,0,60]]) 
print ("\n  Non-zero elements are ", np.nonzero(a))
#numpy.nonzero() function returns the indices of non-zero elements in the input
Our array is:
[3 1 2]


Applying argsort() to x:
[1 2 0]


Reconstruct original array in sorted order:
[1 2 3]


Reconstruct the original array using loop:
1
2
3

 max value index is  0

 min value index is  1

  Non-zero elements are  (array([0, 0, 1, 1, 2, 2]), array([0, 1, 1, 2, 0, 2]))
In [19]:
x = np.arange(12.).reshape(3, 4) 

print ('Our array is:' )
print( x  )

print ('Indices of elements > 3' )
y = np.where(x > 3) # gives tuple for values x and y
print("\n type of y ", type(y))
print (y  )

print ('Use these indices to get elements satisfying the condition' )
print( x[y])
Our array is:
[[ 0.  1.  2.  3.]
 [ 4.  5.  6.  7.]
 [ 8.  9. 10. 11.]]
Indices of elements > 3

 type of y  <class 'tuple'>
(array([1, 1, 1, 1, 2, 2, 2, 2]), array([0, 1, 2, 3, 0, 1, 2, 3]))
Use these indices to get elements satisfying the condition
[ 4.  5.  6.  7.  8.  9. 10. 11.]
In [20]:
x = np.arange(9.).reshape(3, 3) 

print ('Our array is:' )
print(x)
# define a condition 
condition = np.mod(x,2) == 0 

print ('Element-wise value of condition' )
print (condition  )

print ('Extract elements using condition' )
print( np.extract(condition, x))
Our array is:
[[0. 1. 2.]
 [3. 4. 5.]
 [6. 7. 8.]]
Element-wise value of condition
[[ True False  True]
 [False  True False]
 [ True False  True]]
Extract elements using condition
[0. 2. 4. 6. 8.]
In [21]:
from matplotlib import pyplot as plt 
a = np.array([22,87,5,43,56,73,55,54,11,20,51,5,79,31,27]) 
#np.histogram(a,bins = [0,20,40,60,80,100]) 
hist,bins = np.histogram(a,bins = [0,20,40,60,80,100]) #bins provide range for calculation
print (hist) 
print (bins) 
%matplotlib inline
plt.hist(a,bins = [0,20,40,60,80,100]) # histogram using matplotlib
plt.show()
[3 4 5 2 1]
[  0  20  40  60  80 100]

I/O with Numpy

In [22]:
a = np.array([1,2,3,4,5]) 
np.save('outfile',a) #save as outfile.npy
b = np.load('outfile.npy') # load existing numpy file
print (b) 

np.savetxt('out.txt',a) #save as text file
b = np.loadtxt('out.txt') #load as text file
print (b)
[1 2 3 4 5]
[1. 2. 3. 4. 5.]

Scipy Complete Tutorial


SciPy, a scientific library for Python is an open source, BSD-licensed library for mathematics, science and engineering. The SciPy library depends on NumPy, which provides convenient and fast N-dimensional array manipulation. The main reason for building the SciPy library is that, it should work with NumPy arrays. It provides many user-friendly and efficient numerical practices such as routines for numerical integration and optimization. This is an introductory tutorial, which covers the fundamentals of SciPy and describes how to deal with its various modules.

sub-packages

scipy.cluster Vector quantization / Kmeans
scipy.constants Physical and mathematical constants
scipy.fftpack Fourier transform
scipy.integrate Integration routines
scipy.interpolate Interpolation
scipy.io Data input and output
scipy.linalg Linear algebra routines
scipy.ndimage n-dimensional image package
scipy.odr Orthogonal distance regression
scipy.optimize Optimization
scipy.signal Signal processing
scipy.sparse Sparse matrices
scipy.spatial Spatial data structures and algorithms
scipy.special Any special mathematical functions
scipy.stats Statistics

In [23]:
## K-means clustering example using Scipy
import scipy
from scipy.cluster.vq import kmeans,vq,whiten #vq-vector quantization
from numpy import vstack,array
from numpy.random import rand

# data generation with three features
data = vstack((rand(100,3) + array([.5,.5,.5]),rand(100,3)))

# whitening of data
data = whiten(data)
print("\n data \n", data)
# computing K-Means with K = 3 (2 clusters)
centroids,_ = kmeans(data,3)
print("Centroids are \n",centroids)
# assign each sample to a cluster
clx,_ = vq(data,centroids)
print("Cluster: \n", clx)
 data 
 [[3.48587872 2.74389379 2.31904199]
 [3.75494054 2.42481705 3.88430759]
 [1.98839479 3.30904392 2.91091513]
 [2.41843083 3.94049952 1.86343086]
 [1.63318585 1.85730839 2.75470585]
 [2.34246271 2.52206027 2.50047338]
 [1.80542889 2.82413815 1.47040787]
 [1.88820145 2.01490229 1.6425086 ]
 [3.58378659 2.2586092  3.96521523]
 [3.84253632 1.3556426  2.60369111]
 [2.94861214 2.33055111 1.3789749 ]
 [1.33690968 3.37239769 3.23063977]
 [2.51339503 1.92297117 2.02555435]
 [3.60181788 3.34786653 1.56923145]
 [3.44042921 1.77971113 2.91230912]
 [1.86927633 1.99164762 3.53497375]
 [1.48737491 3.62489076 2.22208395]
 [2.13033958 2.94612102 1.83481506]
 [2.40207885 3.63966545 2.26501299]
 [3.82704628 2.15564557 2.0872997 ]
 [2.51156355 2.46388974 1.54769404]
 [2.02306286 3.15821272 1.67897287]
 [2.73607694 1.52335362 3.05985556]
 [3.80876607 1.57947304 2.72690645]
 [2.27451917 3.54966085 3.91767964]
 [2.56109    2.52997087 2.39007842]
 [3.19148576 2.48101761 3.83621402]
 [3.55176653 3.03295709 3.41683826]
 [1.66735489 3.23711677 3.41113761]
 [2.59956108 3.87852205 1.38323417]
 [2.53861626 3.06746681 2.04165062]
 [3.42928292 3.1791854  2.53150295]
 [2.5710894  2.18831334 1.62977206]
 [3.54928274 3.7545655  2.77424802]
 [2.74046462 3.86395018 3.58869953]
 [2.9913189  1.39383357 3.83459256]
 [1.663792   1.76136864 1.58119263]
 [2.31561231 2.17175156 3.08607849]
 [2.34247047 1.59703382 3.03349119]
 [2.54045211 2.42245376 3.49533364]
 [2.43366159 1.40943173 2.11137525]
 [2.20327641 2.80447487 2.76659789]
 [2.75653638 3.91780583 1.60971528]
 [2.43646013 3.0318176  1.64150421]
 [2.68069332 2.23939109 3.25503333]
 [2.45801982 3.79125302 2.37411912]
 [2.60317474 3.01808386 2.38580676]
 [1.96133502 3.24758208 2.93240926]
 [1.61749309 3.94625644 3.62376767]
 [1.75795861 2.54341608 3.75545988]
 [2.16265012 3.52246196 1.37903373]
 [1.93942623 3.43529057 2.72390091]
 [2.16747776 3.12284004 2.44354044]
 [2.41932823 2.747279   3.34583161]
 [1.91230152 1.70223952 1.53178284]
 [2.65207311 3.01703734 3.85810176]
 [1.36734634 3.81783047 2.64775577]
 [3.04638666 2.84539191 3.78018909]
 [1.40218905 2.62912436 2.93556054]
 [2.19999433 3.30989776 2.4578598 ]
 [3.37098734 2.57232931 3.72041061]
 [3.74857287 3.16340909 2.7106259 ]
 [1.84214139 3.93901692 3.81565676]
 [3.08815578 2.52793218 3.77836956]
 [3.0597064  2.24783309 3.4277712 ]
 [3.64061385 2.67143571 1.76708805]
 [2.61946003 3.58979998 2.4058914 ]
 [2.90492979 3.43777936 3.7849459 ]
 [3.21652589 3.43804791 2.78249587]
 [1.66610703 2.22073077 1.34786345]
 [1.5581413  2.96509539 3.40701602]
 [1.88028989 2.0436983  3.13301475]
 [2.33595893 1.35751095 3.31854549]
 [2.62384799 2.05591808 3.37935336]
 [2.09542464 4.06308604 2.20570863]
 [3.21852002 3.81793731 1.57598932]
 [3.0303194  3.79011342 2.86008644]
 [3.71173067 3.92376128 2.52292283]
 [3.06365726 2.53375319 3.16943184]
 [3.69438952 1.60244374 2.13337896]
 [1.47307021 1.41815269 1.94679186]
 [2.60702909 3.48087736 2.50592585]
 [2.92853695 3.43108257 1.53970077]
 [2.10453355 3.97707446 1.53649806]
 [2.97781096 3.60297897 3.04353344]
 [1.3902426  2.15887844 2.38830284]
 [2.70058504 3.59379608 2.27480284]
 [1.43255985 2.07956539 1.73791737]
 [1.42608351 3.97089899 2.31766726]
 [2.83317359 2.31796492 1.48813229]
 [3.47033247 2.05899926 2.77476402]
 [1.98355373 1.69299596 3.42770989]
 [1.84781087 1.45577213 2.70653012]
 [3.65520995 3.8627764  2.60241618]
 [3.57655717 4.0070528  1.77210979]
 [3.0726425  2.92647623 3.58968028]
 [2.01735083 1.78713109 2.56610685]
 [2.56887545 1.6328032  1.9614398 ]
 [1.7920097  1.41964184 3.43249191]
 [3.64307092 3.61237752 2.89428912]
 [0.58741481 2.45863612 0.85337144]
 [1.13779752 1.48221559 0.16403919]
 [1.1975923  1.96115698 0.73347078]
 [2.52700339 2.36059461 1.24021037]
 [2.27957455 2.64735843 0.48652011]
 [2.09838242 2.16498729 2.08237622]
 [1.1896182  1.85491241 1.67047337]
 [1.45328486 1.09057465 0.79744235]
 [0.21261684 1.83191379 1.42516972]
 [2.27025082 1.88274773 0.20154923]
 [1.25219948 2.59172329 2.25207551]
 [0.64516942 0.81233581 1.27828111]
 [0.25760825 2.0994417  2.11978237]
 [0.87701664 1.27655081 2.12703859]
 [0.63241593 2.64010402 1.29051997]
 [1.33569388 2.70908795 1.01169927]
 [1.74092043 2.14749411 0.35993701]
 [1.96836336 2.41180659 0.54959637]
 [1.79625298 0.16411255 0.98933168]
 [1.2255503  1.78051215 1.50617482]
 [1.62078081 1.18323914 1.02323866]
 [2.4145445  1.42851226 2.31894015]
 [1.94490587 1.47082698 2.0819848 ]
 [0.86766965 0.24750464 2.33941342]
 [2.19120602 0.41082526 0.2171667 ]
 [0.46183093 1.87191133 1.67207454]
 [2.05071127 1.13215474 0.07309415]
 [2.16303304 1.26309035 2.37603569]
 [1.78742648 2.45929885 1.96506309]
 [1.93324894 2.3931526  0.58170419]
 [0.1269096  2.03970864 2.56587404]
 [2.25604074 1.04890093 0.55092019]
 [0.15401254 2.15445254 0.76894306]
 [2.14638563 1.98497894 1.11827303]
 [1.51543407 1.634694   1.64681436]
 [0.25335078 0.78083807 2.41064725]
 [0.46127543 1.98950484 0.42018688]
 [1.05961159 2.01819664 0.0092679 ]
 [1.57959998 0.72592229 2.13206321]
 [1.75371965 2.63349893 1.94548353]
 [1.65144377 2.07393679 0.87062245]
 [1.16463461 2.02939811 0.43258096]
 [1.03275237 0.34377211 1.34148108]
 [0.81062604 1.95757613 0.75023439]
 [0.23204189 1.58245825 0.65012265]
 [2.57619499 1.70981665 0.94734117]
 [0.94667029 0.57872994 2.19227235]
 [0.86257572 2.42187003 1.82396678]
 [2.21000706 0.44910105 1.91914643]
 [0.27124157 2.18586213 0.65544702]
 [0.01572698 1.20433536 2.01432178]
 [1.12485125 0.81009972 0.85117509]
 [2.57246171 0.44377435 2.3093374 ]
 [0.48332323 1.40188889 2.35836805]
 [0.20303147 2.35387288 2.21314472]
 [1.57772262 0.34236562 1.14910494]
 [0.73959733 0.39741616 2.54625334]
 [0.00892029 0.90275359 1.22382605]
 [0.94941275 0.8263824  2.17081631]
 [1.16553865 2.07528527 0.20802848]
 [0.15618475 1.58099067 1.27396348]
 [1.71076775 1.65874565 1.32193877]
 [2.47316042 2.33485473 1.42585365]
 [2.08582198 2.17435841 0.36208186]
 [2.44967124 1.02917502 1.13890198]
 [0.0598344  1.86848023 0.99090266]
 [2.16947259 0.99334565 2.04268281]
 [0.03532586 0.12814287 1.66333518]
 [1.14217605 0.39286575 1.8182314 ]
 [0.04589891 2.44522302 0.10568495]
 [2.42345311 1.80239591 1.62274633]
 [2.24193482 1.64055624 1.83126997]
 [2.05347072 0.47722099 1.62129234]
 [0.79623367 1.36189796 2.29741189]
 [2.3138769  1.63907829 2.40636334]
 [1.50328913 1.23345288 1.82676454]
 [0.62430405 1.44728873 0.70265134]
 [2.42383996 0.62496562 2.43447483]
 [1.99005678 0.75241193 2.60102739]
 [1.82161256 0.45128586 1.38993357]
 [1.93250826 2.32516357 1.10608565]
 [1.78263537 0.87495914 2.17191312]
 [1.9976101  1.80811961 0.73464188]
 [0.66914384 1.46981102 2.45689977]
 [0.11856904 1.69364155 1.1504267 ]
 [0.6197423  0.02738986 0.19722054]
 [0.53308648 1.26549418 1.99088935]
 [0.54703248 0.68099235 0.02452312]
 [0.85568816 0.61136702 0.19496127]
 [0.61650401 0.72586147 0.12320324]
 [0.32520009 1.48391169 1.61230815]
 [0.30467302 1.23230064 2.09373494]
 [0.73118031 2.59717234 0.96838327]
 [1.77987437 1.14494926 2.47980594]
 [0.18956732 1.31074354 2.62876692]
 [0.2510283  0.56792475 0.41344728]
 [0.35997502 1.86843845 0.41703341]
 [2.34375274 1.66301611 2.55564562]
 [1.19040293 1.94806998 1.66749274]
 [0.45771978 2.33495961 0.33781381]]
Centroids are 
 [[0.97361954 1.51668303 1.18708869]
 [2.35090675 1.68197547 2.26731003]
 [2.63472909 3.24877122 2.74178755]]
Cluster: 
 [2 2 2 2 1 2 1 1 2 1 1 2 1 2 1 1 2 2 2 1 1 2 1 1 2 2 2 2 2 2 2 2 1 2 2 1 0
 1 1 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 0 2 1 1 1
 2 2 2 2 2 1 0 2 2 2 2 1 2 0 2 1 1 1 1 2 2 2 1 1 1 2 0 0 0 1 0 1 0 0 0 0 1
 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 0 1 0 0
 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 0 1 1 1 0 1 0 0 1 1 0 0 1 0 0 0
 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0]
In [24]:
##Fourier transform using scipy
#Importing the fft and inverse fft functions from fftpackage
#same like we can calculate dct FFT and all
from scipy.fftpack import fft, ifft

#create an array with random n numbers
x = np.array([1.0, 2.0, 1.0, -1.0, 1.5])

#Applying the fft function
y = fft(x)
print( y)
#FFT is already in the workspace, using the same workspace to for inverse transform

yinv = ifft(y)

print (yinv)
[ 4.5       +0.j          2.08155948-1.65109876j -1.83155948+1.60822041j
 -1.83155948-1.60822041j  2.08155948+1.65109876j]
[ 1. +0.j  2. +0.j  1. +0.j -1. +0.j  1.5+0.j]
In [25]:
## Scipy Integration example
## refer https://www.tutorialspoint.com/scipy/scipy_integrate.htm
import scipy.integrate
from numpy import exp

f= lambda x:exp(-x**2)
i = scipy.integrate.quad(f, 0, 1)
#first number integral and the second estimate of absolute error
print( i)


## Double integration
from numpy import exp
from math import sqrt
f = lambda x, y : 16*x*y
g = lambda x : 0
h = lambda y : sqrt(1-4*y**2)
i = scipy.integrate.dblquad(f, 0, 0.5, g, h) # for tripple integration tplquad,for nth nquad
print (i)
(0.7468241328124271, 8.291413475940725e-15)
(0.5, 1.7092350012594845e-14)
In [26]:
##Scipy-Interpolation example
##Interpolation is the process of finding a value between two points on a line or a curve
from scipy import interpolate as intp
import matplotlib.pyplot as plt
x = np.linspace(0, 4, 12)
y = np.cos(x**2/3+4)
print (x,y)
f1 = intp.interp1d(x, y,kind = 'linear') ##i-D interpolation

f2 = intp.interp1d(x, y, kind = 'cubic')

xnew = np.linspace(0, 4,30)

plt.plot(x, y,'o', xnew, f1(xnew), '-', xnew, f2(xnew), '--')

plt.legend(['data', 'linear', 'cubic',], loc = 'best') #legend for multiple graph in single 
##figure 

plt.show()
[0.         0.36363636 0.72727273 1.09090909 1.45454545 1.81818182
 2.18181818 2.54545455 2.90909091 3.27272727 3.63636364 4.        ] [-0.65364362 -0.61966189 -0.51077021 -0.31047698 -0.00715476  0.37976236
  0.76715099  0.99239518  0.85886263  0.27994201 -0.52586509 -0.99582185]
In [27]:
from scipy.interpolate import UnivariateSpline
x = np.linspace(-3, 3, 50)
y = np.exp(-x**2) + 0.1 * np.random.randn(50)
plt.figure()
plt.plot(x, y, 'ro', ms = 5)
#plt.show() #uncomment if want result in seperate figure

spl = UnivariateSpline(x, y)
xs = np.linspace(-3, 3, 1000)
plt.plot(xs, spl(xs), 'g', lw = 3)
#plt.show()

spl.set_smoothing_factor(0.5)
plt.plot(xs, spl(xs), 'b', lw = 3)
plt.show()
In [28]:
##Scipy IO with matlab
import scipy.io as sio


#Save a mat file
vect = np.arange(10)
sio.savemat('array.mat', {'vect':vect})

#Now Load the File
mat_file_content = sio.loadmat('array.mat')
print (mat_file_content)

mat_content = sio.whosmat('array.mat') #checking content of file without loading it
print ("\n",mat_content)
{'__header__': b'MATLAB 5.0 MAT-file Platform: posix, Created on: Wed Nov 27 18:04:28 2019', '__version__': '1.0', '__globals__': [], 'vect': array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])}

 [('vect', (1, 10), 'int64')]

Linear Algebra:

Solving Linear rquation
x + 3y + 5z = 10
2x + 5y + z = 8
2x + 3y + 8z = 3
In [29]:
#Linear equation solving usibg scipy 
#importing the scipy and numpy packages
from scipy import linalg
import numpy as np

#Declaring the numpy arrays
a = np.array([[3, 2, 0], [1, -1, 0], [0, 5, 1]])
b = np.array([2, 4, -1])

#Passing the values to the solve function
x = linalg.solve(a, b)

#printing the result array
print (x)

y = linalg.det(A)

#printing the result
print ("Determinant of matrix a \n",y)

#Passing the values to the eig function
l, v = linalg.eig(a)

#printing the result for eigen values
print("Eigen values of matrix a \n", l)

#printing the result for eigen vectors
print("Eigen vectors of matrix a \n",v)
[ 2. -2.  9.]
Determinant of matrix a 
 -2.0
Eigen values of matrix a 
 [ 1.        +0.j -1.44948974+0.j  3.44948974+0.j]
Eigen vectors of matrix a 
 [[ 0.          0.19399248 -0.89052821]
 [ 0.         -0.43158379 -0.20014165]
 [ 1.          0.88096671 -0.40853743]]

Singular value decomposition

(SVD) can be thought of as an extension of the eigenvalue problem to non square matrixes

svd factorizes the matrix ‘a’ into two unitary matrices ‘U’ and ‘Vh’ and a 1-D array ‘s’ of singular values (real, non-negative) such that a == USVh, where ‘S’ is a suitably shaped matrix of zeros with the main diagonal ‘s’.
In [30]:
#Declaring the numpy array
a = np.random.randn(3, 2) + 1.j*np.random.randn(3, 2)

#Passing the values to the eig function
U, s, Vh = linalg.svd(a)

# printing the result
print (U, Vh, s)
[[-0.2463891 -0.29449235j  0.13919235-0.65344665j  0.63715826-0.01513225j]
 [-0.33334297-0.5252048j   0.43255569+0.29893581j -0.17272255-0.55384317j]
 [ 0.05235255+0.68034422j  0.49758451+0.17197551j  0.39480716-0.31835882j]] [[ 0.61960112+0.j          0.67561263-0.39955228j]
 [ 0.78491684+0.j         -0.53331808+0.31540034j]] [2.37992702 0.79207603]
Eigenvectors make understanding linear transformations easy. They are the "axes" (directions) along which a linear transformation acts simply by "stretching/compressing" and/or "flipping"; eigenvalues give you the factors by which this compression occurs.
Consider a matrix A, for an example one representing a physical transformation (e.g rotation). When this matrix is used to transform a given vector x the result is y=Ax.
Now an interesting question is
Are there any vectors x which does not change it's direction under this transformation, but allow the vector magnitude to vary by scalar λ?
Such a question is of the form
Ax=λx
So, such special x are called eigenvector(s) and the change in magnitude depends on the eigenvalue λ.

SciPy - Ndimage


The SciPy ndimage submodule is dedicated to image processing. Here, ndimage means an n-dimensional image.
Some of the most common tasks in image processing are as follows &miuns;
Input/Output, displaying images Basic manipulations − Cropping, flipping, rotating, etc. Image filtering − De-noising, sharpening, etc. Image segmentation − Labeling pixels corresponding to different objects Classification Feature extraction Registration

SciPy - Optimize

The scipy.optimize package provides several commonly used optimization algorithms.

SciPy - ODR (Orthogonal distance regression)


Why Orthogonal Distance Regression (ODR)? Sometimes one has measurement errors in the explanatory (a.k.a., “independent”) variable(s), not just the response (a.k.a., “dependent”) variable(s). Ordinary Least Squares (OLS) fitting procedures treat the data for explanatory variables as fixed, i.e., not subject to error of any kind. Furthermore, OLS procedures require that the response variables be an explicit function of the explanatory variables; sometimes making the equation explicit is impractical and/or introduces errors. ODR can handle both of these cases with ease, and can even reduce to the OLS case if that is sufficient for the problem.
In [31]:
import numpy as np
import matplotlib.pyplot as plt
from scipy.odr import *
import random

# Initiate some data, giving some randomness using random.random().
x = np.array([0, 1, 2, 3, 4, 5])
y = np.array([i**2 + random.random() for i in x])

# Define a function (quadratic in our case) to fit the data with.
def linear_func(p, x):
    m, c = p
    return m*x + c

# Create a model for fitting.
linear_model = Model(linear_func)

# Create a RealData object using our initiated data from above.
data = RealData(x, y)

# Set up ODR with the model and data.
odr = ODR(data, linear_model, beta0=[0., 1.])

# Run the regression.
out = odr.run()

# Use the in-built pprint method to give us results.
out.pprint()
Beta: [ 5.48224516 -4.06869541]
Beta Std Error: [0.82478086 2.46589803]
Beta Covariance: [[ 1.92505175 -4.81262945]
 [-4.81262945 17.20740925]]
Residual Variance: 0.35337412002372076
Inverse Condition #: 0.1458747661516887
Reason(s) for Halting:
  Sum of squares convergence

No comments:

Post a Comment

Rendering 3D maps on the web using opesource javascript library Cesium

This is a simple 3D viewer using the Cesium javascript library. The example code can be found here . Click on question-mark symbol on upp...