NumPy Beginner's Guide
By Ivan Idris
5/5
()
About this ebook
NumPy is an extension to, and the fundamental package for scientific computing with Python. In today's world of science and technology, it is all about speed and flexibility. When it comes to scientific computing, NumPy is on the top of the list.
NumPy Beginner's Guide will teach you about NumPy, a leading scientific computing library. NumPy replaces a lot of the functionality of Matlab and Mathematica, but in contrast to those products, is free and open source.
Write readable, efficient, and fast code, which is as close to the language of mathematics as is currently possible with the cutting edge open source NumPy software library. Learn all the ins and outs of NumPy that requires you to know basic Python only. Save thousands of dollars on expensive software, while keeping all the flexibility and power of your favourite programming language.You will learn about installing and using NumPy and related concepts. At the end of the book we will explore some related scientific computing projects. This book will give you a solid foundation in NumPy arrays and universal functions. Through examples, you will also learn about plotting with Matplotlib and the related SciPy project. NumPy Beginner's Guide will help you be productive with NumPy and have you writing clean and fast code in no time at all.
ApproachThe book is written in beginner's guide style with each aspect of NumPy demonstrated with real world examples and required screenshots.
Who this book is forIf you are a programmer, scientist, or engineer who has basic Python knowledge and would like to be able to do numerical computations with Python, this book is for you. No prior knowledge of NumPy is required.
Ivan Idris
Ivan Idris has an MSc in Experimental Physics. His graduation thesis had a strong emphasis on Applied Computer Science. After graduating, he worked for several companies as a Java Developer, Data warehouse Developer, and QA Analyst. His main professional interests are Business Intelligence, Big Data, and Cloud Computing. Ivan Idris enjoys writing clean, testable code and interesting technical articles. Ivan Idris is the author of NumPy 1.5 Beginner's Guide and NumPy Cookbook by Packt Publishing. You can find more information and a blog with a few NumPy examples at ivanidris.net.
Read more from Ivan Idris
Python Data Analysis Rating: 4 out of 5 stars4/5Python Data Analysis Cookbook Rating: 5 out of 5 stars5/5NumPy Cookbook Rating: 5 out of 5 stars5/5NumPy: Beginner's Guide - Third Edition Rating: 4 out of 5 stars4/5Learning NumPy Array Rating: 0 out of 5 stars0 ratings
Related to NumPy Beginner's Guide
Related ebooks
Modern Python Cookbook Rating: 5 out of 5 stars5/5Python: Real World Machine Learning Rating: 0 out of 5 stars0 ratingsMastering Python Rating: 0 out of 5 stars0 ratingsBuilding Machine Learning Systems with Python Rating: 4 out of 5 stars4/5Scientific Computing with Python 3 Rating: 0 out of 5 stars0 ratingsPython Parallel Programming Cookbook Rating: 5 out of 5 stars5/5Learning Data Mining with Python Rating: 0 out of 5 stars0 ratingsPython Machine Learning Cookbook Rating: 0 out of 5 stars0 ratingsPython Machine Learning By Example Rating: 4 out of 5 stars4/5Python for Finance Rating: 3 out of 5 stars3/5Python: Journey from Novice to Expert Rating: 5 out of 5 stars5/5Numpy Simply In Depth Rating: 5 out of 5 stars5/5Learning Python Application Development Rating: 0 out of 5 stars0 ratingsPython for Finance - Second Edition Rating: 2 out of 5 stars2/5Deep Learning with Python, Second Edition Rating: 0 out of 5 stars0 ratingsPractical Machine Learning Cookbook Rating: 0 out of 5 stars0 ratingsMastering Python for Finance Rating: 5 out of 5 stars5/5Python for Secret Agents Rating: 0 out of 5 stars0 ratingsMachine Learning with R Rating: 4 out of 5 stars4/5Machine Learning with R - Third Edition: Expert techniques for predictive modeling, 3rd Edition Rating: 0 out of 5 stars0 ratingsmatplotlib Plotting Cookbook Rating: 5 out of 5 stars5/5Python Data Visualization Cookbook Rating: 4 out of 5 stars4/5Web Scraping with Python Rating: 4 out of 5 stars4/5Mastering Objectoriented Python Rating: 5 out of 5 stars5/5NumPy Essentials Rating: 0 out of 5 stars0 ratingsLearning SciPy for Numerical and Scientific Computing - Second Edition Rating: 0 out of 5 stars0 ratingsLearning pandas Rating: 4 out of 5 stars4/5Python GUI Programming Cookbook - Second Edition Rating: 5 out of 5 stars5/5Learning pandas - Second Edition Rating: 4 out of 5 stars4/5
Programming For You
Learn to Code. Get a Job. The Ultimate Guide to Learning and Getting Hired as a Developer. Rating: 5 out of 5 stars5/5Grokking Algorithms: An illustrated guide for programmers and other curious people Rating: 4 out of 5 stars4/5Coding All-in-One For Dummies Rating: 4 out of 5 stars4/5SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL Rating: 4 out of 5 stars4/5Python Programming : How to Code Python Fast In Just 24 Hours With 7 Simple Steps Rating: 4 out of 5 stars4/5Excel : The Ultimate Comprehensive Step-By-Step Guide to the Basics of Excel Programming: 1 Rating: 5 out of 5 stars5/5SQL Guide for Microsoft Access: SQL Basics, Fundamental & Queries Exercise Rating: 5 out of 5 stars5/5Python QuickStart Guide: The Simplified Beginner's Guide to Python Programming Using Hands-On Projects and Real-World Applications Rating: 0 out of 5 stars0 ratingsPython: For Beginners A Crash Course Guide To Learn Python in 1 Week Rating: 4 out of 5 stars4/5Hacking: Ultimate Beginner's Guide for Computer Hacking in 2018 and Beyond: Hacking in 2018, #1 Rating: 4 out of 5 stars4/5Python Projects for Beginners: A Ten-Week Bootcamp Approach to Python Programming Rating: 0 out of 5 stars0 ratingsHTML & CSS: Learn the Fundaments in 7 Days Rating: 4 out of 5 stars4/5Learn JavaScript in 24 Hours Rating: 3 out of 5 stars3/5Web Designer's Idea Book, Volume 4: Inspiration from the Best Web Design Trends, Themes and Styles Rating: 4 out of 5 stars4/5Learn SQL in 24 Hours Rating: 5 out of 5 stars5/5SQL All-in-One For Dummies Rating: 3 out of 5 stars3/5PYTHON: Practical Python Programming For Beginners & Experts With Hands-on Project Rating: 5 out of 5 stars5/5Learn PowerShell in a Month of Lunches, Fourth Edition: Covers Windows, Linux, and macOS Rating: 0 out of 5 stars0 ratingsProgramming Arduino: Getting Started with Sketches Rating: 4 out of 5 stars4/5Python: Learn Python in 24 Hours Rating: 4 out of 5 stars4/5OneNote: The Ultimate Guide on How to Use Microsoft OneNote for Getting Things Done Rating: 1 out of 5 stars1/5
Reviews for NumPy Beginner's Guide
3 ratings0 reviews
Book preview
NumPy Beginner's Guide - Ivan Idris
Table of Contents
Numpy Beginner's Guide Second Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers and more
Why Subscribe?
Free Access for Packt account holders
Preface
What is NumPy?
History
Why use NumPy?
Limitations of NumPy
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. NumPy Quick Start
Python
Time for action – installing Python on different operating systems
What just happened?
Windows
Time for action – installing NumPy, Matplotlib, SciPy, and IPython on Windows
What just happened?
Linux
Time for action – installing NumPy, Matplotlib, SciPy, and IPython on Linux
What just happened?
Mac OS X
Time for action – installing NumPy, Matplotlib, and SciPy on Mac OS X
What just happened?
Time for action – installing NumPy, SciPy, Matplotlib, and IPython with MacPorts or Fink
What just happened?
Building from source
Arrays
Time for action – adding vectors
What just happened?
Pop quiz Functioning of the arange function
Have a go hero – continue the analysis
IPython—an interactive shell
Online resources and help
Summary
2. Beginning with NumPy Fundamentals
NumPy array object
Time for action – creating a multidimensional array
What just happened?
Pop quiz – the shape of ndarray
Have a go hero – create a three-by-three matrix
Selecting elements
NumPy numerical types
Data type objects
Character codes
dtype constructors
dtype attributes
Time for action – creating a record data type
What just happened?
One-dimensional slicing and indexing
Time for action – slicing and indexing multidimensional arrays
What just happened?
Time for action – manipulating array shapes
What just happened?
Stacking
Time for action – stacking arrays
What just happened?
Splitting
Time for action – splitting arrays
What just happened?
Array attributes
Time for action – converting arrays
What just happened?
Summary
3. Get in Terms with Commonly Used Functions
File I/O
Time for action – reading and writing files
What just happened?
CSV files
Time for action – loading from CSV files
What just happened?
Volume-weighted average price
Time for action – calculating volume-weighted average price
What just happened?
The mean function
Time-weighted average price
Pop quiz – computing the weighted average
Have a go hero – calculating other averages
Value range
Time for action – finding highest and lowest values
What just happened?
Statistics
Time for action – doing simple statistics
What just happened?
Stock returns
Time for action – analyzing stock returns
What just happened?
Dates
Time for action – dealing with dates
What just happened?
Have a go hero – looking at VWAP and TWAP
Weekly summary
Time for action – summarizing data
What just happened?
Have a go hero – improving the code
Average true range
Time for action – calculating the average true range
What just happened?
Have a go hero – taking the minimum function for a spin
Simple moving average
Time for action – computing the simple moving average
What just happened?
Exponential moving average
Time for action – calculating the exponential moving average
What just happened?
Bollinger bands
Time for action – enveloping with Bollinger bands
What just happened?
Have a go hero – switching to exponential moving average
Linear model
Time for action – predicting price with a linear model
What just happened?
Trend lines
Time for action – drawing trend lines
What just happened?
Methods of ndarray
Time for action – clipping and compressing arrays
What just happened?
Factorial
Time for action – calculating the factorial
What just happened?
Summary
4. Convenience Functions for Your Convenience
Correlation
Time for action – trading correlated pairs
What just happened?
Pop quiz – calculating covariance
Polynomials
Time for action – fitting to polynomials
What just happened?
Have a go hero – improving the fit
On-balance volume
Time for action – balancing volume
What just happened?
Simulation
Time for action – avoiding loops with vectorize
What just happened?
Have a go hero – analyzing consecutive wins and losses
Smoothing
Time for action – smoothing with the hanning function
What just happened?
Have a go hero – smoothing variations
Summary
5. Working with Matrices and ufuncs
Matrices
Time for action – creating matrices
What just happened?
Creating a matrix from other matrices
Time for action – creating a matrix from other matrices
What just happened?
Pop quiz – defining a matrix with a string
Universal functions
Time for action – creating universal function
What just happened?
Universal function methods
Time for action – applying the ufunc methods on add
What just happened?
Arithmetic functions
Time for action – dividing arrays
What just happened?
Have a go hero – experimenting with __future__.division
Modulo operation
Time for action – computing the modulo
What just happened?
Fibonacci numbers
Time for action – computing Fibonacci numbers
What just happened?
Have a go hero – timing the calculations
Lissajous curves
Time for action – drawing Lissajous curves
What just happened?
Square waves
Time for action – drawing a square wave
What just happened?
Have a go hero – getting rid of the loop
Sawtooth and triangle waves
Time for action – drawing sawtooth and triangle waves
What just happened?
Have a go hero – getting rid of the loop
Bitwise and comparison functions
Time for action – twiddling bits
What just happened?
Summary
6. Move Further with NumPy Modules
Linear algebra
Time for action – inverting matrices
What just happened?
Pop quiz – creating a matrix
Have a go hero – inverting your own matrix
Solving linear systems
Time for action – solving a linear system
What just happened?
Finding eigenvalues and eigenvectors
Time for action – determining eigenvalues and eigenvectors
What just happened?
Singular value decomposition
Time for action – decomposing a matrix
What just happened?
Pseudoinverse
Time for action – computing the pseudo inverse of a matrix
What just happened?
Determinants
Time for action – calculating the determinant of a matrix
What just happened?
Fast Fourier transform
Time for action – calculating the Fourier transform
What just happened?
Shifting
Time for action – shifting frequencies
What just happened?
Random numbers
Time for action – gambling with the binomial
What just happened?
Hypergeometric distribution
Time for action – simulating a game show
What just happened?
Continuous distributions
Time for action – drawing a normal distribution
What just happened?
Lognormal distribution
Time for action – drawing the lognormal distribution
What just happened?
Summary
7. Peeking into Special Routines
Sorting
Time for action – sorting lexically
What just happened?
Have a go hero – trying a different sort order
Complex numbers
Time for action – sorting complex numbers
What just happened?
Pop quiz – generating random numbers
Searching
Time for action – using searchsorted
What just happened?
Array elements' extraction
Time for action – extracting elements from an array
What just happened?
Financial functions
Time for action – determining future value
What just happened?
Present value
Time for action – getting the present value
What just happened?
Net present value
Time for action – calculating the net present value
What just happened?
Internal rate of return
Time for action – determining the internal rate of return
What just happened?
Periodic payments
Time for action – calculating the periodic payments
What just happened?
Number of payments
Time for action – determining the number of periodic payments
What just happened?
Interest rate
Time for action – figuring out the rate
What just happened?
Window functions
Time for action – plotting the Bartlett window
What just happened?
Blackman window
Time for action – smoothing stock prices with the Blackman window
What just happened?
Hamming window
Time for action – plotting the Hamming window
What just happened?
Kaiser window
Time for action – plotting the Kaiser window
What just happened?
Special mathematical functions
Time for action – plotting the modified Bessel function
What just happened?
sinc
Time for action – plotting the sinc function
What just happened?
Summary
8. Assure Quality with Testing
Assert functions
Time for action – asserting almost equal
What just happened?
Pop quiz – specifying decimal precision
Approximately equal arrays
Time for action – asserting approximately equal
What just happened?
Almost equal arrays
Time for action – asserting arrays almost equal
What just happened?
Have a go hero – comparing array with different shapes
Equal arrays
Time for action – comparing arrays
What just happened?
Ordering arrays
Time for action – checking the array order
What just happened?
Objects comparison
Time for action – comparing objects
What just happened?
String comparison
Time for action – comparing strings
What just happened?
Floating point comparisons
Time for action – comparing with assert_array_almost_equal_nulp
What just happened?
Comparison of floats with more ULPs
Time for action – comparing using maxulp of 2
What just happened?
Unit tests
Time for action – writing a unit test
What just happened?
Nose tests decorators
Time for action – decorating tests
What just happened?
Docstrings
Time for action – executing doctests
What just happened?
Summary
9. Plotting with Matplotlib
Simple plots
Time for action – plotting a polynomial function
What just happened?
Pop quiz – the plot function
Plot format string
Time for action – plotting a polynomial and its derivative
What just happened?
Subplots
Time for action – plotting a polynomial and its derivatives
What just happened?
Finance
Time for action – plotting a year’s worth of stock quotes
What just happened?
Histograms
Time for action – charting stock price distributions
What just happened?
Have a go hero – drawing a bell curve
Logarithmic plots
Time for action – plotting stock volume
What just happened?
Scatter plots
Time for action – plotting price and volume returns with scatter plot
What just happened?
Fill between
Time for action – shading plot regions based on a condition
What just happened?
Legend and annotations
Time for action – using legend and annotations
What just happened?
Three dimensional plots
Time for action – plotting in three dimensions
What just happened?
Contour plots
Time for action – drawing a filled contour plot
What just happened?
Animation
Time for action – animating plots
What just happened?
Summary
10. When NumPy is Not Enough – SciPy and Beyond
MATLAB and Octave
Time for action – saving and loading a .mat file
What just happened?
Pop quiz – loading .mat files
Statistics
Time for action – analyzing random values
What just happened?
Have a go hero – improving the data generation
Samples’ comparison and SciKits
Time for action – comparing stock log returns
What just happened?
Signal processing
Time for action – detecting a trend in QQQ
What just happened?
Fourier analysis
Time for action – filtering a detrended signal
What just happened?
Mathematical optimization
Time for action – fitting to a sine
What just happened?
Numerical integration
Time for action – calculating the Gaussian integral
What just happened?
Interpolation
Time for action – interpolating in one dimension
What just happened?
Image processing
Time for action – manipulating Lena
What just happened?
Audio processing
Time for action – replaying audio clips
What just happened?
Summary
11. Playing with Pygame
Pygame
Time for action – installing Pygame
Hello World
Time for action – creating a simple game
What just happened?
Animation
Time for action – animating objects with NumPy and Pygame
What just happened?
Matplotlib
Time for action – using Matplotlib in Pygame
What just happened?
Surface pixels
Time for action – accessing surface pixel data with NumPy
What just happened?
Artificial intelligence
Time for action – clustering points
What just happened?
OpenGL and Pygame
Time for action – drawing the Sierpinski gasket
What just happened?
Simulation game with PyGame
Time for action – simulating life
What just happened?
Summary
A. Pop Quiz Answers
Chapter 1, NumPy Quick Start
Chapter 2, Beginning with NumPy Fundamentals
Chapter 3, Get into Terms with Commonly Used Functions
Chapter 4, Convenience functions for your convenience
Chapter 5, Working with Matrices and ufuncs
Chapter 6, Move further with NumPy modules
Chapter 7, Peeking into special routines
Chapter 8, Assure Quality with Testing
Chapter 9, Plotting with Matplotlib
Chapter 10, When NumPy is not enough Scipy and beyond
Index
NumPy Beginner's Guide Second Edition
Numpy Beginner's Guide Second Edition
Copyright © 2013 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: November 2011
Second edition: April 2013
Production Reference: 1170413
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78216-608-5
www.packtpub.com
Cover Image by Suresh Mogre (<suresh.mogre.99@gmail.com>)
Credits
Author
Ivan Idris
Reviewers
Jaidev Deshpande
Dr. Alexandre Devert
Mark Livingstone
Miklós Prisznyák
Nikolay Karelin
Acquisition Editor
Usha Iyer
Lead Technical Editor
Joel Noronha
Technical Editors
Soumya Kanti
Devdutt Kulkarni
Project Coordinator
Abhishek Kori
Proofreader
Mario Cecere
Indexer
Hemangini Bari
Graphics
Sheetal Aute
Ronak Dhruv
Production Coordinator
Melwyn D'sa
Cover Work
Melwyn D'sa
About the Author
Ivan Idris has an MSc in Experimental Physics. His graduation thesis had a strong emphasis on Applied Computer Science. After graduating, he worked for several companies as a Java Developer, Datawarehouse Developer, and QA Analyst. His main professional interests are Business Intelligence, Big Data, and Cloud Computing. Ivan Idris enjoys writing clean testable code and interesting technical articles. Ivan Idris is the author of NumPy Beginner's Guide & Cookbook. You can find more information and a blog with a few NumPy examples at ivanidris.net.
I would like to take this opportunity to thank the reviewers and the team at Packt Publishing for making this book possible. Also thanks goes to my teachers, professors, and colleagues who taught me about science and programming. Last but not the least, I would like to acknowledge my parents, family, and friends for their support.
About the Reviewers
Jaidev Deshpande is an intern at Enthought, Inc, where he works on software for data analysis and visualization. He is an avid scientific programmer and works on many open source packages in signal processing, data analysis, and machine learning.
Dr. Alexandre Devert is teaching data-mining and software engineering at the University of Science and Technology of China. Alexandre also works as a researcher, both as an academic on optimization problems, and on data-mining problems for a biotechnology startup. In all those contexts, Alexandre very happily uses Python, Numpy, and Scipy.
Mark Livingstone started his career by working for many years for three international computer companies (which no longer exist) in engineering/support/programming/training roles, but got tired of being made redundant. He then graduated from Griffith University on the Gold Coast, Australia, in 2011 with a Bachelor of Information Technology. He is currently in his final semester of his B.InfoTech (Hons) degree researching in the area of Proteomics algorithms with all his research software written in Python on a Mac, and his Supervisor and research group one by one discovering the joys of Python.
Mark enjoys mentoring first year students with special needs, is the Chair of the IEEE Griffith University Gold Coast Student Branch, and volunteers as a Qualified Justice of the Peace at the local District Courthouse, has been a Credit Union Director, and will have completed 100 blood donations by the end of 2013.
In his copious spare time, he co-develops the S2 Salstat Statistics Package available at http://code.google.com/p/salstat-statistics-package-2/ which is multiplatform and uses wxPython, NumPy, SciPy, Scikit, Matplotlib, and a number of other Python modules.
Miklós Prisznyák is a senior software engineer with a scientific background. He graduated as a physicist from the Eötvös Lóránd University, the largest and oldest university in Hungary. He did his MSc thesis on Monte Carlo simulations of non-Abelian lattice quantum field theories in 1992. Having worked three years in the Central Research Institute for Physics of Hungary, he joined MultiRáció Kft. in Budapest, a company founded by physicists, which specialized in mathematical data analysis and forecasting economic data. His main project was the Small Area Unemployment Statistics System which has been in official use at the Hungarian Public Employment Service since then. He learned about the Python programming language here in 2000. He set up his own consulting company in 2002 and then he worked on various projects for insurance, pharmacy and e-commerce companies, using Python whenever he could. He also worked in a European Union research institute in Italy, testing and enhanching a distributed, Python-based Zope/Plone web application. He moved to Great Britain in 2007 and first he worked at a Scottish start-up, using Twisted Python, then in the aerospace industry in England using, among others, the PyQt windowing toolkit, the Enthought application framework, and the NumPy and SciPy libraries. He returned to Hungary in 2012 and he rejoined MultiRáció where now he is working on a Python extension module to OpenOffice/EuroOffice, using NumPy and SciPy again, which will allow users to solve non-linear and stochastic optimization problems. Miklós likes to travel, read, and he is interested in sciences, linguistics, history, politics, the board game of go, and in quite a few other topics. Besides he always enjoys a good cup of coffee. However, nothing beats spending time with his brilliant 10 year old son Zsombor for him.
Nikolay Karelin holds a PhD degree in optics and used various methods of numerical simulations and analysis for nearly 20 years, first in academia and then in the industry (simulation of fiber optics communication links). After initial learning curve with Python and NumPy, these excellent tools became his main choice for almost all numerical analysis and scripting, since past five years.
I wish to thank my family for understanding and keeping patience during long evenings when I was working on reviews for the NumPy Beginner’s Guide.
www.PacktPub.com
Support files, eBooks, discount offers and more
You might want to visit www.PacktPub.com for support files and downloads related to your book.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers