Modern Python Application Development in Practice!

Write a simple Python program and execute it.

• Write a simple program

• Modify our simple program

• Verify user output

Expand upon the first program, and introduce a subtle bug.

• Make our program more complicated

• Verify the output

• Demonstrate error condition

Demonstrate the Python debugger, and PyCharm interface to the debugger mode.

• Run our program in the debug mode.

• Pause execution, explore the program state and discover the error

• Verify that the problem is resolved

Learn how to print justified text and how to format numbers into strings that you define, such as currency, hexadecimal, binary, and scientific notation.

• Learn how to substitute a variable into a string

• Justification and alignment of strings

• Format numbers in a variety of ways

Break a string into parts, using custom delimiters, likewise re-join a series of strings back into a single string.

Learn how to normalize a string and test it against various conditions.

• Learn how to break a string apart into useful pieces to work with and then re-join it

• Normalize a string

• Test various simple conditions

Learn advance string matching with regular expressions in Python.

• Learn simple regular expression pattern

• Discuss about the capture groups

• Learn how to do advanced string replacement with re.sub

Learn how to print datetime objects with user specified formats and manipulate them.

Learn various ways to allow users to input dates.

• Create a datetime for distinct dates in various ways

• Format dates when printing or displaying them

• Learn how to offset dates, and learn about the calendar module

Learn what about checksum and two of the primary uses of checksums.

• Understand a checksum

• Learn about using simple checksums to avoid transcription errors

• Explore cryptographic checksums and their security applications

Reading and writing from files.

• Open a file for writing with a context manager

• Read from a file, as a whole and in chunks

• File encoding and decoding from Unicode to bytes and back

Help the user understand for loops and how to iterate over a list of items.

• Initialize all 3 datatype containers (dict/list/set)

• Iterate over a list of objects adding them to our datatypes

• Understand some of the key differences between the 3 datatypes

Learn how to replace for loops with list comprehensions, as well as how to map items using a list comprehension.

• Convert our for loop into a list comprehension

• Learn to convert for loop into a set comprehension

• Convert our loop into a dictionary comprehension

Learn how to exclude unwanted values in our list comprehensions.

• Learn how to use a list comprehension as a map

• Exclude values within a list comprehension

• Explore Big-O complexities of search, as well as additional resources

List comprehensions, while some are not a silver bullet and still present with some problems that we need to keep in mind.

• Implement a list comprehension that has a long execution time

• Observe the behavior of the software

• Understand some key problems with user experience that list comprehensions can exacerbate

Learn how to use generators to solve some of the problems with list comprehensions, and use generators to help preserve system resources.

• Convert our list comprehension into a generator expression, to preserve computational time

• Learn how to use a generator expression to enumerate future dates

• Learn how to use generators to preserve system space

Explore some of the functions that the itertools library provides in order to count things forever.

• Use itertools to count forever and cycle through a list forever

• Take advantage of these methods to calculate simple probabilities

• Understand the various combinatorics functions of itertools

Learn what a function is and when to think about using them.

• Understand a function and it’s use

• Learn how to pass arguments into a function

• Return a value from a function

Learn about anonymous or unnamed functions, and some common scenarios to use them in, as well as some pitfalls you might encounter, and how to circumvent them.

• Explore anonymous functions and why we might use them

• Analyze potential pitfalls you might encounter with anonymous functions

• Learn ways around the pitfalls that you might encounter

Learn about function decorators, that is functions that modify other functions, and create several example decorators.

• Learn about functions as a first class citizen, create a simple decorator

• Create a memorization decorator

• Create a profile decorator

Learn how to document your code, with type annotations as well as docstrings, both for yourself, and for others.

• Understand the need to document our code

• Explore various levels of code documentation

• Write documentation that also tests out code

Learn about classes and modules by creating a simple blackjack game.

• Learn about classes and modules

• Create some classes and modules

• Play blackjack

This video gives an overview of this section.

In this video, we look at a demo of the final product after the three sections, including a templated Django response and browsing a Django REST Framework API.

• Look at HTML response

• Look at REST API server

In this video, we will install Python, Django and the Django REST Framework onto our development computer

• Install Python 3 based on the Operating System of our computer. Create a Virual Environment

• Use Pip to install Django and the Django REST framework

• Use pip requirements.txt to identify installed python components in a development environment

In this video, we will install PostgreSQL, create a new database and configure Python support

• Install PostgreSQL into our development environment

• Use Psql to check our installation and create a new database

• Use pip to install PostgreSQL support into Python

In this video, we will create a new Django project and then create an application within that project.

• Create a project directory and use django-admin startproject

• Use manage.py startapp to create a Django application

• Review Django external application publishing guidelines

In this video, we will use the development environment we have setup in previous videos to show the Django initial success web page.

• Introduce settings.py and configuring our database

• Highlight the DEBUG flag and placing sensitive information in settings.py

• Use Django’s development webserver via the manage.py runserver command and seeing the Django initial default page

This video gives an overview of MVC and MVT Framework.

• Look at the concept of MVC and MVT Framework.

• Look at Django models

• Look at Django template

In this video, we will discuss Django models, create our first model, create methods on our models and test them.

• Introduce Django model and how relationships between Models can be used.

• Create a Django model

• Create methods on Django models and test them

In this video, we will apply migrations to our Django models, which keep our database structure up to date. We will perform queries on our database.

• Consider what Django migrations are and have Django create schema migrations

• Consider what data migrations are and create one, and then apply all migrations created so far in this video

• Perform some simple queries against our database

In this video, we will create a simple “Hello World” view, consider Django class based views, perform URL configuration and add a parameter to our view.

• Discuss Django class based views and use view to create a simple view that returns a hard coded response

• Perform URL configuration or routing at both the Project and App level to make our new view accessible. Access the view in a browser

• Make use of URL querystring and kwarg parameters. Discuss the pros and cons of each approach

In this video, we will discuss how Django handles requests, Django’s URL routing, matching URLs with path and re_path and HTTP methods.

• Discuss how Django handles requests from the request in to middleware, then URL config, then view and then the reverse for the response

• Review URL configuration from previous video and discuss use of path() and re_path()

• Cover methods (verbs) of Http Requests including Get and Post requests. How Http methods then translate to class methods

In this video, we will cover Django Templates and the TemplateView. We will cover Django’s template inheritance system, how to pass data to templates, where to place template files and template tags and filters.

• Explain how hardcoded responses are undesirable. Introduce Django’s TemplateView

• Learn about using base templates and view specific templates that extend base templates. Learn how to make data available within templates.
  Review template settings in settings.py and see our first set of templates in action

• Explain how templates should not perform core logic, however we can use template tags and filters to perform presentation logic

In this video we will consider why we should test, the core tools Django provides for testing, how to configure PostgreSQL for testing and some integration and unit tests.

• Learn how we need to use automated testing tools and how we can write test cases that place our code under test

• Make use of pqsl to configure PostgreSQL for testing

• Develop Integration tests that make use of the Django test client.
  Develop unit tests that test specific view functions using a RequestFactory as well as class attributes

This video gives an overview of the entire section and then considers what is an API, what is REST, REST resources, methods and CRUD operations and HTTP Status codes.

• Explain the difference between traditional web responses and API responses. Learn how REST provides an API on HTTP

• Learn what Resources, HTTP Methods and CRUD operations are

• Explain how a result of an API request is communicated via a status code

In this video, we will create an API end point by configuring a URL endpoint for a view that generates JSON output. We will also accept input and perform simple validation.

• Make use of a URL pattern configuration to give access to our API view

• Define a View using the base View class that uses JsonResponse to return an API response

• Include in our View definition validation of a name parameter and return an error if this is equal to fred. Show results in a browser

In this video, we will discuss why we use the DRF, what key parts of the DRF are, who uses it and the web browsable API feature of the DRF.

• Learn how the DRF helps us in providing code that we would otherwise have to provide ourselves many times over

• Learn what the key parts of the DRF are, including serializers, DRF views, routers and authentication and permissioning. Learn how these relate to CRUD operations

• Explore who uses the DRF and see the web browsable feature of the DRF

In this video, we will learn about DRF serializers. We will first look at manually defining a serializer and then use ModelSerializer to base a definition on an existing model.

• Review our existing models with focus on the Bookmark Model, then we will define a serializer with create and update methods

• Using the Python console, we will create some initial data in the database and then demonstrate what our serializers do with this data along with simulated received data

• We will define a serializer based on Bookmark using a ModelSerializer.
  We will then demonstrate how this serializer can be used with a list of Bookmarks in the Python console

In this video, we will combine our learnings from the previous two videos with DRF views by building a Bookmarks list and a Bookmarks detail view.

• Learn about basic DRF view URL configuration, including format suffixes.
  Define a Bookmark List view that can build a list and create new Bookmarks

• Learn about basic DRF detail view URL configuration. Define a Bookmark detail view that can get detail, update a Bookmark and delete a Bookmark

• Watch a demonstration of all of the functionality previously developed in this video, including listing existing Bookmarks, creating new Bookmarks, updating existing Bookmarks and deleting them.

In this video, we will replace our views from the previous video by exploiting pre-built functionality of the DRF for common CRUD operations.

• Make use of ListCreate-APIView for Bookmark listing and new Bookmark creation

• Make use of RetriveUpdate-APIView for Bookmark detail queries, updating a Bookmark and deleting a Bookmark

• Take advantage of the ModelViewset that combines all the functionality seen earlier in this video in one convenient package.

In this video we will use DRF routers to clean up our URL configuration and create a specialised API action.

• Learn about Hyperlinked-Model-Serializers and how they can create related links in our API. Significantly simplify our URL configuration with a DRF router

• Create a specialised viewset action that can add a like to a Bookmark. Learn how to define this in the viewset and ensure the DRF router is aware of it. See how it is accessed from the browser

• Consider each of the view strategies covered in this and previous videos and scenarios where each would be the best fit

In this video we will consider how we can use DRF automated testing tools to perform integration and unit tests of our API.

• Learn how to write an integration test of a DRF based viewset and place different actions under test

• Learn how to write a unit test of specific methods of a DRF based viewset, how to isolate dependencies via patching and using mocks to test each operation of the code

• See how the Django test runner is used in combination with DRF test tools

Find the most common causes of performance issues in Python applications.

• Distinguish CPU bound tasks and I/O bound tasks

• Use the time Unix command to diagnose performance issues

• Use the cProfile Python module for advanced profiling

Take an existing I/O intensive Python application and speed it up.

• Understand concurrent I/O task execution in Python

• Use a ThreadPollExecutor to parallelize I/O tasks

• Use the time command to check the speed increase

Take an existing CPU intensive Python application and speed it up.

• Understand concurrent CPU task execution in Python

• Use a multiprocessing Pool to parallelize CPU tasks

• Use the time command to check the speed increase

Take an existing CPU intensive Python application and speed it up using a C/C++ library.

• Create a Python extension that calls C/C++ code

• Parse Python arguments in C/C++ and return C/C++ objects to Python

• Use setuptools to build the extension

Speed up a Python application by using an alternative Python implementation.

• Choose an appropriate Python implementation

• Replace C/C++ extensions in your code with plain Python implementations

• Use PyPy to run the refactored application

Prepare your Python module to be distributed to the Python Package index.

• Write the necessary files for setuptools distribution

• Configure module using the setup.py file

• Choose a license for your code

Distribute your Python module to the Python Package Index in 2 popular formats.

• Create and inspect a source distribution from your Python module

• Create and inspect a wheel distribution from your Python module

• Use twine to upload your distributions to the Python Package Index

Distribute a user-facing Python application to end-users in a friendly way.

• Use PyInstaller to generate executables for a command-line application

• Use PyInstaller to generate executables for a GUI application

• Inspect resulting executables

Distribute a web application using Docker images and containers.

• Docker definitions

• Write a Dockerfile for a simple Flask application

• Build a Docker image and run a container

Ensure your code works as expected by writing unit tests.

• Identify the testable units in your code

• Write unit tests using PyTest

• Calculate the coverage to understand what is missing from your tests

In this video, you’ll deal with the complex part of the testing and also include the coverage part.

• Create complex part of the unit testing

• Measure the amount of code which is tested

Ensure that the code you are writing is working correctly right from the start.

• Translate requirements into unit test cases

• Implement the code after writing the tests

• Change the code until the tests are passed

Ensure that your code integrates properly with external units and that any external changes will be caught early.

• Write tests for functionality not units

• Run the tests in production-like environments

• Check test results regularly

Ensure functionality requirements are met throughout the development lifecycle.

• Define features using the Gherkin syntax

• Implement the scenarios using pytest-bdd

• Build the code until the requirements are met

Install application dependencies separately from each other and from the system environment.

• Identify the components of a typical Python installation

• Create separate Python environments with virtualenv

• Use the virtualenvs to isolate application dependencies

Recreate an entire Python environment to use on a different machine.

• Install dependencies in a virtualenv

• Freeze dependencies in a requirements.txt file

• Use pip to recreate the environment based on requirements.txt

Debug Python applications with built-in tools.

• Run your application using pdb

• Use pdb commands to control the execution

• Use the dis module to inspect Python bytecode

Quickly prototype ideas in Python and share interactive prototypes with other developers.

• Use the Python shell to research a new library

• Use Jupyter to create an interactive prototype

• Use Jupyter to visualize data plots

Find stylistic issues in your Python code.

• Navigate PEP 8 to understand the common guidelines

• Use pylint to check your code for style issues

• Customize pylint messages to fit your needs

Check your code for potential bugs before running it and import all reports in one place.

• Activate SonarQube Python rules

• Use sonar-scanner to check for style issues and potential bugs

• Add unit test coverage and results in SonarQube

Check Python types before running the code. Remove run-time type checking.

• Identify the need for type checking in Python

• Add type hints to your code

• Use mypy to check the types before running

This video gives glimpse of the entire course.

Time is of essence and timeit is your tool. In this video we explore timeit.

• Understand what is timeit

• Set up our timing experiment

• Run our timing experiment

In this video, we will look at profile module and understand where time is spent.

• Understand what is profiling

• Understand how Profiling our Fibonacci sequence generator is done

• Profile the sequence generator using the least recently used cache

In this video we will explore and understand tips and tricks which will allow us to track time more precisely using cProfile.

• Understand the difference between cProfile vs profile

• Explore and create artificially slowed functions

• Understand profiling using cProfile

In this video we will understand the memory_profiler.

• Explore how to go about profiling a simple function

• Explore and understand how profiling manual garbage collection is done

This video is all about reducing the execution time and memory consumption.

• Understand what are slots

• Create classes that use slots

• Inspect the performance of slots vs class dicts

In this video we will use tuples instead of lists when your data does not change.

• Understand what are tuples and how to declare them

• We will see by example that tuples are fasters than lists

• We will understand why are they faster than lists

Save on memory consumption with generators instead of lists.

• What are generators?

• Why are they memory efficient?

• Implementing generators

Generators are designed for a specific purpose, here is when not to use them.

• when you need features from a list

• when you need to iterate through a list multiple times

Chain together a series of processes so one output is another's output.

• Composing two generators

• Using parenthesis to define generator comprehensions

Show how we can use NumPy and the different ways we can create NumPy arrays

• The difference between NumPy arrays and lists

• Creating NumPy arrays from Python lists

• Creating NumPy arrays from NumPy helpers

Show that for large n, NumPy arrays are much faster than list comprehensions

• Setting up our problem with list comprehensions

• Setting up its cousin in NumPy

• Comparing their performance

Show how to use broadcasting - applying binary functions on arrays of varying length

• What is NumPy's broadcasting?

• Broadcasting: an example

• How broadcasting works - three examples

Show the simple interface of numexpr and where we can use it to optimize

• What is numexpr?

• How to use numexpr?

• Use cases of numexpr

If you have a list of URLs, serially requesting the HTML body of each page is super slow

• Using requests to scrape the HTML of a web URL

• Timing 10 web scraping requests

What are coroutines and how they help you request all webpages at once

• What is gevent?

• What are semaphores?

• Creating a gevent scraper

What is event driven concurrency, and how that can speed up web scraping

• What is tornado?

• What is an IO Loop?

• Creating a tornado based scraper

Futures and asyncio driven concurrent web scraping

• What is asyncio?

• await and async

• Creating an asyncio based scraper

Introduce the audience to parallel programming concepts.

• What is parallel processing?

• Embarrassingly parallel and less embarrassingly parallel

• The global interpreter lock of Python

Show how the decorate-sort-undecorate pattern can be beaten

• The decorate-sort-undecorate pattern

• Beating the DSU pattern with a sort key

Show how we can use Pool to achieve multiprocessing

• What is Pool?

• Our first process Pool

• Asynchronous process Pools

In this video we will understand how we can stop processes from interfering with each other using locks.

• We will be using multiprocessing.Value to share variables

• Why does it not add up?

• Using a lock to remove overwrites

This video covers the effects of many processes.

• Using the logging module

• Creating the right logger format for multiprocessing

• Logging multi-processes

This video is focused on understanding how to copy and clone objects the right way.

• Identify the wrong object instance

• Use the correct object clone

This video will show us how to mini-classes of Python: namedtuples

• Learn about mini-classes of Python

In this video we will understand how to create smart values with static methods and properties.

• Create smart values with static methods and properties

In this video we will be understanding how to compare two different objects.

• Understanding how to compare two different objects