One of Python's strengths is that it's approachable and easy to learrn; anyone can learn to write a basic Python program. The Python Journeyman will take you beyond those basics. We'll lead you through the deeper aspects of Python, and give you the skills you need to write great Python programs.

The Python Journeyman

Preface

In this chapter we’ll be covering more of Python’s techniques for organizing programs. Specifically we’ll be looking at Python’s concept of packages and how these can be used to add structure to your program as it grows beyond simple modules.

Organizing Larger Programs

By this point you should have plenty of experience with functions in Python. They are a fundamental unit of reuse in the language, and calling functions is one of the most common things you do in most Python code. But when it comes to calling things in Python, functions are just the tip of the iceberg! In this chapter we’ll cover a generalisation of functions known as callable objects , and we’ll explore some other types of callable objects including callable instances and lambdas .

Beyond Basic Functions

The functions we’ve looked at so far have been defined either at module scope, as a member of a class, or as anonymous “lambda” functions. Python provides us with more flexibility than that though. In this chapter we’ll look at local functions , functions which we define within the scopes of other functions. We’ll also look at the related concept of closures which are key to really understanding local functions. We’ll close off the chapter with a look at Python’s function decorators, a powerful and elegant way to augment existing functions without changing their implementation.

Closures and Decorators

In this chapter we’re going to look at a number of topics we’ve already covered — including decorators from the previous chapter — and see how we can use them to improve the design of our classes. First, though, we need to look at class scope attributes since they are the foundation for some of the techniques we’ll cover.

Properties and Class Methods

In this chapter we’ll look at string representations of objects in Python, and in particular we’ll cover the important but oft-confused differences between repr() and str() . Understanding the various string representations in Python is important for writing maintainable, debuggable, and human-friendly programs. We’ll show you what you need to know to use them properly.

Strings and Representations

In this chapter we’ll dig deeper into some of the fundamentals of numerical computing. We’ll take a look at the numeric types included in the Python language and the Python standard library, including those for dates and times. Let’s start though, by reviewing and looking in a little more detail at some of the scalar types we have already encountered.

Numeric and Scalar Types

In this chapter we’ll be taking a deeper look at iterables and iteration in Python, including topics such as more advanced comprehensions, some language and library features which support a more functional style of programming 35 , and the protocols underlying iteration. This chapter builds upon the contents of chapter 7 of “The Python Apprentice” , and, as with that previous material, you’ll find that these techniques and tools can help you write more expressive, elegant, and even beautiful code.

Iterables and Iteration

In chapter 8 of “The Python Apprentice” we covered the basics of inheritance in Python. That chapter was limited, however, to single-inheritance and the fundamental concepts of class hierarchies. In this chapter we’ll take a deeper look at Python’s support for inheritance — including multiple-inheritance — and the underlying mechanisms by which Python dispatches method calls. We’ll also look at techniques for runtime type checking for those times when it’s necessary.

Inheritance and Subtype Polymorphism

In chapter 5 of The Python Apprentice we showed how the different built-in collections can be categorised according to which protocols they support. In this chapter we’ll build on that and demonstrate how to go beyond the built-in collection types of list , tuple , dict and set by creating your own collections. We’ll build a new and fully functioning collection type of our own design. This will require us to implement a series of different protocols, each of which represents a different capability of collections. The protocols we will cover are:

Implementing Collections with Protocols

In Chapter 6 of The Python Apprentice we introduced exceptions and exception techniques. There we covered the basic concepts of exceptions and the specifics of how to handle them in Python. We also gave advice on when and how to deploy exception raising and exception handling in your code. This chapter builds directly on the foundation we established in The Python Apprentice , and here we seek to deepen our understanding of some of the tools Python provide for sophisticated error handling and reporting.

Errors and Exceptions in Depth

In Chapter 9 of The Python Apprentice we took our first look at Python’s with-blocks . We only looked at how to use them with close() -able objects, such as files, but in fact with-blocks are a powerful tool for managing all sorts of resources and object lifetimes.

Defining Context Managers

Introspection is the ability of a program to examine its own structure and state; a process of looking inward to perform self-examination. We’ve already encountered many tools which allow programs to examine themselves, both in earlier chapters of this book and in The Python Apprentice . We’ll quickly review some of those tools here to bring them under the umbrella of introspection, and then will move on to more tools in this category, and some uses for them.

Introspection

Having digested in the material in The Python Apprentice and The Python Journeyman , you have arrived at a significant point on your journey as a Python programmer. You should have the all core Python language tools at hand to tackle almost any Python development project.

Afterword: Levelling Up

This is a Python implementation of the ISO6346 standard for BIC codes.

Appendix A - Python implementation of ISO6346

See Chapter 3 in The Python Apprentice ↩ In the spirit of “batteries included”, the Python standard library include support for reading both gzip and bz2.↩ See chapter 9 of The Python Apprentice for background on files and file-like-objects.↩ A subpackage may seem like an unnecessary amount of ceremony for something as simple as this. In production code that would likely be true, but for teaching purposes it’s helpful to have simple code.↩ It turns out that leading underscores in Python are a bit more than just a convention to indicate implementation details for the benefit of humans. Only module scope ( i.e. global) attributes without any leading underscores will be imported, unless they are explicitly named.↩ Python has a flexible import system, and you are not limited to just importing from files. For example, PEP 273 describes how to Python allows you to import modules from zip-files. We cover this in detail in Chapter 13 of “The Python Master” .↩ Remember that packages are just a particular kind of module.↩ For example, the mutation testing tool Cosmic Ray is designed to apply mutations to Python modules and then run tests against them. If your test code is part of your package, Cosmic Ray will try to mutate your tests, resulting in very strange behavior indeed!↩ In fact, in the past we’ve recommended including tests as subpackages rather than outside the package. Our experience since then has convinced us otherwise.↩ The singleton pattern has its detractors, and for good reason. Singletons are often misused in a way that introduces unnecessary restrictions in situation where in multiple instances of a class could be useful. Be aware that singletons introduce shared global state into your program.↩ See Chapter 5 in The Python Master ↩ As of Python 3.6, which incorporates PEP 468, **kwargs in a function signature is guaranteed to be an insertion-order-preserving mapping. In other words, the arguments in the received dictionary will be in the same order as the arguments at the call-site.↩ See Chapter 4 in The Python Apprentice ↩ See Chapter 4 in The Python Apprentice ↩ For more details on the LEGB rule, see Chapter 4 of The Python Apprentice .↩ Remember that classes don’t introduce scopes.↩ BIC is the Bureau International des Conteneurs (International Container Bureau) ↩ In BIC codes the fourth character specifies the “equipment category”. The default category is ‘U’, but refrigerated shipping containers use an equipment category of ‘R’. We must specify this when creating the BIC code by passing an additional category argument to the iso6346.create() function. We do this in the overriden _make_bic_code() static method in the derived class.↩ For the details, see the discussion on Descriptors in chapter 4 in our book The Python Master ↩ We’re not making this up — such things do exist, and their purpose is to maintain a temperatures within a wide range of ambient conditions.↩ Short for D on’t R epeat Y ourself↩ We’ll discuss super() in some detail in chapter 8. You can find a thorough discussion of more of the details in our book The Python Master .↩ https://en.wikipedia.org/wiki/Template_method_pattern↩ As the old saying goes, “there’s no problem in computer science which can’t be solved by an additional level of indirection”.↩ “repr” is an abbreviation of “representation”.↩ reprlib is part of Python’s standard library. You can read all about it in the Python docs ↩ The section “Subclassing Repr Objects” gives an example.↩ The assertion in this code would probably be an exception in production code, but the assertion is more expressive for the purposes of this example.↩ A double-precision floating point number has 64 bits of precision; twice that of a 32-bit single-precision float. The details are mostly important for interoperability with other programming languages which use C-compatiable types such the the single-precision float and the double-precision double . One Python context where these distinctions crop up is in libraries such as Numpy for handling large arrays of floating-point numbers. This is because Numpy is implemented in C.↩ The value to which the fraction is raised.↩ Also known as the mantissa or significand.↩ Note that the type assumes a proleptic Gregorian calendar that extends backwards for all eternity and into the infinite future. For historical dates this must be used with some care. The last country to adopt the Gregorian calendar was Turkey, in 1927.↩ Both time and datetime can be used in so-called ‘naïve’ or ‘aware’ modes. In naïve mode, the values lack time zone and daylight saving time information, and their meaning with respect to other time values is purely by convention within a particular program. In other words part of the meaning of the time is implicit. On the other hand, in ‘aware’ mode these objects have knowledge of both time zone and daylight saving time and so can be located with respect to other time objects.↩ Calculating determinants is a fairly straightforward operation .↩ In imperative programming languages, such as Python, the program makes progress by executing statements. In functional programming languages the program progresses purely by evaluating expressions. Since Python can evaluate expressions too, it is possible to program in a functional style in Python.↩ In some cases you may find significant differences in speed, though. If the performance of a piece of code is critical, it might be worth timing both versions.↩ As with map , filter is a really a class.↩ You can, in fact, implement both map and filter in terms of reduce ↩ Using techniques of introspection we introduce in chapter 12, the methods can figure out the name of the class of the object on which they have been invoked. This means that even the __repr__() override can be removed, leaving us with an empty class block, save for a syntactically necessary pass statement.↩ You may also discover a method called mro() . This is called once, when a class is first created, to populate the __mro__ attribute. You shouldn’t call it yourself, preferring the __mro__ attribute. We cover this further in The Python Master when we discuss metaclasses .↩ We cover some more aspects of object in “The Python Master” .↩ The machinery of the in and not in operators contains a fallback to exhaustive search through the iterable series of items.↩ It is common to use a temporary set in this way to eliminate duplicates. Although it may seem wasteful to construct a collection just for this purpose, the underlying implementation of set in C will be much faster than anything you can write in Python.↩ The yield from syntax was introduced in↩ We won’t cover slice objects in detail here, except to point out that they can be manually constructed by calling the slice constructor which accepts integer start , stop and step arguments. These arguments can also be retrieved through attributes of the same name. You can read more about slice objects in the Python documentation ↩ See chapter 5 for all of the details on string representations.↩ This is the case for Python 3, but not for Python 2 where you must walk the extra mile to implement __ne__() manually.↩ abc is an acronym for Abstract Base Class. We cover these in detail in chapter 8 of The Python Master . For now, the mechanism is unimportant; it’s sufficient to know that the base class we’re using implements useful behavior which we can inherit.↩ is an example of Big O Notation . It is used to classify algorithms according to how their running time grows in relation to the input data size . For an algorithm, the running time is directly proportional to , so doubling the input size will double the run time. For an algorithm the running time is proportional to the logarithm of the size of the input data. For example, as rises by a factor to ten from 100 to 1000, goes only from from 2 to 3, an increase in running time of only one-half. A few minutes playing with the log() and log10() functions in the Python math module should convince you that the base of the logarithm doesn’t matter.↩ Some people call such except clauses “Pokemon excepts” because they “catch them all”.↩ From his Turing Award Lecture: “I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.”↩ Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch is the longest place name in Europe.↩ For tuples with more than one element the delimiting parentheses are optional.↩ 

Community Questions