Steven Lott is an American software developer and author. He is an associate for the bank holding company Capital One and uses Python to build APIs for new products. Previously, he worked as a solution architect for CTG, which provides IT services. In 2003, Steven started using his talent for solving problems with Python to write books. He has since authored titles including Modern Python Cookbook, Python for Secrets Agents, and Functional Python Programming. Steven creates educational content for the Python community and writes a tech blog.

Mike Driscoll: So why did you become a programmer?

Steven Lott: I started programming in the 1970s, when computers were rare. My school had two Olivetti Programma 101 calculators and an IBM 1620 computer.

It was empowering being able to create useful behavior on these machines, such as simulating random events, drawing things and trying to design new kinds of games. A responsive and autonomous device was the ultimate toy, even when doing math homework. The idea of building things that were new and useful via software was compelling. Also, I had a bunch of friends who hung around in the computer room.

Driscoll: How did you start using Python?

Lott: In the late 90s, as object-oriented programming was building momentum, I started tracking the popular languages.

I had a Macintosh with the port of Smalltalk-80, the THINK C++ compiler and a JDK 1.1. I made regular searches for emerging object-oriented programming technology and eventually found Python.

The barriers to entry for Python were so much lower than the other languages that I had learned. There was only a runtime and no complex toolchain required to build software. Python was replacing Perl, AWK, sed, and grep with one tool that handled a variety of use cases. By 2000, I was trying to build useful and working applications in Python.

Driscoll: What did you like about Python?

Lott: At first, I was drawn to the elegant simplicity of Python. The standard library provided an amazing array of tools. As I learned more, the vast ecosystem of modules and packages outside of the standard library showed me how much could be done.

I used Python at work because I could solve a problem quickly. The language was wonderful for complex data wrangling problems. In many cases, success stemmed from getting started quickly and discovering the nuances and complications of a problem as early as possible. Python encourages you to fail quickly and start again on a new course.

The more that I learn about NumPy, the more that I see Python as a kind of universal container for code. The NumPy libraries are based on C (and Fortran), so having a Python wrapper makes them widely available and useful.

The underlying reason for using Python wasn't clear to me until Guido van Rossum's keynote speech at PyCon 2016. Python's biggest strength stems from the community. Python's open source nature creates and encourages a community effort to build cool new things.

Python has numerous other strengths, such as its wide adoption as a language. Python is used in numerous contexts: scientists are using it to analyze truly gigantic datasets and it's used to build scalable web services too. Python is also used recreationally by home hackers who are integrating their Alexa, Nest, and Arduino-based temperature sensors.

Another strength of Python is sometimes called batteries included. With a single download, you have all the tools you want. If you want to learn the language, then you can start with the distribution for your computer. If you want to do data science, then you can start with the Anaconda distribution, which is where lots of packages are bundled.

The Python Software Foundation (PSF) makes active steps to be as inclusive as possible. The philosophy is that everyone should be able to learn and share their findings. Python's community believes that nobody should be excluded. We're all using Python to solve problems, so we all need help.

Driscoll: What are Python's weaknesses as a language?

Lott: I've collected a few lists of Python's weaknesses. Some of them are utterly farcical and I've seen many sentiments which make no sense at all. A few complaints about Python are meaningful.

Overall, I've learned that most problems that are blamed on the Python language being slow are more often than not due to ineffective algorithm and data structure choices.

Python's core runtime is remarkably fast. Fortran and C are considerably faster because they have optimizing compilers, that produce code focused on the underlying chipset. The SciPy and NumPy use of binary code wrapped in Python addresses this concern nicely.

Another issue is the opportunity for confusion when using Python. The orthogonality between language statements and data structures means that lists, sets, and dictionaries have some overlapping features. The immensely sophisticated implementation of Python data structures makes it possible to make a bad choice and get correct answers, but have horribly inefficient code.

Lastly, a weakness for Python is the possibility of creating inheritance problems. Everything is dynamic, so it can be difficult for tools like Pylint to discern meaningful method redefinitions from spelling mistakes with similar-looking method names and plain bad design.

The collections.abc module has decorators that can be used to organize code and provide some help with checking redefinitions. The type definitions in the typing module allow mypy to locate potential problems.

Driscoll: So how did you end up becoming an author of Python books?

Lott: Most roles in my career more or less just happened to me, but becoming a writer was a conscious decision.

In this case, I had decided that there could be value in teaching the Python language and the associated software engineering skills. I started to collect notes for a book in 2002. By 2010, I had tried self-publishing several books on Python.

When Stack Overflow started, I was an early participant. There were many interesting Python questions. The questions showed gaps where more information was needed about Python specifically and software engineering in general. Over a few years, I answered thousands of questions about Python and somehow built up a large reputation.

Driscoll: What have you learned in the writing process?

Lott: I've learned about the difficulty of creating meaningful and interesting examples. An example needs to have a story arc and a problem that requires a solution.

Stories require drama and conflict, and that doesn't often surface when thinking about data structures and algorithms. I spend more time wandering around trying to think of examples than doing any other part of the writing process. A lot of the problems that I come up with are too large and complex.

A snippet of code is difficult to describe if it doesn't solve a problem.

For example, the traveling salesman problem has a compelling story arc that characterizes graph traversal. Having a story provides a framework for remembering the essential problem and seeing how the solution works. Pure code doesn't help anyone to understand why the language construct is important. Code only exists to solve a problem, so it's imperative to describe the problem.

Creating stories requires the time to view the problem from a distance, which is essential for summarizing and abstracting out needless details. Finding the right details requires a deep understanding. I know that I've failed when the description of the code becomes long and complex, involving tangential topics.

Driscoll: What are the pros and cons of self-publishing your books versus using a regular publisher?

Lott: The difference between self-publishing and using a publisher is editing. The way that Python handles documentation testing (via the doctest module) means that the technical aspects of the content can be validated automatically. I've become better at this, but there are still some testing gaps in my published code.

Other challenges are grammar, usage, clarity, precision, color, unity, coherence, and concision. With Packt Publishing, there's a pipeline of editors who ask questions and notice the incomprehensible parts, long before my book lands in the hands of a reader.

When I self-published, I did what seemed best to me. Publishers manage costs, prices, and revenue streams adroitly. My job is to know Python and Packt Publishing handles the rest.

Driscoll: Have you learned anything from your readers? If so, what?

Lott: My readers have taught me the importance of using the Python doctest tool for checking each example in the body of a book. Readers have spotted numerous errors from code that I didn't check properly.

Driscoll: What has been your favorite interaction with a reader?

Lott: I work for a tech company in Northern Virginia. A co-worker was surprised to find out that I'd written Mastering Object-Oriented Python. They had bought the book based on recommendations and read the outline, without really looking at the author's name.

Driscoll: So which of your books has been the most popular? Why do you think that people buy one book over another?

Lott: My most successful book has been Python for Secret Agents. It seems like the fun factor is part of that. If a book has a wide variety of fun exercises and problems, then readers can see how Python applies to the problems that they know and want to solve. If the book is too narrowly focused on one problem domain, or too abstract, then the practical applications become hard to envision.

Driscoll: What new and exciting trends are you seeing in Python?

Lott: Python 3.6 is fast and getting faster. The developers working on foundational algorithms have done impressive things.

The new internal data structures for the dict save memory and run faster. This kind of internal re-engineering is exciting. There are huge benefits that come from having an upgrade with few visible changes to the language.

Another exciting direction that Python is going in is connected to the mypy project and the type hints. You have a handy quality tool that doesn't involve a profound change to the language, or the development tools. This can help you to write more reliable code, without introducing significant overheads. If mypy becomes part of Pylint or Pyflakes, then that will help even more.

As an Arduino maker, I often collect data for later analysis using Python-based tools. My current project involves a customized GPS tracker, which will be used on a boat to monitor its position while at anchor. An alarm will sound when the vessel is drifting. There are numerous other examples of Internet of things (IoT) projects, where Python is an important part of the overall effort to build something new and useful.

Driscoll: Do you see Python becoming a popular language for embedded programming now that MicroPython is becoming popular?

Lott: Yes, MicroPython and the pyboard are exciting new developments. Raspberry Pis also run Python nicely.

Processors continue to become faster and smaller, which means that more sophisticated languages can be used. One of the first computers that I ever used had 20K of memory and was the size of an upright piano. My first Apple II Plus had 64K of memory and covered the top of a desk. A pyboard has 1M of ROM and 192K of RAM in a package which is just over two square inches.

Driscoll: Thank you, Steven Lott.