How to handle exceptions in Python - About Software Development and real life ..

Basics of Exception handling in Python

One of the most important pinpoints in production is not to return 500 Internal Server Error to customers/users. In order to avoid uncontrolled server errors, we can catch and handle errors in production.

Basic idea of handling exceptions:

# This code snippet shows basic idea how to catch exceptions
# For more information check: https://docs.python.org/3.8/library/exceptions.html
try:
    ...  # some code here might raise an exception
except Exception as e:  # catching exceptions
    print("Exception occurred:", repr(e))

In order to catch exceptions, we need to put our code snippet, which might throw an exception, into try ... except block. In the example shown above, we are catching exceptions of class Exception.

Why not to catch BaseException and Exception

Let's take a look into following example:

# This code snippet explains how Exception hierarchy works
# For more information check: https://docs.python.org/3.8/library/exceptions.html
try:
    raise Exception("My custom exception")
except Exception as e:  # catching Exception is not the best idea
    print("Exception occurred:", repr(e))
except BaseException as e:  # catching BaseException is even worse,
    print("BaseException occurred:", repr(e))

In the example above we are trying to catch exceptions of the classes Exception and BaseException. Usually it's not a really good idea to catch Exception, as we will catch a very broad range of exceptions. It's better to be more specific and then handle specific exceptions based on the class of exception. Catching BaseException is even worse, as we will catch all exceptions of class Exception plus some more, amongst which there is a KeyboardInterrupt exception. In our example above we will see this output:

> Exception occurred: Exception('My custom exception')

I highly recommend to take a closer look into a hierarchy of exceptions from official documentation:

BaseException
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- GeneratorExit
 +-- Exception
      +-- StopIteration
      +-- StopAsyncIteration
      +-- ArithmeticError
      |    +-- FloatingPointError
      |    +-- OverflowError
      |    +-- ZeroDivisionError
      +-- AssertionError
      +-- AttributeError
      +-- BufferError
      +-- EOFError
      +-- ImportError
      |    +-- ModuleNotFoundError
      +-- LookupError
      |    +-- IndexError
      |    +-- KeyError
      +-- MemoryError
      +-- NameError
      |    +-- UnboundLocalError
      +-- OSError
      |    +-- BlockingIOError
      |    +-- ChildProcessError
      |    +-- ConnectionError
      |    |    +-- BrokenPipeError
      |    |    +-- ConnectionAbortedError
      |    |    +-- ConnectionRefusedError
      |    |    +-- ConnectionResetError
      |    +-- FileExistsError
      |    +-- FileNotFoundError
      |    +-- InterruptedError
      |    +-- IsADirectoryError
      |    +-- NotADirectoryError
      |    +-- PermissionError
      |    +-- ProcessLookupError
      |    +-- TimeoutError
      +-- ReferenceError
      +-- RuntimeError
      |    +-- NotImplementedError
      |    +-- RecursionError
      +-- SyntaxError
      |    +-- IndentationError
      |         +-- TabError
      +-- SystemError
      +-- TypeError
      +-- ValueError
      |    +-- UnicodeError
      |         +-- UnicodeDecodeError
      |         +-- UnicodeEncodeError
      |         +-- UnicodeTranslateError
      +-- Warning
           +-- DeprecationWarning
           +-- PendingDeprecationWarning
           +-- RuntimeWarning
           +-- SyntaxWarning
           +-- UserWarning
           +-- FutureWarning
           +-- ImportWarning
           +-- UnicodeWarning
           +-- BytesWarning
           +-- ResourceWarning

In this hierrarchy we can see that Exception is derived from BaseException class. And a few more exceptions are derived from BaseException class:

SystemExit
KeyboardInterrupt
GeneratorExit
Exception

It means that if we will try to catch BaseException, we might catch any of these 4 exceptions.

Run this code snippet and try to interrupt it with keyboard interrupt:

# This code snippet explains how Exception hierarchy works
try:
    input()  # try to interrupt it and check which exception will be called
except Exception as e:
    print("Exception occurred:", repr(e))
except BaseException as e:
    # KeyboardInterrupt inherits from BaseException so as to not be accidentally caught by code that catches Exception
    # and thus prevent the interpreter from exiting. -> from official Python documentation
    print("BaseException occurred:", repr(e))

Guessed already which message will be printed? Right, this one:

> BaseException occurred: KeyboardInterrupt()

How to handle exceptions properly?

What happens if you run this code snippet?

# This code snippet explains why we should not raise a new exception
# while trying to handle already one exception
try:
    raise Exception("Something custom happened!!!")
except Exception as e:
    print("Printing exception", repr(e))

    # if we raise a new exception here, we will get a message that
    # while handling our exception another exception occurred
    raise Exception("I want my custom message!!!")

In the code snippet above we are trying to catch and handle exceptions of class Exception. But while trying to handle occurred exception we are raising a new one. As a result, we will get output:

> Traceback (most recent call last):
>   File "my_awesome_code.py", line 4, in <module>
>     raise Exception("Something custom happened!!!")
> Exception: Something custom happened!!!
>
> During handling of the above exception, another exception occurred:

> Traceback (most recent call last):
>   File "my_awesome_code.py", line 10, in <module>
>     raise Exception("I want my custom message!!!")
> Exception: I want my custom message!!!
> Printing exception Exception('Something custom happened!!!')
>
> Process finished with exit code 1

Hence the question, how can we raise an exception with some custom message without raising a new exception? Answer to this question is pretty simple - we can use already caught exception:

# This code snippet explains why we should not raise a new exception
# while trying to handle already one exception
try:
    raise Exception("Something custom happened!!!")
except Exception as e:
    print("Printing exception", repr(e))

    # if we need to rise a new exception, we can use this trick,
    # as a result we will get a message: "The above exception was the direct cause of the following exception"
    raise Exception("I want my custom message!!!") from e

As a result, we will get the following message:

Traceback (most recent call last):
  File "my_awesome_code.py", line 4, in <module>
    raise Exception("Something custom happened!!!")
Exception: Something custom happened!!!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "my_awesome_code.py", line 10, in <module>
    raise Exception("I want my custom message!!!") from e
Exception: I want my custom message!!!
Printing exception Exception('Something custom happened!!!')

Custom exceptions

The best way to catch exceptions is to define your custom exceptions and then focus on catching these:

class MyCustomException(Exception):
    pass


try:
    raise MyCustomException("Something custom happened!!!")
except MyCustomException as e:  # always a good idea to be more specific in catching exceptions
    print("We are handling this exception here!", repr(e))

For more examples, visit official documentation

Used sources and books to read

Official documentation
"Learning Python" by Mark Lutz
"Python Tricks" by Dan Bader
Talk "Exceptional Exceptions" by Mario Corchero
Photo by John Torcasio on Unsplash