Error Handling in Python

Understanding Python Errors and Exceptions

Python is an interpreted language, which means errors can occur at two main stages: compile-time (syntax errors) and runtime (exceptions). Syntax errors are caught before your code even runs – think missing colons or mismatched parentheses. But exceptions are the real runtime beasts: they happen when your code is executing and something goes wrong, like dividing by zero or trying to open a non-existent file.

In Python, everything is an object, and exceptions are no different. They inherit from the BaseException class, with Exception being the base for most user-facing ones. When an exception occurs, Python stops normal execution and looks for a handler (like a try/except block). If none is found, the program crashes with a traceback – that scary stack of error messages showing where things went south.

Why care? Poor error handling leads to bugs that are hard to debug, security vulnerabilities (e.g., exposing sensitive data in logs), or even data corruption. Good handling makes your code resilient: it can recover from issues or fail informatively.

Let’s look at a simple example. Suppose you’re writing a function to divide two numbers:

Python

def divide(a, b):
    return a / b

result = divide(10, 0)  # Boom! ZeroDivisionError

Running this raises a ZeroDivisionError. Without handling, your program halts. That’s fine for a script, but in a web app, it could crash the server.

Exceptions can be caught, but understanding their hierarchy is key. For instance, ArithmeticError is a parent of ZeroDivisionError and OverflowError. Catching a parent catches its children too – useful for broad handling, but risky if overused.

Key takeaway: Errors aren’t enemies; they’re signals. Learn to read tracebacks – they tell you the exception type, message, and line number. Tools like pdb or IDE debuggers help step through them.

Common Built-in Exceptions

Python has dozens of built-in exceptions, but you’ll encounter a handful most often. Knowing them helps you anticipate issues and handle them specifically.

• ZeroDivisionError: As above, dividing by zero. Common in math ops.
• TypeError: Mixing incompatible types, like adding a string and int: “hello” + 5.
• ValueError: Right type, wrong value, e.g., int(“abc”).
• IndexError: Accessing a list index out of range: my_list[10] when len=5.
• KeyError: Missing dict key: my_dict[‘nonexistent’].
• FileNotFoundError: Opening a missing file: open(‘ghost.txt’).
• AttributeError: Calling a non-existent attribute: None.upper().
• ImportError: Failing to import a module.
• OSError: OS-level issues, like permission denied.
• RuntimeError: Catch-all for generic runtime problems.

There’s also KeyboardInterrupt (Ctrl+C) and SystemExit (from sys.exit()), which inherit from BaseException – avoid catching these unless you know what you’re doing.

Example: Handling a few in a file reader:

Python

def read_file(filename):
    try:
        with open(filename, 'r') as f:
            return f.read()
    except FileNotFoundError:
        print(f"File {filename} not found!")
    except PermissionError:  # Subclass of OSError
        print("Permission denied.")
    except OSError as e:  # Catch other OS errors
        print(f"OS error: {e}")

content = read_file('missing.txt')  # Triggers FileNotFoundError

Here, we catch specific exceptions first, then broader ones. This is the “specific to general” rule – prevents masking errors.

Pro tip: Use except Exception as e: sparingly; it catches almost everything, hiding bugs. Always prefer named exceptions.

try / except / else / finally Explained

The try/except block is your core tool for handling exceptions. But it has optional clauses: else and finally, which add power.

• try: Wrap code that might raise an exception.
• except: Catch and handle the exception. You can specify types: except ValueError: or multiple: except (TypeError, ValueError):. Use as e to access the exception object.
• else: Runs if no exception in try. Great for code that should only execute on success.
• finally: Always runs, exception or not. Ideal for cleanup, like closing files or connections.

Full example: A function to parse user input and compute square root.

Python

import math

def safe_sqrt(user_input):
    try:
        num = float(user_input)  # Might raise ValueError
        if num < 0:
            raise ValueError("Negative number!")  # We'll cover raising later
        result = math.sqrt(num)
    except ValueError as e:
        print(f"Invalid input: {e}")
        return None
    except TypeError as e:
        print(f"Type error: {e}")
        return None
    else:
        print("Calculation successful!")
        return result
    finally:
        print("Cleaning up...")  # Always runs

print(safe_sqrt("4"))   # Outputs: Calculation successful! Cleaning up... 2.0
print(safe_sqrt("-1"))  # Outputs: Invalid input: Negative number! Cleaning up... None
print(safe_sqrt([]))    # Outputs: Type error: float() argument must be a string or a real number, not 'list' Cleaning up... None

See how else separates success logic, and finally ensures cleanup? In real apps, finally might close a database cursor.

You can nest try blocks, but keep them shallow to avoid complexity. Also, exceptions in except or finally can propagate or be caught further up.

Remember: Don’t use try/except for control flow (e.g., checking if a key exists with except KeyError). Use if statements or dict’s get() – it’s faster and clearer.

Raising Custom Exceptions

Sometimes built-in exceptions don’t fit. That’s when you raise your own. Use raise Exception(“Message”) for simple cases, but better: define custom classes.

Custom exceptions inherit from Exception or a subclass. They can add attributes for more info.

Why custom? Clarity: raise UserNotFoundError(“ID: 123”) is better than raise ValueError(“User not found”). Also, callers can catch them specifically.

Example: A user management system.

Python

class UserNotFoundError(Exception):
    def __init__(self, user_id, message="User not found"):
        self.user_id = user_id
        self.message = message
        super().__init__(f"{message} (ID: {user_id})")

class InvalidUserDataError(ValueError):
    pass  # Inherits from ValueError for semantic fit

def get_user(user_id):
    users = {1: "Alice", 2: "Bob"}
    if user_id not in users:
        raise UserNotFoundError(user_id)
    return users[user_id]

def update_user(user_id, new_name):
    if not isinstance(new_name, str) or not new_name:
        raise InvalidUserDataError("Name must be a non-empty string")
    # Update logic...

try:
    user = get_user(3)  # Raises UserNotFoundError
except UserNotFoundError as e:
    print(f"Error: {e} for user {e.user_id}")
except InvalidUserDataError as e:
    print(f"Data error: {e}")

Here, UserNotFoundError carries extra data. In large codebases, this helps logging and debugging.

You can re-raise with raise in an except block to propagate, or use raise NewException() from e to chain exceptions (Python 3+).

Tip: Keep custom exceptions simple; don’t over-engineer. Document them in docstrings.

When to Catch Errors and When Not To

This is crucial: Not every error should be caught. Catching too broadly (e.g., except Exception:) swallows bugs, leading to “silent failures” where code continues with bad data.

When to catch:

• Recoverable errors: User input validation, network timeouts (retry).
• Expected failures: File might not exist? Handle and inform.
• Graceful degradation: In a web app, catch DB errors and show a friendly message.

When not to catch:

• Programming errors: IndexError, TypeError – these indicate bugs; let them crash for debugging.
• Critical failures: Memory exhaustion (MemoryError) – recovery is impossible.
• At top level: In scripts, let unhandled exceptions print tracebacks.

Rule of thumb: Catch only what you can handle meaningfully. If you catch and just pass or print, reconsider.

Example: Bad – broad catch:

Python

try:
    # Some code
    dict['key']  # Typo: should be my_dict
except Exception:
    pass  # Silent failure – bug hidden!

Good – specific, actionable:

Python

import requests

def fetch_data(url):
    try:
        response = requests.get(url)
        response.raise_for_status()  # Raises HTTPError for bad status
    except requests.exceptions.HTTPError as e:
        print(f"HTTP error: {e}")
        return None  # Or retry
    except requests.exceptions.RequestException as e:
        print(f"Network error: {e}")
        return None
    return response.json()

Here, we catch network issues but let other bugs (e.g., undefined variables) propagate.

In libraries, raise exceptions for invalid use; let callers handle. In apps, wrap entry points with broad handlers for logging.

Logging vs Printing Errors

Printing errors with print() is fine for scripts, but in production, use logging. Python’s logging module is built-in and powerful.

Why logging?

• Levels: DEBUG, INFO, WARNING, ERROR, CRITICAL – control verbosity.
• Configurable: Log to files, console, emails.
• Structured: Add timestamps, module names.
• Thread-safe: Important for concurrent code.

Setup basics:

Python

import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    filename='app.log'
)

logger = logging.getLogger(__name__)

def risky_function():
    try:
/ 0
    except ZeroDivisionError as e:
        logger.error("Division error occurred", exc_info=True)  # Logs traceback
        raise  # Re-raise if needed

risky_function()

This logs to app.log with timestamp, etc. exc_info=True adds the traceback.

Vs printing: print(“Error:”, e) goes to stdout, no persistence, no levels. In daemons, prints vanish.

Advanced: Use logging.exception(“Msg”) in except blocks – auto-adds exc_info.

Integrate with frameworks: Flask/Django have loggers. For distributed systems, use ELK stack or Sentry for error tracking.

Tip: Log at ERROR for exceptions, WARNING for potential issues. Avoid logging sensitive data (e.g., passwords) – use redaction.

Anti-Patterns in Error Handling

From experience, here are pitfalls to avoid:

1. Bare except:except: catches everything, including KeyboardInterrupt. Use at least except Exception:.
2. Swallowing exceptions:except: pass – hides problems. Always log or raise.
3. Overly broad catches: Catching Exception everywhere. Specific is better.
4. Using exceptions for control flow: E.g., try: dict[key] except KeyError: default. Use get(key, default) instead.
5. Nested try/excepts without need: Leads to “exception spaghetti.” Factor into functions.
6. Ignoring finally: Forget cleanup? Leaks resources like open files.
7. Printing instead of logging: As above, not scalable.
8. Not handling context managers properly:with statements handle finally for you, but custom ones need __exit__.

Example of anti-pattern:

Python

# Bad
try:
    file = open('file.txt')
    data = file.read()
except:
    print("Oops")
# No close, broad catch

Fixed:

Python

# Good
import logging

logger = logging.getLogger(__name__)

try:
    with open('file.txt') as file:  # Auto-closes
        data = file.read()
except FileNotFoundError:
    logger.warning("File not found")
except OSError as e:
    logger.error("OS error", exc_info=True)
    raise

Learn from open-source: Check how libraries like requests handle errors – they raise specific ones and document them.

Writing Fail-Safe Code

Fail-safe means your code handles failures without cascading disasters. Principles:

• Defensive programming: Validate inputs early. Use type hints (Python 3.5+), assertions.
• Resource management: Always use with for files, locks, connections.
• Retries: For transient errors (e.g., networks), use libraries like tenacity.
• Fallbacks: Provide defaults or alternatives.
• Testing errors: Write unit tests that assert exceptions are raised/handled.
• Monitoring: Integrate with tools like Prometheus for error rates.

Example: A fail-safe API caller with retry.

Python

import requests
from tenacity import retry, stop_after_attempt, wait_fixed

@retry(stop=stop_after_attempt(3), wait=wait_fixed(2))
def fetch_with_retry(url):
    try:
        response = requests.get(url, timeout=5)
        response.raise_for_status()
    except requests.exceptions.Timeout:
        raise  # Will retry
    except requests.exceptions.HTTPError as e:
        if 500 <= e.response.status_code < 600:
            raise  # Retry server errors
        else:
            raise ValueError(f"Client error: {e}")  # Don't retry
    return response.json()

# Usage
try:
    data = fetch_with_retry("https://api.example.com")
except Exception as e:  # Top-level catch
    logging.error("Failed after retries", exc_info=True)
    data = {"fallback": "data"}  # Graceful degradation

Here, @retry handles transients. We distinguish retryable errors.

For async code (asyncio), use try/async except.

In data pipelines, use transactions: Roll back on errors.

Finally, code reviews: Always check error paths. Tools like pylint flag bare excepts.

Conclusion

Error handling in Python isn’t just syntax – it’s a mindset. By understanding exceptions, using try/except wisely, raising customs when needed, and avoiding anti-patterns, you’ll write code that’s robust and safe.