Python for AI Engineers
Python is the daily driver for AI engineers. Behind every model and deployment is Python code handling data, configuration, and orchestration. This tutorial covers the subset of Python that actually matters for AI work.
Tutorial Goals
- Understand fundamental Python data structures for AI
- Use functional programming and comprehensions for data manipulation
- Structure and validate data effectively
- Parse and handle common data formats like JSON
- Perform efficient numerical operations with NumPy
- Manipulate and analyze tabular data using Pandas
Why Python?
While the heavy computation happens in C++ or CUDA, Python is where you'll spend 90% of your time as an AI engineer. Most AI bugs come from poor data handling, not algorithmic issues. Solid Python fundamentals mean faster debugging and more reliable systems.
Data Structures
Four core data structures cover ~90% of what you'll need in AI code.
Lists: Your Go-To for Sequences
You'll use lists everywhere-storing feature vectors, batch data, token sequences, or any ordered collection that needs to change.
Create with [], grow with append(), access by index, slice with start:stop:step, iterate with for.
Building feature vectors with a bias term:
# Feature vector for a data pointfeature_vector = [0.1, -0.5, 1.2, 0.9]print(f"Original features: {feature_vector}")
# Add a bias term (common in linear models)feature_vector.append(1.0)print(f"Features with bias: {feature_vector}")
# Access a specific featureprint(f"Second feature: {feature_vector[1]}")
# Get first three featuresprint(f"First three features: {feature_vector[:3]}")Exercise: Create a list containing the numbers 1 through 5. Then, append the number 6 and print the element at index 2.
Dictionaries: Configuration and Mapping
Dictionaries map names to values: model configs, hyperparameters, word embeddings. Use {} to create, dict[key] to access (or dict.get(key) for safety), and dict.items() to iterate pairs.
Every training run needs hyperparameters:
hyperparams = { "model_type": "LSTM", "learning_rate": 0.001, "epochs": 10, "batch_size": 32, "dropout": 0.2}print(f"Learning Rate: {hyperparams['learning_rate']}")print(f"Epochs: {hyperparams.get('epochs', 5)}") # Get with default
# Update a parameterhyperparams["learning_rate"] = 0.0005print(f"Updated Hyperparameters: {hyperparams}")Exercise: Create a dictionary representing a simple model's performance with keys 'accuracy' and 'loss' and corresponding float values. Print the 'loss'.
Sets: Uniqueness and Fast Lookups
Sets do two things well: remove duplicates and test membership in constant time with in — much faster than lists for lookups. Common for building vocabularies and tracking unique IDs.
Building a vocabulary from text (a standard NLP preprocessing step):
sentence = "the quick brown fox jumps over the lazy dog the"words = sentence.split()unique_words = set(words)
print(f"Original words: {words}")print(f"Unique words: {unique_words}")print(f"Is 'fox' in the unique words? {'fox' in unique_words}")Exercise: Create two sets, set1 = {1, 2, 3} and set2 = {3, 4, 5}. Find and print their intersection.
Tuples: When Things Shouldn't Change
Tuples are immutable sequences: coordinates, RGB values, return-multiple-values patterns. The immutability prevents accidental modifications and makes them usable as dictionary keys. In AI, some data represents fixed concepts (image dimensions, coordinate pairs). Tuples enforce that.
Image dimensions are a classic tuple use case:
image_dimensions = (256, 256, 3) # Width, Height, Channelswidth = image_dimensions[0]height = image_dimensions[1]channels = image_dimensions[2]
print(f"Image Width: {width}, Height: {height}, Channels: {channels}")
# Attempting to change will raise TypeError# image_dimensions[0] = 512 # This line would cause an errorExercise: Create a tuple representing a point in 3D space (x, y, z). Unpack the tuple into three separate variables x_coord, y_coord, z_coord and print them.
Functional Programming
Functional patterns show up constantly in preprocessing and feature engineering. Worth internalizing.
List Comprehensions: The Python Way
List comprehensions are faster than loops and more readable once you get used to them.
The pattern: [expression for item in iterable if condition]
Filtering model predictions above a confidence threshold:
predictions = [0.1, 0.9, 0.85, 0.3, 0.95, 0.6]threshold = 0.8high_confidence_preds = [p for p in predictions if p >= threshold]
print(f"High confidence predictions: {high_confidence_preds}")
# Example with transformationscaled_preds = [p * 100 for p in predictions]print(f"Scaled predictions (%): {scaled_preds}")Exercise: Given numbers = [1, 2, 3, 4, 5, 6], use a list comprehension to create a new list containing the squares of the even numbers only.
Lambda Functions
Lambdas are one-off functions for small operations, especially useful with sorted(), map(), and filter().
Sorting model results by score:
training_logs = [ {'epoch': 1, 'loss': 0.5, 'accuracy': 0.8}, {'epoch': 3, 'loss': 0.3, 'accuracy': 0.9}, {'epoch': 2, 'loss': 0.4, 'accuracy': 0.85},]# Sort logs by epoch numbersorted_by_epoch = sorted(training_logs, key=lambda log: log['epoch'])print(f"Sorted by Epoch: {sorted_by_epoch}")
# Sort logs by loss (descending)sorted_by_loss = sorted( training_logs, key=lambda log: log["loss"], reverse=True)print(f"Sorted by Loss (Desc): {sorted_by_loss}")Exercise: Create a lambda function that takes one argument x and returns x + 10. Call it with the value 5 and print the result.
map() & filter(): Functional Tools
List comprehensions are often cleaner, but map and filter have their place, especially when composing existing functions. Note: they return iterators, so wrap with list() if you need the results immediately.
Converting and filtering data in one pipeline:
raw_values = ["1.2", "-0.5", "3.0", "0.0", "-2.1"]float_values = map(float, raw_values) # Returns a map object (iterator)# Returns a filter object (iterator)positive_values = filter(lambda x: x > 0, float_values)
# Convert iterators to lists to see the resultsprint(f"Positive values: {list(positive_values)}")Note that float_values is consumed by filter.
If you need to reuse the values later, convert the iterator to a list first: float_values_list = list(map(float, raw_values))
Exercise: Use map and a lambda function to convert a list of temperatures in Celsius [0, 10, 20, 30] to Fahrenheit using the formula . Convert the result to a list and print it.
Data Classes
Data classes1 give you structured data without the boilerplate: automatic __init__, __repr__, and __eq__ methods, plus type hint integration. Use them for configs, experiment results, or any data with a consistent shape.
Every ML experiment needs configuration:
from dataclasses import dataclass, fieldfrom typing import Dict, List
@dataclassclass ExperimentConfig: experiment_name: str model_name: str learning_rate: float = 0.01 # Use default_factory for mutable defaults layers: List[int] = field(default_factory=lambda: [64, 32]) params: Dict[str, str] = field(default_factory=dict)
# Create an instanceconfig = ExperimentConfig( experiment_name="ResNet_FineTune", model_name="ResNet50", learning_rate=0.005, params={"optimizer": "AdamW"})print(config)print(f"Model: {config.model_name}, LR: {config.learning_rate}")
# Instance with default layersdefault_config = ExperimentConfig("DefaultRun", "SimpleCNN")print(default_config)Exercise: Create a DataPoint data class with attributes features (a list of floats) and label (an integer). Instantiate it with sample data.
Type Hinting
Type hints prevent bugs, improve autocomplete, and make code self-documenting. Essential for larger AI projects. Start with function parameters and return types — you don't need perfect coverage everywhere, but the critical paths should be typed.
Type hints in data processing functions:
from typing import List, Optional
def tokenize_text(text: str, lowercase: bool = True) -> List[str]: """Splits text into tokens.""" if lowercase: text = text.lower() tokens = text.split() return tokens
def process_batch(batch: List[str]) -> Optional[List[List[str]]]: """Tokenizes a batch of texts.""" if not batch: return None # Indicate empty batch with None processed_batch: List[List[str]] = [] for text in batch: processed_batch.append(tokenize_text(text)) return processed_batch
# Usagemy_texts = ["Hello World", "Python for AI"]tokenized_texts = process_batch(my_texts)if tokenized_texts: # Check if not None print(tokenized_texts)
empty_batch_result = process_batch([])print(f"Empty batch result: {empty_batch_result}")Exercise: Add type hints to the DataPoint data class you created in the previous exercise. Also, create a simple function add_numbers(a: int, b: int) -> int that adds two integers and returns the result, including type hints.
JSON Handling
JSON is the lingua franca of AI systems: API responses, model configs, experiment logs. The core API: json.loads() for strings to Python, json.dumps() for Python to strings. Add indent=2 for readability.
Loading model settings and saving results:
import json
# Loading configurationconfig_json_str = """{ "model_name": "BERT-base", "max_seq_length": 128, "learning_rate": 2e-5}"""config = json.loads(config_json_str)print(f"Model Name: {config['model_name']}")print(f"Max Sequence Length: {config['max_seq_length']}")
# Saving evaluation resultsresults = { "test_accuracy": 0.88, "f1_score": 0.85, "predictions": [1, 0, 1, 1, 0] # Example prediction list}results_json_str = json.dumps(results, indent=4) # Pretty print with indentprint("\nEvaluation Results (JSON):")print(results_json_str)Exercise: Create a Python dictionary with at least two key-value pairs. Convert this dictionary into a JSON string with an indentation of 2 spaces and print it.
File Handling with pathlib
pathlib replaced os.path as the clean, cross-platform way to handle file paths. The / operator for joining paths — Path('data') / 'models' / 'best.pth' — is reason enough to switch.
Organizing model outputs:
import jsonfrom pathlib import Path
# Define base directory for artifactsartifacts_dir = Path("model_artifacts")experiment_id = "exp_001"experiment_dir = artifacts_dir / experiment_id# Create ./model_artifacts/exp_001/experiment_dir.mkdir(parents=True, exist_ok=True)
# Dummy model weights and config# In reality, loaded/saved via frameworkmodel_weights = b"dummy_binary_weights_data"config = {"layers": 2, "activation": "relu"}
# Save weights (as bytes)weights_file = experiment_dir / "weights.bin"weights_file.write_bytes(model_weights)print(f"Saved weights to: {weights_file}")
# Save config (as JSON text)config_file = experiment_dir / "config.json"config_file.write_text(json.dumps(config, indent=2))print(f"Saved config to: {config_file}")
# Verify creationprint(f"Does weights file exist? {weights_file.exists()}")print(f"Does config file exist? {config_file.exists()}")Exercise: Create a directory named output. Inside output, create a file named summary.txt and write the text "Analysis complete." into it using pathlib.
NumPy & Pandas
NumPy and Pandas sit under most AI libraries.
NumPy: Fast Numerical Operations
NumPy2 gives you efficient arrays and vectorized operations, much faster than Python loops for numerical work.
Basic operations you'll use daily:
import numpy as np
a = np.array([[1, 2], [3, 4]])b = np.array([[5, 6], [7, 8]])
print("Sum:\n", a + b)print("Mean of a:", np.mean(a))Exercise: Create two 1D NumPy arrays and compute their dot product.
Pandas: Tabular Data
Pandas3 handles tabular data — loading CSVs, cleaning data, exploring datasets. The essential workflow: load with pd.read_csv(), explore with .head() and .info(), select with .loc[], clean as needed.
Working with typical ML data:
import pandas as pd
# Create sample data using a dictionarydata = { "participant_id": [101, 102, 103, 104, 105], "age": [25, 34, 29, 45, 31], "treatment_group": ["A", "B", "A", "B", "A"], "baseline_score": [85.5, 92.1, 88.0, 75.3, 89.9], "outcome_score": [88.2, 90.5, 91.0, 78.1, 93.4],}
# Create a DataFramedf = pd.DataFrame(data)
print("DataFrame Head:")print(df.head())
print("\nDataFrame Info:")df.info()
print("\nDataFrame Description (Numerical Columns):")print(df.describe())
# Select participants older than 30older_participants = df[df["age"] > 30]print("\nParticipants Older Than 30:")print(older_participants)
# Select specific columns for treatment group 'A'group_a_outcomes = df.loc[ df["treatment_group"] == "A", ["participant_id", "outcome_score"]]print("\nOutcome Scores for Group A:")print(group_a_outcomes)Exercise: Using the DataFrame created in the example above, calculate the average outcome_score for participants in treatment_group 'B'.
Next Steps
You now have the Python toolkit that powers every AI system worth shipping. Data structures for wrangling features, functional patterns for clean pipelines, type hints for code that doesn't break at 2 AM, and NumPy/Pandas for the heavy numerical lifting.
Up next: the mathematical foundations that make AI tick. You'll see how vectors become feature representations, how gradients drive learning, and how probability quantifies uncertainty. The data structures you just mastered become the building blocks for implementing actual algorithms.
You can manipulate Python data structures, write functional transformations, handle JSON and files with pathlib, and perform vectorized operations with NumPy and Pandas. You're ready for the math.