📅 Today’s Picks

PydanticAI: Control OpenAI Data Retention with openai_store

Problem

By default, OpenAI may retain your API request data for internal review and model improvement. For healthcare, finance, and legal applications, this default creates compliance risks you can’t afford.

Solution

PydanticAI provides the openai_store setting to explicitly disable data retention in one line.

📖 View Full Article

DuckDB - read an entire CSV folder in one line

Problem

Working with multiple CSV files often means writing extra code to load, combine, and process each file.

Solution

DuckDB makes this simpler by letting you query multiple CSV files directly with SQL and pattern matching, so you can analyze them in one efficient query.

📖 View Full Article

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

A quick personal update: I welcomed my baby boy last week, and both mom and baby are doing well.

I’ll be taking some time to enjoy this special chapter with my family, but I’ve prepared plenty of content in advance, so you can still expect the same high-quality posts and newsletters during my maternity leave.

Introduction

pandas has been the standard tool for working with tabular data in Python for over a decade. But as datasets grow larger and performance requirements increase, two modern alternatives have emerged: Polars, a DataFrame library written in Rust, and DuckDB, an embedded SQL database optimized for analytics.

Each tool excels in different scenarios:

Tool    Backend   Execution Model             Best For
pandas  C/Python  Eager, single-threaded      Small datasets, prototyping, ML integration
Polars  Rust      Lazy/Eager, multi-threaded  Large-scale analytics, data pipelines
DuckDB  C++       SQL-first, multi-threaded   SQL workflows, embedded analytics, file queries

This guide compares all three tools with practical examples, helping you choose the right one for your workflow.

This is a condensed version focused on the performance benchmarks. For the full guide with side-by-side syntax comparisons, interoperability examples, and a decision matrix, see the complete comparison.

Tool Strengths at a Glance

pandas

pandas is the original DataFrame library for Python that excels at interactive data exploration and integrates seamlessly with the ML ecosystem. Key capabilities include:

• Direct compatibility with scikit-learn, statsmodels, and visualization libraries

• Rich ecosystem of extensions (pandas-profiling, pandasql, etc.)

• Mature time series functionality

• Familiar syntax that most data scientists already know

Polars

Polars is a Rust-powered DataFrame library designed for speed that brings multi-threaded execution and query optimization to Python. Key capabilities include:

• Speeds up operations by using all available CPU cores by default

• Builds a query plan first, then executes only what’s needed

• Streaming mode for processing datasets larger than RAM

• Expressive method chaining with a pandas-like API

DuckDB

DuckDB is an embedded SQL database optimized for analytics that brings database-level query optimization to local files. Key capabilities include:

• Native SQL syntax with full analytical query support

• Queries CSV, Parquet, and JSON files directly without loading

• Uses disk storage automatically when data exceeds available memory

• Zero-configuration embedded database requiring no server setup

Setup

Install all three libraries:

pip install pandas polars duckdb

Generate sample data for benchmarking:

import pandas as pd
import numpy as np

np.random.seed(42)
n_rows = 5_000_000

data = {
    "category": np.random.choice(["Electronics", "Clothing", "Food", "Books"], size=n_rows),
    "region": np.random.choice(["North", "South", "East", "West"], size=n_rows),
    "amount": np.random.rand(n_rows) * 1000,
    "quantity": np.random.randint(1, 100, size=n_rows),
}

df_pandas = pd.DataFrame(data)
df_pandas.to_csv("sales_data.csv", index=False)
print(f"Created sales_data.csv with {n_rows:,} rows")

Created sales_data.csv with 5,000,000 rows

Data Loading Performance

pandas reads CSV files on a single CPU core. Polars and DuckDB use multi-threaded execution, distributing the work across all available cores to read different parts of the file simultaneously.

pandas

Single-threaded CSV parsing loads data sequentially.

┌─────────────────────────────────────────────┐
│ CPU Core 1                                  │
│ ┌─────────────────────────────────────────┐ │
│ │ Chunk 1 → Chunk 2 → Chunk 3 → ... → End │ │
│ └─────────────────────────────────────────┘ │
│ CPU Core 2  [idle]                          │
│ CPU Core 3  [idle]                          │
│ CPU Core 4  [idle]                          │
└─────────────────────────────────────────────┘

pandas_time = %timeit -o pd.read_csv("sales_data.csv")

1.05 s ± 26.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Polars

Multi-threaded parsing distributes file reading across all available cores.

┌─────────────────────────────────────────────┐
│ CPU Core 1  ┌────────────────┐              │
│             │ ████████████   │              │
│ CPU Core 2  ┌────────────────┐              │
│             │ ████████████   │              │
│ CPU Core 3  ┌────────────────┐              │
│             │ ████████████   │              │
│ CPU Core 4  ┌────────────────┐              │
│             │ ████████████   │              │
└─────────────────────────────────────────────┘

import polars as pl

polars_time = %timeit -o pl.read_csv("sales_data.csv")

137 ms ± 34 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

DuckDB

Similar to Polars, file reading is distributed across all available cores.

┌─────────────────────────────────────────────┐
│ CPU Core 1  ┌────────────────┐              │
│             │ ████████████   │              │
│ CPU Core 2  ┌────────────────┐              │
│             │ ████████████   │              │
│ CPU Core 3  ┌────────────────┐              │
│             │ ████████████   │              │
│ CPU Core 4  ┌────────────────┐              │
│             │ ████████████   │              │
└─────────────────────────────────────────────┘

import duckdb

duckdb_time = %timeit -o duckdb.sql("SELECT * FROM 'sales_data.csv'").df()

762 ms ± 77.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

print(f"Polars is {pandas_time.average / polars_time.average:.1f}× faster than pandas")
print(f"DuckDB is {pandas_time.average / duckdb_time.average:.1f}× faster than pandas")

Polars is 7.7× faster than pandas
DuckDB is 1.4× faster than pandas

While Polars leads with a 7.7× speedup in CSV reading, DuckDB’s 1.4× improvement shows parsing isn’t its focus. DuckDB shines when querying files directly or running complex analytical queries.

Query Optimization

pandas: No Optimization

pandas executes operations eagerly, creating intermediate DataFrames at each step. This wastes memory and prevents optimization.

┌─────────────────────────────────────────────────────────────┐
│ Step 1: Load ALL rows          → 10M rows in memory         │
│ Step 2: Filter (amount > 100)  → 5M rows in memory          │
│ Step 3: GroupBy                → New DataFrame              │
│ Step 4: Mean                   → Final result               │
└─────────────────────────────────────────────────────────────┘
Memory: ████████████████████████████████ (high - stores all intermediates)

def pandas_query():
    return (
        pd.read_csv("sales_data.csv")
        .query('amount > 100')
        .groupby('category')['amount']
        .mean()
    )

pandas_opt_time = %timeit -o pandas_query()

1.46 s ± 88.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

This approach has three problems:

• Full CSV load: All rows are read before filtering

• No predicate pushdown: Rows are filtered after loading the entire file into memory

• No projection pushdown: All columns are loaded, even unused ones

You can manually add usecols to load fewer columns:

def pandas_query_optimized():
    return (
        pd.read_csv("sales_data.csv", usecols=["category", "amount"])
        .query('amount > 100')
        .groupby('category')['amount']
        .mean()
    )

pandas_usecols_time = %timeit -o pandas_query_optimized()

1.06 s ± 48.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

This is faster, but has two drawbacks:

• Manual tracking: You must specify columns yourself; change the query, update usecols

• No row filtering: All rows still load before the filter applies

Polars and DuckDB handle both automatically by analyzing your query before execution.

Polars: Lazy Evaluation

Polars supports lazy evaluation, which builds a query plan and optimizes it before execution:

┌─────────────────────────────────────────────────────────────┐
│ Query Plan Built:                                           │
│   scan_csv → filter → group_by → agg                        │
│                                                             │
│ Optimizations Applied:                                      │
│   • Predicate pushdown (filter during scan)                 │
│   • Projection pushdown (read only needed columns)          │
│   • Multi-threaded execution (parallel across CPU cores)    │
└─────────────────────────────────────────────────────────────┘
Memory: ████████ (low - no intermediate DataFrames)

query_pl = (
    pl.scan_csv("sales_data.csv")
    .filter(pl.col("amount") > 100)
    .group_by("category")
    .agg(pl.col("amount").mean().alias("avg_amount"))
)

# View the optimized query plan
print(query_pl.explain())

AGGREGATE[maintain_order: false]
  [col("amount").mean().alias("avg_amount")] BY [col("category")]
  FROM
  Csv SCAN [sales_data.csv] [id: 4687118704]
  PROJECT 2/4 COLUMNS
  SELECTION: [(col("amount")) > (100.0)]

The query plan shows these optimizations:

• Predicate pushdown: SELECTION filters during scan, not after loading

• Projection pushdown: PROJECT 2/4 COLUMNS reads only what’s needed

• Operation reordering: Aggregate runs on filtered data, not the full dataset

Execute the optimized query:

def polars_query():
    return (
        pl.scan_csv("sales_data.csv")
        .filter(pl.col("amount") > 100)
        .group_by("category")
        .agg(pl.col("amount").mean().alias("avg_amount"))
        .collect()
    )

polars_opt_time = %timeit -o polars_query()

148 ms ± 32.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

DuckDB: SQL Optimizer

DuckDB’s SQL optimizer applies similar optimizations automatically:

┌─────────────────────────────────────────────────────────────┐
│ Query Plan Built:                                           │
│   SQL → Parser → Optimizer → Execution Plan                 │
│                                                             │
│ Optimizations Applied:                                      │
│   • Predicate pushdown (WHERE during scan)                  │
│   • Projection pushdown (SELECT only needed columns)        │
│   • Vectorized execution (process 1024 rows per batch)      │
└─────────────────────────────────────────────────────────────┘
Memory: ████████ (low - streaming execution)

def duckdb_query():
    return duckdb.sql("""
        SELECT category, AVG(amount) as avg_amount
        FROM 'sales_data.csv'
        WHERE amount > 100
        GROUP BY category
    """).df()

duckdb_opt_time = %timeit -o duckdb_query()

245 ms ± 12.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Let’s compare the performance of the optimized queries:

print(f"Polars is {pandas_opt_time.average / polars_opt_time.average:.1f}× faster than pandas")
print(f"DuckDB is {pandas_opt_time.average / duckdb_opt_time.average:.1f}× faster than pandas")

Polars is 9.9× faster than pandas
DuckDB is 6.0× faster than pandas

Polars outperforms DuckDB (9.9× vs 6.0×) in this benchmark because its Rust-based engine handles the filter-then-aggregate pattern efficiently. DuckDB’s strength lies in complex SQL queries with joins and subqueries.

Memory Efficiency

pandas: Full Memory Load

pandas loads the entire dataset into RAM:

┌─────────────────────────────────────────────────────────────┐
│  RAM                                                        │
│  ┌────────────────────────────────────────────────────────┐ │
│  │████████████████████████████████████████████████████████│ │
│  │██████████████████ ALL 10M ROWS ████████████████████████│ │
│  │████████████████████████████████████████████████████████│ │
│  └────────────────────────────────────────────────────────┘ │
│  Usage: 707,495 KB (entire dataset in memory)               │
└─────────────────────────────────────────────────────────────┘

df_pd_mem = pd.read_csv("sales_data.csv")
pandas_mem = df_pd_mem.memory_usage(deep=True).sum() / 1e3
print(f"pandas memory usage: {pandas_mem:,.0f} KB")

pandas memory usage: 707,495 KB

For larger-than-RAM datasets, pandas throws an out-of-memory error.

Polars: Streaming Mode

Polars can process data in streaming mode, handling chunks without loading everything:

┌─────────────────────────────────────────────────────────────┐
│  RAM                                                        │
│  ┌────────────────────────────────────────────────────────┐ │
│  │█                                                       │ │
│  │                    (result only)                       │ │
│  │                                                        │ │
│  └────────────────────────────────────────────────────────┘ │
│  Usage: 0.06 KB (streams chunks, keeps only result)         │
└─────────────────────────────────────────────────────────────┘

result_pl_stream = (
    pl.scan_csv("sales_data.csv")
    .group_by("category")
    .agg(pl.col("amount").mean())
    .collect(streaming=True)
)

polars_mem = result_pl_stream.estimated_size() / 1e3
print(f"Polars result memory: {polars_mem:.2f} KB")

Polars result memory: 0.06 KB

For larger-than-RAM files, use sink_parquet instead of collect(). It writes results directly to disk as chunks are processed, never holding the full dataset in memory:

(
    pl.scan_csv("sales_data.csv")
    .filter(pl.col("amount") > 500)
    .sink_parquet("filtered_sales.parquet")
)

DuckDB: Automatic Spill-to-Disk

DuckDB automatically writes intermediate results to temporary files when data exceeds available RAM:

┌─────────────────────────────────────────────────────────────┐
│  RAM                              Disk (if needed)          │
│  ┌──────────────────────────┐     ┌──────────────────────┐  │
│  │█                         │     │░░░░░░░░░░░░░░░░░░░░░░│  │
│  │     (up to 500MB)        │  →  │    (overflow here)   │  │
│  │                          │     │                      │  │
│  └──────────────────────────┘     └──────────────────────┘  │
│  Usage: 0.42 KB (spills to disk when RAM full)              │
└─────────────────────────────────────────────────────────────┘

# Configure memory limit and temp directory
duckdb.sql("SET memory_limit = '500MB'")
duckdb.sql("SET temp_directory = '/tmp/duckdb_temp'")

# DuckDB handles larger-than-RAM automatically
result_duckdb_mem = duckdb.sql("""
    SELECT category, AVG(amount) as avg_amount
    FROM 'sales_data.csv'
    GROUP BY category
""").df()

duckdb_mem = result_duckdb_mem.memory_usage(deep=True).sum() / 1e3
print(f"DuckDB result memory: {duckdb_mem:.2f} KB")

DuckDB result memory: 0.42 KB

DuckDB’s out-of-core processing makes it ideal for embedded analytics where memory is limited.

print(f"pandas: {pandas_mem:,.0f} KB (full dataset)")
print(f"Polars: {polars_mem:.2f} KB (result only)")
print(f"DuckDB: {duckdb_mem:.2f} KB (result only)")
print(f"\nPolars uses {pandas_mem / polars_mem:,.0f}× less memory than pandas")
print(f"DuckDB uses {pandas_mem / duckdb_mem:,.0f}× less memory than pandas")

pandas: 707,495 KB (full dataset)
Polars: 0.06 KB (result only)
DuckDB: 0.42 KB (result only)

Polars uses 11,791,583× less memory than pandas
DuckDB uses 1,684,512× less memory than pandas

The million-fold reduction comes from streaming: Polars and DuckDB process data in chunks and only keep the 4-row result in memory, while pandas must hold all 10 million rows to compute the same aggregation.

Final Thoughts

If your code is all written in pandas, you don’t need to rewrite it all. You can migrate where it matters:

• Profile first: Find which pandas operations are slow

• Replace with Polars: CSV reads, groupbys, and joins see the biggest gains

• Add DuckDB: When SQL is cleaner than chained DataFrame operations

Keep pandas for final ML steps. Convert with df.to_pandas() when needed.

For side-by-side syntax comparisons, interoperability examples, and a full decision matrix with recommendations by data size and workflow, see the complete comparison.

Originally published on CodeCut.

📅 Today’s Picks

dotenvx - Separate Dev, Staging, and Prod Configs Safely

Problem

A single .env file is fine for a small project.

Once you add dev, staging, production, and CI, those settings need to be separated because each environment connects to different resources and should not share the same secrets.

Solution

dotenvx makes each environment explicit: load .env.production, .env.staging, or .env.ci directly with -f.

Because each file can be encrypted, you can commit the config safely without sharing the private keys across environments.

Ideogram 4 - Run Open-Weight Image Generation Locally

Problem

Most image generation tools are hosted behind an API. That works for quick experiments, but it limits control over cost, infrastructure, privacy, and customization.

Solution

Ideogram 4 changes that with an open-weight release. You can download the 9.3B parameter model files and run them on your own machine or server.

Key features:

• Strong text rendering inside images, with a 0.97 score on an English text-accuracy benchmark

• Structured JSON prompts instead of only free-form sentences

• Control over subject, style, lighting, typography, and color palettes by hex code

• Bounding-box coordinates for placing visual elements exactly where you want them

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

Have you ever copied a table from a PDF into a spreadsheet only to find the formatting completely broken? These issues include cells shifting, values landing in the wrong columns, and merged headers losing their structure.

While doing research, I came across three Python tools for extracting tables from PDFs: Docling, Marker, and LlamaParse. To compare them fairly, I ran each tool on the same difficult table and evaluated the results.

In this article, I’ll walk through what I found on one dense numeric table that exposes how each tool handles tight layouts. For the full comparison across simple, medium, and hard tables, see the long version: PDF Table Extraction: Docling vs Marker vs LlamaParse Compared.

The Test Document

All examples use the same PDF: the Docling Technical Report from arXiv:

source = "https://arxiv.org/pdf/2408.09869"

Some tools require a local file path instead of a URL, so let’s download the PDF first:

import urllib.request

# Download PDF locally (used by Marker later)
local_pdf = "docling_report.pdf"
urllib.request.urlretrieve(source, local_pdf)

Docling: Vision-Language Model Pipeline

Docling is IBM’s open-source document converter built specifically for structured extraction. It ships with two pipelines:

• Default pipeline uses two small AI models trained specifically for tables. One spots tables on the page, the other reads the grid inside

• VLM pipeline uses one larger AI model that can understand images, similar to how ChatGPT can describe a photo. It reads the whole page and outputs the table structure directly

The default pipeline is fast, but it can struggle with complex layouts like multi-level headers and merged cells. The VLM pipeline trades some speed for better accuracy on tricky tables, which is what we want for this comparison.

We’ll use GraniteDocling, IBM’s vision model built specifically for documents.

The result is a pandas DataFrame for each table, ready for analysis.

For Docling’s full document processing capabilities beyond tables, including chunking and RAG integration, see Transform Any PDF into Searchable AI Data with Docling.

To install Docling, pick the variant that matches your hardware:

Platform                       Install command                     Model spec
Apple Silicon (M1+)            pip install "docling[vlm]" mlx-vlm  GRANITEDOCLING_MLX
Linux / Windows (CUDA or CPU)  pip install "docling[vlm]"          GRANITEDOCLING_TRANSFORMERS

This article uses docling v2.93.0.

Table Extraction

To use the VLM pipeline, we configure DocumentConverter with VlmPipeline and select GraniteDocling as the model:

from docling.datamodel import vlm_model_specs
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import VlmPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline

pipeline_options = VlmPipelineOptions(
    vlm_options=vlm_model_specs.GRANITEDOCLING_MLX,           # Apple Silicon
    # vlm_options=vlm_model_specs.GRANITEDOCLING_TRANSFORMERS,  # Linux / Windows
)

converter = DocumentConverter(
    format_options={
        InputFormat.PDF: PdfFormatOption(
            pipeline_cls=VlmPipeline,
            pipeline_options=pipeline_options,
        )
    }
)

Now we can convert the PDF and measure how long it takes:

%%time
result = converter.convert(source)

Wall time: 1min 50s

Here’s the original table from the PDF:

And here’s what Docling extracted:

# Export the table as a DataFrame
table = result.document.tables[1]
df = table.export_to_dataframe(doc=result.document)
df

The VLM pipeline struggled badly with this denser table.

Worked:

• The 12 row labels (Caption, Footnote, ..., Title, All) match the original

Didn’t work:

• Column headers are hallucinated: the original has “MRCNN R50”, “MRCNN R101”, “FRCNN R101”, “YOLO v5x6”, but the VLM output shows “R-CNN”, “R-CNN10-FPRN 3x”, “V1S”, “V2S”

• Numeric values don’t match the original. The Footnote row reads “70.1 70.1 70.1 70.1 70.1” instead of “83-91 70.9 71.8 73.7 77.2”

• Column 4 shows “64.4” repeating across 7 consecutive rows

This happens because the VLM writes cells one at a time, similar to how ChatGPT writes a response word by word. When the table has many similar-looking numbers, the model can get stuck and keep repeating the same value, which is why “64.4” appears 7 times in a row.

Conclusion: Docling’s VLM pipeline produces unreliable results on dense numeric data, where it can hallucinate column names, repeat values across rows, and lose track of merged cells.

Performance

Docling took about 1 minute 50 seconds for the full 6-page PDF on an Apple M5 Pro (64 GB RAM). Most of that time is spent on the GPU: GraniteDocling reads each page as an image and generates the table structure one token at a time, which pins the GPU at near-full utilization.

Marker: Vision Transformer Pipeline

Marker is an open-source PDF-to-Markdown converter built on the Surya layout engine. Unlike Docling’s two-stage pipeline, Marker runs five stages for table extraction:

• Layout detection: a Vision Transformer identifies table regions on each page

• OCR error detection: flags misrecognized text

• Bounding box detection: locates individual cell boundaries

• Table recognition: reconstructs row/column structure from detected cells

• Text recognition: extracts text from all detected regions

To install Marker, run:

pip install marker-pdf

This article uses marker v1.10.2.

Table Extraction

Marker provides a dedicated TableConverter that extracts only tables from a document, returning them as Markdown:

from marker.converters.table import TableConverter
from marker.models import create_model_dict
from marker.output import text_from_rendered

models = create_model_dict()
converter = TableConverter(artifact_dict=models)

Convert the PDF and measure how long it takes:

%%time
rendered = converter(local_pdf)
table_md, _, images = text_from_rendered(rendered)

Wall time: 47.1 s

Since TableConverter returns all tables as a single Markdown string, we split them on blank lines:

tables = table_md.strip().split("\n\n")

Here’s the original table from the PDF:

And here’s what Marker extracted:

print(tables[1])

Marker struggled with this denser table.

Worked:

• All 12 row labels are preserved (Caption, Footnote, ..., Title, All)

• Values for the FRCNN R101 and YOLO v5x6 columns extracted correctly

Didn’t work:

• Header parents merged: “human” and “MRCNN” share a column header, “FRCNN” and “YOLO” merged into one cell

• The human, MRCNN R50, and MRCNN R101 values are packed into one cell per row (e.g., “84-89 68.4 71.5”), leaving the MRCNN columns empty

• The Section-header row label merged with its data (”Section-header 83-84 67.6 69.3”), breaking that row’s alignment

Conclusion: Marker needs visible cell borders. When the same data appears redrawn with borders later in the PDF, Marker extracts it perfectly. See the long version for that result.

Performance

Marker took about 47 seconds for the full 6-page PDF on an Apple M5 Pro (64 GB RAM), more than twice as fast as Docling’s VLM pipeline. The speed difference comes down to architecture:

• Docling runs a single large vision-language model that reads each page as an image and generates the table structure one token at a time. Large models take time per token, so the total runtime adds up.

• Marker runs a 5-stage pipeline of smaller specialized models that mostly do classification or detection, avoiding the slow token-by-token generation that VLMs need.

LlamaParse: LLM-Guided Extraction

LlamaParse is a cloud-hosted document parser by LlamaIndex that takes a different approach:

• Cloud-based: the PDF is uploaded to LlamaCloud instead of being processed locally

• LLM-guided: an LLM interprets each page and identifies tables, returning structured row data

For extracting structured data from images like receipts using the same LlamaIndex ecosystem, see Turn Receipt Images into Spreadsheets with LlamaIndex.

To install LlamaParse, run:

pip install llama-parse

This article uses llama-parse v0.6.54.

LlamaParse requires an API key from LlamaIndex Cloud. The free tier includes 10,000 credits per month (basic parsing costs 1 credit per page; advanced modes like parse_page_with_agent cost more).

Create a .env file with your API key:

LLAMA_CLOUD_API_KEY=llx-...

from dotenv import load_dotenv

load_dotenv()

Table Extraction

To extract tables, we create a LlamaParse instance with two key settings:

• parse_page_with_agent: tells LlamaCloud to use an LLM agent that reads each page and returns structured items (tables, text, figures)

• output_tables_as_HTML=True: returns tables as HTML instead of Markdown, which better preserves multi-level headers

from llama_cloud_services import LlamaParse

parser = LlamaParse(
    parse_mode="parse_page_with_agent",
    output_tables_as_HTML=True,
)

Now let’s convert the PDF and measure how long it takes:

%%time
result = parser.parse(local_pdf)

Wall time: 8.54 s

We can then iterate through each page’s items and collect only the tables:

all_tables = []
for page in result.pages:
    for item in page.items:
        if item.type == "table":
            all_tables.append(item)

Not every item LlamaParse tagged as a table is actually a table. We filter out the title page and a figure that were misclassified, then pick our target table:

incorrect_table_indices = (1, 3)
tables = [t for i, t in enumerate(all_tables) if i not in incorrect_table_indices]

Here’s the original table from the PDF:

And here’s what LlamaParse extracted:

print(tables[1].md)

LlamaParse handled this table perfectly:

• All 12 row labels match the original (Caption, Footnote, ..., All)

• All 5 columns are correctly named: human, MRCNN R50, MRCNN R101, FRCNN R101, YOLO v5x6

• All numeric values match the source, including the “human” inter-annotator range column (84-89, 83-91, etc.)

Conclusion: LlamaParse produces the most accurate extraction of the three tools on this dense layout.

Performance

LlamaParse finished in 8.54 seconds, the fastest of the three tools (Docling took 1 min 50s, Marker took 47s).

Unlike Docling and Marker, LlamaParse runs no models on your machine. It uploads the PDF to LlamaCloud, an LLM agent reads each page, and the result comes back:

The runtime is mostly network upload time and server processing, so it depends on your internet speed and current LlamaCloud load rather than your local hardware.

Summary

In short:

• LlamaParse wins on speed and accuracy. It’s the fastest overall and produces the cleanest output, but it requires sending PDFs to LlamaCloud.

• Marker is the best local option. It’s faster than Docling and handles tables with clear visual separation well, but it merges columns on dense layouts.

• Docling is the slowest of the three and prone to hallucinating values on dense tables.

When to use each:

• Use LlamaParse if your documents aren’t sensitive and you want the best accuracy.

• Use Marker if you must stay local.

• Use Docling for its broader document conversion features beyond just table extraction like chunking and RAG.

For the full comparison across other tables with different layouts, see PDF Table Extraction: Docling vs Marker vs LlamaParse Compared.

Originally published on CodeCut.

📅 Today’s Picks

LM Studio - Pick Local Models That Fit Your Machine

Problem

Choosing a local LLM is not just about picking the smartest model.

You also need to know whether your machine can actually run it.

Model size, quantization, GPU memory, and system RAM all affect whether the model will load successfully.

Solution

LM Studio makes local model selection easier by showing the details that affect whether a model will run well.

What LM Studio shows:

• Download options: choose among model variants by comparing format, quantization, and download size.

• Fit signal: see whether the model is likely to fit your machine before downloading.

• README: review model-specific instructions and benchmarks from the model page.

PP-OCRv6 - Extract Text from Receipts with Small OCR Models

Problem

When extracting text from messy documents, many teams now reach for large vision-language models.

But if you only need to find and read text, a smaller OCR-specific model may be faster and cheaper to run.

Solution

PP-OCRv6 is designed for reading text from images, rather than broad vision-language reasoning, so it can focus on text extraction efficiently.

It delivers strong text detection and recognition while staying much smaller than billion-parameter VLMs.

With tiny, small, and medium variants, you can start lightweight and switch to a larger model only when the extracted text needs better accuracy.

🧪 Run code

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

Model choice matters, but prompt quality matters too. If the prompt is vague or hard to maintain, the classifier can still produce wrong labels.

A typical example is a prompt written as one string:

prompt = """
Classify this banking query as:
- card_arrival
- card_delivery_estimate
- card_not_working
- card_swallowed

Return only the label.

Query: My new card still has not arrived after two weeks.
Intent:
"""

This works for a simple demo, but real queries quickly reveal cases the prompt does not handle well.

For example:

My new card arrived, but it does not work at the ATM.

Because the prompt does not clarify this edge case, the model may focus on “new card” and return:

card_arrival

But the correct intent is:

card_not_working

You can patch the prompt with another rule, but that creates a new problem: every change needs to be retested. A fix for one visible mistake can hide new failures elsewhere.

Without a dataset and metric, you cannot tell whether the classifier improved overall.

DSPy replaces manual prompt tweaking with four repeatable steps:

1. Define the task as a program.

2. Evaluate the program with examples and a metric.

3. Let an optimizer improve the program.

4. Compare the score before and after.

This article walks through that loop by building a small banking intent classifier.

Full version: This is a condensed version. For the complete walkthrough, see the full DSPy classifier tutorial.

What Is DSPy?

DSPy is a Python framework for programming LLM workflows instead of hand-writing prompts.

It breaks an LLM workflow into explicit parts:

• Signatures define the inputs and outputs.

• Modules run the task with strategies such as Predict or ChainOfThought.

• Metrics score the outputs.

• Optimizers improve the program using examples and metrics.

This structure makes prompt engineering measurable. You can compare versions, optimize against a metric, and reuse the improved program.

Manual prompt                    DSPy program
-------------                    ------------
Task description       --->      Signature
Prompting style        --->      Module
Manual inspection      --->      Metric
Prompt tweaking        --->      Optimizer

Setup: Banking Query Classification

Install the libraries used in this tutorial:

pip install -U dspy pandas python-dotenv

This article uses dspy v3.2.1, pandas v2.3.1, and python-dotenv v1.1.1.

This tutorial uses OpenAI’s gpt-4o-mini through DSPy’s language model interface. Store your API key in a .env file:

OPENAI_API_KEY=your-openai-api-key

Then load the environment variables and configure DSPy:

from typing import Literal

import dspy
import pandas as pd
from dotenv import load_dotenv

load_dotenv()

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

We will use BANKING77, a dataset of banking support questions labeled with customer intents. To keep loading simple, this tutorial reads the raw CSV files from the original PolyAI repository.

TRAIN_URL = "https://raw.githubusercontent.com/PolyAI-LDN/task-specific-datasets/master/banking_data/train.csv"
TEST_URL = "https://raw.githubusercontent.com/PolyAI-LDN/task-specific-datasets/master/banking_data/test.csv"

train_df = pd.read_csv(TRAIN_URL)
test_df = pd.read_csv(TEST_URL)

print(train_df.head())

                                                text      category
0                     I am still waiting on my card?  card_arrival
1  What can I do if my card still hasn't arrived ...  card_arrival
2  I have been waiting over a week. Is the card s...  card_arrival
3  Can I track my card while it is in the process...  card_arrival
4  How do I know if I will get my card, or if it ...  card_arrival

To keep the example small, we will use four card-support intents instead of all 77 labels. The subset is still useful because card_arrival and card_delivery_estimate are similar enough to create meaningful mistakes.

INTENTS = [
    "card_arrival",
    "card_delivery_estimate",
    "card_not_working",
    "card_swallowed",
]


def sample_intents(data: pd.DataFrame, examples_per_intent: int) -> pd.DataFrame:
    return (
        data[data["category"].isin(INTENTS)]
        .groupby("category", group_keys=False)
        .sample(n=examples_per_intent, random_state=42)
        .reset_index(drop=True)
    )


train_sample = sample_intents(train_df, examples_per_intent=8)
dev_sample = sample_intents(test_df, examples_per_intent=10)

Before evaluation, prepare the data for DSPy:

• Store each query-label pair as a dspy.Example.

• Mark query as the input field with .with_inputs("query").

• Keep intent as the target label DSPy will compare against the prediction.

def to_dspy_examples(data: pd.DataFrame) -> list[dspy.Example]:
    return [
        dspy.Example(query=row.text, intent=row.category).with_inputs("query")
        for row in data.itertuples(index=False)
    ]

trainset = to_dspy_examples(train_sample)
devset = to_dspy_examples(dev_sample)

Define the Task with a Signature

A DSPy signature makes the task explicit. Instead of hiding the task inside a prompt string, you define the input fields, output fields, and output constraints in code.

The signature below defines the task schema:

• Input field: query

• Output field: intent

• Allowed outputs: card_arrival, card_delivery_estimate, card_not_working, card_swallowed

• Field descriptions: short hints DSPy can use when prompting the model

class ClassifyBankingIntent(dspy.Signature):
    """Classify a banking support query into one of the allowed intents."""

    query: str = dspy.InputField(desc="Customer support query")
    intent: Literal[
        "card_arrival",
        "card_delivery_estimate",
        "card_not_working",
        "card_swallowed",
    ] = dspy.OutputField(desc="Predicted banking intent")

The typed intent field is how DSPy keeps outputs within the allowed labels. For a dedicated way to enforce and validate typed LLM outputs with Python types, see Enforce Structured Outputs from LLMs with PydanticAI.

Run the Task with DSPy Modules

A DSPy module turns the signature into callable code.

Different modules run the same task in different ways:

• Predict returns the output directly.

• ChainOfThought adds a reasoning step before the output.

Because they can share the same signature, you can switch strategies without redefining the task.

Predict: Direct Prediction

Predict is the simplest module. It asks the model to return the output directly.

predict_classifier = dspy.Predict(ClassifyBankingIntent)

prediction = predict_classifier(
    query="The ATM kept my card and did not return it. How do I get it back?"
)

print(f'Intent: {prediction.intent}')

Intent: card_swallowed

This matches the query and stays within the allowed intent labels.

ChainOfThought: Reason Before Predicting

ChainOfThought keeps the same input and output fields, but adds a reasoning step before the prediction:

cot_classifier = dspy.ChainOfThought(ClassifyBankingIntent)

prediction = cot_classifier(
    query="The ATM kept my card and did not return it. How do I get it back?"
)

print(f'Reasoning: {prediction.reasoning}')
print(f'Intent: {prediction.intent}')

Reasoning: The customer's query indicates that their card was not returned by an ATM, which suggests that the card was likely swallowed by the machine. The customer is seeking information on how to retrieve their card, which aligns with the intent of a card being swallowed by the ATM.
Intent: card_swallowed

Unlike Predict, ChainOfThought exposes the reasoning before the final label. The predicted intent is still card_swallowed.

Now that the modules are defined, the next step is to score the classifier versions and optimize one of them.

Evaluate the Baseline

Before optimizing the classifier, we need to measure the baseline. Here, the metric is simple: a prediction is correct when the predicted intent matches the expected label.

def intent_exact_match(example: dspy.Example, prediction: dspy.Prediction, trace=None) -> bool:
    return example.intent == prediction.intent

Now create a DSPy evaluator:

evaluate = dspy.Evaluate(
    devset=devset,  # examples to score
    metric=intent_exact_match,  # scoring function
    num_threads=4,  # parallel model calls
    display_progress=True,  # show progress bar
    display_table=5,  # show sample predictions
)

Use the same evaluator to compare Predict and ChainOfThought on the dev set:

predict_score = evaluate(predict_classifier)
print(f"Predict score: {predict_score.score}")

Because display_table=5 is set, the evaluator prints a sample of predictions before the score:

... 35 more rows not displayed ...

Predict score: 77.5

Run the same evaluator on ChainOfThought:

cot_score = evaluate(cot_classifier)
print(f"ChainOfThought score: {cot_score.score}")

... 35 more rows not displayed ...

ChainOfThought score: 80.0

The displayed rows make the comparison easier to inspect: you can see which examples matched the expected intent and which ones failed. In this run, ChainOfThought scores higher because the reasoning step helps with some ambiguous delivery queries.

Optimize the Classifier with Examples

Once the metric shows where the baseline fails, DSPy can use training examples to search for a better version of the program.

DSPy provides several optimizer options depending on how much search you want:

• BootstrapFewShot improves the prompt by adding better examples.

• MIPROv2 improves the prompt by tuning both instructions and examples.

• GEPA improves the prompt by using feedback from previous attempts.

This article uses BootstrapFewShot because it is the simplest optimizer for this setup. It uses the training set and metric to choose examples that make the prompt stronger.

Few-shot examples are useful when the label name alone is not enough.

For example, card_arrival could sound like a successful delivery, but this example shows what it means in the dataset:

Query: My card has not arrived yet.
Intent: card_arrival

The label refers to questions or problems about card delivery. BootstrapFewShot helps find examples like this and add them to the prompt:

from dspy.teleprompt import BootstrapFewShot

optimizer = BootstrapFewShot(
    metric=intent_exact_match,  # score each candidate
    max_bootstrapped_demos=4,  # generated examples to keep
    max_labeled_demos=8,  # labeled examples to include
    max_rounds=1,  # bootstrap attempts per example
)

optimized_classifier = optimizer.compile(
    student=cot_classifier,
    trainset=trainset,
)

Compare Before vs. After

Evaluate the optimized classifier on the same dev set:

optimized_score = evaluate(optimized_classifier)

scores = pd.DataFrame(
    [
        {"program": "Predict", "score": predict_score.score},
        {"program": "ChainOfThought", "score": cot_score.score},
        {"program": "BootstrapFewShot + ChainOfThought", "score": optimized_score.score},
    ]
)

print(scores)

                             program  score
0                            Predict   77.5
1                     ChainOfThought   80.0
2  BootstrapFewShot + ChainOfThought   87.5

Nice! The optimized classifier performs best in this run, improving from 80.0 with ChainOfThought to 87.5 after adding optimized few-shot examples.

You can also inspect individual misses to understand what still fails:

for example in devset:
    prediction = optimized_classifier(query=example.query)

    if prediction.intent != example.intent:
        print("Query:", example.query)
        print("Expected:", example.intent)
        print("Predicted:", prediction.intent)
        print()

Query: Is there tracking info available?
Expected: card_arrival
Predicted: card_delivery_estimate

Query: Where is the tracking number for the card you sent me?
Expected: card_arrival
Predicted: card_delivery_estimate

Query: Do you know if there is a tracking number for the new card you sent me?
Expected: card_arrival
Predicted: card_delivery_estimate

Query: I'm just wondering when my card will get here.
Expected: card_delivery_estimate
Predicted: card_arrival

Query: I am waiting for my card to arrive.
Expected: card_delivery_estimate
Predicted: card_arrival

Most misses are between card_arrival and card_delivery_estimate. That makes sense: both intents mention waiting for a card, tracking, or delivery timing.

To improve this, we could add more labeled examples that separate “my card has not arrived” from “how long does delivery take?”

Final Thoughts

DSPy is worth using when an LLM workflow will run repeatedly and quality matters. It is especially useful when you have:

• Labeled examples

• A metric

• Several prompt or module versions to compare

• A task that will evolve over time

It is probably too much for one-off prompts, quick brainstorming, or tasks where you do not have examples to evaluate against.

For the complete walkthrough, see the full DSPy classifier tutorial.

Related Tutorials

• Structured Output Tools for LLMs: Instructor, PydanticAI, LangChain, Outlines, and Guidance Compared: Compares libraries that force LLMs to return valid, typed outputs, the same problem DSPy signatures solve.

• Build Production-Ready RAG Systems with MLflow Quality Metrics: Measures LLM output quality with metrics, complementing DSPy’s evaluate-and-optimize loop.

Originally published on CodeCut.

📅 Today’s Picks

LiteLLM - One interface for OpenAI, Anthropic, and Gemini

Problem

Frameworks like LangChain and LlamaIndex are useful when you need chains, agents, retrieval, memory, or orchestration.

But if you only need to call different providers through the same interface, you may not want to restructure your app around a full LLM framework.

Solution

LiteLLM solves this as a lightweight interface layer.

You can keep one completion() call across 100+ providers, then switch models by changing only the model string.

📖 View Full Article

Polars 1.41 - Speed Up Wide Parquet Scans Without Code Changes

Problem

scan_parquet is Polars’ lazy way to read a Parquet file, allowing it to optimize the query before loading data.

Before the query runs, Polars still needs to decode the Parquet footer to understand the schema, row groups, and column statistics. For very wide files, that setup step can take noticeable time.

Solution

Polars 1.41 makes scan_parquet faster by replacing the old metadata decoder with one built specifically for Parquet files.

In the Polars 1.41 release benchmark, footer decoding was up to 3.29x faster on a 10,000-column file.

📖 Read the release

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

LLM applications are often stateless by default. Each prompt only knows what you include at that moment, so useful context from earlier interactions can disappear between sessions.

For simple questions, that is often enough. For returning users, it can lead to repeated explanations, missing preferences, and less helpful responses.

Mem0 helps by extracting important facts from conversations, storing them by user, and retrieving relevant memories when needed.

To see how Mem0 works, we will build a simple e-commerce support workflow, first without memory and then with Mem0, so you can compare how stored context changes the response.

📖 Full version: This is a condensed version. For memory management operations like inspecting and deleting stored memories, see the complete tutorial.

Vector Search vs Memory

A vector database is a good starting point for retrieval, but memory requires more than finding similar text. You still need a way to decide which details are worth saving for future interactions.

In this customer support example, the vector database may retrieve the entire message, including the coffee detail even though it is unrelated to the delayed order:

My order #4821 still has not arrived. I contacted support last week and got no update. Also, I was checking this while making coffee this morning.

A memory layer should keep only the support-relevant facts:

Order #4821 is delayed, and the customer contacted support last week without receiving an update.

To get from raw retrieval to useful memory, your application still needs logic for:

• Deciding which details from the message should be remembered for future replies.

• Making sure each memory is tied to the right user or conversation.

• Deciding how to handle newer memories that conflict with or supersede earlier ones.

• Formatting retrieved messages into prompt-ready context.

You can implement this yourself, but the extra extraction and memory-management logic adds up. Mem0 packages that workflow behind a simpler memory API, which we will use in the next section.

Setup

Install the libraries used in this tutorial:

• mem0ai: Adds long-term memory for storing and retrieving user context.

• litellm: Provides a unified interface for calling LLM providers.

• python-dotenv: Loads API keys and configuration from a .env file.

pip install mem0ai litellm python-dotenv

This article uses mem0ai v2.0.4, litellm v1.87.0, and python-dotenv v1.2.2.

Since this tutorial uses OpenAI’s GPT-4o-mini, store your OPENAI_API_KEY in a .env file in your project folder so Mem0 and LiteLLM can access it.

OPENAI_API_KEY=your-openai-api-key

Start by loading the environment variables and imports used throughout the tutorial:

from dotenv import load_dotenv
from litellm import completion
from mem0 import Memory

load_dotenv()

Next, define a helper that sends a list of chat messages to GPT-4o-mini through LiteLLM and returns only the generated text:

def ask_model(messages: list[dict[str, str]]) -> str:
    response = completion(model="openai/gpt-4o-mini", messages=messages)
    return response.choices[0].message.content

To use another model, replace "openai/gpt-4o-mini" with a model name from the LiteLLM provider docs.

Finally, configure Mem0 with an LLM for extracting memories from messages. This example uses GPT-4o-mini, but Mem0 and LiteLLM do not have to use the same model:

memory = Memory.from_config(
    {
        "llm": {
            "provider": "openai",
            "config": {"model": "gpt-4o-mini"},
        },
    }
)

Build a Stateless Support Bot

First, create a support bot with no memory. It only sees the current message.

SYSTEM_PROMPT = (
    "You are a helpful e-commerce customer support agent. "
    "Be polite, concise, and practical."
)


def ask_stateless_bot(question: str) -> str:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": question},
    ]
    return ask_model(messages)

The customer opens a support chat about a delayed order. We will also include an irrelevant detail in the message, so that we can later show that Mem0 can focus on the useful support facts:

initial_message = (
    "My order #4821 still has not arrived. "
    "I contacted support last week and got no update. "
    "Also, I was checking this while making coffee this morning." # irrelevant detail
)

print(ask_stateless_bot(initial_message))

I'm sorry to hear that your order #4821 hasn't arrived yet. I understand how frustrating this can be. Let me assist you by checking the status for you. Please give me a moment.

The reply is reasonable for a first message. The problem shows up when the customer returns later.

Test the Stateless Support Bot

Let’s see how the bot responds to a follow-up question without passing the earlier conversation:

follow_up = "Hi, any update on my delayed order?"

print(ask_stateless_bot(follow_up))

I’d be happy to help with your order. Could you please provide me with your order number? This will help me look up the status for you.

The response is polite, but it loses the context that matters most: the order number, the prior support contact, and the customer’s frustration.

Store Support Interactions in Mem0

To preserve the missing order context, add Mem0 as a memory layer. Store the customer message under a stable customer_id so it can be retrieved in later interactions.

customer_id = "cust_4821"
order_id = "4821"
issue_type = "shipping"
sentiment = "frustrated"

memory_extraction_prompt = (
    "Extract only useful customer-support facts. Ignore unimportant information."
)

add_result = memory.add(
    messages=[{"role": "user", "content": initial_message}],
    user_id=customer_id,
    metadata={
        "issue_type": issue_type,
        "sentiment": sentiment,
        "order_id": order_id,
    },
    prompt=memory_extraction_prompt,
)

print(add_result["results"][0]["memory"])

User's order #4821 has not arrived as of June 3, 2026, and they contacted support last week but received no update.

Note that the saved memory keeps the support-relevant details: the order number, the delayed delivery, and the prior support contact. It does not include the unrelated coffee detail from the original message.

Raw customer message
   |
   v
LLM-based extraction step
   |
   +--> Save: order #4821 has not arrived
   +--> Save: customer contacted support last week
   +--> Drop: customer was making coffee

This has a few practical benefits:

• Cleaner retrieval: Future searches return useful support context instead of unrelated conversation details.

• Lower API cost: Passing compact memories instead of noisy chat history can reduce the number of tokens sent to the model.

• Better privacy boundaries: Incidental personal details are less likely to be retained when they are not needed.

If you do not want Mem0 to infer or rewrite memories, set infer=False. In that mode, Mem0 stores the provided message directly instead of extracting selected facts from it.

Retrieve Memories Before Responding

Before answering a new message, define a helper that searches Mem0 for memories related to the current question and limits the search to the current customer_id:

def retrieve_customer_memories(question: str, customer_id: str) -> dict:
    return memory.search(
        query=question,
        filters={"user_id": customer_id},
        top_k=3,
    )


for item in retrieve_customer_memories(follow_up, customer_id)["results"]:
    print(item["memory"])

User's order #4821 has not arrived and they contacted support last week but received no update.

Next, convert the dictionary returned by Mem0 into plain text so it can be inserted into the system prompt:

def format_memories(search_result: dict) -> str:
    memories = search_result["results"]
    if not memories:
        return "No relevant memories found."

    return "\n".join(f"- {item['memory']}" for item in memories)


print(format_memories(retrieve_customer_memories(follow_up, customer_id)))

- User's order #4821 has not arrived and they contacted support last week but received no update.

Compare Stateless vs Memory-Aware Responses

Now we are ready to create a memory-aware support bot. The function below does three things:

• Retrieves relevant customer memories from Mem0.

• Adds those memories to the system prompt.

• Sends the updated prompt to the model.

def ask_memory_aware_bot(question: str, customer_id: str) -> str:
    customer_memories = format_memories(
        retrieve_customer_memories(question, customer_id)
    )

    messages = [
        {
            "role": "system",
            "content": (
                f"{SYSTEM_PROMPT}\n\n"
                f"Relevant customer memories:\n{customer_memories}"
            ),
        },
        {"role": "user", "content": question},
    ]
    return ask_model(messages)

Ask the same follow-up again:

print(ask_memory_aware_bot(follow_up, customer_id))

Thanks for checking back on order #4821. I can see it is still delayed and that you contacted us last week without an update. I am escalating this with shipping now and will email you a new delivery estimate today.

This response now uses the stored memory. It remembers the order number, the earlier support contact, and the unresolved shipping issue without asking the customer to repeat them.

Add a Status Update

Next, add a new message that changes the order status. This lets us see how the memory-aware bot handles newer information:

status_update_result = memory.add(
    messages=[{"role": "user", "content": "Order #4821 arrived today."}],
    user_id=customer_id,
    metadata={"order_id": order_id, "issue_type": issue_type},
)

Mem0 stores this as a new memory. Ask the same follow-up again:

print(ask_memory_aware_bot(follow_up, customer_id))

Hello! I see that your order #4821 has actually arrived on June 3, 2026. If you need any further assistance or have questions about your order, feel free to let me know!

The answer changes because the model now receives both the earlier delay and the newer arrival status in the retrieved memories.

For inspecting and deleting stored memories with get_all(), delete(), and delete_all(), see the complete tutorial.

Final Thoughts

Stateless LLM applications are easy to build, but they only know what is included in the current prompt.

Mem0 adds a memory layer for applications that need continuity across sessions. The same pattern can support customer support bots, personal assistants, tutoring apps, healthcare intake tools, CRM workflows, and onboarding assistants.

With Mem0, applications can save important details from past interactions and retrieve the most relevant context for future responses.

Use memory when you don’t want returning users to have to repeat important details. Skip it when each request is self-contained or when storing user history adds unnecessary complexity.

Related Tutorials

• Implement Semantic Search in Postgres Using pgvector and Ollama: Learn how to store embeddings in Postgres and retrieve semantically similar records with pgvector.

• Enforce Structured Outputs from LLMs with PydanticAI: Learn how to validate extracted LLM outputs with typed Pydantic models.

Originally published on CodeCut.

📅 Today’s Picks

LM Studio - Try local LLMs without setup code

Problem

Tools like Ollama and Hugging Face Transformers are powerful, but they can still require CLI commands, Python setup, model configuration, or server management before you send your first prompt.

That friction makes cloud APIs feel easier, even when local models are better for privacy, cost, or offline experimentation.

Solution

LM Studio gives you a clean desktop interface for browsing, downloading, and running local models without writing setup code.

Clean Code Skills - Turn AI-generated code into cleaner Python

Problem

AI-generated code often works on the first run, but the structure can be hard to follow.

You might see long functions, duplicated logic, vague names, or deeply nested conditionals that make future changes slower.

Solution

Clean Code Skills gives your AI agent focused guidance based on Robert C. Martin’s Clean Code rules for Python and TypeScript.

Each skill targets a maintenance problem:

• boy-scout: improve the code it touches

• clean-functions: keep functions small and focused

• clean-names: choose names that explain intent

• clean-tests: write tests around clear behavior

• clean-general: reduce duplication, magic numbers, long branching paths, and more

I use this skill when asking my agent to write Python code, refactor existing code, review a change, or add tests.

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

In a previous article, we tested three Python tools for PDF table extraction: Docling, Marker, and LlamaParse. None of them handled the test document perfectly: Docling hallucinated values, Marker merged columns on borderless rows, and LlamaParse added a duplicate empty column.

After publishing part 1, I came across two more tools that target the same problem and wanted to see how they perform compared to the ones we already tested:

• olmOCR-2 from Allen Institute for AI, a 7B fine-tune of Qwen2.5-VL

• PaddleOCR-VL 1.6 from Baidu, a 1B model with a layout-detection pipeline

Both claim state-of-the-art table extraction. We’ll test them on a Mac (Apple M5 Pro), using the same PDF as part 1, to see if they fix the failures we saw there. Both tools run on Apple Silicon via GGUF quantizations with llama.cpp or native CPU PaddlePaddle.

📖 Full version: This is a condensed version focusing on the hardest table in the document. For the first-table walkthrough and a full discussion of the tradeoffs, see the complete comparison.

The Test Document

For a fair comparison, we will use the same PDF as part 1: the Docling Technical Report from arXiv:

import urllib.request

source = "https://arxiv.org/pdf/2408.09869"
local_pdf = "docling_report.pdf"
urllib.request.urlretrieve(source, local_pdf)

olmOCR-2: Qwen2.5-VL Fine-Tune

olmOCR-2 is Allen AI’s open-weight OCR model. It stands out for three reasons:

• A 7B fine-tune of Qwen2.5-VL reads each PDF page as an image

• Cheap to run at scale: on a rented NVIDIA H100, olmOCR-2 processes a few pages per second, working out to about $2 per 10,000 pages in cloud costs

• Strongest table benchmark: scores 84.9 on tables on its own olmOCR-Bench, the highest among open VLM-OCR models at release

olmOCR-2 takes the whole PDF page as an image and produces structured output in a single step. This is the same architecture as Docling’s VLM pipeline from part 1, just with a different model.

PDF page rendered as image
┌─────────────────────┐
│ Text paragraph...   │
│ Name  Score         │
│ Alice  92           │
│ Bob    85           │
└─────────────────────┘
         │
         ▼
One model reads the page
and writes the output
         │
         ▼
| Name  | Score |
|-------|-------|
| Alice | 92    |
| Bob   | 85    |

Download the GGUF and vision projector

To use olmOCR-2 with llama.cpp on Apple Silicon, install llama.cpp and download two files: the model weights and the vision projector (mmproj).

brew install llama.cpp

# Language model (Q8_0, ~8 GB)
curl -L -O https://huggingface.co/lmstudio-community/olmOCR-2-7B-1025-GGUF/resolve/main/olmOCR-2-7B-1025-Q8_0.gguf

# Vision projector (F16, ~1.4 GB)
curl -L -O https://huggingface.co/lmstudio-community/olmOCR-2-7B-1025-GGUF/resolve/main/mmproj-olmOCR-2-7B-1025-F16.gguf

Table extraction

olmOCR-2 reads images, not PDFs, so we’ll extract tables in three steps:

1. Convert each PDF page to an image

2. Run olmOCR-2 on each image and collect the output

3. Extract the tables from the combined output with a regex

For step 1, we will use pdf2image, which depends on the poppler system binary. Install both:

brew install poppler
pip install pdf2image

Now convert each page to a JPEG:

import subprocess
from pathlib import Path
from pdf2image import convert_from_path

images_dir = Path("images")
images_dir.mkdir(exist_ok=True)

pages = convert_from_path(local_pdf, dpi=200)
for i, page in enumerate(pages):
    page.save(images_dir / f"page_{i}.jpg")

olmOCR-2 doesn’t have a pure-Python API that runs on Apple Silicon, so we shell out to llama-mtmd-cli via subprocess for each page. The command for one page looks like this:

llama-mtmd-cli \
  -m olmOCR-2-7B-1025-Q8_0.gguf \
  --mmproj mmproj-olmOCR-2-7B-1025-F16.gguf \
  --image page_0.jpg \
  -p "Convert this page to markdown. Preserve tables exactly. Output tables in HTML format." \
  --n-predict 3072

What each flag does:

• -m: the language model weights (the .gguf we downloaded)

• --mmproj: the vision encoder (the mmproj we downloaded)

• --image: the input image to process

• -p: the prompt sent to the model

• --n-predict: the maximum number of tokens to generate (3072 is enough for most table-heavy pages)

Wrap it in a Python helper so we can loop over pages:

import re

def extract_with_olmocr(page_path: str) -> str:
    result = subprocess.run(
        [
            "llama-mtmd-cli",
            "-m", "olmOCR-2-7B-1025-Q8_0.gguf",
            "--mmproj", "mmproj-olmOCR-2-7B-1025-F16.gguf",
            "--image", page_path,
            "-p", "Convert this page to markdown. Preserve tables exactly. Output tables in HTML format.",
            "--n-predict", "3072",
        ],
        capture_output=True,
        text=True,
    )
    return result.stdout

Run the helper on every page and combine the outputs:

%%time
olmocr_output = "\n".join(
    extract_with_olmocr(str(images_dir / f"page_{i}.jpg")) for i in range(len(pages))
)

Wall time: 5min 34s

olmOCR-2’s output is mostly Markdown but tables come out as HTML blocks. Extract them with a regex:

all_tables = re.findall(r"<table>.*?</table>", olmocr_output, re.DOTALL)
print(f"Items tagged as table: {len(all_tables)}")

Items tagged as table: 4

Not every block tagged <table> is actually a table. olmOCR-2 misreads the author block on the title page as a table and outputs two copies of it. We filter both out:

incorrect_table_indices = (1, 2)

tables = [t for i, t in enumerate(all_tables) if i not in incorrect_table_indices]
print(f"Actual tables: {len(tables)}")

Actual tables: 2

The output is HTML, so use IPython.display.HTML to see it rendered:

from IPython.display import display, HTML

Let’s look at the hardest table in the document: 12 rows of similar-looking numbers and no cell borders to mark column boundaries. Here’s the original from the PDF:

And here’s what olmOCR-2 extracted:

display(HTML(tables[1]))

Worked:

• All 12 row labels (Caption, Footnote, ..., All) preserved

• 12 data rows extracted with one numeric value per cell

Didn’t work:

• Two column headers are missing: Only 4 of the 6 columns have headers, so the class-label column and one of the model columns appear unlabeled.

• MRCNN R101 is dropped from the header row: The numeric values in that column still appear, but they sit under the wrong header name.

• Hyphenated ranges become decimals: Every entry in the “human” range column is wrong: 84-89 becomes 84.89, 83-91 becomes 83.91, and so on.

• Numeric values drift in several cells: Most rows have at least one digit substitution (Page-footer 61.6 → 74.6, List-item 81.2 → 81.6, All-row 72.4 → 77.4).

Conclusion: olmOCR-2’s output looks clean but can be quietly wrong. It introduces character-level errors on dense numeric tables. Verify numeric values before trusting them.

Performance

olmOCR-2 took 5 min 34 s for the 9-page PDF on an Apple M5 Pro (64 GB RAM), about 37 seconds per page through GGUF + llama.cpp.

For production on a Mac, switch to the native MLX build (mlx-community/olmOCR-2-7B-1025-8bit), which runs about 20% faster than GGUF.

PaddleOCR-VL 1.6: Pipeline VLM

PaddleOCR-VL is Baidu’s open-weight document parser. It stands out for three reasons:

• A 1B fine-tune of ERNIE-4.5, the smallest model of the new VLM-OCR generation

• Strong multilingual support including Chinese ancient documents, scans, and stamps (not tested in this article)

• Mature ecosystem: PaddleOCR has 78.9k stars on GitHub and a long history of production deployment

Unlike olmOCR-2’s single-pass approach, PaddleOCR-VL splits table extraction into two stages:

• Layout detection locates each text block, table, and figure on the page

• Element-level VL recognition reads each detected region and converts it to text or structured Markdown

PDF page
┌─────────────────────┐
│ Text paragraph...   │
│ Name  Score         │
│ Alice  92           │
│ Bob    85           │
└─────────────────────┘
         │
         ▼
1. Layout detection identifies [TABLE] region
         │
         ▼
2. Element-level VL reads only the table region
         │
         ▼
| Name  | Score |
|-------|-------|
| Alice | 92    |
| Bob   | 85    |

Install

Pick the install that matches your hardware.

Apple Silicon (Mac):

pip install paddlepaddle
pip install -U "paddleocr[doc-parser]>=3.6.0"

Linux / Windows (NVIDIA):

pip install paddlepaddle-gpu==3.2.1
pip install -U "paddleocr[doc-parser]>=3.6.0"

This article uses PaddleOCR v3.6.0.

Table extraction

Unlike olmOCR-2, PaddleOCR-VL accepts a PDF path directly and returns a result object per page. No PDF-to-image conversion or subprocess loop required:

from paddleocr import PaddleOCRVL

pipeline = PaddleOCRVL(pipeline_version="v1.6")

Run the pipeline on the PDF:

%%time
results = pipeline.predict(local_pdf)

Wall time: 7min 56s

Each entry in results corresponds to one page of the PDF. Loop through them and collect the tables:

# Create an output directory for the per-page markdown files
paddle_output_dir = Path("paddle_output")
paddle_output_dir.mkdir(exist_ok=True)

# Save each page's markdown to disk
for res in results:
    res.save_to_markdown(save_path=str(paddle_output_dir))

# Find every HTML table block
all_paddle_tables = []
for md_file in sorted(paddle_output_dir.glob("*.md")):
    content = md_file.read_text()
    all_paddle_tables.extend(re.findall(r"<table[^>]*>.*?</table>", content, re.DOTALL))

print(f"Items tagged as table: {len(all_paddle_tables)}")

Items tagged as table: 3

Not every block PaddleOCR-VL tagged as a table is a unique table. The third item is a malformed near-duplicate of the second. Let’s filter it out:

incorrect_table_indices = (2,)

paddle_tables = [t for i, t in enumerate(all_paddle_tables) if i not in incorrect_table_indices]
print(f"Actual tables: {len(paddle_tables)}")

Actual tables: 2

Now the same hard table. Here’s the original from the PDF:

And here’s what PaddleOCR-VL extracted:

display(HTML(paddle_tables[1]))

Worked:

• All 12 class-label rows plus the Total row are present (truncated above for space)

• Hyphenated ranges preserved correctly as “84-89”, “40-61”, exactly where olmOCR-2 misread them as decimals

• “n/a” entries preserved

• All numeric values match the source

Didn’t work:

• Header grouping is wrong: The two parent headers in the original PDF get split into three in the extraction: “Count” is absorbed into “% of Total”, and “triple inter-annotator mAP @ 0.5-0.95 (%)” is split into two separate parents.

Conclusion: PaddleOCR-VL is 7x smaller than olmOCR-2 (1B vs 7B parameters) and still more accurate on this PDF. All numeric values match the source, and the only real flaw is the mis-grouped multi-tier headers.

Performance

PaddleOCR-VL 1.6 took about 7 min 56 s for the full 9-page PDF on an Apple M5 Pro running CPU PaddlePaddle, roughly 53 seconds per page.

Even though the model is smaller than olmOCR-2, the pipeline overhead (layout detection plus element-level recognition) makes it slower per page than olmOCR-2 on this hardware.

Try It Yourself

These benchmarks are based on a single academic PDF tested on an Apple M5 Pro (64 GB RAM) using GGUF Q8_0 quantizations via llama.cpp. Table complexity, document language, scan quality, and hardware all affect the results. The best way to pick the right tool is to run each one on a sample of your own PDFs.

For the first-table walkthrough on each tool and the full runtime setup, see the complete comparison.

Originally published on CodeCut.

📅 Today’s Picks

Python contextlib - Reuse Context Managers as Function Decorators

Problem

Many developers know @contextmanager as a convenient way to add setup and teardown logic around a with block.

But fewer realize it can also be used as a function decorator.

A common pattern for timing or logging functions is to write a separate decorator using functools.wraps. But this means maintaining two versions of the same logic, which can drift over time.

Solution

Since Python 3.3, any function decorated with @contextmanager can be used both ways:

• Need to time a code block? with duration("..."):

• Need to time a whole function? @duration("...")

Claude Code - Move Session Context Across Folders with /export

Problem

Each Claude session lives in the folder where it was started. When you open Claude somewhere else, the previous context does not carry over.

You could ask Claude to summarize the old session and paste it into the new one, but summaries can miss important details, decisions, or tool outputs.

Solution

Claude Code’s /export command gives you the full conversation as a plain text file, including tool calls.

How to use it:

• Run /export filename.md in the original folder before switching

• Open a new Claude session in the other folder

• Ask Claude to read filename.md and continue the task

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

💡 Sponsored by Tiger Data. This article was produced in partnership with the team behind TimescaleDB. All technical content, examples, and opinions are my own.

Introduction

PostgreSQL is optimized for transactional workloads where individual rows are read and updated. Common operations include updating user profiles, processing payments, or changing order states. Access is random and row-focused.

PostgreSQL: designed for row-level access

┌────┐
│ r1 │ ◄── read/update
├────┤
│ r2 │
├────┤
│ r3 │ ◄── delete
├────┤
│ r4 │
├────┤
│ r5 │ ◄── read/update
└────┘

Time-series data is the opposite: you constantly append new readings, query by time range, and periodically delete old data in bulk. Instead of touching individual rows, every operation works on ranges of time.

Time-series: needs range-level access

┌────┬────┬────┬────┐
│ t1 │ t2 │ t3 │ t4 │ ◄── append
├────┼────┼────┼────┤
│    │    │    │    │
│    query range    │
│    ◄──────────►   │
├────┼────┼────┼────┤
│ t1 │ t2 │         │
│ ▼  │ ▼  │         │
│drop│drop│         │
└────┴────┴─────────┘
     ▲
     └── bulk delete old chunks

PostgreSQL can handle this pattern at smaller scales, but time-series tables don’t remain small. They grow continuously, and operations that were efficient early on becomes less efficient over time.

As datasets scale into the hundreds of millions of rows, inserts, queries, and deletes all start to slow down.

Why PostgreSQL Struggles with Time-Series Data

Writes: Every Insert Updates Every Index

As a PostgreSQL table grows, inserts get progressively slower because every insert does two things:

• Append the row to the table, which stores the actual data in insertion order (fast)

• Update every index by finding the correct sorted position for the new value, so queries can locate rows without scanning the full table (gets slower as the index grows)

For example, inserting a sensor reading at 10:30:00 triggers both steps:

INSERT INTO sensor_data VALUES ('10:30:00', 'sensor_3', 22.5)

Step 1: Append row to table
┌───────────┬──────────┬──────┐
│ 10:00:01  │ sensor_1 │ 21.3 │
│ 10:00:05  │ sensor_2 │ 22.1 │
│ 10:29:58  │ sensor_1 │ 21.8 │
│ 10:30:00  │ sensor_3 │ 22.5 │ ◄ append to end (fast)
└───────────┴──────────┴──────┘

Step 2: Update time index
┌──────────┬────────┐
│ 10:00:01 │ row 1  │
│ 10:00:05 │ row 2  │
│ 10:29:58 │ row 3  │
│ 10:30:00 │ row 4  │ ◄ find sorted position, insert (slow)
│ 10:30:02 │ row 5  │
└──────────┴────────┘

With more rows, the index grows larger and finding the correct position takes more work. Multiply this by every index on the table, and inserts get progressively slower.

Reads: No Way to Skip Irrelevant Time Ranges

Time-series queries almost always filter by time, but PostgreSQL stores all rows together with no time-based grouping. To find rows matching a time range, it must search through the entire index, even when you only need the most recent hour.

For example, if your table holds three months of sensor data and you query the last hour, PostgreSQL still searches from the beginning:

SELECT * FROM sensor_data WHERE time > '10:00:00'

┌──────────────────────────────────────────────┐
│ Jan    Feb    Mar    Apr ... Apr 25 10:00+   │
│ ◄──────── scanned ────────► ◄── needed ──►   │
└──────────────────────────────────────────────┘

Deletes: Row-by-Row Removal Locks the Table

Time-series tables grow continuously, and at some point you need to delete old data. But PostgreSQL’s DELETE operates one row at a time, removing each entry from the table, updating every index, and logging the change for crash recovery.

For example, if you have three months of sensor data and decide to keep only April, PostgreSQL processes every row in January, February, and March one at a time:

DELETE FROM sensor_data WHERE time < '2026-04-01'

┌────────┬────────┬────────┬────────┐
│  Jan   │  Feb   │  Mar   │  Apr   │
│  drop  │  drop  │  drop  │  keep  │
└────────┴────────┴────────┴────────┘
 Millions of rows, deleted one at a time

TimescaleDB is a PostgreSQL extension designed to solve all three of these problems. Let’s see how it solves each of these problems.

What Is TimescaleDB?

TimescaleDB is a PostgreSQL extension. It installs into your existing PostgreSQL instance without replacing anything. Your queries, connections, backups, and monitoring all stay the same.

What it adds is a set of features designed specifically for time-series workloads:

• Hypertables: Automatically split large tables into smaller time-based chunks

• time_bucket(): Group data by any time interval (15 minutes, 6 hours, 3 days)

• Continuous aggregates: Pre-computed views that only process new data on refresh

• Columnar compression: Reduce storage by 90-95% without changing how you query the data

• Retention policies: Drop old data instantly without row-by-row deletion

To see how these features work in practice, let’s set up TimescaleDB and try each one.

Setup

The fastest way to start is with Docker. This runs TimescaleDB with PostgreSQL 18:

docker run -d --name timescaledb \
  -p 5432:5432 \
  -e POSTGRES_PASSWORD=password \
  timescale/timescaledb-ha:pg18

Connect to the PostgreSQL instance running in the container:

psql -d "postgres://postgres:password@localhost:5432/postgres"

Verify TimescaleDB is installed:

SELECT default_version
FROM pg_available_extensions
WHERE name = 'timescaledb';

 default_version
-----------------
 2.26.3

Note: If you’re not using Docker, see Install TimescaleDB for other setup options.

Hypertables: Automatic Time-Based Partitioning

A hypertable looks and behaves like a regular PostgreSQL table. You insert, query, and join it the same way.

The difference is behind the scenes: TimescaleDB automatically splits the data into time-based “chunks” and skips irrelevant chunks during queries.

SELECT * FROM sensor_data WHERE time > 'Jan 13'

┌─────────┐  ┌─────────┐  ┌─────────┐
│ Chunk 1 │  │ Chunk 2 │  │ Chunk 3 │
│ Jan 1-7 │  │ Jan 8-14│  │ Jan 15+ │
│  skip   │  │ partial │  │  scan   │
└─────────┘  └─────────┘  └─────────┘
     ✗            ✓            ✓

Only chunks overlapping the time range are read.

Creating a Hypertable

Creating a hypertable requires only one change to a standard CREATE TABLE statement: adding WITH (tsdb.hypertable).

Let’s create one for storing sensor readings:

CREATE TABLE sensor_data (
    time TIMESTAMPTZ NOT NULL,
    sensor_id TEXT NOT NULL,
    temperature DOUBLE PRECISION,
    humidity DOUBLE PRECISION
) WITH (tsdb.hypertable);

That’s it. TimescaleDB automatically:

• Uses the time column to decide how to split data into chunks

• Creates time-based chunks (default: 7-day intervals)

• Automatically indexes the time column for fast range queries

You can customize the chunk interval and partition column if needed:

CREATE TABLE metrics (
    time TIMESTAMPTZ NOT NULL,
    device_id TEXT NOT NULL,
    cpu_usage DOUBLE PRECISION,
    memory_usage DOUBLE PRECISION
) WITH (
    tsdb.hypertable,
    tsdb.partition_column = 'time',
    tsdb.chunk_interval = '1 day'
);

Loading Data

Let’s generate a realistic dataset: 5 sensors reporting temperature and humidity every 10 seconds for 7 days. This produces about 300,000 rows.

INSERT INTO sensor_data (time, sensor_id, temperature, humidity)
SELECT
    ts,
    'sensor_' || device_num,
    20 + 10 * sin(extract(epoch FROM ts) / 43200) + random() * 2,
    40 + 20 * cos(extract(epoch FROM ts) / 43200) + random() * 5
FROM generate_series(
    '2026-04-21 00:00:00'::timestamptz,
    '2026-04-28 00:00:00'::timestamptz,
    INTERVAL '10 seconds'
) AS ts
CROSS JOIN generate_series(1, 5) AS device_num;

INSERT 0 302405

The sin() and cos() functions create realistic daily temperature and humidity cycles, with random() adding noise.

Inspecting Chunks

Query the chunk metadata to see how TimescaleDB partitioned the data:

SELECT chunk_name, range_start, range_end, is_compressed
FROM timescaledb_information.chunks
WHERE hypertable_name = 'sensor_data'
ORDER BY range_start;

    chunk_name    |      range_start       |       range_end        | is_compressed
------------------+------------------------+------------------------+---------------
 _hyper_1_1_chunk | 2026-04-16 00:00:00+00 | 2026-04-23 00:00:00+00 | f
 _hyper_1_2_chunk | 2026-04-23 00:00:00+00 | 2026-04-30 00:00:00+00 | f

Each chunk spans exactly 7 days (2026-04-16 to 2026-04-23, then 2026-04-23 to 2026-04-30). Let’s query the last 2 days and see which chunks get used:

-- First 3 rows
SELECT * FROM sensor_data
WHERE time > '2026-04-26 00:00:00'
ORDER BY time ASC LIMIT 3;

-- Last 3 rows
SELECT * FROM sensor_data
WHERE time > '2026-04-26 00:00:00'
ORDER BY time DESC LIMIT 3;

          time          | sensor_id |    temperature     |      humidity
------------------------+-----------+--------------------+--------------------
 2026-04-26 00:00:10+00 | sensor_5  | 31.067574603033847 |  32.42945016679141
 2026-04-26 00:00:10+00 | sensor_1  | 29.770821180002446 |  33.87789610251711
 2026-04-26 00:00:10+00 | sensor_2  |   30.5820979427979 | 36.640107625668165
(3 rows)

          time          | sensor_id |    temperature     |      humidity
------------------------+-----------+--------------------+--------------------
 2026-04-28 00:00:00+00 | sensor_1  |  18.82289094155276 | 61.290098283252824
 2026-04-28 00:00:00+00 | sensor_2  | 17.298999086810714 |  60.28357262461673
 2026-04-28 00:00:00+00 | sensor_3  |  18.31179563705799 | 61.943984006436224
(3 rows)

All rows fall between April 26 and April 28. To verify that only the second chunk was scanned, use EXPLAIN to explain the query plan:

EXPLAIN SELECT * FROM sensor_data
WHERE time > '2026-04-26 00:00:00';

                                                     QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------
 Index Scan using _hyper_1_2_chunk_sensor_data_time_idx on _hyper_1_2_chunk  (cost=0.42..3618.97 rows=85792 width=33)
   Index Cond: ("time" > '2026-04-26 00:00:00+00')

The plan only references _hyper_1_2_chunk (Apr 23-30). The first chunk _hyper_1_1_chunk (Apr 16-23) doesn’t appear at all, meaning TimescaleDB skipped it entirely.

time_bucket(): Flexible Time-Based Aggregations

Time-series analysis often requires grouping data into fixed time windows: average temperature every 15 minutes, total requests per 6-hour block, or peak usage per 3-day period.

PostgreSQL’s built-in date_trunc() only supports fixed calendar intervals like hour, day, or month. For custom intervals like 15 minutes, you have to manually calculate which bucket each timestamp falls into:

-- PostgreSQL workaround for 15-minute buckets
SELECT
    date_trunc('hour', time)
        + INTERVAL '15 min' * FLOOR(EXTRACT(MINUTE FROM time) / 15)
        AS bucket,
    sensor_id,
    AVG(temperature) AS avg_temp
FROM sensor_data
WHERE sensor_id = 'sensor_1'
  AND time BETWEEN '2026-04-27 00:00:00' AND '2026-04-27 02:00:00'
GROUP BY bucket, sensor_id
ORDER BY bucket;

         bucket         | sensor_id |      avg_temp
------------------------+-----------+--------------------
 2026-04-27 00:00:00+00 | sensor_1  | 13.542631344509111
 2026-04-27 00:15:00+00 | sensor_1  |  13.23800229831265
 2026-04-27 00:30:00+00 | sensor_1  |  13.20352104441286
 2026-04-27 00:45:00+00 | sensor_1  | 13.119135414712746
 2026-04-27 01:00:00+00 | sensor_1  | 12.889897733150352
 2026-04-27 01:15:00+00 | sensor_1  | 12.876690040717648
 2026-04-27 01:30:00+00 | sensor_1  |  12.72343003629354
 2026-04-27 01:45:00+00 | sensor_1  |  12.58139781130447
 2026-04-27 02:00:00+00 | sensor_1  |   11.8816092609178

TimescaleDB’s time_bucket() produces the same result with a single function call:

SELECT
    time_bucket('15 minutes', time) AS bucket,
    sensor_id,
    AVG(temperature) AS avg_temp,
    AVG(humidity) AS avg_humidity
FROM sensor_data
WHERE sensor_id = 'sensor_1'
  AND time BETWEEN '2026-04-27 00:00:00' AND '2026-04-27 02:00:00'
GROUP BY bucket, sensor_id
ORDER BY bucket;

         bucket         | sensor_id |      avg_temp      |    avg_humidity
------------------------+-----------+--------------------+--------------------
 2026-04-27 00:00:00+00 | sensor_1  | 13.542631344509111 | 29.297833301485014
 2026-04-27 00:15:00+00 | sensor_1  |  13.23800229831265 |  29.57904716854577
 2026-04-27 00:30:00+00 | sensor_1  |  13.20352104441286 |  30.02442153150598
 2026-04-27 00:45:00+00 | sensor_1  | 13.119135414712746 | 30.389413838577834
 2026-04-27 01:00:00+00 | sensor_1  | 12.889897733150352 |  30.57742656702146
 2026-04-27 01:15:00+00 | sensor_1  | 12.876690040717648 | 30.799536994969888
 2026-04-27 01:30:00+00 | sensor_1  |  12.72343003629354 | 31.520854251589334
 2026-04-27 01:45:00+00 | sensor_1  |  12.58139781130447 | 31.613018125020453
 2026-04-27 02:00:00+00 | sensor_1  |   11.8816092609178 |   33.0892996678552

Continuous Aggregates: Pre-Computed Views That Refresh Incrementally

When you run the same aggregation query repeatedly (e.g., hourly averages for a dashboard), PostgreSQL recomputes it from raw data every time. A materialized view solves this by storing the query result in a table, so future reads are instant.

The problem is that materialized views don’t update themselves. Every time new data arrives, you have to manually run REFRESH MATERIALIZED VIEW to keep the results current. This rebuilds the entire view from scratch, which can take minutes to hours on large tables.

TimescaleDB’s continuous aggregates take a smarter approach: they remember where they left off and only process new data on each refresh.

Let’s compare the two approaches with an example:

Materialized view:    recompute ALL buckets on refresh
┌──────┬──────┬──────┬──────┬──────┬──────┐
│ Jan  │ Feb  │ Mar  │ Apr  │ Apr  │ Apr  │
│ redo │ redo │ redo │ redo │ redo │ redo │
└──────┴──────┴──────┴──────┴──────┴──────┘

Continuous aggregate: only recompute CHANGED buckets
┌──────┬──────┬──────┬──────┬──────┬──────┐
│ Jan  │ Feb  │ Mar  │ Apr  │ Apr  │ Apr  │
│ skip │ skip │ skip │ skip │ skip │ redo │
└──────┴──────┴──────┴──────┴──────┴──────┘
                                     ▲
                                 new data here

With a materialized view, all six buckets are recomputed even though only the last one has new data. With a continuous aggregate, the first five are skipped entirely.

Creating a Continuous Aggregate

Let’s create a continuous aggregate that computes hourly averages, min, max, and reading count per sensor. The syntax is similar to a materialized view, with timescaledb.continuous added:

CREATE MATERIALIZED VIEW sensor_hourly
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 hour', time) AS hour,
    sensor_id,
    AVG(temperature) AS avg_temp,
    MIN(temperature) AS min_temp,
    MAX(temperature) AS max_temp,
    AVG(humidity) AS avg_humidity,
    COUNT(*) AS readings
FROM sensor_data
GROUP BY hour, sensor_id;

This creates the aggregate and materializes the existing data. You can query it like any view:

SELECT hour, sensor_id, avg_temp, readings
FROM sensor_hourly
WHERE sensor_id = 'sensor_1'
ORDER BY hour DESC
LIMIT 5;

          hour          | sensor_id |      avg_temp      | readings
------------------------+-----------+--------------------+----------
 2026-04-28 00:00:00+00 | sensor_1  |  18.82289094155276 |        1
 2026-04-27 23:00:00+00 | sensor_1  |  17.62569258253398 |      360
 2026-04-27 22:00:00+00 | sensor_1  | 16.916727264064487 |      360
 2026-04-27 21:00:00+00 | sensor_1  | 16.140865785926938 |      360
 2026-04-27 20:00:00+00 | sensor_1  | 15.457800483137094 |      360

Each hour has 360 readings (one every 10 seconds = 6 per minute x 60 minutes). The midnight bucket has just 1 reading because that’s the last timestamp in our dataset.

Refreshing After New Data Arrives

Now let’s insert a new reading and see how the aggregate handles it:

INSERT INTO sensor_data (time, sensor_id, temperature, humidity)
VALUES ('2026-04-28 00:10:00', 'sensor_1', 25.3, 51.2);

Then refresh only the time range that changed:

CALL refresh_continuous_aggregate('sensor_hourly', '2026-04-28 00:00:00', '2026-04-28 02:00:00');

Now query the aggregate to see the update:

SELECT hour, sensor_id, avg_temp, readings
FROM sensor_hourly
WHERE sensor_id = 'sensor_1'
ORDER BY hour DESC
LIMIT 5;

          hour          | sensor_id |      avg_temp      | readings
------------------------+-----------+--------------------+----------
 2026-04-28 00:00:00+00 | sensor_1  |  22.06144547077638 |        2
 2026-04-27 23:00:00+00 | sensor_1  |  17.62569258253398 |      360
 2026-04-27 22:00:00+00 | sensor_1  | 16.916727264064487 |      360
 2026-04-27 21:00:00+00 | sensor_1  | 16.140865785926938 |      360
 2026-04-27 20:00:00+00 | sensor_1  | 15.457800483137094 |      360

The midnight bucket went from 1 reading to 2, with its average updated to (18.82289 + 25.3) / 2 ≈ 22.06. The 20:00 through 23:00 buckets are unchanged because they fall outside the refresh window (2026-04-28 00:00:00 to 2026-04-28 02:00:00).

Adding a Refresh Policy

In practice, you wouldn’t refresh manually every time new data arrives. Instead, set up a policy that runs the refresh automatically:

SELECT add_continuous_aggregate_policy(
    'sensor_hourly',
    start_offset  => INTERVAL '3 hours',
    end_offset    => INTERVAL '1 hour',
    schedule_interval => INTERVAL '1 hour'
);

This tells TimescaleDB:

• schedule_interval: Run the refresh job every hour

• start_offset: Look back 3 hours for any data that changed

• end_offset: Don’t process the most recent hour (it’s likely still receiving data)

The refresh processes only the buckets that fall within the start_offset to end_offset window. On a table with months of data, this takes seconds instead of minutes:

Table with 3 months of data, policy runs at 10:00:

│ Jan │ Feb │ Mar │ ... │ 7:00 │ 8:00 │ 9:00 │10:00│
│skip │skip │skip │     │ redo │ redo │ redo │skip │
│◄──    untouched    ──►│◄──     3 hrs    ──►│     │

Monitoring Refresh Jobs

Check the status of refresh jobs:

SELECT job_id, application_name, schedule_interval, next_start
FROM timescaledb_information.jobs
WHERE application_name LIKE '%Continuous%';

 job_id |              application_name              | schedule_interval |          next_start
--------+--------------------------------------------+-------------------+-------------------------------
   1002 | Refresh Continuous Aggregate Policy [1002] | 01:00:00          | 2026-04-30 05:37:23.960309+00

This confirms the policy is active: job 1002 refreshes the sensor_hourly aggregate every hour, with the next run scheduled for the time shown in next_start.

Compression: Columnar Storage with 90%+ Reduction

As time-series tables grow, storage becomes a concern. A sensor writing every 10 seconds across 5 devices generates over 300,000 rows per week. Over months, that adds up to gigabytes of data.

The good news is that time-series data compresses far better than transactional data. In a users table, every row has a different name, email, and address. In a sensor table, the same sensor IDs appear over and over, timestamps increment by the same interval, and readings barely change from one row to the next. This repetition means the database can store the data in a fraction of the space:

10 rows of sensor data:

│ sensor_1 │ 10:00:10 │ 22.1 │
│ sensor_1 │ 10:00:20 │ 22.3 │
│ sensor_1 │ 10:00:30 │ 22.2 │
│ sensor_2 │ 10:00:10 │ 45.0 │
│ sensor_2 │ 10:00:20 │ 45.2 │
│ sensor_2 │ 10:00:30 │ 44.8 │

After compression:

  sensor_id: "sensor_1" x3, "sensor_2" x3
  time:      10:00:10, +10s, +10s, 10:00:10, +10s, +10s
  temp:      22.1, +0.2, -0.1, 45.0, +0.2, -0.4

TimescaleDB applies different compression strategies depending on the data type:

• Timestamps: Stores the difference between consecutive values (like the +10s, +10s, +10s pattern above)

• Repeated strings (like sensor IDs): Stores the value once and references it across all rows

• Floating-point values: Stores the difference between consecutive readings (like the +0.2, -0.1 pattern above)

When Compression Happens

You wouldn’t want to compress data that’s still being written to. Each insert into a compressed chunk requires decompression, which is slow. TimescaleDB avoids this by default: hypertables created with WITH (tsdb.hypertable) automatically compress only chunks older than 7 days, leaving recent data uncompressed for fast writes.

┌─────────────┐ ┌─────────────┐ ┌────────────--─┐
│  Apr 7-14   │ │  Apr 14-21  │ │  Apr 21-28    │
│ compressed  │ │ compressed  │ │ uncompressed  │
│ (storage ↓) │ │ (storage ↓) │ │ (fast writes) │
└─────────────┘ └─────────────┘ └────────────-─-┘
│◄──   older than 7 days   ──►│ │◄──  recent ──►│
        auto-compressed          still receiving
                                   new inserts

To use a different threshold, remove the default policy and add a new one. For example, to compress data older than 30 days instead of 7:

-- Remove the default policy created with the hypertable
CALL remove_columnstore_policy('sensor_data');

-- Add a new policy with the desired interval
CALL add_columnstore_policy('sensor_data', after => INTERVAL '30 days');

Configuring Compression

Compression is already enabled by default when you create a hypertable. No extra setup is needed for basic compression. However, you can improve query performance on compressed data by telling TimescaleDB how to organize the rows:

CREATE TABLE sensor_data_v2 (
    time TIMESTAMPTZ NOT NULL,
    sensor_id TEXT NOT NULL,
    temperature DOUBLE PRECISION,
    humidity DOUBLE PRECISION
) WITH (
    tsdb.hypertable,
    tsdb.segmentby = 'sensor_id',
    tsdb.orderby = 'time DESC'
);

• segmentby = 'sensor_id': Compresses each sensor’s data into a separate block. A query for sensor_1 only reads that block, skipping all other sensors.

• orderby = 'time DESC': Sorts each block newest first. A query for the latest reading finds it at the top without scanning the entire block.

For example, if you query WHERE sensor_id = 'sensor_1', here’s what happens with and without segmentby:

Without segmentby (one block):

┌──────────────────────────────────────────┐
│ sensor_1, sensor_2 data mixed together   │
│ query for sensor_1 decompresses ALL      │
└──────────────────────────────────────────┘

With segmentby = 'sensor_id' (one block per sensor):

┌─ sensor_1 ─────┐  ┌─ sensor_2 ─────┐
│ time DESC      │  │ time DESC      │
│ 10:00:30 22.2  │  │ 10:00:30 44.8  │
│ 10:00:20 22.3  │  │ 10:00:20 45.2  │
│ 10:00:10 22.1  │  │ 10:00:10 45.0  │
└────────────────┘  └────────────────┘
      read ✓              skip

With 100 sensors, this means a query for one sensor decompresses 1% of the data instead of all of it.

Checking Compression Results

The automatic policy runs daily, so let’s compress a chunk manually to see the results immediately:

SELECT compress_chunk('_timescaledb_internal._hyper_1_1_chunk');

Now check the chunk status:

SELECT chunk_name, range_start, range_end, is_compressed
FROM timescaledb_information.chunks
WHERE hypertable_name = 'sensor_data'
ORDER BY range_start;

    chunk_name    |      range_start       |       range_end        | is_compressed
------------------+------------------------+------------------------+---------------
 _hyper_1_1_chunk | 2026-04-16 00:00:00+00 | 2026-04-23 00:00:00+00 | t
 _hyper_1_2_chunk | 2026-04-23 00:00:00+00 | 2026-04-30 00:00:00+00 | f

is_compressed = t confirms the first chunk is now compressed.

Compression doesn’t change how you write queries. The same SQL works on both compressed and uncompressed data:

-- This queries the compressed chunk (_hyper_1_1_chunk) directly
SELECT sensor_id, AVG(temperature) AS avg_temp, COUNT(*) AS readings
FROM sensor_data
WHERE time BETWEEN '2026-04-21 00:00:00' AND '2026-04-22 00:00:00'
  AND sensor_id = 'sensor_1'
GROUP BY sensor_id;

 sensor_id |      avg_temp      | readings
-----------+--------------------+----------
 sensor_1  | 15.388563698009191 |     8641

Retention Policies: Automated Data Lifecycle Management

Time-series tables grow continuously, and at some point old data needs to go. In plain PostgreSQL, DELETE removes rows one at a time, which is slow on large tables.

TimescaleDB solves this by dropping entire chunks instead of deleting individual rows. A chunk is a separate table internally, so dropping it is instant regardless of how many rows it contains.

Retention policy: drop_after => 6 months

│  Oct   │  Nov   │  Dec   │ ... │  Mar   │  Apr   │  May   │
│  DROP  │  DROP  │  DROP  │     │  keep  │  keep  │  keep  │
│◄──── older than 6 months  ────►│◄─── within 6 months  ───►│
        dropped instantly                 untouched

To have TimescaleDB drop old chunks automatically on a schedule, add a retention policy:

SELECT add_retention_policy('sensor_data', drop_after => INTERVAL '6 months');

This runs a background job that automatically drops chunks containing data older than 6 months.

Combining Retention with Continuous Aggregates

Deleting all old data saves storage but means you can no longer answer questions like “how does this month compare to last year?” A common solution is to delete the raw readings after a short period, but keep the pre-computed summaries (hourly averages, daily totals) for much longer.

Here, raw data from sensor_data is deleted after 30 days to save storage, while the sensor_hourly continuous aggregate retains hourly averages for 1 year:

-- Keep raw data for 30 days
SELECT add_retention_policy('sensor_data', drop_after => INTERVAL '30 days');

-- Keep hourly aggregates for 1 year
SELECT add_retention_policy('sensor_hourly', drop_after => INTERVAL '1 year');

This gives you detailed data for recent debugging and long-term trends for historical analysis, without the storage cost of keeping every raw reading forever.

Key Takeaways

TimescaleDB adds time-series capabilities to PostgreSQL without requiring you to switch databases or learn a new query language.

TimescaleDB is worth considering when your PostgreSQL tables are growing with timestamped data and you’re noticing slower inserts, expensive aggregation queries, or increasing storage costs. Since it’s an extension, you can try it on a single table without changing anything else in your database.

Related Tutorials

• Implement Semantic Search in Postgres Using pgvector and Ollama: Another PostgreSQL extension that specializes Postgres for a workload it wasn’t originally built for, this time vector search for RAG.

• A Deep Dive into DuckDB for Data Scientists: An embedded analytical alternative for cases where you need fast aggregations on a single machine without running a database server.

• Polars vs. Pandas: A Fast, Multi-Core Alternative for DataFrames: A columnar processing engine in Python, useful for analyzing data pulled out of TimescaleDB.

Originally published on CodeCut.

📅 Today’s Picks

Claude Code: Run Tasks to Completion with /goal

Problem

Starting a task in Claude Code often means spending multiple turns going back and forth before it finally reaches the result you wanted.

Solution

Claude Code’s /goal command lets you define a completion condition once, then continues working until that condition is met.

After each turn, a lightweight model evaluates the conversation against your condition. If the goal is not satisfied, Claude gets sent back for another turn with a short explanation of what still needs to be done.

To get the most out of /goal, write a condition the evaluator can actually verify.

• Name a measurable end state instead of a vague target (e.g. “all tests in test/auth pass” instead of “auth is working”)

• List constraints that must hold along the way (e.g. “no other test file is modified” or “no tests are skipped or deleted”)

• Add a safety limit like “stop after 20 turns” to avoid endless iterations.

📖 View The Docs

smolagents: Build an AI Agent in 4 Lines of Code

Problem

LangChain and LangGraph are powerful, but they also come with concepts like chains, runnables, and state graphs.

That is useful for complex workflows, but overkill when you just want to give a model a tool and let it solve a task.

Solution

smolagents from Hugging Face is a barebones agent library designed for simplicity.

In roughly 1,000 lines of code, it gives you the core pieces needed to build a working agent with a single import and three lines of setup.

Key features:

• Agents act by writing Python in a sandbox, the same way you’d script a task in a notebook

• Model-agnostic design, swap between OpenAI, Anthropic, local Ollama, or Hugging Face Inference

• Push agents to the Hugging Face Hub to share with the community

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

Introduction

Most Docker images contain far more than a Python application actually needs at runtime. They include full OS layers with shells, compilers, and utilities that often go completely unused, leading to unnecessarily large images that consume storage and slow deployment pipelines.

SlimToolkit analyzes your container at runtime, identifies which files are actually used, and builds a minimal image with only those dependencies.

This article walks through slimming a Chainlit LLM chatbot, but the same approach works on any Python container.

What Is SlimToolkit?

SlimToolkit is a command-line tool that strips unused files from a container image without touching your Dockerfile. It works in two steps:

1. Static analysis. Looks at the image’s contents without running it.

2. Dynamic analysis. Runs the image to see which files the app actually uses.

The first step lists everything in the image. The second narrows that list to what the app actually needs. Everything outside the second list gets stripped.

Install it via the official install script (works on Linux and macOS):

curl -sL https://raw.githubusercontent.com/slimtoolkit/slim/master/scripts/install-slim.sh | sudo -E bash -

Or with Homebrew on macOS:

brew install docker-slim

Verify the install:

slim --version

Output

mint version darwin/arm64|Aurora|1.41.8|latest|latest

💻 Get the Code: The complete source code and Jupyter notebook for this tutorial are available on GitHub. Clone it to follow along!

Build the Chatbot Image

To test SlimToolkit, build a small Chainlit chatbot image first. We’ll write a small Chainlit chatbot, package it with a Dockerfile, and build it.

The Chainlit App

The chatbot app uses two libraries:

• Chainlit: an open-source Python framework for building LLM chat UIs; provides the web interface and message handling

• OpenAI SDK: calls gpt-4o-mini for responses

# app.py
import os
import chainlit as cl
from openai import AsyncOpenAI

client = AsyncOpenAI(api_key=os.environ.get("OPENAI_API_KEY"))


@cl.on_chat_start
async def start():
    cl.user_session.set("messages", [])


@cl.on_message
async def main(message: cl.Message):
    messages = cl.user_session.get("messages")
    messages.append({"role": "user", "content": message.content})

    response = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
    )
    reply = response.choices[0].message.content

    messages.append({"role": "assistant", "content": reply})
    cl.user_session.set("messages", messages)

    await cl.Message(content=reply).send()

Here’s what happens when someone uses the chatbot:

• At import time, AsyncOpenAI reads the OPENAI_API_KEY environment variable and constructs the client

• When a new chat session opens, @cl.on_chat_start initializes an empty message list for that session

• When the user types something, @cl.on_message appends it to the history, sends the whole conversation to gpt-4o-mini, stores the reply, and displays it

Before containerizing, let’s test it locally first. Export your OpenAI key:

export OPENAI_API_KEY=sk-...

Then run the app:

chainlit run app.py

Open http://localhost:8000, you should see the Chainlit welcome screen.

The Dockerfile

Pin every dependency in a requirements.txt so the build is reproducible:

# requirements.txt
chainlit==2.11.1
openai==2.16.0

Create a Dockerfile to build the image:

# Dockerfile
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

EXPOSE 8000
CMD ["chainlit", "run", "app.py", "--host", "0.0.0.0", "--port", "8000", "-h"]

Here’s what each step does:

• Pulls python:3.11-slim as the base image

• Installs the pinned dependencies from requirements.txt

• Copies in app.py

• Starts Chainlit on port 8000 when the container runs

Build the image:

docker build -t llm-chatbot:fat .

Verify the image size:

docker images llm-chatbot:fat

Output

IMAGE             ID             DISK USAGE   CONTENT SIZE   EXTRA
llm-chatbot:fat   e8de32dd85d4        308MB             0B

Around 300 MB. Let’s see if we can shrink it with SlimToolkit.

Slim the Image

To slim the image, start with the basic command:

slim build \
    --target llm-chatbot:fat \
    --tag llm-chatbot:slim \
    --env OPENAI_API_KEY=$OPENAI_API_KEY

Each flag plays a role in the slim build:

• --target llm-chatbot:fat: tells slim which image to minify

• --tag llm-chatbot:slim: names the output image

• --env OPENAI_API_KEY=$OPENAI_API_KEY: sets the env var inside slim’s probe container so the module-level AsyncOpenAI() can construct at import time

Here’s what happens when you run the command:

1. Slim inspects the fat image

2. Starts it in a sandbox

3. Sends a GET / probe and records every file the container touches

4. Builds a slim image containing only those files

But the default GET / only loads the chat UI shell. It doesn’t actually send a chat message, so files needed for the chat path (OpenAI’s lazy submodules, httpcore, etc.) get stripped. To trace the chat path too, add --continue-after enter:

slim build \
    --target llm-chatbot:fat \
    --tag llm-chatbot:slim \
    --env OPENAI_API_KEY=$OPENAI_API_KEY \
    --continue-after enter

--continue-after enter pauses slim after the default probe so you can open the chatbot in a browser and send a message; this tells slim what files the chat actually needs.

Now that the chat path runs during the probe, slim keeps openai.resources and httpcore in the final image:

Let’s compare both images:

docker images llm-chatbot

Output

IMAGE              ID             DISK USAGE   CONTENT SIZE   EXTRA
llm-chatbot:fat    e8de32dd85d4        308MB             0B    U
llm-chatbot:slim   e1f5e9b31e53        123MB             0B

Nice! We reduced the image size from 308 MB to 123 MB, about a 2.5x reduction.

Let’s run the slim image with the environment variable and see what happens.

docker run -p 8000:8000 -e OPENAI_API_KEY=$OPENAI_API_KEY llm-chatbot:slim

Output

File "/usr/local/lib/python3.11/site-packages/chainlit/server.py", line 181, in get_build_dir
    raise FileNotFoundError(f"{local_target} built UI dir not found")
FileNotFoundError: libs/copilot built UI dir not found

The error comes from a mismatch between Chainlit and slim:

• Chainlit expects chainlit/copilot/ to exist when the server starts

• Even with --continue-after enter, slim removed chainlit/copilot/ because the chat you exercised in the browser only loaded the main chat UI. The Copilot widget is a separate Chainlit feature; no file inside chainlit/copilot/ was opened during the probe

• Now Chainlit’s startup check finds the directory missing and the container crashes

This is not unique to Chainlit or chat-UI frameworks. Any framework that loads features lazily but checks for them at startup is vulnerable. Django’s admin, FastAPI apps with multiple routers, ML serving frameworks with embedded UIs, and plugin systems all hit the same problem.

Add --include-path for the Chainlit Package

The fix is --include-path, which tells SlimToolkit to preserve a path regardless of whether probing touched it. The path that matters is the whole Chainlit package directory, which contains every feature bundle Chainlit ships:

slim build \
    --target llm-chatbot:fat \
    --tag llm-chatbot:slim \
    --include-path /usr/local/lib/python3.11/site-packages/chainlit \
    --continue-after enter \
    --env OPENAI_API_KEY=$OPENAI_API_KEY

Compare the images again:

docker images llm-chatbot

Output

IMAGE              ID             DISK USAGE   CONTENT SIZE   EXTRA
llm-chatbot:fat    e8de32dd85d4        308MB             0B    U
llm-chatbot:slim   952b6b44df9f        163MB             0B    U

Re-run the image to confirm it works:

docker run -p 8000:8000 -e OPENAI_API_KEY=$OPENAI_API_KEY llm-chatbot:slim

With the chainlit directory preserved, the slim container starts and runs as expected.

Inspecting the Result with slim xray

To see exactly which files slim stripped, run slim xray against both images. It reverse-engineers a built image into a JSON report listing every file, its size, and the layer it came from. Slim always writes its output to slim.report.json, so rename the first report before the second run overwrites it:

slim xray --target llm-chatbot:fat
mv slim.report.json fat.report.json

slim xray --target llm-chatbot:slim

Each report is several megabytes of JSON, which makes manual comparison painful. To handle that, I packaged the diff and summary steps into compare.sh:

bash compare.sh

Here are the biggest deletions, grouped by bucket:

Removed                          Size    Bucket
/usr/bin/perl                    3.8 MB  OS cruft
libapt-pkg.so                    2.4 MB  OS cruft (apt)
ensurepip/pip-24.0.whl           2.1 MB  Python build leftover
libdb-5.3.so                     1.8 MB  OS cruft (apt)
/usr/bin/sqv                     1.6 MB  OS cruft (apt’s PGP verifier)
/usr/bin/bash                    1.4 MB  OS cruft
ensurepip/setuptools-79.0.1.whl  1.3 MB  Python build leftover
multidict/_multidict.so          923 kB  Unused part of aiohttp
jiter/jiter.so                   880 kB  Unused part of openai
pydoc_data/topics.py             775 kB  Python build leftover
aiohttp/_http_writer.so          600 kB  Unused part of aiohttp

The biggest savings come from the Debian base image, not from the Python packages. Of the top 11 deletions:

• Five are base-image binaries: perl, bash, libapt-pkg, libdb-5.3, sqv

• Three are install-time leftovers: bundled pip and setuptools wheels, plus the pydoc help-text database

• Three are write-side or helper modules of runtime libraries the probe didn’t exercise: aiohttp/_http_writer, multidict, jiter

Final Thoughts

A single slim build command took this chatbot from 308 MB to 163 MB. That is one data point on one image. Your numbers will likely look different. The wider the gap between what your image installs and what it actually runs at runtime, the bigger the reduction tends to be. Give it a try on your own images and see what kind of improvement you get.

Related Tutorials

• How to Test GitHub Actions Locally with act: Iterate on a slim-build CI workflow locally before pushing.

• Unregistry: Skip the Registry, Deploy Docker Images Directly: Deploy the slimmed image straight to a server, no registry required.

📚 Want to go deeper? Learning new techniques is the easy part. Knowing how to structure, test, and deploy them is what separates side projects from real work. My book shows you how to build data science projects that actually make it to production. Get the book →

Originally published on CodeCut.

📅 Today’s Picks

Never Use yaml.load() in Python. Here’s Why.

Problem

Has your AI assistant ever suggested yaml.load() to parse a config file?

It’s an easy mistake: yaml.load() looks like a clean way to turn YAML into a Python dict, and AI tools generate it freely because it appears in plenty of legacy code.

But yaml.load() can do more than parse text. It can execute Python objects embedded in the YAML, meaning anyone who controls that file could run shell commands on your machine.

Solution

Use yaml.safe_load() instead. It only supports standard YAML types like mappings, lists, strings, numbers, booleans, and null, and rejects anything that tries to execute code.

To catch unsafe yaml.load() calls automatically, scan your codebase with Bandit.

📖 View Full Article

Context7: Stop AI Agents from Generating Deprecated Syntax

Problem

AI coding assistants often generate code using training data that may already be outdated.

That means if you ask for Polars 1.x code, the model may still generate deprecated 0.x APIs from older training data.

Solution

Context7 solves this by pulling the latest, version-specific library docs directly into the prompt before the assistant writes any code. That way, it generates code against the API that actually exists today.

Other capabilities:

• One command sets it up across Cursor, Claude Code, Copilot, and 30+ clients

• Trigger with use context7 in any prompt

• Runs as either an MCP server or a CLI + skill, so it works with or without MCP support

• Automatically loads version-matched docs like “Polars 1.0

☕️ Weekly Finds

beartype [Code Quality] - Near-real-time pure-Python runtime type-checker. Decorate a function and catch type violations the moment they happen, not after a stack trace.

vulture [Code Quality] - Find dead Python code. Scans your project for unused functions, classes, imports, and variables so you can safely delete them.

pyupgrade [Code Quality] - A tool (and pre-commit hook) that automatically upgrades Python syntax to newer language versions. Drop f-strings, dict literals, and modern type hints in one sweep.

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

📅 Today’s Picks

Bandit: Catch Insecure Patterns in AI-Generated Python Code

Problem

LLMs have become extremely good at generating syntactically valid Python, but security quality has barely improved.

Veracode’s Spring 2026 GenAI Code Security Report found that security pass rates have remained stuck near 55% since 2024.

That gap exists because models learn from public code full of insecure patterns and reproduce them when prompted.

At the same time, reviewers usually verify whether code works, not whether it introduces vulnerabilities.

Solution

Bandit is a static analyzer for Python that identifies insecure patterns by matching code against 60+ Common Weakness Enumeration (CWE) rules.

Key capabilities:

• Flags hardcoded secrets, weak hashes (MD5, SHA1), and unsafe deserialization

• Detects SQL string concatenation, eval, and exec injection risks

• Catches empty except blocks and unpinned Hugging Face downloads

• Integrates with pre-commit hooks and GitHub Actions out of the box

• Maps every finding to a CWE ID so issues stay auditable

📖 View Full Article

llama.cpp: Run LLMs 70% Faster Than Ollama on the Same GPU

Problem

An r/LocalLLaMA benchmark on Qwen-3 Coder 32B (FP16) measured Ollama at 30 tokens/sec and llama.cpp at 52 tokens/sec on the same model and hardware.

Ollama is built on top of llama.cpp, so the core inference engine is identical. The performance gap comes from configuration.

Solution

Ollama ships with conservative defaults designed to work reliably across many systems. llama.cpp exposes the low-level tuning flags so you can optimize specifically for your hardware.

Key flags to tune:

• GPU offload (-ngl 99): put the entire model on your GPU instead of partial offload

• Throughput (--batch-size): bigger means faster prompts but more GPU memory

• Context length (--cache-type-k q4): use less GPU memory per token so longer prompts fit

• Generation speed (-md): run a small helper model alongside your main model for 2-3x faster generation

📖 View Source

☕️ Weekly Finds

scalene [Code Quality] - A high-performance CPU, GPU, and memory profiler for Python with AI-powered optimization proposals.

pymc-marketing [Data Analysis] - Bayesian marketing toolbox in PyMC. Includes Media Mix Modeling (MMM), customer lifetime value (CLV), and buy-till-you-die (BTYD) models.

cronboard [Workflow Automation] - A terminal-based dashboard for managing cron jobs locally and on remote servers. Add, edit, pause, resume, and search jobs from the terminal.

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

📅 Today’s Picks

Slim: Shrink Docker Images by 30x with One Command

Problem

Docker images include the entire OS layer. For a simple Python app, this is unnecessary because it never touches the shells, compilers, and system utilities bundled inside.

This inflates images to hundreds of megabytes, wasting storage and adding time to every deploy.

Solution

Slim automatically analyzes your container at runtime to identify which files are actually used, then builds a minimal image with only essential components.

Slim works alongside Docker, not instead of it:

• Step 1: Build your image with docker build

• Step 2: Minify with slim build your-image

• Step 3: Push the .slim image to your registry

• Your Dockerfile and workflow stay the same

Polars: Vectorize List Column Transformations with list.eval

Problem

When working with list columns, most people reach for apply with a Python lambda that loops over every list row by row.

That approach breaks vectorization, so performance quickly degrades as the dataset grows.

Solution

Polars solves this with list.eval, which runs a full expression against each list using pl.element() and stays fully vectorized.

Key benefits:

• Vectorized per-element transformations without Python loops

• Support for a wide range of expressions, including aggregations like max, mean, and sum

• Composable with other Polars expressions for clean, readable pipelines

📖 View Full Article

☕️ Weekly Finds

pyupgrade [Code Quality] - A tool (and pre-commit hook) to automatically upgrade syntax for newer versions of Python.

doxx [Python Utils] - Expose the contents of .docx files without leaving your terminal. Fast, safe, and smart, with no Office required.

ingestr [Data Processing] - ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

📅 Today’s Picks

Mem0: Turn AI Conversations into a Knowledge Graph

Problem

Vector databases store each memory as an independent embedding. You can search for similar text, but they cannot link entities across facts or follow relationships between them.

Store “Alice and Bob are collaborating” and “Alice leads payments” separately, and a vector store cannot recognize that both facts refer to the same Alice.

Solution

Mem0 fixes this by extracting entities and relationships from each conversation.

When you store “Alice and Bob are collaborating” and “Alice leads payments,” Mem0 recognizes that both facts refer to the same Alice and links them in a knowledge graph.

Key capabilities:

• Reads raw conversations and identifies the entities for you

• Relationship tracking between people, projects, and concepts

• A memory layer that grows smarter with every interaction

🧪 Run code

lakeFS: Bring Git-Style Branches and Commits to Cloud Storage

Problem

Cloud storage like S3, Google Cloud Storage, and Azure Blob has no concept of versions, branches, or commits like Git does.

As a result, testing changes can mean copying terabytes of data, and fixing mistakes requires tedious file-by-file restores.

Solution

lakeFS turns your cloud storage into a Git-like repository. You get branches, commits, merges, and instant rollback, all without copying data.

☕️ Weekly Finds

unsloth [LLM] - Fine-tune LLMs 5x faster with 60% less memory using QLoRA, no accuracy loss.

context7 [RAG] - Pulls up-to-date, version-specific docs and code examples straight into your LLM prompts and AI code editors.

Repomix [LLM] - Packs your entire repository into a single AI-friendly file you can paste into Claude, ChatGPT, or Gemini.

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

📅 Today’s Picks

Generate Time-Sortable IDs with Python 3.14’s UUID v7

Problem

UUID4 generates purely random identifiers that lack chronological ordering.

Without embedded timestamps, you need separate timestamp fields and custom sorting logic to organize records by creation time.

Solution

Python 3.14 introduces UUID version 7 with built-in timestamp ordering.

Key features:

• Determine creation order by comparing two UUIDs directly

• Retrieve exact creation time by extracting the embedded timestamp

Pyrefly: Automatic Type Inference for Python

Problem

By default, MyPy ignores untyped function bodies, forcing you to manually annotate variables and return types to get meaningful type checking.

This adds overhead and clutters your code with types that don’t add much value.

Solution

Pyrefly automatically infers return types and local variable types from usage, so you only annotate function parameters and public APIs where it matters.

Other features:

• Lightning-fast type checking built in Rust

• Works with VSCode, Neovim, Helix, Emacs, and Zed

• Recognizes dataclass transforms from attrs, Pydantic, and similar libraries

☕️ Weekly Finds

pyrsistent [Python Utils] - Persistent/Immutable/Functional data structures for Python

wrapt [Python Utils] - A Python module for decorators, wrappers and monkey patching

Shiny for Python [Dashboard] - Build fast, beautiful reactive web applications in Python

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production

📅 Today’s Picks

kotaemon: Self-Hosted Document QA with Citations in One Command

Problem

Building a RAG app for document Q&A usually means assembling a parser, vector database, retrieval pipeline, and UI from scratch.

Each piece has its own setup, and getting everything to work together can take hours of debugging.

Solution

kotaemon packages the entire RAG stack into a single Docker image, letting you skip the setup and go straight to asking questions.

Key features:

• Citations linked to exact PDF pages for verifiable answers

• Question answering across multiple documents with figures and tables

• Works with local models or cloud APIs like OpenAI, Azure, and Groq

• Extensible Gradio-based UI with multi-user document management

gws: Replace Bulky MCP Tools with 100+ Modular Skill Files

Problem

Connecting AI agents to Google Workspace through MCP often means injecting every tool definition into each request, even if only a couple are needed.

That overhead quickly eats into the token budget, leaving less room for reasoning and task execution.

Solution

gws solves this by replacing bulky tool definitions with 100+ modular SKILL.md files.

Agents load only the skills they need, keeping the context lean and efficient.

Key features:

• Works with Claude Code, Cursor, Gemini CLI, and other AI agents out of the box

• 100+ skill files covering Google Docs, Sheets, Drive, Calendar, and more

• Agents load only relevant skills instead of full tool definitions

☕️ Weekly Finds

unstructured [RAG] - Turn any document into clean, structured data ready for RAG pipelines and LLM applications.

json_repair [LLM] - Repair malformed JSON from LLMs, APIs, and logs. A drop-in replacement for json.loads() that auto-fixes broken output.

MindsDB [AI Agents] - Query AI models directly from your database using SQL. Connect 200+ data sources to LLMs, ML, and vector operations.

💬 Rate Your Experience

How would you rate your newsletter experience? Share your feedback →

🔍 Explore More on CodeCut

• Tool Selector - Discover 70+ Python tools for AI and data science

• Production Ready Data Science - A practical book for taking projects from prototype to production